Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Whisper is an automatic speech recognition (ASR) system trained on 680,000 hours of multilingual and multitask supervised data collected from the web. We show that the use of such a large and diverse dataset leads to improved robustness to accents, background noise, and technical language. Moreover, it enables transcription in multiple languages, as well as translation from those languages into English. We are open-sourcing models and inference code to serve as a foundation for building useful applications and for further research on robust speech processing. In this example, we will transcribe an audio clip locally, containerize the script and then run the container on Bacalhau.
The advantage of using Bacalhau over managed Automatic Speech Recognition services is that you can run your own containers which can scale to do batch process petabytes of videos or audio for automatic speech recognition
Using OpenAI whisper with Bacalhau to process audio files
To get started, you need to install:
Bacalhau client, see more information here
Whisper,
pytorch
pandas
Before we create and run the script we need a sample audio file to test the code for that we download a sample audio clip.
We will create a script that accepts parameters (input file path, output file path, temperature, etc.) and set the default parameters. Also:
If the input file is in mp4 format, then the script converts it to wav format.
Save the transcript in various formats,
We load the large model
Then pass it the required parameters. This model is not only limited to English and transcription, it supports other languages and also does translation, into the following languages:
Next, let's create a openai-whisper script:
Let's run the script with the default parameters:
Viewing the outputs
To build your own docker container, create a Dockerfile, which contains instructions on how the image will be built, and what extra requirements will be included.
We choose pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime as our base image
And then install all the dependencies, after that we will add the test audio file and our openai-whisper script to the container, we will also run a test command to check whether our script works inside the container and if the container builds successfully
:::info See more information on how to containerize your script/app here :::
We will run docker build command to build the container;
Before running the command replace;
hub-user with your docker hub username, If you don’t have a docker hub account follow these instructions to create a Docker account, and use the username of the account you created
repo-name with the name of the container, you can name it anything you want
tag this is not required but you can use the latest tag
In our case
Next, upload the image to the registry. This can be done by using the Docker hub username, repo name or tag.
In our case
We will transcribe the moon landing video, which can be found here: https://www.nasa.gov/multimedia/hd/apollo11_hdpage.html
Since the downloaded video is in mov format we convert the video to mp4 format and then upload it to our public storage in this case IPFS. We will be using NFT.Storage (Recommended Option). To upload your dataset using NFTup just drag and drop your directory it will upload it to IPFS
After the dataset has been uploaded, copy the CID:
bafybeielf6z4cd2nuey5arckect5bjmelhouvn5rhbjlvpvhp7erkrc4nu
To submit a job, run the following Bacalhau command:
Let's look closely at the command above:
-i ipfs://bafybeielf6z4cd2nuey5arckect5bjmelhouvn5r: flag to mount the CID which contains our file to the container at the path /inputs
-p inputs/Apollo_11_moonwalk_montage_720p.mp4 : the input path of our file
-o outputs: the path where to store the outputs
--gpu : here we request 1 GPU
jsacex/whisper: the name and the tag of the docker image we are using
Job status: You can check the status of the job using bacalhau list.
When it says Published or Completed, that means the job is done, and we can get the results.
Job information: You can find out more information about your job by using bacalhau describe.
Job download: You can download your job results directly by using bacalhau get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
To view the file, run the following command:
In this example tutorial, we will show you how to generate realistic images with StyleGAN3 and Bacalhau. StyleGAN is based on Generative Adversarial Networks (GANs), which include a generator and discriminator network that has been trained to differentiate images generated by the generator from real images. However, during the training, the generator tries to fool the discriminator, which results in the generation of realistic-looking images. With StyleGAN3 we can generate realistic-looking images or videos. It can generate not only human faces but also animals, cars, and landscapes.
Generative images with Bacalhau
To get started, you need to install the Bacalhau client, see more information here
To run StyleGAN3 locally, you'll need to clone the repo, install dependencies and download the model weights.
Generate an image using a pre-trained AFHQv2 model
Viewing the output image
To build your own docker container, create a Dockerfile, which contains instructions to build your image.
:::info See more information on how to containerize your script/apphere :::
We will run docker build command to build the container;
Before running the command replace;
hub-user with your docker hub username, If you don’t have a docker hub account follow these instructions to create docker account, and use the username of the account you created
repo-name with the name of the container, you can name it anything you want
tag this is not required but you can use the latest tag
In our case
Next, upload the image to the registry. This can be done by using the Docker hub username, repo name or tag.
In our case
To submit a job, run the following Bacalhau command:
Let's look closely at the command above:
bacalhau docker run: call to bacalhau
--gpu 1: No of GPUs
jsacex/stylegan3: the name and the tag of the docker image we are using
../outputs: path to the output
python gen_images.py: execute the script
--trunc=1 --seeds=2 --network=stylegan3-r-afhqv2-512x512.pkl: The animation length is either determined based on the --seeds value or explicitly specified using the --num-keyframes option. When num keyframes are specified with --num-keyframes, the output video length will be 'num_keyframes*w_frames' frames.
You can also run variations of this command to generate videos and other things. In the following command below, we will render a latent vector interpolation video. This will render a 4x2 grid of interpolations for seeds 0 through 31.
Let's look closely at the command above:
bacalhau docker run: call to bacalhau
--gpu 1: No of GPUs
jsacex/stylegan3: the name and the tag of the docker image we are using
../outputs: path to the output
python gen_images.py: execute the script
--trunc=1 --seeds=2 --network=stylegan3-r-afhqv2-512x512.pkl: The animation length is either determined based on the --seeds value or explicitly specified using the --num-keyframes option. When num keyframes is specified with --num-keyframes, the output video length will be 'num_keyframes_w_frames' frames. If --num-keyframes is not specified, the number of seeds given with --seeds must be divisible by grid size W_H (--grid). In this case, the output video length will be '# seeds/(w*h)*w_frames' frames.
When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.
Job status: You can check the status of the job using bacalhau list.
When it says Completed, that means the job is done, and we can get the results.
Job information: You can find out more information about your job by using bacalhau describe.
Job download: You can download your job results directly by using bacalhau get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
After the download has finished you should see the following contents in the results directory
To view the file, run the following command:
Stable Diffusion is a state of the art text-to-image model that generates images from text and was developed as an open-source alternative to DALL·E 2. It is based on a Diffusion Probabilistic Model and uses a Transformer to generate images from text.
This example demonstrates how to use stable diffusion on a CPU and run it on the Bacalhau network. The first section describes the development of the code and the container. The section demonstrates how to run the job using Bacalhau.
The following image is an example generated by this model.
The original text-to-image stable diffusion model was trained on a fleet of GPU machines, at great cost. To use this trained model for inference, you also need to run it on a GPU.
However, this isn't always desired or possible. One alternative is to use a project called OpenVINO from Intel that allows you to convert and optimize models from a variety of frameworks (and ONNX if your framework isn't directly supported) to run on a supported Intel CPU. This is what we will do in this example.
:::tip Heads up! This example takes about 10 minutes to generate an image on an average CPU. Whilst this demonstrates it is possible, it might not be practical. :::
In order to run this example you need:
A Debian-flavoured Linux (although you might be able to get it working on M1 macs)
The first step is to convert the trained stable diffusion models so that they work efficiently on a CPU using OpenVINO. The example is quite complex, so we have created a separate repository (which is a fork from Github user Sergei Belousov) to host the code. In summary, the code:
Downloads a pre-optimized OpenVINO version of ...
the original pre-trained stable diffusion model ...
which also leverages OpenAI's CLIP transformer ...
and is then wrapped inside an OpenVINO runtime, which reads in and executes the model.
The core code representing these tasks can be found in the stable_diffusion_engine.py file. This is a mashup that creates a pipeline necessary to tokenize the text and run the stable diffusion model. This boilerplate could be simplified by leveraging the more recent version of the diffusers library. But let's crack on.
Note that these dependencies are only known to work on Ubuntu-based x64 machines.
The following commands clone the example repository, and other required repositories, and install the Python dependencies.
Now that we have all the dependencies installed, we can call the demo.py wrapper, which is a simple CLI, to generate an image from a prompt.
Now we have a working example, we can convert it into a format that allows us to perform inference in a distributed environment.
First we will create a Dockerfile to containerize the inference code. The Dockerfile can be found in the repository, but is presented here to aid understanding.
This container is using the python:3.9.9-bullseye image and the working directory is set. Next, the Dockerfile installs the same dependencies from earlier in this notebook. Then we add our custom code and pull the dependent repositories.
We've already pushed this image to GHCR, but for posterity, you'd use a command like this to update it:
To run this example you will need:
Bacalhau installed and running
Bacalhau is a distributed computing platform that allows you to run jobs on a network of computers. It is designed to be easy to use and to run on a variety of hardware. In this example, we will use it to run the stable diffusion model on a CPU.
To submit a job, you can use the Bacalhau CLI. The following command passes a prompt to the model and generates an image in the outputs directory.
:::tip
This will take about 10 minutes to complete. Go grab a coffee. Or a beer. Or both. If you want to block and wait for the job to complete, add the --wait flag.
Furthermore, the container itself is about 15GB, so it might take a while to download on the node if it isn't cached.
:::
Running the commands will output a UUID that represents the job that was created. You can check the status of the job with the following command:
Wait until it says Completed and then get the results.
To find out more information about your job, run the following command:
If you see that the job has completed and there are no errors, then you can download the results with the following command:
After the download has finished you should see the following contents in the results directory:
Dolly 2.0, the groundbreaking, open-source, instruction-following Large Language Model (LLM) that has been fine-tuned on a human-generated instruction dataset, licensed for both research and commercial purposes. Developed using the EleutherAI Pythia model family, this 12-billion-parameter language model is built exclusively on a high-quality, human-generated instruction following dataset, contributed by Databricks employees.
Dolly 2.0 package is open source, including the training code, dataset, and model weights, all available for commercial use. This unprecedented move empowers organizations to create, own, and customize robust LLMs capable of engaging in human-like interactions, without the need for API access fees or sharing data with third parties.
A NVIDIA GPU
Python
Creating the inference script
Install Docker on your local machine.
Sign up for a DockerHub account if you don't already have one. Steps
Step 1: Create a Dockerfile Create a new file named Dockerfile in your project directory with the following content:
This Dockerfile sets up a container with the necessary dependencies and installs the Segment Anything Model from its GitHub repository.
Step 2: Build the Docker Image In your terminal, navigate to the directory containing the Dockerfile and run the following command to build the Docker image:
Replace your-dockerhub-username with your actual DockerHub username. This command will build the Docker image and tag it with your DockerHub username and the name "sam".
Step 3: Push the Docker Image to DockerHub Once the build process is complete, Next, push the Docker image to DockerHub using the following command:
Again, replace your-dockerhub-username with your actual DockerHub username. This command will push the Docker image to your DockerHub repository.
To get started, you need to install the Bacalhau client, see more information here
docker run: Docker command to run a container from a specified image.
--gpu 1: Flag to specify the number of GPUs to use for the execution. In this case, 1 GPU will be used.
-w /inputs: Flag to set the working directory inside the container to /inputs.
-i gitlfs://huggingface.co/databricks/dolly-v2-3b.git: Flag to clone the Dolly V2-3B model from Hugging Face's repository using Git LFS. The files will be mounted to /inputs/databricks/dolly-v2-3b.
-i https://gist.githubusercontent.com/js-ts/d35e2caa98b1c9a8f176b0b877e0c892/raw/3f020a6e789ceef0274c28fc522ebf91059a09a9/inference.py: Flag to download the inference.py script from the provided URL. The file will be mounted to /inputs/inference.py.
jsacex/dolly_inference:latest: The name and the tag of the Docker image.
The command to run inference on the model: python inference.py --prompt "Where is Earth located ?" --model_version "./databricks/dolly-v2-3b".
inference.py: The Python script that runs the inference process using the Dolly V2-3B model.
--prompt "Where is Earth located ?": Specifies the text prompt to be used for the inference.
--model_version "./databricks/dolly-v2-3b": Specifies the path to the Dolly V2-3B model. In this case, the model files are mounted to /inputs/databricks/dolly-v2-3b.
In this example tutorial, we use Bacalhau and Easy OCR to digitize paper records or for recognizing characters or extract text data from images stored on IPFS/Filecoin or on the web. EasyOCR is a ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc. With easy OCR you use the pre-trained models or use your own fine-tuned model.
Using Bacalhau and Easy OCR to extract text data from images stored on the web.
To get started, you need to install the Bacalhau client, see more information here
Install the required dependencies
Load the different example images
List all the images
To displaying an image from the list
Next, we create a reader to do OCR to get coordinates which represent a rectangle containing text and the text itself
:::tip You can skip this step and go straight to running a Bacalhau job :::
We will use the Dockerfile that is already created in the Easy OCR repo. Use the command below to clone the repo
The docker build command builds Docker images from a Dockerfile.
Before running the command replace;
hub-user with your docker hub username, If you don’t have a docker hub account follow these instructions to create docker account, and use the username of the account you created
repo-name with the name of the container, you can name it anything you want
tag this is not required but you can use the latest tag
Next, upload the image to the registry. This can be done by using the Docker hub username, repo name, or tag.
After the repo image has been pushed to Docker Hub, we can now use the container for running on Bacalhau. To submit a job, run the following Bacalhau command:
Since the model and the image aren't present in the container we will mount the image from an URL and the model from IPFS. You can find models to download from here. You can choose the model you want to use in this case we will be using the zh_sim_g2 model
-i ipfs://bafybeibvc......: Mounting the model from IPFS
-i https://raw.githubusercontent.com......... Mounting the Input Image from a URL
--gpu 1: Specifying the no of GPUs
jsacex/easyocr: Name of the Docker image
Breaking up the easyocr command
-- easyocr -l ch_sim en -f ./inputs/chinese.jpg --detail=1 --gpu=True
-l: the name of the model which is ch_sim
-f: path to the input Image or directory
--detail=1: level of detail
--gpu=True: we set this flag to true since we are running inference on a GPU, if you run this on a CPU you set this to false
When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.
Job status: You can check the status of the job using bacalhau list.
When it says Completed, that means the job is done, and we can get the results.
Job information: You can find out more information about your job by using bacalhau describe.
Job download: You can download your job results directly by using bacalhau get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
After the download has finished you should see the following contents in the results directory
To view the file, run the following command:
The identification and localization of objects in images and videos is a computer vision task called object detection. Several algorithms have emerged in the past few years to tackle the problem. One of the most popular algorithms to date for real-time object detection is YOLO (You Only Look Once), initially proposed by Redmond et al.[1]
Traditionally, models like YOLO required enormous amounts of training data to yield reasonable results. People might not have access to such high-quality labeled data. Thankfully, open-source communities and researchers have made it possible to utilize pre-trained models to perform inference. In other words, you can use models that have already been trained on large datasets to perform object detection on your own data.
In this tutorial you will perform an end-to-end object detection inference, using the YOLOv5 Docker Image developed by Ultralytics.
Performing object detection inference using Yolov5 and Bacalhau
To get started, you need to install the Bacalhau client, see more information here
Bacalhau is a highly scalable decentralized computing platform and is well suited to running massive object detection jobs. In this example, you can take advantage of the GPUs available on the Bacalhau network.
To get started, let's run a test job with a small sample dataset that is included in the YOLOv5 Docker Image. This will give you a chance to familiarise yourself with the process of running a job on Bacalhau.
In addition to the usual Bacalhau flags, you will also see:
--gpu 1 to specify the use of a GPU
:::tip Remember that Bacalhau does not provide any network connectivity when running a job. All assets must be provided at submission time. :::
The model requires pre-trained weights to run and by default downloads them from within the container. Bacalhau jobs don't have network access so we will pass in the weights at submission time, saving them to /usr/src/app/yolov5s.pt. You may also provide your own weights here.
The container has its own options that we must specify:
--input to select which pre-trained weights you desire with details on the yolov5 release page
--project specifies the output volume that the model will save its results to. Bacalhau defaults to using /outputs as the output directory, so we save it there.
For more container flags refer to the yolov5/detect.py file in the YOLO repository.
One final additional hack that we have to do is move the weights file to a location with the standard name. As of writing this, Bacalhau downloads the file to a UUID-named file, which the model is not expecting. This is because GitHub 302 redirects the request to a random file in its backend.
This should output a UUID (like 59c59bfb-4ef8-45ac-9f4b-f0e9afd26e70). This is the ID of the job that was created. You can check the status of the job with the following command:
Job status: You can check the status of the job using bacalhau list.
When it says Completed, that means the job is done, and we can get the results.
Job information: You can find out more information about your job by using bacalhau describe.
Job download: You can download your job results directly by using bacalhau get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
After the download has finished we can see the results:
Now let's use some custom images. First, you will need to ingest your images onto IPFS/Filecoin. For more information about how to do that see the data ingestion section.
This example will use the Cyclist Dataset for Object Detection | Kaggle dataset.
We have already uploaded this dataset to Filecoin under the CID: bafybeicyuddgg4iliqzkx57twgshjluo2jtmlovovlx5lmgp5uoh3zrvpm. You can browse to this dataset via a HTTP IPFS proxy.
Let's run a the same job again, but this time use the images above.
When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.
Job status: You can check the status of the job using bacalhau list.
Job download: You can download your job results directly by using bacalhau get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
This example tutorial demonstrates how to use stable diffusion on a GPU and run it on the Bacalhau network.Stable Diffusion is a state of the art text-to-image model that generates images from text and was developed as an open-source alternative to DALL·E 2. It is based on a Diffusion Probabilistic Model and uses a Transformer to generate images from text.
To get started, you need to install the Bacalhau client, see more information here
Here is an example of an image generated by this model.
This stable diffusion example is based on the Keras/Tensorflow implementation. You might also be interested in the Pytorch oriented diffusers library.
:::info When you run this code for the first time, it will download the pre-trained weights, which may add a short delay. :::
Based on the requirements here, we will install the following:
We have a sample code from this the Stable Diffusion in TensorFlow/Keras repo which we will use to check if the code is working as expected. Our output for this code will be a DSLR photograph of an astronaut riding a horse.
:::tip When you run this code for the first time, it will download the pre-trained weights, which may add a short delay. :::
When running this code, if you check the GPU RAM usage, you'll see that it's sucked up many GBs, and depending on what GPU you're running, it may OOM (Out of memory) if you run this again.
You can try and reduce RAM usage by playing with batch sizes (although it is only set to 1 above!) or more carefully controlling the TensorFlow session.
To clear the GPU memory we will use numba. This won't be required when running in a single-shot manner.
You need a script to execute when we submit jobs. The code below is a slightly modified version of the code we ran above which we got from here, however, this includes more things such as argument parsing argument parsing to be able to customize the generator.
:::info For a full list of arguments that you can pass to the script, see more information here :::
After writing the code the next step is to run the script.
The following presents additional parameters you can try:
python main.py --p "cat with three eyes - to set prompt
python main.py --p "cat with three eyes --n 100 - to set the number of iterations to 100
python stable-diffusion.py --p "cat with three eyes" --b 2 to set batch size to 2 (No of images to generate)
Docker is the easiest way to run TensorFlow on a GPU since the host machine only requires the NVIDIA® driver. To containerize the inference code, we will create a Dockerfile. The Dockerfile is a text document that contains the commands that specify how the image will be built.
The Dockerfile leverages the latest official TensorFlow GPU image and then installs other dependencies like git, CUDA packages, and other image-related necessities. See the original repository for the expected requirements.
:::info See more information on how to containerize your script/apphere :::
We will run docker build command to build the container;
Before running the command replace;
hub-user with your docker hub username, If you don’t have a docker hub account follow these instructions to create a Docker account, and use the username of the account you created
repo-name with the name of the container, you can name it anything you want
tag this is not required but you can use the latest tag
In our case
Next, upload the image to the registry. This can be done by using the Docker hub username, repo name or tag.
In our case
To submit a job, run the following Bacalhau command:
Let's look closely at the command above:
bacalhau docker run: call to bacalhau
--gpu 1: No of GPUs
ghcr.io/bacalhau-project/examples/stable-diffusion-gpu:0.0.1: the name and the tag of the docker image we are using
../outputs: path to the output
python main.py: execute the script
The Bacalhau command passes a prompt to the model and generates an image in the outputs directory. The main difference in the example below compared to all the other examples is the addition of the --gpu X flag, which tells Bacalhau to only schedule the job on nodes that have X GPUs free. You can read more about GPU support in the documentation.
:::tip This will take about 5 minutes to complete and is mainly due to the cold-start GPU setup time. This is faster than the CPU version, but you might still want to grab some fruit or plan your lunchtime run.
Furthermore, the container itself is about 10GB, so it might take a while to download on the node if it isn't cached. :::
When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.
Job status: You can check the status of the job using bacalhau list.
When it says Completed, that means the job is done, and we can get the results.
Job information: You can find out more information about your job by using bacalhau describe.
Job download: You can download your job results directly by using bacalhau get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
After the download has finished you should see the following contents in results directory
To view the file, run the following command:
To display and view your image run the code below:
In this example, we will demonstrate how to run inference on a model stored on Amazon S3. We will use a PyTorch model trained on the MNIST dataset.
Python
PyTorch
This script is designed to load a pretrained PyTorch model for MNIST digit classification from a tar.gz file, extract it, and use the model to perform inference on a given input image.
To use this script, you need to provide the paths to the tar.gz file containing the pre-trained model, the output directory where the model will be extracted, and the input image file for which you want to perform inference. The script will output the predicted digit (class) for the given input image.
To get started, you need to install the Bacalhau client, see more information here
-w /inputs Setting the current working directory at /inputs in the container
-i src=s3://sagemaker-sample-files/datasets/image/MNIST/model/pytorch-training-2020-11-21-22-02-56-203/model.tar.gz,dst=/model/,opt=region=us-east-1: Mounting the s3 bucket at the destination path provided /model/ and specifying the region where the bucket is located opt=region=us-east-1
-i git://github.com/js-ts/mnist-test.git: Flag to mount the source code repo from GitHub. It would mount the repo at /inputs/js-ts/mnist-test in this case it also contains the test image.
pytorch/pytorch: The name of the Docker image.
-- python /inputs/js-ts/mnist-test/inference.py --tar_gz_file_path /model/model.tar.gz --output_directory /model-pth --image_path /inputs/js-ts/mnist-test/image.png: The command to run inference on the model.
/model/model.tar.gz is the path to the model file.
/model-pth is the output directory for the model.
/inputs/js-ts/mnist-test/image.png is the path to the input image.
The job has been submitted and Bacalhau has printed out the related job id. We store that in an environment variable so that we can reuse it later on.
