Introduction​

In this example tutorial, we use Bacalhau and Easy OCR to digitize paper records or for recognizing characters or extract text data from images stored on IPFS, S3 or on the web. EasyOCR is a ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic etc. With easy OCR, you use the pre-trained models or use your own fine-tuned model.

TL;DR​

bacalhau docker run \
    -i ipfs://bafybeibvcllzpfviggluobcfassm3vy4x2a4yanfxtmn4ir7olyzfrgq64:/root/.EasyOCR/model/zh_sim_g2.pth  \
    -i https://raw.githubusercontent.com/JaidedAI/EasyOCR/ae773d693c3f355aac2e58f0d8142c600172f016/examples/chinese.jpg \
    --timeout 3600 \
    --wait-timeout-secs 3600 \
    --gpu 1  \
    --memory 10Gb \
    --cpu 3 \
    --id-only \
    --wait \
    jsacex/easyocr \
    --  easyocr -l ch_sim  en -f ./inputs/chinese.jpg --detail=1 --gpu=True

Running Easy OCR Locally​​

Install the required dependencies

pip install --upgrade easyocr

Load the different example images

npx degit JaidedAI/EasyOCR/examples -f

List all the images. You'll see an output like this:

ls -l

total 3508
-rw-r--r-- 1 root root   59898 Jun 16 22:36 chinese.jpg
-rw-r--r-- 1 root root   97910 Jun 16 22:36 easyocr_framework.jpeg
-rw-r--r-- 1 root root 1740957 Jun 16 22:36 english.png
-rw-r--r-- 1 root root  487995 Jun 16 22:36 example2.png
-rw-r--r-- 1 root root  127454 Jun 16 22:36 example3.png
-rw-r--r-- 1 root root  488641 Jun 16 22:36 example.png
-rw-r--r-- 1 root root  168376 Jun 16 22:36 french.jpg
-rw-r--r-- 1 root root   42159 Jun 16 22:36 japanese.jpg
-rw-r--r-- 1 root root  225531 Jun 16 22:36 korean.png
drwxr-xr-x 1 root root    4096 Jun 15 13:37 sample_data
-rw-r--r-- 1 root root   82229 Jun 16 22:36 thai.jpg
-rw-r--r-- 1 root root   34706 Jun 16 22:36 width_ths.png

Next, we create a reader to do OCR to get coordinates which represent a rectangle containing text and the text itself:

import easyocr
reader = easyocr.Reader(['th','en'])
# Doing OCR. Get bounding boxes.
bounds = reader.readtext('thai.jpg')
bounds

Containerize your Script using Docker​

We will use the Dockerfile that is already created in the Easy OCR repo. Use the command below to clone the repo

git clone https://github.com/JaidedAI/EasyOCR
cd EasyOCR

Build the Container​

The docker build command builds Docker images from a Dockerfile.

docker build -t <hub-user>/<repo-name>:<tag> .

Before running the command replace:

1. hub-user with your docker hub username, If you don’t have a docker hub account follow these instructions to create docker account, and use the username of the account you created

2. repo-name with the name of the container, you can name it anything you want

3. tag this is not required but you can use the latest tag

Push the container​

Next, upload the image to the registry. This can be done by using the Docker hub username, repo name, or tag.

docker push <hub-user>/<repo-name>:<tag>

Running a Bacalhau Job to Generate Easy OCR output​

Prerequisite​

To get started, you need to install the Bacalhau client, see more information here.

Now that we have an image in the docker hub (your own or an example image from the manual), we can use the container for running on Bacalhau.

Structure of the imperative command​

Let's look closely at the command below:

1. export JOB_ID=$( ... ) exports the job ID as environment variable

2. bacalhau docker run: call to bacalhau

3. The --gpu 1 flag is set to specify hardware requirements, a GPU is needed to run such a job

4. The --id-only flag is set to print only job id

5. -i ipfs://bafybeibvc...... Mounts the model from IPFS

6. -i https://raw.githubusercontent.com... Mounts the Input Image from a URL

7. jsacex/easyocr the name and the tag of the docker image we are using

8. -- easyocr -l ch_sim en -f ./inputs/chinese.jpg --detail=1 --gpu=True execute script with following paramters:

   1. -l ch_sim: the name of the model

   2. -f ./inputs/chinese.jpg: path to the input Image or directory

   3. --detail=1: level of detail

   4. --gpu=True: we set this flag to true since we are running inference on a GPU. If you run this on a CPU - set this flag to false

Since the model and the image aren't present in the container we will mount the image from an URL and the model from IPFS. You can find models to download from here. You can choose the model you want to use in this case we will be using the zh_sim_g2 model

export JOB_ID=$(bacalhau docker run \
    -i ipfs://bafybeibvcllzpfviggluobcfassm3vy4x2a4yanfxtmn4ir7olyzfrgq64:/root/.EasyOCR/model/zh_sim_g2.pth  \
    -i https://raw.githubusercontent.com/JaidedAI/EasyOCR/ae773d693c3f355aac2e58f0d8142c600172f016/examples/chinese.jpg \
    --timeout 3600 \
    --wait-timeout-secs 3600 \
    --gpu 1  \
    --memory 10Gb \
    --cpu 3 \
    --id-only \
    --wait \
    jsacex/easyocr \
    --  easyocr -l ch_sim  en -f ./inputs/chinese.jpg --detail=1 --gpu=True)

When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.

Declarative job description​

The same job can be presented in the declarative format. In this case, the description will look like this:

name: EasyOCR
type: batch
count: 1
tasks:
  - name: My main task
    Engine:
      type: docker
      params:
        Image: "jsacex/easyocr" 
        Entrypoint:
          - /bin/bash
        Parameters:
          - -c
          - easyocr -l ch_sim  en -f ./inputs/chinese.jpg --detail=1 --gpu=True
    InputSources:
    - Source:
        Type: "urlDownload"
        Params:
          URL: "https://raw.githubusercontent.com/JaidedAI/EasyOCR/ae773d693c3f355aac2e58f0d8142c600172f016/examples/chinese.jpg"
      Target: "/inputs/chinese.jpg"
    - Source:
        Type: "s3"
        Params:
          Bucket: "landsat-image-processing"
          Key: "*"
          Region: "us-east-1"
      Target: "/root/.EasyOCR/model/zh_sim_g2.pth"
    Resources:
      GPU: "1"

The job description should be saved in .yaml format, e.g. easyocr.yaml, and then run with the command:

bacalhau job run easyocr.yaml

Checking the State of your Jobs​

Job status​

You can check the status of the job using bacalhau job list.

bacalhau job list --id-filter ${JOB_ID}

When it says Completed, that means the job is done, and we can get the results.

Job information​

You can find out more information about your job by using bacalhau job describe.

bacalhau job describe ${JOB_ID}

Job download​

You can download your job results directly by using bacalhau job get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.

rm -rf results && mkdir -p results
bacalhau job get $JOB_ID --output-dir results

After the download has finished you should see the following contents in results directory

Viewing your Job Output​

Now you can find the file in the results/outputs folder. You can view results by running following commands:

ls results # list the contents of the current directory 

cat results/stdout # displays the contents of the current directory 

Introduction​

Stable Diffusion is a state of the art text-to-image model that generates images from text and was developed as an open-source alternative to DALL·E 2. It is based on a Diffusion Probabilistic Model and uses a Transformer to generate images from text.

This example demonstrates how to use stable diffusion using a finetuned model and run inference on it. The first section describes the development of the code and the container - it is optional as users don't need to build their own containers to use their own custom model. The second section demonstrates how to convert your model weights to ckpt. The third section demonstrates how to run the job using Bacalhau.

The following guide is using the fine-tuned model, which was finetuned on Bacalhau. To learn how to finetune your own stable diffusion model refer to this guide.

TL;DR​

1. Convert your existing model weights to the ckpt format and upload to the IPFS storage.

2. Create a job using bacalhau docker run, relevant docker image, model weights and any prompt.

3. Download results using bacalhau job get and the job id.

Prerequisite​

To get started, you need to install:

1. Bacalhau client, see more information here

2. NVIDIA GPU

3. CUDA drivers

4. NVIDIA docker

Running Locally​

Containerize your Script using Docker​

To build your own docker container, create a Dockerfile, which contains instructions to containerize the code for inference.

FROM  pytorch/pytorch:1.13.0-cuda11.6-cudnn8-runtime

WORKDIR /

RUN apt update &&  apt install -y git

RUN git clone https://github.com/runwayml/stable-diffusion.git

WORKDIR /stable-diffusion

RUN conda env create -f environment.yaml

SHELL ["conda", "run", "-n", "ldm", "/bin/bash", "-c"]

RUN pip install opencv-python

RUN apt update

RUN apt-get install ffmpeg libsm6 libxext6 libxrender-dev  -y

This container is using the pytorch/pytorch:1.13.0-cuda11.6-cudnn8-runtime image and the working directory is set. Next the Dockerfile installs required dependencies. Then we add our custom code and pull the dependent repositories.

Build the container​

We will run docker build command to build the container.

docker build -t <hub-user>/<repo-name>:<tag> .

Before running the command replace:

1. hub-user with your docker hub username, If you don’t have a docker hub account follow these instructions to create the Docker account, and use the username of the account you created

2. repo-name with the name of the container, you can name it anything you want

3. tag this is not required but you can use the latest tag

So in our case, the command will look like this:

docker build -t jsacex/stable-diffusion-ckpt

Push the container​

Next, upload the image to the registry. This can be done by using the Docker hub username, repo name or tag.

docker push <hub-user>/<repo-name>:<tag>

Thus, in this case, the command would look this way:

docker push jsacex/stable-diffusion-ckpt

After the repo image has been pushed to Docker Hub, you can now use the container for running on Bacalhau. But before that you need to check whether your model is a ckpt file or not. If your model is a ckpt file you can skip to the running on Bacalhau, and if not - the next section describes how to convert your model into the ckpt format.

Converting model weights to CKPT​

To download the convert script:

wget -q https://github.com/TheLastBen/diffusers/raw/main/scripts/convert_diffusers_to_original_stable_diffusion.py

To convert the model weights into ckpt format, the --half flag cuts the size of the output model from 4GB to 2GB:

python3 convert_diffusers_to_original_stable_diffusion.py \
    --model_path <path-to-the-model-weights>  \
    --checkpoint_path <path-to-save-the-checkpoint>/model.ckpt \
    --half

Running a Bacalhau Job​

To do inference on your own checkpoint on Bacalhau you need to first upload it to your public storage, which can be mounted anywhere on your machine. In this case, we will be using NFT.Storage (Recommended Option). To upload your dataset using NFTup drag and drop your directory and it will upload it to IPFS.

After the checkpoint file has been uploaded copy its CID.

Structure of the command​

Let's look closely at the command above:

1. export JOB_ID=$( ... ): Export results of a command execution as environment variable

2. The --gpu 1 flag is set to specify hardware requirements, a GPU is needed to run such a job

3. -i ipfs://QmUCJuFZ2v7KvjBGHRP2K1TMPFce3reTkKVGF2BJY5bXdZ:/model.ckpt: Path to mount the checkpoint

4. -- conda run --no-capture-output -n ldm: since we are using conda we need to specify the name of the environment which we are going to use, in this case it is ldm

5. scripts/txt2img.py: running the python script

6. --prompt "a photo of a person drinking coffee": the prompt you need to specify the session name in the prompt.

7. --plms: the sampler you want to use. In this case we will use the plms sampler

8. --ckpt ../model.ckpt: here we specify the path to our checkpoint

9. --n_samples 1: no of samples we want to produce

10. --skip_grid: skip creating a grid of images

11. --outdir ../outputs: path to store the outputs

12. --seed $RANDOM: The output generated on the same prompt will always be the same for different outputs on the same prompt set the seed parameter to random

When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.

export JOB_ID=$(bacalhau docker run \
--gpu 1 \
--timeout 3600 \
--wait-timeout-secs 3600 \
--wait \
--id-only \
-i ipfs://QmUCJuFZ2v7KvjBGHRP2K1TMPFce3reTkKVGF2BJY5bXdZ:/model.ckpt \
jsacex/stable-diffusion-ckpt \
-- conda run --no-capture-output -n ldm python scripts/txt2img.py --prompt "a photo of a person drinking coffee" --plms --ckpt ../model.ckpt --skip_grid --n_samples 1 --skip_grid --outdir ../outputs) 

Checking the State of your Jobs​

Job status: You can check the status of the job using bacalhau job list:

bacalhau job list --id-filter ${JOB_ID}

When it says Completed, that means the job is done, and we can get the results.

Job information: You can find out more information about your job by using bacalhau job describe:

bacalhau job describe ${JOB_ID}

Job download: You can download your job results directly by using bacalhau job get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.

rm -rf results && mkdir results
bacalhau job get ${JOB_ID} --output-dir results

Viewing your Job Output​

After the download has finished we can see the results in the results/outputs folder. We received following image for our prompt:

Introduction​

Dolly 2.0, the groundbreaking, open-source, instruction-following Large Language Model (LLM) that has been fine-tuned on a human-generated instruction dataset, licensed for both research and commercial purposes. Developed using the EleutherAI Pythia model family, this 12-billion-parameter language model is built exclusively on a high-quality, human-generated instruction following dataset, contributed by Databricks employees.

Dolly 2.0 package is open source, including the training code, dataset, and model weights, all available for commercial use. This unprecedented move empowers organizations to create, own, and customize robust LLMs capable of engaging in human-like interactions, without the need for API access fees or sharing data with third parties.

Running locally​

Prerequisites​

1. A NVIDIA GPU

2. Python

Installing dependencies​

pip -q install git+https://github.com/huggingface/transformers # need to install from github
pip -q --upgrade install accelerate # ensure you are using version higher than 0.12.0

Create an inference.py file with following code:

# content of the inference.py file
import argparse
import torch
from transformers import pipeline

def main(prompt_string, model_version):

    # use dolly-v2-12b if you're using Colab Pro+, using pythia-2.8b for Free Colab
    generate_text = pipeline(model=model_version, 
                            torch_dtype=torch.bfloat16, 
                            trust_remote_code=True,
                            device_map="auto")

    print(generate_text(prompt_string))

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--prompt", type=str, required=True, help="The prompt to be used in the GPT model")
    parser.add_argument("--model_version", type=str, default="./databricks/dolly-v2-12b", help="The model version to be used")
    args = parser.parse_args()
    main(args.prompt, args.model_version)

Building the container (optional)​

You may want to create your own container for this kind of task. In that case, use the instructions for creating and publishing your own image in the docker hub. Use huggingface/transformers-pytorch-deepspeed-nightly-gpu as base image, install dependencies listed above and copy the inference.py into it. So your Dockerfile will look like this:

FROM huggingface/transformers-pytorch-deepspeed-nightly-gpu
RUN apt-get update -y
RUN pip -q install git+https://github.com/huggingface/transformers
RUN pip -q install accelerate>=0.12.0 
COPY ./inference.py .

Running Inference on Bacalhau​

Prerequisite​

To get started, you need to install the Bacalhau client, see more information here

Structure of the command​

1. export JOB_ID=$( ... ): Export results of a command execution as environment variable

2. bacalhau docker run: Run a job using docker executor.

3. --gpu 1: Flag to specify the number of GPUs to use for the execution. In this case, 1 GPU will be used.

4. -w /inputs: Flag to set the working directory inside the container to /inputs.

5. -i gitlfs://huggingface.co/databricks/dolly-v2-3b.git: Flag to clone the Dolly V2-3B model from Hugging Face's repository using Git LFS. The files will be mounted to /inputs/databricks/dolly-v2-3b.

6. -i https://gist.githubusercontent.com/js-ts/d35e2caa98b1c9a8f176b0b877e0c892/raw/3f020a6e789ceef0274c28fc522ebf91059a09a9/inference.py: Flag to download the inference.py script from the provided URL. The file will be mounted to /inputs/inference.py.

7. jsacex/dolly_inference:latest: The name and the tag of the Docker image.

8. The command to run inference on the model: python inference.py --prompt "Where is Earth located ?" --model_version "./databricks/dolly-v2-3b". It consists of:

   1. inference.py: The Python script that runs the inference process using the Dolly V2-3B model.

   2. --prompt "Where is Earth located ?": Specifies the text prompt to be used for the inference.

   3. --model_version "./databricks/dolly-v2-3b": Specifies the path to the Dolly V2-3B model. In this case, the model files are mounted to /inputs/databricks/dolly-v2-3b.

When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.

export JOB_ID=$(bacalhau docker run \
    --gpu 1 \
    --id-only \
    -w /inputs \
    -i gitlfs://huggingface.co/databricks/dolly-v2-3b.git \
    -i https://gist.githubusercontent.com/js-ts/d35e2caa98b1c9a8f176b0b877e0c892/raw/3f020a6e789ceef0274c28fc522ebf91059a09a9/inference.py \
    jsacex/dolly_inference:latest \
    -- python inference.py --prompt "Where is Earth located ?" --model_version "./databricks/dolly-v2-3b")

Checking the State of your Jobs​

Job status: You can check the status of the job using bacalhau job list:

bacalhau job list --id-filter ${JOB_ID}

When it says Completed, that means the job is done, and we can get the results.

Job information: You can find out more information about your job by using bacalhau job describe:

bacalhau job describe ${JOB_ID}

Job download: You can download your job results directly by using bacalhau job get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.

rm -rf results && mkdir results
bacalhau job get ${JOB_ID} --output-dir results

Viewing your Job Output​

After the download has finished, we can see the results in the results/outputs folder.

This example tutorial demonstrates how to use Stable Diffusion on a GPU and run it on the demo network. is a state of the art text-to-image model that generates images from text and was developed as an open-source alternative to . It is based on a and uses a to generate images from text.

TL;DR​

Prerequisite​

To get started, you need to install the Bacalhau client, see more information .

Quick Test​

Here is an example of an image generated by this model.

Development​

Installing dependencies​

Testing the Code​

When running this code, if you check the GPU RAM usage, you'll see that it's sucked up many GBs, and depending on what GPU you're running, it may OOM (Out of memory) if you run this again.

You can try and reduce RAM usage by playing with batch sizes (although it is only set to 1 above!) or more carefully controlling the TensorFlow session.

To clear the GPU memory we will use numba. This won't be required when running in a single-shot manner.

Write the Script​

Run the Script​

After writing the code the next step is to run the script.

As a result, you will get something like this:

The following presents additional parameters you can try:

1. python main.py --p "cat with three eyes - to set prompt

2. python main.py --p "cat with three eyes" --n 100 - to set the number of iterations to 100

3. python stable-diffusion.py --p "cat with three eyes" --b 2 to set batch size to 2 (№ of images to generate)

Containerize Script using Docker​

Build the container​

We will run docker build command to build the container;

Before running the command replace following:

2. repo-name with the name of the container, you can name it anything you want

3. tag this is not required but you can use the latest tag

In our case:

Push the container​

Next, upload the image to the registry. This can be done by using the Docker hub username, repo name or tag.

In our case:

Running a Bacalhau Job​

Structure of the command​

To submit a job run the Bacalhau command with following structure:

1. export JOB_ID=$( ... ) exports the job ID as environment variable

2. The --gpu 1 flag is set to specify hardware requirements, a GPU is needed to run such a job

3. The --id-only flag is set to print only job id

4. ghcr.io/bacalhau-project/examples/stable-diffusion-gpu:0.0.1: the name and the tag of the docker image we are using

5. -- python main.py --o ./outputs --p "meme about tensorflow": The command to run inference on the model. It consists of:

   1. main.py path to the script

   2. --o ./outputs specifies the output directory

   3. --p "meme about tensorflow" specifies the prompt

When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.

Checking the State of your Jobs​

Job status: You can check the status of the job using bacalhau job list.

When it says Completed, that means the job is done, and we can get the results.

Job information: You can find out more information about your job by using bacalhau job describe.

Job download: You can download your job results directly by using bacalhau job get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.

After the download has finished you should see the following contents in results directory

Viewing your Job Output​

Now you can find the file in the results/outputsfolder:

Introduction​

In this example, we will demonstrate how to run inference on a model stored on Amazon S3. We will use a PyTorch model trained on the MNIST dataset.

Running Locally​

Prerequisites​

Consider using the latest versions or use the docker method listed below in the article.

1. Python

2. PyTorch

Downloading the Datasets​

Use the following commands to download the model and test image:

Creating the Inference Script​

This script is designed to load a pretrained PyTorch model for MNIST digit classification from a tar.gz file, extract it, and use the model to perform inference on a given input image. Ensure you have all required dependencies installed:

Running the Inference Script​

To use this script, you need to provide the paths to the tar.gz file containing the pre-trained model, the output directory where the model will be extracted, and the input image file for which you want to perform inference. The script will output the predicted digit (class) for the given input image.

Running Inference on Bacalhau​

Prerequisite​

Structure of the Command​

1. export JOB_ID=$( ... ): Export results of a command execution as environment variable

2. -w /inputs Set the current working directory at /inputs in the container

3. -i src=s3://sagemaker-sample-files/datasets/image/MNIST/model/pytorch-training-2020-11-21-22-02-56-203/model.tar.gz,dst=/model/,opt=region=us-east-1: Mount the s3 bucket at the destination path provided - /model/ and specifying the region where the bucket is located opt=region=us-east-1

4. -i git://github.com/js-ts/mnist-test.git: Flag to mount the source code repo from GitHub. It would mount the repo at /inputs/js-ts/mnist-test in this case it also contains the test image

5. pytorch/pytorch: The name of the Docker image

6. -- python3 /inputs/js-ts/mnist-test/inference.py --tar_gz_file_path /model/model.tar.gz --output_directory /model-pth --image_path /inputs/js-ts/mnist-test/image.png: The command to run inference on the model. It consists of:

   1. /model/model.tar.gz is the path to the model file

   2. /model-pth is the output directory for the model

   3. /inputs/js-ts/mnist-test/image.png is the path to the input image

When the job is submitted Bacalhau prints out the related job id. We store that in an environment variable JOB_ID so that we can reuse it later on.

Viewing the Output​

Use the bacalhau job logs command to view the job output, since the script prints the result of execution to the stdout:

You can also use bacalhau job get to download job results:

Support

Introduction

Whisper is an automatic speech recognition (ASR) system trained on 680,000 hours of multilingual and multitask supervised data collected from the web. It shows that the use of such a large and diverse dataset leads to improved robustness to accents, background noise, and technical language. Moreover, it enables transcription in multiple languages, as well as translation from those languages into English. Creators are open-sourcing models and inference code to serve as a foundation for building useful applications and for further research on robust speech processing. In this example, we will transcribe an audio clip locally, containerize the script and then run the container on Bacalhau.

The advantage of using Bacalhau over managed Automatic Speech Recognition services is that you can run your own containers which can scale to do batch process petabytes of videos or audio for automatic speech recognition

TL;DR

Prerequisite

To get started, you need to install:

1. Bacalhau client, see more information .

2. Whisper.

3. PyTorch.

4. Python Pandas.

Before we create and run the script we need a sample audio file to test the code. For that we download a sample audio clip:

We will create a script that accepts parameters (input file path, output file path, temperature, etc.) and set the default parameters. Also if the input file is in mp4 format, then the script converts it to wav format. The transcript can be saved in various formats. Then the large model is loaded and we pass it the required parameters.

This model is not only limited to English and transcription, it supports many other languages.

Next, let's create an openai-whisper script:

Let's run the script with the default parameters:

To view the outputs, execute following:

To build your own docker container, create a Dockerfile, which contains instructions on how the image will be built, and what extra requirements will be included.

We choose pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime as our base image.

And then install all the dependencies, after that we will add the test audio file and our openai-whisper script to the container, we will also run a test command to check whether our script works inside the container and if the container builds successfully

We will run docker build command to build the container;

Before running the command replace:

2. repo-name with the name of the container, you can name it anything you want

3. tag this is not required but you can use the latest tag

In our case:

Next, upload the image to the registry. This can be done by using the Docker hub username, repo name or tag.

In our case:

After the dataset has been uploaded, copy the CID:

Let's look closely at the command below:

1. export JOB_ID=$( ... ) exports the job ID as environment variable

2. bacalhau docker run: call to bacalhau

3. The-i ipfs://bafybeielf6z4cd2nuey5arckect5bjmelhouvn5r: flag to mount the CID which contains our file to the container at the path /inputs

4. The --gpu 1 flag is set to specify hardware requirements, a GPU is needed to run such a job

5. jsacex/whisper: the name and the tag of the docker image we are using

6. python openai-whisper.py: execute the script with following parameters:

   1. -p inputs/Apollo_11_moonwalk_montage_720p.mp4 : the input path of our file

   2. -o outputs: the path where to store the outputs

When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.

You can check the status of the job using bacalhau job list.

When it says Completed, that means the job is done, and we can get the results.

You can find out more information about your job by using bacalhau job describe.

You can download your job results directly by using bacalhau job get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.

After the download has finished you should see the following contents in results directory

Now you can find the file in the results/outputs folder. To view it, run the following command:

Introduction

In this example tutorial, we will show you how to generate realistic images with and Bacalhau. StyleGAN is based on Generative Adversarial Networks (GANs), which include a generator and discriminator network that has been trained to differentiate images generated by the generator from real images. However, during the training, the generator tries to fool the discriminator, which results in the generation of realistic-looking images. With StyleGAN3 we can generate realistic-looking images or videos. It can generate not only human faces but also animals, cars, and landscapes.

TL;DR​

Prerequisite​

To get started, you need to install the Bacalhau client, see more information

Running StyleGAN3 locally​

To run StyleGAN3 locally, you'll need to clone the repo, install dependencies and download the model weights.

Now you can generate an image using a pre-trained AFHQv2 model. Here is an example of the image we generated:

Containerize Script with Docker​

To build your own docker container, create a Dockerfile, which contains instructions to build your image.

Build the container​

We will run docker build command to build the container:

Before running the command replace:

2. repo-name with the name of the container, you can name it anything you want

3. tag this is not required but you can use the latest tag

In our case:

Push the container​

Next, upload the image to the registry. This can be done by using the Docker hub username, repo name or tag.

In our case:

Running a Bacalhau Job​

Structure of the command​

To submit a job run the Bacalhau command with following structure:

1. export JOB_ID=$( ... ) exports the job ID as environment variable

2. bacalhau docker run: call to Bacalhau

3. The --gpu 1 flag is set to specify hardware requirements, a GPU is needed to run such a job

4. The --id-only flag is set to print only job id

5. jsacex/stylegan3: the name and the tag of the docker image we are using

6. python gen_images.py: execute the script with following parameters:

   1. --trunc=1 --seeds=2 --network=stylegan3-r-afhqv2-512x512.pkl: The animation length is either determined based on the --seeds value or explicitly specified using the --num-keyframes option. When num keyframes are specified with --num-keyframes, the output video length will be num_keyframes * w_frames frames.

   2. ../outputs: path to the output

Declarative job description​

The job description should be saved in .yaml format, e.g. stylegan3.yaml, and then run with the command:

Render a latent vector interpolation video​

You can also run variations of this command to generate videos and other things. In the following command below, we will render a latent vector interpolation video. This will render a 4x2 grid of interpolations for seeds 0 through 31.

Structure of the command​

Let's look closely at the command below:

1. export JOB_ID=$( ... ) exports the job ID as environment variable

2. bacalhau docker run: call to bacalhau

3. The --gpu 1 flag is set to specify hardware requirements, a GPU is needed to run such a job

4. The --id-only flag is set to print only job id

5. jsacex/stylegan3 the name and the tag of the docker image we are using

6. python gen_images.py: execute the script with following parameters:

   1. --trunc=1 --seeds=2 --network=stylegan3-r-afhqv2-512x512.pkl: The animation length is either determined based on the --seeds value or explicitly specified using the --num-keyframes option. When num keyframes is specified with --num-keyframes, the output video length will be num_keyframes * w_frames frames. If --num-keyframes is not specified, the number of seeds given with --seeds must be divisible by grid size W*H (--grid). In this case, the output video length will be # seeds/(w*h)*w_frames frames.

   2. ../outputs: path to the output

When a job is submitted, Bacalhau prints out the related job_id. We store that in an environment variable so that we can reuse it later on.

Checking the State of your Jobs​

Job status​

You can check the status of the job using bacalhau job list.

When it says Completed, that means the job is done, and we can get the results.

Job information​

You can find out more information about your job by using bacalhau job describe.

Job download​

You can download your job results directly by using bacalhau job get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.

After the download has finished you should see the following contents in results directory

Viewing your Job Output​

Now you can find the file in the results/outputs folder.

Support

Introduction​

The identification and localization of objects in images and videos is a computer vision task called object detection. Several algorithms have emerged in the past few years to tackle the problem. One of the most popular algorithms to date for real-time object detection is , initially proposed

Traditionally, models like YOLO required enormous amounts of training data to yield reasonable results. People might not have access to such high-quality labeled data. Thankfully, open-source communities and researchers have made it possible to utilize pre-trained models to perform inference. In other words, you can use models that have already been trained on large datasets to perform object detection on your own data.

Bacalhau is a highly scalable decentralized computing platform and is well suited to running massive object detection jobs. In this example, you can take advantage of the GPUs available on the Bacalhau Network and perform an end-to-end object detection inference, using the

TL;DR​

Load your dataset into S3/IPFS, specify it and pre-trained weights via the --input flags, choose a suitable container, specify the command and path to save the results - done!

Prerequisite​

To get started, you need to install the Bacalhau client, see more information

Test Run with Sample Data​

To get started, let's run a test job with a small sample dataset that is included in the YOLOv5 Docker Image. This will give you a chance to familiarise yourself with the process of running a job on Bacalhau.

In addition to the usual Bacalhau flags, you will also see example of using the --gpu 1 flag in order to specify the use of a GPU.

The model requires pre-trained weights to run and by default downloads them from within the container. Bacalhau jobs don't have network access so we will pass in the weights at submission time, saving them to /usr/src/app/yolov5s.pt. You may also provide your own weights here.

The container has its own options that we must specify:

2. --project specifies the output volume that the model will save its results to. Bacalhau defaults to using /outputs as the output directory, so we save it there.

One final additional hack that we have to do is move the weights file to a location with the standard name. As of writing this, Bacalhau downloads the file to a UUID-named file, which the model is not expecting. This is because GitHub 302 redirects the request to a random file in its backend.

Structure of the command​

1. export JOB_ID=$( ... ) exports the job ID as environment variable

2. The --gpu 1 flag is set to specify hardware requirements, a GPU is needed to run such a job

3. The --timeout flag is set to make sure that if the job is not completed in the specified time, it will be terminated

4. The --wait flag is set to wait for the job to complete before return

5. The --wait-timeout-secs flag is set together with --wait to define how long should app wait for the job to complete

6. The --id-only flag is set to print only job id

7. The --input flags are used to specify the sources of input data

8. -- /bin/bash -c 'find /inputs -type f -exec cp {} /outputs/yolov5s.pt \; ; python detect.py --weights /outputs/yolov5s.pt --source $(pwd)/data/images --project /outputs' tells the model where to find input data and where to write output

This should output a UUID (like 59c59bfb-4ef8-45ac-9f4b-f0e9afd26e70), which will be stored in the environment variable JOB_ID. This is the ID of the job that was created. You can check the status of the job using the commands below.

Declarative job description​

Checking the State of your Jobs​

Job status: You can check the status of the job using bacalhau job list:

When it says Completed, that means the job is done, and we can get the results.

Job information: You can find out more information about your job by using bacalhau job describe:

Job download: You can download your job results directly by using bacalhau job get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.

Viewing Output​

After the download has finished we can see the results in the results/outputs/exp folder.

Using Custom Images as an Input​

Let's run a the same job again, but this time use the images above.

Just as in the example above, this should output a UUID, which will be stored in the environment variable JOB_ID. You can check the status of the job using the commands below.

Support​

Introduction

Stable Diffusion is an open-source text-to-image model, which generates images from text. It's a cutting-edge alternative to DALL·E 2 and uses the Diffusion Probabilistic Model for image generation. At the core the model generates graphics from text using a Transformer.

This example demonstrates how to use stable diffusion online on a CPU and run it on the Bacalhau demo network. The first section describes the development of the code and the container. The second section demonstrates how to run the job using Bacalhau.

This model generated the images presented on this page.

TL;DR​

bacalhau docker run ghcr.io/bacalhau-project/examples/stable-diffusion-cpu:0.0.1 \
  -- python demo.py \
  --prompt "cod in space" \
  --output ../outputs/cod.png

Development​

The original text-to-image stable diffusion model was trained on a fleet of GPU machines, at great cost. To use this trained model for inference, you also need to run it on a GPU.

However, this isn't always desired or possible. One alternative is to use a project called OpenVINO from Intel that allows you to convert and optimize models from a variety of frameworks (and ONNX if your framework isn't directly supported) to run on a supported Intel CPU. This is what we will do in this example.

Prerequisites​

In order to run this example you need:

1. A Debian-flavoured Linux (although you might be able to get it working on the newest machines)

2. Docker

Converting Stable Diffusion to a CPU Model Using OpenVINO​

First we convert the trained stable diffusion models so that they work efficiently on a CPU with OpenVINO. Choose the fine tuned version of Stable Diffusion you want to use. The example is quite complex, so we have created a separate repository to host the code. This is a fork from this Github repository.

In summary, the code downloads a pre-optimized OpenVINO version of the original pre-trained stable diffusion model. This model leverages OpenAI's CLIP transformer and is wrapped inside an OpenVINO runtime, which executes the model.

The core code representing these tasks can be found in the stable_diffusion_engine.py file. This is a mashup that creates a pipeline necessary to tokenize the text and run the stable diffusion model. This boilerplate could be simplified by leveraging the more recent version of the diffusers library. But let's continue.

Install Dependencies​

Note that these dependencies are only known to work on Ubuntu-based x64 machines.

sudo apt-get update
sudo apt-get install -y libgl1 libglib2.0-0 git-lfs

Clone the Repository and Dependencies​

The following commands clone the example repository, and other required repositories, and install the Python dependencies.

git clone https://github.com/js-ts/stable_diffusion.openvino
cd stable_diffusion.openvino
git lfs install
git clone https://huggingface.co/openai/clip-vit-large-patch14
git clone https://huggingface.co/bes-dev/stable-diffusion-v1-4-openvino
pip3 install -r requirements.txt

Generate an Image​

Now that we have all the dependencies installed, we can call the demo.py wrapper, which is a simple CLI, to generate an image from a prompt.

cd stable_diffusion.openvino && \
  python3 demo.py \
  --prompt "hello" \
  --output hello.png

When the generation is complete, you can open the generated hello.png and see something like this:

Lets try another prompt and see what we get:

cd stable_diffusion.openvino && \
  python3 demo.py \
  --prompt "cat driving a car" \
  --output cat.png

Running Stable Diffusion (CPU) on Bacalhau​

Now we have a working example, we can convert it into a format that allows us to perform inference in a distributed environment.

First we will create a Dockerfile to containerize the inference code. The Dockerfile can be found in the repository, but is presented here to aid understanding.

FROM python:3.9.9-bullseye

WORKDIR /src

RUN apt-get update && \
    apt-get install -y \
    libgl1 libglib2.0-0 git-lfs

RUN git lfs install

COPY requirements.txt /src/

RUN pip3 install -r requirements.txt

COPY stable_diffusion_engine.py demo.py demo_web.py /src/
COPY data/ /src/data/

RUN git clone https://huggingface.co/openai/clip-vit-large-patch14
RUN git clone https://huggingface.co/bes-dev/stable-diffusion-v1-4-openvino

# download models
RUN python3 demo.py --num-inference-steps 1 --prompt "test" --output /tmp/test.jpg