Running locally in 'devstack'

You can run a stand-alone Bacalhau and IPFS network on your computer with the following guide.

The devstack command of bacalhau will start a 3 node cluster alongside isolated ipfs servers.

This is useful to kick the tires and/or developing on the codebase. It's also the tool used by some tests.

Pre-requisites

• x86_64 or ARM64 architecture

  • Ubuntu 20.0+ has most often been used for development and testing

• Go >= 1.21

• (Optional) Docker Engine

• (Optional) A build of the latest Bacalhau release

(Optional) Building Bacalhau from source

sudo apt-get update && sudo apt-get install -y make gcc zip nodejs npm
sudo snap install go --classic
wget https://github.com/bacalhau-project/bacalhau/archive/refs/heads/main.zip
unzip main.zip
cd bacalhau-main
make build

Start the cluster

./bacalhau devstack

This will start a 3 node Bacalhau cluster.

Each node has its own IPFS server isolated using the IPFS_PATH environment variable and its own API RPC server isolated using a random port. These IPFS nodes are not connected to the public IPFS network. If you wish to connect the devstack to the public IPFS network, you can include --public-ipfs flag.

You can also use your own IPFS node and connect it to the devstack by running (after starting the devstack):

# $BACALHAU_IPFS_SWARM_ADDRESSES is an environment variable given after starting devstack
ipfs swarm connect $BACALHAU_IPFS_SWARM_ADDRESSES

If you would like to make it a bit more predictable and/or ignore errors (such as during CI), you can add the following before your execution:

IGNORE_PID_AND_PORT_FILES=true PREDICTABLE_API_PORT=1

Once everything has started up - you will see output like the following:

Devstack is ready!
To use the devstack, run the following commands in your shell:
export BACALHAU_IPFS_SWARM_ADDRESSES=/ip4/127.0.0.1/tcp/33033/p2p/QmNp5XqbkePNYtRzB2MXZPo6MxkeH6N2fYZRCLT57VsACn
export BACALHAU_API_HOST=0.0.0.0
export BACALHAU_API_PORT=39763

By default devstack is not running on public IPFS network.
If you wish to connect devstack to public IPFS network consider running new IPFS node daemon locally
and then connecting it to bacalhau using the command below or by adding --public-ipfs flag:

ipfs swarm connect $BACALHAU_IPFS_SWARM_ADDRESSES

The message above contains the environment variables you need for a new window. You can paste these into a new terminal so that bacalhau will use your local devstack.

Alternatively, to remove the need to copy and paste, you can set DEVSTACK_ENV_FILE environment variable to the name of a .env file that devstack will write to, and bacalhau commands will read from, before launching the devstack e.g.:

DEVSTACK_ENV_FILE=.devstack.env bacalhau devstack

When the devstack is shut down, the local env file (if configured) will be removed. It is suggested you use .devstack.env to avoid clashing with longer lived .env files.

New Terminal Window

• Open an additional terminal window to be used for submitting jobs.

• Copy and paste environment variables from previous message into this window. EG:

export BACALHAU_IPFS_SWARM_ADDRESSES=/ip4/127.0.0.1/tcp/33033/p2p/QmNp5XqbkePNYtRzB2MXZPo6MxkeH6N2fYZRCLT57VsACn
export BACALHAU_API_HOST=0.0.0.0
export BACALHAU_API_PORT=62406

You are now ready to submit a job to your local devstack.

Submit a simple job

This will submit a simple job to a single node:

./bacalhau docker run ubuntu echo "hello devstack test"

This should output something like the following:

15:01:00.638 | INF bacalhau/utils.go:69 > Development client version, skipping version check
d7d4d23d-08ff-46f4-a695-f37647da67cc

After a short while - the job should be in complete state.

./bacalhau job list --wide
 CREATION_TIME      ID                                    JOB                             STATE      RESULT
 22-08-29-15:01:00  d7d4d23d-08ff-46f4-a695-f37647da67cc  Docker ubuntu echo hello world  Published  /ipfs/QmW7TdjNEMzqmWxm5WPK1p6QCkeChxMLpvhLxyUW2wpjCf

Download the results to the current directory:

./bacalhau job get d7d4d23d-08ff-46f4-a695-f37647da67cc # Works with partial IDs - just the first 8 characters

You should now have the following files and directories:

• stdout

• stderr

• volumes/output

If you cat stdout it should read "hello devstack test". If you write any files in your job, they will appear in volumes/output.