It is possible to run Bacalhau completely disconnected from the main Bacalhau network so that you can run private workloads without risking running on public nodes or inadvertently sharing your data outside of your organization. The isolated network will not connect to the public Bacalhau network nor connect to a public network. To do this, we will run our network in-process rather than externally.
:::info A private network and storage is easier to set up, but a separate public server is better for production. The private network and storage will use a temporary directory for its repository and so the contents will be lost on shutdown. :::
The first step is to start up the initial node, which we will use as the requester node. This node will connect to nothing but will listen for connections.
This will produce output similar to this:
To connect another node to this private one, run the following command in your shell:
To connect another node to this private one, run the following command in your shell:
:::tip The exact command will be different on each computer and is outputted by the bacalhau serve --node-type requester ... command :::
The command bacalhau serve --private-internal-ipfs --peer ... starts up a compute node and adds it to the cluster.
To use this cluster from the client, run the following commands in your shell:
:::tip The exact command will be different on each computer and is outputted by the bacalhau serve --node-type requester ... command :::
The command export BACALHAU_IPFS_SWARM_ADDRESSES=... sends jobs into the cluster from the command line client.
Instructions for connecting to the public IPFS network via the private Bacalhau cluster:
On all nodes, start ipfs:
Then run the following command in your shell:
On the first node execute the following:
Monitor the output log for: 11:16:03.827 | DBG pkg/transport/bprotocol/compute_handler.go:39 > ComputeHandler started on host QmWXAaSHbbP7mU4GrqDhkgUkX9EscfAHPMCHbrBSUi4A35
On all other nodes execute the following:
Replace the values in the command above with your own value
Here is our example:
Then from any client set the following before invoking your Bacalhau job:
A private cluster is a network of Bacalhau nodes completely isolated from any public node. That means you can safely process private jobs and data on your cloud or on-premise hosts!
Install Bacalhau curl -sL https://get.bacalhau.org/install.sh | bash on every host
Run bacalhau serve only on one host, this will be our "bootstrap" machine
Copy and paste the command it outputs under the "To connect another node to this private one, run the following command in your shell..." line to the other hosts
Copy and paste the env vars it outputs under the "To use this requester node from the client, run the following commands in your shell..." line to a client machine
Run bacalhau docker run ubuntu echo hello on the client machine
Optionally, set up systemd units make Bacalhau daemons permanent, here's an example systemd service file.
Please contact us on Slack #bacalhau channel for questions and feedback!
Good news. Spinning up a private cluster is really a piece of cake :
That's all folks!