Private IPFS Network Setup

Set up private IPFS network

Note that currently Bacalhau v1.4.0 supports IPFS v0.27 and below. Support for later versions of IPFS will be added in the next versions.

Introduction

Support for the embedded IPFS node was discontinued in v1.4.0 to streamline communication and reduce overhead. Therefore, now in order to use a private IPFS network, it is necessary to create it yourself and then connect to it with nodes. This manual describes how to:

1. Install and configure IPFS

2. Create Private IPFS network

3. Configure your Bacalhau network to use the private IPFS network

4. Pin your data to private IPFS network

TL;DR

1. Install Go on all nodes

2. Install IPFS

3. Initialize Private IPFS network

4. Connect all nodes to the same private network

5. Connect Bacalhau network to use private IPFS network

Download and Install

In this manual Kubo (the earliest and most widely used implementation of IPFS) will be used, so first of all, Go should be installed.

1. See the Go Downloads page for latest Go version.

wget https://go.dev/dl/go1.23.0.linux-amd64.tar.gz

2. Remove any previous Go installation by deleting the /usr/local/go folder (if it exists), then extract the archive you downloaded into /usr/local, creating a fresh Go tree in /usr/local/go:

rm -rf /usr/local/go && tar -C /usr/local -xzf go1.23.0.linux-amd64.tar.gz

3. Add /usr/local/go/bin to the PATH environment variable. You can do this by adding the following line to your $HOME/.profile or /etc/profile (for a system-wide installation):

export PATH=$PATH:/usr/local/go/bin

Changes made to a profile file may not apply until the next time you log into the system. To apply the changes immediately, just run the shell commands directly or execute them from the profile using a command such as source $HOME/.profile.

4. Verify that Go is installed correctly by checking its version:

go version

The next step is to download and install Kubo. Select and download the appropriate version for your system. It is recommended to use the latest stable version.

wget https://dist.ipfs.tech/kubo/v0.29.0/kubo_v0.29.0_linux-amd64.tar.gz
tar -xvzf kubo_v0.29.0_linux-amd64.tar.gz
sudo bash kubo/install.sh

Verify that IPFS is installed correctly by checking its version:

ipfs --version

Configure Bootstrap IPFS Node

A bootstrap node is used by client nodes to connect to the private IPFS network. The bootstrap connects clients to other nodes available on the network.

Execute the ipfs init command to initialize an IPFS node:

ipfs init

# example output

generating ED25519 keypair...done
peer identity: 12D3KooWQqr8BLHDUaZvYG59KnrfYJ1PbbzCq3pzfpQ6QrKP5yz7
initializing IPFS node at /home/username/.ipfs

The next step is to generate the swarm key - a cryptographic key that is used to control access to an IPFS network, and export the key into a swarm.key file, located in the ~/ipfs folder.

echo -e "/key/swarm/psk/1.0.0/\n/base16/" > swarm.key
ipfs key gen swarmkey >> ~/.ipfs/swarm.key

# example swarm.key content:

/key/swarm/psk/1.0.0/
/base16/
k51qzi5uqu5dli3yce3powa8pme8yc2mcwc3gpfwh7hzkzrvp5c6l0um99kiw2

Now the default entries of bootstrap nodes should be removed. Execute the command on all nodes:

ipfs bootstrap rm --all

Check that bootstrap config does not contain default values:

ipfs config show | grep Bootstrap

# expected output:

  "Bootstrap": null,

Configure IPFS to listen for incoming connections on specific network addresses and ports, making the IPFS Gateway and API services accessible. Consider changing addresses and ports depending on the specifics of your network.

ipfs config Addresses.Gateway /ip4/0.0.0.0/tcp/8080

ipfs config Addresses.API /ip4/0.0.0.0/tcp/5001

Start the IPFS daemon:

ipfs daemon

Configure Client Nodes

Copy the swarm.key file from the bootstrap node to client nodes into the ~/.ipfs/ folder and initialize IPFS:

ipfs init

Apply same config as on bootstrap node and start the daemon:

ipfs bootstrap rm — all

ipfs config Addresses.Gateway /ip4/0.0.0.0/tcp/8080

ipfs config Addresses.API /ip4/0.0.0.0/tcp/5001

ipfs daemon

Done! Now you can check that private IPFS network works properly:

1. List peers on the bootstrap node. It should list all connected nodes:

ipfs swarm peers

# example output for single connected node

/ip4/10.0.2.15/tcp/4001/p2p/12D3KooWQqr8BLHDUaZvYG59KnrfYJ1PbbzCq3pzfpQ6QrKP5yz7

2. Pin some files and check their availability across the network:

# Create a sample text file and pin it
echo “Hello from the private IPFS network!” > sample.txt

# Pin file:
ipfs add sample.txt

# example output:

added QmWQeYip3JuwhDFmkDkx9mXG3p83a3zMFfiMfhjS2Zvnms sample.txt
 25 B / 25 B [=========================================] 100.00%

# Retrieve and display the content of a pinned file
# Execute this on any node of your private network
ipfs cat QmWQeYip3JuwhDFmkDkx9mXG3p83a3zMFfiMfhjS2Zvnms

# expected output:

Hello from the private IPFS network!

Configure the IPFS Daemon as systemd Service

Finally, make the IPFS daemon run at system startup. To do this:

1. Create new service unit file in the /etc/systemd/system/

sudo nano /etc/systemd/system/ipfs.service

2. Add following content to the file, replacing /path/to/your/ipfs/executable with the actual path

[Unit]
Description=IPFS Daemon
After=network.target

[Service]
User=username
ExecStart=/path/to/your/ipfs/executable daemon
Restart=on-failure

[Install]
WantedBy=multi-user.target

Use which ipfs command to locate the executable.

Usually path to the executable is /usr/local/bin/ipfs

For security purposes, consider creating a separate user to run the service. In this case, specify its name in the User= line. Without specifying user, the ipfs service will be launched with root, which means that you will need to copy the ipfs binary to the /root directory

3. Reload and enable the service

sudo systemctl daemon-reload
sudo systemctl enable ipfs

4. Done! Now reboot the machine to ensure that daemon starts correctly. Use systemctl status ipfs command to check that service is running:

sudo systemctl status ipfs

#example output

● ipfs.service - IPFS Daemon
     Loaded: loaded (/etc/systemd/system/ipfs.service; enabled; preset: enabled)
     Active: active (running) since Wed 2024-09-10 13:24:09 CEST; 16min ago

Configure Bacalhau Nodes

Now to connect your private Bacalhau network to the private IPFS network, the IPFS API address should be specified using the --ipfs-connect flag. It can be found in the ~/.ipfs/api file:

bacalhau serve \
# any other flags
--ipfs-connect /ip4/0.0.0.0/tcp/5001

Done! Now your private Bacalhau network is connected to the private IPFS network!

Test Configured Networks

To verify that everything works correctly:

1. Pin the file to the private IPFS network

2. Run the job, which takes the pinned file as input and publishes result to the private IPFS network

3. View and download job results

Create and Pin Sample File

Create any file and pin it. Use the ipfs add command:

# create file
echo "Hello from private IPFS network!" > file.txt

# pin the file
ipfs add file.txt

# example output:

added QmWQK2Rz4Ng1RPFPyiHECvQGrJb5ZbSwjpLeuWpDuCZAbQ file.txt
 33 B / 33 B

Run a Bacalhau Job

Run a simple job, which fetches the pinned file via its CID, lists its content and publishes results back into the private IPFS network:

bacalhau docker run \
-i ipfs://QmWQK2Rz4Ng1RPFPyiHECvQGrJb5ZbSwjpLeuWpDuCZAbQ
--publisher ipfs \
alpine cat inputs

# example output

Job successfully submitted. Job ID: j-0402f760-70e3-404a-99a9-c87e200f9dde
Checking job status... (Enter Ctrl+C to exit at any time, your job will continue running):

	Communicating with the network  ................  done ✅  0.0s
	   Creating job for submission  ................  done ✅  0.5s
	               Job in progress  ................  done ✅  1.0s

To get more details about the run, execute:
	bacalhau job describe j-0402f760-70e3-404a-99a9-c87e200f9dde

To get more details about the run executions, execute:
	bacalhau job executions j-0402f760-70e3-404a-99a9-c87e200f9dde

To download the results, execute:
	bacalhau job get j-0402f760-70e3-404a-99a9-c87e200f9dde

View and Download Job Results

Use bacalhau job describe command to view job execution results:

bacalhau job describe j-0402f760-70e3-404a-99a9-c87e200f9dde        

# example output (was truncated for brevity)

...
Standard Output
Hello from private IPFS network!

Use bacalhau job get command to download job results. In this particular case, ipfs publisher was used, so the get command will print the CID of the job results:

bacalhau job get j-0402f760-70e3-404a-99a9-c87e200f9dde

# example output

Fetching results of job 'j-0402f760-70e3-404a-99a9-c87e200f9dde'...
No supported downloader found for the published results. You will have to download the results differently.
[
    {
        "Type": "ipfs",
        "Params": {
            "CID": "QmSskRNnbbw8rNtkLdcJrUS2uC2mhiKofVJsahKRPgbGGj"
        }
    }
]

Use the ipfs ls command to view the results:

ipfs ls QmSskRNnbbw8rNtkLdcJrUS2uC2mhiKofVJsahKRPgbGGj

# example output

QmS6mcrMTFsZnT3wAptqEb8NpBPnv1H6WwZBMzEjT8SSDv 1  exitCode
QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH 0  stderr
QmWQK2Rz4Ng1RPFPyiHECvQGrJb5ZbSwjpLeuWpDuCZAbQ 33 stdout

Use the ipfs cat command to view the file content. In our case, the file of interest is the stdout:

ipfs cat QmWQK2Rz4Ng1RPFPyiHECvQGrJb5ZbSwjpLeuWpDuCZAbQ

# example output

Hello from private IPFS network!

Use the ipfs get command to download the file using its CID:

ipfs get --output stdout QmWQK2Rz4Ng1RPFPyiHECvQGrJb5ZbSwjpLeuWpDuCZAbQ
Saving file(s) to stdout
 33 B / 33 B [===============================================] 100.00% 0s

Need Support?​

For questions and feedback, please reach out in our Slack