Description

The bacalhau agent command is a parent command that offers sub-commands to query information about the Bacalhau agent. This can be useful for debugging, monitoring, or managing the agent's behavior and health.

Usage

bacalhau agent [command]

Available Commands

1. alive:

   • Description: Retrieves the agent's liveness and health information. This can be helpful to determine if the agent is running and healthy.

   • Usage:

     Copy

     bacalhau agent alive

2. node:

   • Description: Gathers the agent's node-related information. This might include details about the machine or environment where the agent is running, available resources, supported engines, etc.

   • Usage:

     Copy

     bacalhau agent node

3. version:

   • Description: Retrieves the Bacalhau version of the agent. This can be beneficial for ensuring compatibility or checking for updates.

   • Usage:

     Copy

     bacalhau agent version

For more detailed information on any of the sub-commands, you can use the command:

bacalhau agent [command] --help

Flags

• -h, --help:

  • Description: Displays help information for the agent command.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Specifies the port for REST communication. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Sets the desired log format. Options are: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: ``$HOME/.bacalhau`

Description

The bacalhau agent alive command provides information about the agent's liveness and health. This is essential for monitoring and ensuring that the agent is active and functioning correctly.

Usage

bacalhau agent alive [flags]

Flags

• -h, --help:

  • Description: Displays help information for the alive sub-command.

• --output format:

  • Description: Determines the format in which the output is displayed. Available formats include JSON and YAML.

  • Options: json, yaml

  • Default: yaml

• --pretty:

  • Description: Formats the output for enhanced readability. This flag is relevant only when using JSON or YAML output formats.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Specifies the port for REST communication. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Sets the desired log format. Options are: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: ``$HOME/.bacalhau`

Examples

Checking the Agent's Liveness and Health Info

1. Basic Usage:

   Command:

   Copy

   → bacalhau agent alive

   Output:

   Copy

   status: OK

2. Output in JSON format:

   Command:

   Copy

   → bacalhau agent alive --output json --pretty

   Output:

   Copy

   {
     "Status": "OK"
   }

The bacalhau agent version command is used to obtain the version of the bacalhau agent.

Description:

Using this command, users can quickly retrieve the version of the agent, allowing them to confirm the specific release of the software they are using.

Usage:

bacalhau agent version [flags]

Flags:

• -h, --help:

  • Show help for the version command.

• --output format:

  • Defines the output format of the command's results. Accepted formats include "json" and "yaml".

• --pretty:

  • Used for pretty printing the output, enhancing readability. This flag is applicable only for the "json" and "yaml" output formats.

Global Flags:

• --api-host string:

  • Designates the host for client-server communication via REST. If the BACALHAU_API_HOST environment variable is present, this flag will be disregarded.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Defines the port for client-server communication through REST. This flag becomes irrelevant if the BACALHAU_API_PORT environment variable is specified.

  • Default: 1234

• --log-mode logging-mode:

  • Specifies the desired logging format.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Indicates the path to the bacalhau repository.

  • Default: "$HOME/.bacalhau"`

Examples

1. Retrieve the agent version:

   Execute the command to get the agent version:

   Copy

   bacalhau agent version

   Expected output:

   Copy

   Bacalhau v0.0.0-xxxxxxx
   BuildDate 2023-09-22 16:03:44 +0000 UTC
   GitCommit 0fe81cb488f666845ac72c73a4b804aaa658e511

2. Retrieve the agent version in JSON format:

   Copy

   bacalhau agent version --output json

   Expected output:

   Copy

   {"major":"0","minor":"0","gitversion":"v0.0.0-xxxxxxx","gitcommit":"0fe81cb488f666845ac72c73a4b804aaa658e511","builddate":"2023-09-22T16:03:44Z","goos":"linux","goarch":"amd64"}

3. Retrieve the agent version in Pretty-printed JSON format:

   Copy

   bacalhau agent version --output json --pretty

   Expected output:

   Copy

   {
     "major": "0",
     "minor": "0",
     "gitversion": "v0.0.0-xxxxxxx",
     "gitcommit": "0fe81cb488f666845ac72c73a4b804aaa658e511",
     "builddate": "2023-09-22T16:03:44Z",
     "goos": "linux",
     "goarch": "amd64"
   }

Description

The bacalhau agent node command retrieves information about the agent's node, providing insights into the agent's environment and aiding in debugging.

Usage

bacalhau agent node [flags]

Flags

• -h, --help:

  • Displays help information for the node sub-command.

• --output format:

  • Defines the output format (either JSON or YAML).

  • Options: json, yaml

  • Default: yaml

• --pretty:

  • Beautifies the output when using JSON or YAML formats.

Global Flags

• --api-host string:

  • The host for REST communication. Overrides the BACALHAU_API_HOST environment variable.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • The port for REST communication. Overridden if BACALHAU_API_PORT environment variable is set.

  • Default: 1234

• --log-mode logging-mode:

  • Specifies the log format. Choices are: default, station, json, combined, event.

  • Default: default

• --repo string:

  • Path to the bacalhau repository.

  • Default: ``$HOME/.bacalhau`

Examples

1. Retrieve Node Information in Default Format (YAML)

   Copy

   bacalhau agent node

2. Retrieve Node Information in JSON Format

   Copy

   bacalhau agent node --output json

3. Retrieve Node Information in Pretty-printed JSON Format

   Copy

   bacalhau agent node --output json --pretty

Description:

The bacalhau config default command prints the default configuration of a bacalhau node to the standard output (stdout). This command is advantageous for users to view the baseline settings a bacalhau node will use in the absence of any user-defined configuration changes. It provides a clear view of the default operational parameters of the node, aiding users in understanding and customizing their configuration from a known baseline.

Note: The output of this command shows the initial default settings for a new bacalhau node and is useful for understanding the foundational settings for customization. To apply these default settings, you can redirect the output to your configuration file using bacalhau config default > ~/.bacalhau/config.yaml, which overwrites your current configuration file with the default settings. However, if you wish to always use the latest default settings, especially if the defaults are updated over time, consider deleting your existing configuration file (e.g., ~/.bacalhau/config.yaml). This approach ensures that your bacalhau node uses the most current defaults, circumventing potential discrepancies between the latest defaults and those captured in an older configuration file created with bacalhau config default.

Usage

Flags

• -h, --help:

  • Description: Displays help information for the list sub-command.

• --path:

  • Description: Sets path dependent config fields

  • Default: $HOME/.bacalhau

Examples

Redirecting Default Configuration to a File

Description

The bacalhau config command is a parent command that offers sub-commands to modify and query information about the Bacalhau config. This can be useful for debugging, monitoring, or managing the nodes configuration.

Usage

Available Commands

1. :

   • Description: Lists the configuration keys and values of the bacalhau node. This command is useful for understanding how configuration keys map to their respective values, aiding in the use of the bacalhau config set command.

   • Usage:

2. :

   • Description: Sets a value in the bacalhau node's configuration file. This command is used to modify the configuration file that the bacalhau node will reference for its settings.

   • Usage:

3. :

   • Description: Prints the default configuration of a bacalhau node to the standard output (stdout). This command is beneficial for viewing the baseline settings a bacalhau node will use before any user-defined configuration changes are applied.

   • Usage:

4. :

   • Description: Automatically sets compute resource values in the bacalhau node's configuration file based on the hardware resources of the user's machine. This command simplifies the process of allocating resources for jobs by dynamically adjusting the settings to match the machine's capabilities.

   • Usage:

Description

The bacalhau job command provides a suite of sub-commands to submit, query, and manage jobs within Bacalhau. Users can deploy jobs, obtain job details, track execution logs, and more.

Usage

Available Commands

1. :

   • Description: Retrieves detailed information of a job using its ID.

   • Usage:

2. :

   • Description: Lists all executions associated with a job, identified by its ID.

   • Usage:

3. :

   • Description: Enumerates the historical events related to a job, identified by its ID.

   • Usage:

4. :

   • Description: Provides an overview of all submitted jobs.

   • Usage:

5. :

   • Description: Fetches and streams the logs from a currently executing job.

   • Usage:

6. :

   • Description: Submits a job for execution using either a JSON or YAML configuration file.

   • Usage:

7. :

   • Description: Halts a previously submitted job.

   • Usage:

For comprehensive details on any of the sub-commands, run:

Flags

• -h, --help:

  • Description: Shows the help information for the job command.

Global Flags

1. --api-host string:

   • Description: Determines the host for RESTful communication between the client and server. This flag is overlooked if the BACALHAU_API_HOST environment variable is set.

   • Default: bootstrap.production.bacalhau.org

2. --api-port int:

   • Description: Designates the port for RESTful communication. This flag is bypassed if the BACALHAU_API_PORT environment variable is active.

   • Default: 1234

3. --log-mode logging-mode:

   • Description: Chooses the preferred log format. Available choices are: default, station, json, combined, and event.

   • Default: default

4. --repo string:

   • Description: Specifies the path to the bacalhau repository.

   • Default: $HOME/.bacalhau

Description:

The bacalhau config auto-resources command automatically configures compute resource values in the bacalhau node's configuration file based on the hardware resources of the user's machine. This command streamlines the process of resource allocation for jobs, dynamically adjusting settings to align with the capabilities of the machine. It is designed to simplify the task of resource management, ensuring that the node operates efficiently and effectively within the hardware's limits.

Note: The bacalhau config auto-resources command intelligently adjusts resource allocation settings based on the specific hardware configuration of your machine, promoting optimal utilization for bacalhau jobs. Due to the dynamic nature of this command, the specific values set in the configuration will vary depending on the available hardware resources of the machine in use. This functionality is particularly beneficial for users who seek to optimize their node's performance without the need for manual calculations of resource limits. It is important for users to understand that these settings will directly impact the number and types of jobs their node can manage at any given time, based on the machine's resource capacity.

Usage

Flags

• --default-job-percentage int:

  • Description: Sets the default percentage of resources allocated for each job when specific limits are not defined. Acceptable values range from 1 to 100 (values over 100 are rejected).

  • Default: 75

• --job-percentage int:

  • Description: Determines the percentage of resources that can be utilized at one time for a single job. Accept values from 1 to 100 (values over 100 are rejected).

  • Default: 75

• --queue-job-percentage int:

  • Description: Specifies the total percentage of resources that the system can allocate for all jobs queued at one time. Accept values from 1 to 100 (values over 100 are accepted).

  • Default: 150

• --total-percentage int:

  • Description: Indicates the total percentage of resources that the system can utilize at one time across all jobs. Accept values from 1 to 100 (values over 100 are rejected).

  • Default: 75

Examples

(Ran on an Apple M1 Max with 10 Cores and 64GB RAM)

1. Basic Usage:

   Command:

   Config File:
2. Queue 500% system resources:

   Command:

   Config File:
3. With 25% of system resources:

   Command:

   Config File:

Description:

The bacalhau config set command sets a value in the bacalhau node's configuration file. This command is used to modify the configuration file that the bacalhau node will reference for its settings. Key names in the configuration are case-insensitive. Additionally, the command validates the value being set based on the type of the configuration key, ensuring that only appropriate and valid configurations are applied.

Note: Changes made using this command will be applied to the configuration file, but they do not immediately affect the running configuration of an active bacalhau node. The modifications will take effect only after the node is restarted.

Usage

Flags

• -h, --help:

  • Description: Displays help information for the set sub-command.

Examples

Configuring the Server API Port Value

Configuring the Logging Mode Value

Multiple Set commands append to the file

Set command value validation

Example of invalid logging mode value

Example of invalid time duration value

Description

The bacalhau node command provides a set of sub-commands to query and manage node-related information within Bacalhau. With these tools, users can access specific details about nodes, list all network nodes, and more.

Usage

Available Commands

1. :

   • Description: Approves a single node to join the cluster.

   • Usage:

2. :

   • Description: Deletes a node from the cluster using its ID.

   • Usage:

3. :

   • Description: Retrieves detailed information of a node using its ID.

   • Usage:

4. :

   • Description: Lists the details of all nodes present in the network.

   • Usage:

5. :

• Description: Reject a specific node's request to join the cluster.

• Usage:

For comprehensive details on any of the sub-commands, run:

Flags

• -h, --help:

  • Description: Shows the help information for the node command.

Global Flags

• --api-host string:

  • Description: Specifies the host for RESTful communication between the client and server. The flag will be ignored if the BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Designates the port for RESTful communication. The flag will be bypassed if the BACALHAU_API_PORT environment variable is active.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Chooses the preferred log format. Available choices are: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Specifies the path to the bacalhau repository.

  • Default: /Users/walid/.bacalhau

Description

The bacalhau job list command provides a listing of all submitted jobs. This command offers an overview of all tasks and processes registered in the system, allowing users to monitor and manage their jobs effectively.

Usage

bacalhau job list [flags]

Flags

• -h, --help:

  • Description: Display help for the list command.

• --hide-header:

  • Description: Opts out of printing the column headers in the results.

• --labels string:

  • Description: Filters jobs by labels. It's designed to function similar to Kubernetes label selectors.

  • Default: bacalhau_canary != true

• --limit uint32:

  • Description: Limits the number of results returned.

  • Default: 10

• --next-token string:

  • Description: Uses the provided token for pagination.

• --no-style:

  • Description: Strips all styling from the table output.

• --order-by string:

  • Description: Organizes results based on a chosen field. Valid fields are id and created_at.

• --order-reversed:

  • Description: Reverses the order of the displayed results.

• --output format:

  • Description: Dictates the desired output format for the command. Options are table, csv, json, and yaml.

  • Default: table

• --pretty:

  • Description: Offers a more visually pleasing output for json and yaml formats.

• --wide:

  • Description: Presents full values in the table results, preventing truncation.

Global Flags

• --api-host string:

  • Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST environment variable, if set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT environment variable takes precedence if set.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Designates the desired log format. Options include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Points to the bacalhau repository location.

  • Default: $HOME/.bacalhau

Examples

1. List all jobs:

   Execute the command to list all the jobs:

   Copy

   bacalhau job list

   Expected output:

   Copy

   CREATED   ID          JOB     TYPE   STATE
   08:19:07  d78a4cb4    docker  batch  Completed
   04:17:21  e45f31a7    docker  batch  Completed
   04:53:50  f4993f62    docker  batch  Completed
   ... (trimmed for brevity) ...

2. Limit the list to the last two jobs:

   Limit the list to display only the last two jobs:

   Copy

   bacalhau job list --limit 2

   Expected output:

   Copy

   CREATED   ID          JOB     TYPE   STATE
   03:14:16  19a26187    docker  batch  Completed
   21:47:21  2a53a13b    docker  batch  Completed

3. Order the list by creation date in descending order:

   Order the jobs by their creation date in a descending manner:

   Copy

   bacalhau job list --order-by created_at --order-reversed

   Expected output:

   Copy

   CREATED   ID          JOB     TYPE   STATE
   17:44:16  90e14efd    docker  batch  Completed
   17:44:08  8204570c    docker  batch  Completed
   17:43:50  f196521d    docker  batch  Completed
   ... (trimmed for brevity) ...

4. Filter the jobs by specific labels:

   Display jobs that have specific labels:

   Copy

   bacalhau job list --labels "region in (us-east-1, us-east-2),env = prod"

   Expected output:

   Copy

   ... (filtered jobs) ...

5. Display the list in JSON format with pretty printing:

   Get a limited list of jobs in a formatted JSON output:

   Copy

   bacalhau job list --limit 3 --output json --pretty

   Expected output:

   Copy

   ... [The JSON formatted output] ...

Description

The bacalhau job logs command allows users to retrieve logs from a job that has been previously submitted. This command is useful for tracking and debugging the progress and state of a running or completed job.

Usage

bacalhau job logs [id] [flags]

Flags

1. -f, --follow:

   • Description: This flag allows the user to follow the logs in real-time after fetching the current logs. It provides a continuous stream of log updates, similar to tail -f in Unix-like systems.

2. -h, --help:

   • Description: Display help information for the logs command.

Global Flags

1. --api-host string:

   • Description: Specifies the host for the client and server to communicate through REST. This flag is disregarded if the BACALHAU_API_HOST environment variable is set.

   • Default: bootstrap.production.bacalhau.org

2. --api-port int:

   • Description: Sets the port for RESTful communication between the client and server. If the BACALHAU_API_PORT environment variable is available, this flag is ignored.

   • Default: 1234

3. --log-mode logging-mode:

   • Description: Determines the desired log format. Available options include default, station, json, combined, and event.

   • Default: default

4. --repo string:

   • Description: Specifies the path to the bacalhau repository.

   • Default: $HOME/.bacalhau

Examples

1. Display Logs for a Previously Submitted Job with Full ID:

   Command:

   Copy

   bacalhau job logs j-51225160-807e-48b8-88c9-28311c7899e1

   Expected Output:

   Copy

   [2023-09-24 09:01:32] INFO - Application started successfully.
   [2023-09-24 09:01:33] DEBUG - Initializing database connections.
   [2023-09-24 09:01:35] WARN - API rate limit approaching.
   [2023-09-24 09:02:01] ERROR - Failed to retrieve data from endpoint: /api/v1/data.
   [2023-09-24 09:05:00] INFO - Data sync completed with 4500 new records.

2. Follow Logs in Real-Time:

   Command:

   Copy

   bacalhau job logs --follow j-51225160-807e-48b8-88c9-28311c7899e1

   Expected Output:

   Copy

   [2023-09-24 11:30:02] INFO - User 'john_doe' logged in successfully.
   [2023-09-24 11:30:15] DEBUG - Fetching data from cache for key: userSettings_john_doe.
   [2023-09-24 11:31:05] WARN - High memory usage detected: 85% of allocated resources.
   ... [Logs continue to appear in real-time] ...

3. Display Logs Using a Shortened ID:

   Command:

   Copy

   bacalhau job logs j-ebd9bf2f

   Expected Output:

   Copy

   [2023-09-24 10:15:12] INFO - Application initialization sequence started.
   [2023-09-24 10:15:13] DEBUG - Loading configurations from /config/app.json.
   [2023-09-24 10:15:14] INFO - Connected to message broker successfully.
   [2023-09-24 10:16:00] ERROR - Failed to send email notification to user@example.com.

Description

The bacalhau job executions command retrieves a list of executions for a specific job based on its ID. This can be essential when tracking the various runs and their respective states for a particular job.

Usage

bacalhau job executions [id] [flags]

Flags

• -h, --help:

  • Description: Display help for the executions command.

• --hide-header:

  • Description: Do not print the column headers when displaying the results.

• --limit uint32:

  • Description: Restricts the number of results returned.

  • Default: 20

• --next-token string:

  • Description: Uses the specified token for pagination. Useful for fetching the next set of results.

• --no-style:

  • Description: Removes all styling from the table output, displaying raw data.

• --order-by string:

  • Description: Orders results based on a specific field. Valid fields are: modify_time, create_time, id, and state.

• --order-reversed:

  • Description: Reverses the order of the results. Useful in conjunction with --order-by.

• --output format:

  • Description: Specifies the desired output format for the command. Supported values are table, csv, json, and yaml.

  • Default: table

• --pretty:

  • Description: Pretty prints the output. This option is applicable only to json and yaml output formats.

• --wide:

  • Description: Prints full values in the table results without truncating any information.

Global Flags

• --api-host string:

  • Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST environment variable is set, this flag will be ignored.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Specifies the desired log format. Supported values include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

1. List executions for a specific Job:

   Copy

   bacalhau job executions j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881

   Expected output:

   Copy

   CREATED   MODIFIED  ID          NODE ID   REV.  COMPUTE    DESIRED  COMMENT
                                                   STATE      STATE
   16:46:03  16:46:04  e-99362435  QmTSJgdN  6     Completed  Stopped
   16:46:03  16:46:04  e-75dd20bb  QmXRdLru  6     Completed  Stopped
   16:46:03  16:46:04  e-03870df5  QmVXwmdZ  6     Completed  Stopped

2. Order executions by state for a specific job:

   Execute the command:

   Copy

   bacalhau job executions j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --order-by state

   Expected output:

   Copy

   CREATED   MODIFIED  ID          NODE ID   REV.  COMPUTE    DESIRED  COMMENT
                                                   STATE      STATE
   16:46:03  16:46:04  e-03870df5  QmVXwmdZ  6     Completed  Stopped
   16:46:03  16:46:04  e-75dd20bb  QmXRdLru  6     Completed  Stopped
   16:46:03  16:46:04  e-99362435  QmTSJgdN  6     Completed  Stopped

3. List executions with YAML output:

   Copy

   bacalhau job executions j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --output yaml

   Expected output:

   Copy

   ... [The YAML formatted output] ...

Description

The bacalhau exec command allows for the specification of jobs to be executed from the command line without the need for a job specification file (see job run).

Usage

bacalhau exec [flags] [job-type] arguments

Flags

1. -h, --help:

   • Description: Displays help information for the exec sub-command.

2. --code:

   • Includes the specified code in the job. This can be a single file or a directory containing many files. There is a limit of 10Mb on the size of the uploaded code.

3. -f, --follow:

   • Description: If provided, the command will continuously display the output from the job as it runs.

4. --wait

   • Description: Wait for the job to finish. Use --wait=false to return as soon as the job is submitted.

5. --wait-timeout-secs

   • Description: When using --wait, how many seconds to wait for the job to complete before giving up.

6. --node-details

   • Description: Print out details of all nodes (overridden by --id-only).

7. --id-only:

   • Description: On successful job submission, only the Job ID will be printed.

8. -p, --publisher

   • Description: Where to publish the result of the job.

     Examples:

     Publish to IPFS

     -p ipfs

     Publish to S3

     -p s3://bucket/key

9. -i, --input

   • Description: Mount URIs as inputs to the job. Can be specified multiple times. Format: src=URI,dst=PATH[,opt=key=value]

     Examples:

     Mount IPFS CID to /inputs directory

     -i ipfs://QmeZRGhe4PmjctYVSVHuEiA9oSXnqmYa4kQubSHgWbjv72

     Mount S3 object to a specific path

     -i s3://bucket/key,dst=/my/input/path

     Mount S3 object with specific endpoint and region

     -i src=s3://bucket/key,dst=/my/input/path,opt=endpoint=https://s3.example.com,opt=region=us-east-1

10. -o, --output

    • Description: name:path of the output data volumes. 'outputs:/outputs' is always added unless '/outputs' is mapped to a different name.

11. -e, --env

    • Description: The environment variables to supply to the job (e.g. --env FOO=bar --env BAR=baz)

12. --timeout

    • Description: Job execution timeout in seconds (e.g. 300 for 5 minutes)

13. -l, --labels

    • Description: List of labels for the job. Enter multiple in the format '-l a -l 2'. All characters not matching /a-zA-Z0-9_:|-/ and all emojis will be stripped.

14. -s, --selector

    • Description: Selector (label query) to filter nodes on which this job can be executed, supports '=', '==', and '!='.(e.g. -s key1=value1,key2=value2). Matching objects must satisfy all of the specified label constraints.

Global Flags

1. --api-host string:

   • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST environment variable is set.

   • Default: bootstrap.production.bacalhau.org

2. --api-port int:

   • Description: Specifies the port for REST communication. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

   • Default: 1234

3. --log-mode logging-mode:

   • Description: Sets the desired log format. Options are: default, station, json, combined, and event.

   • Default: default

4. --repo string:

   • Description: Defines the path to the bacalhau repository.

   • Default: ``$HOME/.bacalhau`

Examples

Running python tasks

1. Basic Usage:

   Command:

   Copy

   bacalhau exec python -- -c "import this"

   Output:

   Copy

   The Zen of Python, by Tim Peters
   
   Beautiful is better than ugly.
   Explicit is better than implicit.
   Simple is better than complex.
   Complex is better than complicated.
   Flat is better than nested.
   ....

2. Single file Python:

   Command:

   Copy

   bacalhau exec --code=app.py python app.py

   where app.py is

   Copy

   """
   pip install colorama
   """
   
   from colorama import Fore
   print(Fore.RED + "Hello World")

   Output:

   As red text

   Copy

   Hello World

Running duckdb queries

1. Basic Usage:

   Command:

   Copy

   cat describe.sql
     DESCRIBE TABLE '/inputs/world-cities_csv.csv';
   
   bacalhau exec --code=describe.sql -i src=https://datahub.io/core/world-cities/r/world-cities.csv,dst=/inputs duckdb -- -init /code/describe.sql

   Output:

   Copy

        ┌─────────────┬─────────────┬─────────┬─────────┬─────────┬─────────┐
        │ column_name │ column_type │  null   │   key   │ default │  extra  │
        │   varchar   │   varchar   │ varchar │ varchar │ varchar │ varchar │
        ├─────────────┼─────────────┼─────────┼─────────┼─────────┼─────────┤
        │ name        │ VARCHAR     │ YES     │         │         │         │
        │ country     │ VARCHAR     │ YES     │         │         │         │
        │ subcountry  │ VARCHAR     │ YES     │         │         │         │
        │ geonameid   │ BIGINT      │ YES     │         │         │         │
        └─────────────┴─────────────┴─────────┴─────────┴─────────┴─────────┘

Description

The bacalhau config list command lists the configuration keys and values of the bacalhau node. This command is useful for understanding how configuration keys map to their respective values, aiding in the use of the bacalhau config set command.

Note: Configuration values displayed by this command represent the settings that will be applied when the bacalhau node is next restarted. It is important to note that these values may not reflect the current operational configuration of an active bacalhau node. The displayed configuration is relevant and accurate for a node that is either not currently running or that has been restarted after the execution of this command.

Usage

bacalhau config list [flags]

Flags

• -h, --help:

  • Description: Displays help information for the list sub-command.

• --hide-header:

  • Description: Do not print the column headers when displaying the results.

  • Default: false

• --no-style:

  • Description: Removes all styling from the table output, displaying raw data.

  • Default: false

• --output format:

  • Description: Determines the format in which the output is displayed. Available formats include Table, JSON, and YAML.

  • Options: json, yaml, table

  • Default: table

• --pretty:

  • Description: Formats the output for enhanced readability. This flag is relevant only when using JSON or YAML output formats.

  • Default: true

• --wide:

  • Description: Prints full values in the table results without truncating any information.

  • Default: false

Examples

Listing the Bacalhau nodes configuration settings

1. Basic Usage:

   Command:

   Copy

   bacalhau config list

   Output:

   Copy

    KEY          VALUE
    <key_name> <key_value>
   ...

2. Output in JSON format:

   Command:

   Copy

   bacalhau config list --output json --pretty

   Output:

   Copy

   [
     {
       "Key": "<key_name>",
       "Value": <key_value>
     },
     ...
   ]

Description

The bacalhau job run command facilitates the initiation of a job from a file or directly from the standard input (stdin). The command supports both JSON and YAML data formats. This command is particularly useful for quickly executing a job without the need for manual configurations.

Usage

Flags

• --dry-run:

  • Description: With this flag, the job will not be submitted. Instead, it will display what would have been submitted, providing a way to preview before actual submission.

• -f, --follow:

  • Description: If provided, the command will continuously display the output from the job as it runs.

• --id-only:

  • Description: On successful job submission, only the Job ID will be printed.

• --no-template:

  • Disable the templating feature. When this flag is set, the job spec will be used as-is, without any placeholder replacements

• --node-details:

  • Description: Displays details of all nodes. Note that this flag is overridden if --id-only is provided.

• --show-warnings:

  • Description: Shows any warnings that occur during the job submission.

• -E, --template-envs:

  • Specify a regular expression pattern for selecting environment variables to be included as template variables in the job spec. e.g. --template-envs ".*" will include all environment variables.

• -V, --template-vars:

  • Replace a placeholder in the job spec with a value. e.g. --template-vars foo=bar

• --wait:

  • Description: Waits for the job to finish execution. To set this to false, use --wait=false

  • Default: true

• --wait-timeout-secs int:

  • Description: If --wait is provided, this flag sets the maximum time (in seconds) the command will wait for the job to finish before it terminates.

  • Default: 600 seconds

• -h, --help:

  • Description: Displays help information for the run command.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for REST communication. If BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Selects the desired log format. Options include: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

Sample Job (job.yaml)

A sample job used in the following examples is provided below:

This configuration describes a batch job that runs a Docker task. It utilizes the ubuntu:latest image and executes the command echo Hello Bacalhau!.

1. Running a Job using a YAML Configuration:

   To run a job with a configuration provided in a job.yaml file:

   Command:

   Expected Output:
2. Running a Job and Following its Logs:

   Command:

   Expected Output:
3. Running a Job Without Waiting:

   Command:

   Expected Output:
4. Fetching Only the Job ID Upon Submission:

   Command:

   Expected Output:
5. Fetching Only the Job ID and Wait for Completion:

   Command:

   Expected Output:
6. Running a Job with Node Details:

   Command:

   Expected Output:
7. Rerunning a previously submitting job:

   Command:

   Expected Output:

Templating

Description

The bacalhau job stop command allows users to terminate a previously submitted job. This is useful in scenarios where there's a need to halt a running job, perhaps due to misconfiguration or changed priorities.

Usage

Flags

• --quiet:

  • Description: If provided, the command will not display any output, neither to the standard output (stdout) nor to the standard error (stderr).

• -h, --help:

  • Description: Displays help information for the stop command.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for REST communication. If BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Selects the desired log format. Options include: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

1. Stop a Specific Job:

   If you wish to halt the execution of a job, you can utilize the stop command. Here's how you can achieve that:

   Command:

   Expected Output:
2. Silently Stop a Job:

   If you prefer to terminate a job without seeing any verbose feedback or messages, the --quiet option can be used.

   Command:

   Expected Output:

The bacalhau node reject command offers administrators the ability to reject a compute node's request to join the cluster.

Description:

Using the reject sub-command, administrators can reject a node in the pending state from joining the cluster and receiving work. This feature is crucial for system administrators to manage the cluster and will stop the node from taking part in the cluster until approved.

Usage:

Flags:

• [id]:

  • The unique identifier of the node you wish to describe.

• -h, --help:

  • Displays the help documentation for the describe command.

• -m message:

  • A message to be attached to the rejection action.

Global Flags:

• --api-host string:

  • Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST environment variable is set.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT environment variable is defined.

  • Default: 1234

• --log-mode logging-mode:

  • Determines the log format preference.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Points to the bacalhau repository's path.

  • Default: "$HOME/.bacalhau"`

Examples:

1. Reject a Node with ID nodeID123:
2. Reject a Node with an audit message:

The bacalhau node describe command offers users the ability to retrieve detailed information about a specific node using its unique identifier.

Description:

Using the describe sub-command under the bacalhau node umbrella, users can get comprehensive details of a node by providing its ID. This information is crucial for system administrators and network managers to understand the state, specifications, and other attributes of nodes in their infrastructure.

Usage:

Flags:

• [id]:

  • The unique identifier of the node you wish to describe.

• -h, --help:

  • Displays the help documentation for the describe command.

• --output format:

  • Defines the desired format for the command's output.

  • Options: "json" or "yaml"

  • Default: "yaml"

• --pretty:

  • When this flag is used, the command will pretty print the output. This is applicable only for outputs in json and yaml formats.

Global Flags:

• --api-host string:

  • Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST environment variable is set.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT environment variable is defined.

  • Default: 1234

• --log-mode logging-mode:

  • Determines the log format preference.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Points to the bacalhau repository's path.

  • Default: "$HOME/.bacalhau"`

Examples:

1. Describing a Node with ID nodeID123:
2. Describing a Node with Output in JSON Format:
3. Pretty Printing the Description of a Node:

Description

The bacalhau job describe command provides a detailed description of a specific job in YAML format. This description can be particularly useful when wanting to understand the attributes and current status of a specific job. To list all available jobs, the bacalhau job list command can be used.

Usage

bacalhau job describe [id] [flags]

Flags

• -h, --help:

  • Description: Display help for the describe command.

• --output format:

  • Description: Specifies the desired output format for the command. Supported values are json and yaml.

  • Default: yaml

• --pretty:

  • Description: Pretty prints the output. This option is applicable only to json and yaml output formats.

Global Flags

• --api-host string:

  • Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST environment variable is set, this flag will be ignored.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Specifies the desired log format. Supported values include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

1. Describe a Job with Full ID:

   Copy

   bacalhau job describe j-e3f8c209-d683-4a41-b840-f09b88d087b9

2. Describe a Job with Shortened ID:

   Copy

   bacalhau job describe j-47805f5c

3. Describe a Job with JSON Output:

   Copy

   bacalhau job describe --output json --pretty j-b6ad164a

The bacalhau node approve command offers administrators the ability to approve the cluster membership for a node using its name.

Description:

Using the approve sub-command under the bacalhau node umbrella, users can allow a node in the pending state to join the cluster and receive work. This feature is crucial for system administrators to manage the cluster.

Usage:

bacalhau node approve [id] [flags]

Flags:

• [id]:

  • The unique identifier of the node you wish to describe.

• -h, --help:

  • Displays the help documentation for the describe command.

• -m message:

  • A message to be attached to the approval action.

Global Flags:

• --api-host string:

  • Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST environment variable is set.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT environment variable is defined.

  • Default: 1234

• --log-mode logging-mode:

  • Determines the log format preference.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Points to the bacalhau repository's path.

  • Default: "$HOME/.bacalhau"`

Examples:

1. Approve a Node with ID nodeID123:

   Copy

   bacalhau node approve nodeID123

2. Approve a Node with an audit message:

   Copy

   bacalhau node approve nodeID123 -m "okay"

The bacalhau node delete command offers administrators the ability to remove a node from the cluster using its name.

Description:

Using the delete sub-command, administrators can remove a node from the list of available compute nodes in the cluster. This feature is necessary for the management of the infrastructure.

Usage:

bacalhau node delete [id] [flags]

Flags:

• [id]:

  • The unique identifier of the node you wish to describe.

• -h, --help:

  • Displays the help documentation for the describe command.

• -m message:

  • A message to be attached to the deletion action.

Global Flags:

• --api-host string:

  • Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST environment variable is set.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT environment variable is defined.

  • Default: 1234

• --log-mode logging-mode:

  • Determines the log format preference.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Points to the bacalhau repository's path.

  • Default: "$HOME/.bacalhau"`

Examples:

1. Delete the Node with ID nodeID123:

   Copy

   bacalhau node delete nodeID123

2. Delete a Node with an audit message:

   Copy

   bacalhau node delete nodeID123 -m "bad actor"

In version 1.4.0 of Bacalhau, all legacy commands will be removed. Here’s a breakdown of the old commands and their new equivalents:

For some commands there are actions required to migrate to Bacalhau v.1.4.0. In your network. In the following view these actions are specified.

Special Attention to create , validate and describe Commands

Guidance for Users Not Ready for the Changes

Users who are not prepared for the changes in CLI behavior and job specification definitions are advised to remain on Bacalhau v1.3.1. This version continues to support the legacy commands and job specifications. Users can maintain their own private Bacalhau cluster using v1.3.1.

When users are ready to transition to the new CLI behavior and job specification requirements, they can upgrade to Bacalhau v1.4.

Description

The bacalhau job history command lists the history events of a specific job based on its ID. This feature allows users to track changes, executions, and other significant milestones associated with a particular job.

Usage

Flags

• --event-type string:

  • Description: Specifies the type of history events to retrieve. Available options include all, job, and execution.

  • Default: all

• --execution-id string:

  • Description: Filters results by a specific execution ID.

• -h, --help:

  • Description: Display help for the history command.

• --hide-header:

  • Description: Opts out of printing the column headers in the results.

• --limit uint32:

  • Description: Limits the number of results returned.

• --next-token string:

  • Description: Uses the provided token for pagination.

• --no-style:

  • Description: Strips all styling from the table output.

• --node-id string:

  • Description: Filters the results by a specific node ID.

• --order-by string:

  • Description: Organizes results based on a chosen field.

• --order-reversed:

  • Description: Reverses the order of the displayed results.

• --output format:

  • Description: Dictates the desired output format for the command. Options are table, csv, json, and yaml.

  • Default: table

• --pretty:

  • Description: Offers a more visually pleasing output for json and yaml formats.

• --wide:

  • Description: Presents full values in the table results, preventing truncation.

Global Flags

• --api-host string:

  • Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST environment variable, if set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT environment variable takes precedence if set.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Designates the desired log format. Options include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Points to the bacalhau repository location.

  • Default: $HOME/.bacalhau

Examples

1. Retrieve the history of a specific job:

   Execute the command to get the job history:

   Expected output:
2. Filter the history by event type:

   Filter the job history by the event type:

   Expected output:
3. Filter the history by execution ID:

   Filter the job history by a specific execution ID:

   Expected output:
4. Retrieve the history in YAML format:

   Get the job history in YAML format:

   Expected output: