Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
The bacalhau agent version
command is used to obtain the version of the bacalhau agent.
Using this command, users can quickly retrieve the version of the agent, allowing them to confirm the specific release of the software they are using.
-h
, --help
:
Show help for the version
command.
--output format
:
Defines the output format of the command's results. Accepted formats include "json" and "yaml".
--pretty
:
Used for pretty printing the output, enhancing readability. This flag is applicable only for the "json" and "yaml" output formats.
--api-host string
:
Designates the host for client-server communication via REST. If the BACALHAU_API_HOST
environment variable is present, this flag will be disregarded.
Default: "bootstrap.production.bacalhau.org"
--api-port int
:
Defines the port for client-server communication through REST. This flag becomes irrelevant if the BACALHAU_API_PORT
environment variable is specified.
Default: 1234
--log-mode logging-mode
:
Specifies the desired logging format.
Options: 'default','station','json','combined','event'
Default: 'default'
--repo string
:
Indicates the path to the bacalhau repository.
Default: "
$HOME/.bacalhau"`
Retrieve the agent version:
Execute the command to get the agent version:
Expected output:
Retrieve the agent version in JSON format:
Expected output:
Retrieve the agent version in Pretty-printed JSON format:
Expected output:
The following commands refer to experimental bacalhau cli version v1.1.0
. Experimental means that the commands are not stable and may change in future versions. More stable commands, with limited functionality, are documented here.
For installing or upgrading a client, follow the instructions in the installation page. Run bacalhau version
in a terminal to check what version you have.
The bacalhau agent alive
command provides information about the agent's liveness and health. This is essential for monitoring and ensuring that the agent is active and functioning correctly.
-h
, --help
:
Description: Displays help information for the alive
sub-command.
--output format
:
Description: Determines the format in which the output is displayed. Available formats include JSON and YAML.
Options: json
, yaml
Default: yaml
--pretty
:
Description: Formats the output for enhanced readability. This flag is relevant only when using JSON or YAML output formats.
--api-host string
:
Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Specifies the port for REST communication. If the BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Sets the desired log format. Options are: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: ``$HOME/.bacalhau`
Basic Usage:
Command:
Output:
Output in JSON format:
Command:
Output:
The bacalhau agent
command is a parent command that offers sub-commands to query information about the Bacalhau agent. This can be useful for debugging, monitoring, or managing the agent's behavior and health.
Description: Retrieves the agent's liveness and health information. This can be helpful to determine if the agent is running and healthy.
Usage:
node:
Description: Gathers the agent's node-related information. This might include details about the machine or environment where the agent is running, available resources, supported engines, etc.
Usage:
Description: Retrieves the Bacalhau version of the agent. This can be beneficial for ensuring compatibility or checking for updates.
Usage:
For more detailed information on any of the sub-commands, you can use the command:
-h
, --help
:
Description: Displays help information for the agent
command.
--api-host string
:
Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Specifies the port for REST communication. If the BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Sets the desired log format. Options are: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: ``$HOME/.bacalhau`
The bacalhau agent node
command retrieves information about the agent's node, providing insights into the agent's environment and aiding in debugging.
-h
, --help
:
Displays help information for the node
sub-command.
--output format
:
Defines the output format (either JSON or YAML).
Options: json
, yaml
Default: yaml
--pretty
:
Beautifies the output when using JSON or YAML formats.
--api-host string
:
The host for REST communication. Overrides the BACALHAU_API_HOST
environment variable.
Default: bootstrap.production.bacalhau.org
--api-port int
:
The port for REST communication. Overridden if BACALHAU_API_PORT
environment variable is set.
Default: 1234
--log-mode logging-mode
:
Specifies the log format. Choices are: default
, station
, json
, combined
, event
.
Default: default
--repo string
:
Path to the bacalhau repository.
Default: ``$HOME/.bacalhau`
Retrieve Node Information in Default Format (YAML)
Retrieve Node Information in JSON Format
Retrieve Node Information in Pretty-printed JSON Format
The bacalhau config list
command lists the configuration keys and values of the bacalhau node. This command is useful for understanding how configuration keys map to their respective values, aiding in the use of the bacalhau config set
command.
Note: Configuration values displayed by this command represent the settings that will be applied when the bacalhau node is next restarted. It is important to note that these values may not reflect the current operational configuration of an active bacalhau node. The displayed configuration is relevant and accurate for a node that is either not currently running or that has been restarted after the execution of this command.
-h
, --help
:
Description: Displays help information for the list
sub-command.
--hide-header
:
Description: Do not print the column headers when displaying the results.
Default: false
--no-style
:
Description: Removes all styling from the table output, displaying raw data.
Default: false
--output format
:
Description: Determines the format in which the output is displayed. Available formats include Table, JSON, and YAML.
Options: json
, yaml
, table
Default: table
--pretty
:
Description: Formats the output for enhanced readability. This flag is relevant only when using JSON or YAML output formats.
Default: true
--wide
:
Description: Prints full values in the table results without truncating any information.
Default: false
Basic Usage:
Command:
Output:
Output in JSON format:
Command:
Output:
The bacalhau job describe
command provides a detailed description of a specific job in YAML format. This description can be particularly useful when wanting to understand the attributes and current status of a specific job. To list all available jobs, the bacalhau job list
command can be used.
-h
, --help
:
Description: Display help for the describe
command.
--output format
:
Description: Specifies the desired output format for the command. Supported values are json
and yaml
.
Default: yaml
--pretty
:
Description: Pretty prints the output. This option is applicable only to json
and yaml
output formats.
--api-host string
:
Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST
environment variable is set, this flag will be ignored.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Specifies the desired log format. Supported values include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: $HOME/.bacalhau
Describe a Job with Full ID:
Describe a Job with Shortened ID:
Describe a Job with JSON Output:
The bacalhau config
command is a parent command that offers sub-commands to modify and query information about the Bacalhau config. This can be useful for debugging, monitoring, or managing the nodes configuration.
list:
Description: Lists the configuration keys and values of the bacalhau node. This command is useful for understanding how configuration keys map to their respective values, aiding in the use of the bacalhau config set
command.
Usage:
set:
Description: Sets a value in the bacalhau node's configuration file. This command is used to modify the configuration file that the bacalhau node will reference for its settings.
Usage:
Description: Prints the default configuration of a bacalhau node to the standard output (stdout). This command is beneficial for viewing the baseline settings a bacalhau node will use before any user-defined configuration changes are applied.
Usage:
Description: Automatically sets compute resource values in the bacalhau node's configuration file based on the hardware resources of the user's machine. This command simplifies the process of allocating resources for jobs by dynamically adjusting the settings to match the machine's capabilities.
Usage:
The bacalhau job executions
command retrieves a list of executions for a specific job based on its ID. This can be essential when tracking the various runs and their respective states for a particular job.
-h
, --help
:
Description: Display help for the executions
command.
--hide-header
:
Description: Do not print the column headers when displaying the results.
--limit uint32
:
Description: Restricts the number of results returned.
Default: 20
--next-token string
:
Description: Uses the specified token for pagination. Useful for fetching the next set of results.
--no-style
:
Description: Removes all styling from the table output, displaying raw data.
--order-by string
:
Description: Orders results based on a specific field. Valid fields are: modify_time
, create_time
, id
, and state
.
--order-reversed
:
Description: Reverses the order of the results. Useful in conjunction with --order-by
.
--output format
:
Description: Specifies the desired output format for the command. Supported values are table
, csv
, json
, and yaml
.
Default: table
--pretty
:
Description: Pretty prints the output. This option is applicable only to json
and yaml
output formats.
--wide
:
Description: Prints full values in the table results without truncating any information.
--api-host string
:
Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST
environment variable is set, this flag will be ignored.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Specifies the desired log format. Supported values include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: $HOME/.bacalhau
List executions for a specific Job:
Expected output:
Order executions by state for a specific job:
Execute the command:
Expected output:
List executions with YAML output:
Expected output:
The bacalhau config auto-resources
command automatically configures compute resource values in the bacalhau node's configuration file based on the hardware resources of the user's machine. This command streamlines the process of resource allocation for jobs, dynamically adjusting settings to align with the capabilities of the machine. It is designed to simplify the task of resource management, ensuring that the node operates efficiently and effectively within the hardware's limits.
Note: The bacalhau config auto-resources
command intelligently adjusts resource allocation settings based on the specific hardware configuration of your machine, promoting optimal utilization for bacalhau jobs. Due to the dynamic nature of this command, the specific values set in the configuration will vary depending on the available hardware resources of the machine in use. This functionality is particularly beneficial for users who seek to optimize their node's performance without the need for manual calculations of resource limits. It is important for users to understand that these settings will directly impact the number and types of jobs their node can manage at any given time, based on the machine's resource capacity.
--default-job-percentage int
:
Description: Sets the default percentage of resources allocated for each job when specific limits are not defined. Acceptable values range from 1 to 100 (values over 100 are rejected).
Default: 75
--job-percentage int
:
Description: Determines the percentage of resources that can be utilized at one time for a single job. Accept values from 1 to 100 (values over 100 are rejected).
Default: 75
--queue-job-percentage int
:
Description: Specifies the total percentage of resources that the system can allocate for all jobs queued at one time. Accept values from 1 to 100 (values over 100 are accepted).
Default: 150
--total-percentage int
:
Description: Indicates the total percentage of resources that the system can utilize at one time across all jobs. Accept values from 1 to 100 (values over 100 are rejected).
Default: 75
(Ran on an Apple M1 Max with 10 Cores and 64GB RAM)
Basic Usage:
Command:
Config File:
Queue 500% system resources:
Command:
Config File:
With 25% of system resources:
Command:
Config File:
The bacalhau config set
command sets a value in the bacalhau node's configuration file. This command is used to modify the configuration file that the bacalhau node will reference for its settings. Key names in the configuration are case-insensitive. Additionally, the command validates the value being set based on the type of the configuration key, ensuring that only appropriate and valid configurations are applied.
Note: Changes made using this command will be applied to the configuration file, but they do not immediately affect the running configuration of an active bacalhau node. The modifications will take effect only after the node is restarted.
-h
, --help
:
Description: Displays help information for the set
sub-command.
Example of invalid logging mode value
Example of invalid time duration value
The bacalhau exec
command allows for the specification of jobs to be executed from the command line, without the need for a job specification file (see job run).
-h
, --help
:
Description: Displays help information for the exec
sub-command.
--code
:
Includes the specified code in the job. This can be a single file, or a directory containing many files. There is a limit of 10Mb on the size of the uploaded code.
-f
, --follow
:
Description: If provided, the command will continuously display the output from the job as it runs.
--wait
Description: Wait for the job to finish. Use --wait=false to return as soon as the job is submitted.
--wait-timeout-secs
Description: When using --wait, how many seconds to wait for the job to complete before giving up.
--node-details
Description: Print out details of all nodes (overridden by --id-only).
--id-only
:
Description: On successful job submission, only the Job ID will be printed.
-p
, --publisher
Description: Where to publish the result of the job.
Examples:
Publish to IPFS
-p ipfs
Publish to S3
-p s3://bucket/key
-i
, --input
Description: Mount URIs as inputs to the job. Can be specified multiple times. Format: src=URI,dst=PATH[,opt=key=value]
Examples:
Mount IPFS CID to /inputs directory
-i ipfs://QmeZRGhe4PmjctYVSVHuEiA9oSXnqmYa4kQubSHgWbjv72
Mount S3 object to a specific path
-i s3://bucket/key,dst=/my/input/path
Mount S3 object with specific endpoint and region
-i src=s3://bucket/key,dst=/my/input/path,opt=endpoint=https://s3.example.com,opt=region=us-east-1
-o
, --output
Description: name:path of the output data volumes. 'outputs:/outputs' is always added unless '/outputs' is mapped to a different name.
-e
, --env
Description: The environment variables to supply to the job (e.g. --env FOO=bar --env BAR=baz)
--timeout
Description: Job execution timeout in seconds (e.g. 300 for 5 minutes)
-l
, --labels
Description: List of labels for the job. Enter multiple in the format '-l a -l 2'. All characters not matching /a-zA-Z0-9_:|-/ and all emojis will be stripped.
-s
, --selector
Description: Selector (label query) to filter nodes on which this job can be executed, supports '=', '==', and '!='.(e.g. -s key1=value1,key2=value2). Matching objects must satisfy all of the specified label constraints.
--api-host string
:
Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Specifies the port for REST communication. If the BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Sets the desired log format. Options are: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: ``$HOME/.bacalhau`
Basic Usage:
Command:
Output:
Single file Python:
Command:
where app.py is
Output:
As red text
Basic Usage:
Command:
Output:
The bacalhau job logs
command allows users to retrieve logs from a job that has been previously submitted. This command is useful for tracking and debugging the progress and state of a running or completed job.
-f
, --follow
:
Description: This flag allows the user to follow the logs in real-time after fetching the current logs. It provides a continuous stream of log updates, similar to tail -f
in Unix-like systems.
-h
, --help
:
Description: Display help information for the logs
command.
--api-host string
:
Description: Specifies the host for the client and server to communicate through REST. This flag is disregarded if the BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Sets the port for RESTful communication between the client and server. If the BACALHAU_API_PORT
environment variable is available, this flag is ignored.
Default: 1234
--log-mode logging-mode
:
Description: Determines the desired log format. Available options include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Specifies the path to the bacalhau repository.
Default: $HOME/.bacalhau
Display Logs for a Previously Submitted Job with Full ID:
Command:
Expected Output:
Follow Logs in Real-Time:
Command:
Expected Output:
Display Logs Using a Shortened ID:
Command:
Expected Output:
The bacalhau node
command provides a set of sub-commands to query and manage node-related information within Bacalhau. With these tools, users can access specific details about nodes, list all network nodes, and more.
Description: Approves a single node to join the cluster.
Usage:
Description: Deletes a node from the cluster using its ID.
Usage:
Description: Retrieves detailed information of a node using its ID.
Usage:
list:
Description: Lists the details of all nodes present in the network.
Usage:
Description: Reject a specific node's request to join the cluster.
Usage:
For comprehensive details on any of the sub-commands, run:
-h
, --help
:
Description: Shows the help information for the node
command.
--api-host string
:
Description: Specifies the host for RESTful communication between the client and server. The flag will be ignored if the BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Designates the port for RESTful communication. The flag will be bypassed if the BACALHAU_API_PORT
environment variable is active.
Default: 1234
--log-mode logging-mode
:
Description: Chooses the preferred log format. Available choices are: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Specifies the path to the bacalhau repository.
Default: /Users/walid/.bacalhau
The bacalhau job list
command provides a listing of all submitted jobs. This command offers an overview of all tasks and processes registered in the system, allowing users to monitor and manage their jobs effectively.
-h
, --help
:
Description: Display help for the list
command.
--hide-header
:
Description: Opts out of printing the column headers in the results.
--labels string
:
Description: Filters jobs by labels. It's designed to function similar to Kubernetes label selectors.
Default: bacalhau_canary != true
--limit uint32
:
Description: Limits the number of results returned.
Default: 10
--next-token string
:
Description: Uses the provided token for pagination.
--no-style
:
Description: Strips all styling from the table output.
--order-by string
:
Description: Organizes results based on a chosen field. Valid fields are id
and created_at
.
--order-reversed
:
Description: Reverses the order of the displayed results.
--output format
:
Description: Dictates the desired output format for the command. Options are table
, csv
, json
, and yaml
.
Default: table
--pretty
:
Description: Offers a more visually pleasing output for json
and yaml
formats.
--wide
:
Description: Presents full values in the table results, preventing truncation.
--api-host string
:
Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST
environment variable, if set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT
environment variable takes precedence if set.
Default: 1234
--log-mode logging-mode
:
Description: Designates the desired log format. Options include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Points to the bacalhau repository location.
Default: $HOME/.bacalhau
List all jobs:
Execute the command to list all the jobs:
Expected output:
Limit the list to the last two jobs:
Limit the list to display only the last two jobs:
Expected output:
Order the list by creation date in descending order:
Order the jobs by their creation date in a descending manner:
Expected output:
Filter the jobs by specific labels:
Display jobs that have specific labels:
Expected output:
Display the list in JSON format with pretty printing:
Get a limited list of jobs in a formatted JSON output:
Expected output:
The bacalhau job run
command facilitates the initiation of a job from a file or directly from the standard input (stdin). The command supports both JSON and YAML data formats. This command is particularly useful for quickly executing a job without the need for manual configurations.
--dry-run
:
Description: With this flag, the job will not be submitted. Instead, it will display what would have been submitted, providing a way to preview before actual submission.
-f
, --follow
:
Description: If provided, the command will continuously display the output from the job as it runs.
--id-only
:
Description: On successful job submission, only the Job ID will be printed.
--no-template
:
Disable the templating feature. When this flag is set, the job spec will be used as-is, without any placeholder replacements
--node-details
:
Description: Displays details of all nodes. Note that this flag is overridden if --id-only
is provided.
--show-warnings
:
Description: Shows any warnings that occur during the job submission.
-E
, --template-envs
:
Specify a regular expression pattern for selecting environment variables to be included as template variables in the job spec. e.g. --template-envs ".*"
will include all environment variables.
-V
, --template-vars
:
Replace a placeholder in the job spec with a value. e.g. --template-vars foo=bar
--wait
:
Description: Waits for the job to finish execution. To set this to false, use --wait=false
Default: true
--wait-timeout-secs int
:
Description: If --wait
is provided, this flag sets the maximum time (in seconds) the command will wait for the job to finish before it terminates.
Default: 600
seconds
-h
, --help
:
Description: Displays help information for the run
command.
--api-host string
:
Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Determines the port for REST communication. If BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Selects the desired log format. Options include: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: $HOME/.bacalhau
Sample Job (job.yaml
)
A sample job used in the following examples is provided below:
This configuration describes a batch job that runs a Docker task. It utilizes the ubuntu:latest
image and executes the command echo Hello Bacalhau!
.
Running a Job using a YAML Configuration:
To run a job with a configuration provided in a job.yaml
file:
Command:
Expected Output:
Running a Job and Following its Logs:
Command:
Expected Output:
Running a Job Without Waiting:
Command:
Expected Output:
Fetching Only the Job ID Upon Submission:
Command:
Expected Output:
Fetching Only the Job ID and Wait for Completion:
Command:
Expected Output:
Running a Job with Node Details:
Command:
Expected Output:
Rerunning a previously submitting job:
Command:
Expected Output:
bacalhau job run
providing users with the ability to dynamically inject variables into their job specifications. This feature is particularly useful when running multiple jobs with varying parameters such as S3 buckets, prefixes, and time ranges without the need to edit each job specification file manually. You can find more information about templating here.
The bacalhau job stop
command allows users to terminate a previously submitted job. This is useful in scenarios where there's a need to halt a running job, perhaps due to misconfiguration or changed priorities.
--quiet
:
Description: If provided, the command will not display any output, neither to the standard output (stdout) nor to the standard error (stderr).
-h
, --help
:
Description: Displays help information for the stop
command.
--api-host string
:
Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Determines the port for REST communication. If BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Selects the desired log format. Options include: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: $HOME/.bacalhau
Stop a Specific Job:
If you wish to halt the execution of a job, you can utilize the stop
command. Here's how you can achieve that:
Command:
Expected Output:
Silently Stop a Job:
If you prefer to terminate a job without seeing any verbose feedback or messages, the --quiet
option can be used.
Command:
Expected Output:
The bacalhau node delete
command offers administrators the ability to remove a node from the cluster using its name.
Using the delete
sub-command, administrators can remove a node from the list of available compute nodes in the cluster. This feature is necessary for the management of the infrastructure.
[id]
:
The unique identifier of the node you wish to describe.
-h
, --help
:
Displays the help documentation for the describe
command.
-m message
:
A message to be attached to the deletion action.
--api-host string
:
Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST
environment variable is set.
Default: "bootstrap.production.bacalhau.org"
--api-port int
:
Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT
environment variable is defined.
Default: 1234
--log-mode logging-mode
:
Determines the log format preference.
Options: 'default','station','json','combined','event'
Default: 'default'
--repo string
:
Points to the bacalhau repository's path.
Default: "
$HOME/.bacalhau"`
Delete the Node with ID nodeID123
:
Delete a Node with an audit message:
The bacalhau config default
command prints the default configuration of a bacalhau node to the standard output (stdout). This command is advantageous for users to view the baseline settings a bacalhau node will use in the absence of any user-defined configuration changes. It provides a clear view of the default operational parameters of the node, aiding users in understanding and customizing their configuration from a known baseline.
Note: The output of this command shows the initial default settings for a new bacalhau node and is useful for understanding the foundational settings for customization. To apply these default settings, you can redirect the output to your configuration file using bacalhau config default > ~/.bacalhau/config.yaml
, which overwrites your current configuration file with the default settings. However, if you wish to always use the latest default settings, especially if the defaults are updated over time, consider deleting your existing configuration file (e.g., ~/.bacalhau/config.yaml
). This approach ensures that your bacalhau node uses the most current defaults, circumventing potential discrepancies between the latest defaults and those captured in an older configuration file created with bacalhau config default
.
-h
, --help
:
Description: Displays help information for the list
sub-command.
--path
:
Description: Sets path dependent config fields
Default: $HOME/.bacalhau
The bacalhau node list
command is designed to provide users with a comprehensive list of network nodes along with details based on specified flags.
The list
sub-command under the bacalhau node
category enumerates information about nodes in the network. It supports various filtering, ordering, and output formatting options, allowing users to tailor the output to their needs.
-h
, --help
:
Show the help message for the list
command.
--hide-header
:
Do not display the column headers in the output.
--filter-approval
:
Only show nodes with the specified approval status. Valid values are: approved
, pending
, rejected
.
--filter-status
:
Only show nodes with the specified state. Valid values are: healthy
, unhealthy
, unknown
.
--labels string
:
Filter nodes based on labels. This follows the filtering format provided by Kubernetes, as shown in their documentation about labels.
--limit uint32
:
Restrict the number of results displayed.
--next-token string
:
Provide the next token for pagination.
--no-style
:
Output the table without any style.
--order-by string
:
Sort the results based on a specific field. Valid sorting fields are: id
, type
, available_cpu
, available_memory
, available_disk
, available_gpu
.
--order-reversed
:
Display the results in reverse order.
--output format
:
Choose the output format. Available options: table
, csv
, json
, yaml
.
Default: table
.
--pretty
:
Enhance the visual appeal of the output. This is applicable only to json
and yaml
formats.
--show strings
:
Determine the column groups to be displayed. Acceptable values are: labels
, version
, features
, capacity
.
Default: labels
, capacity
.
--wide
:
Display full values in the output table, without truncation.
--api-host string
:
Specify the host for client-server communication via REST. This gets ignored if the BACALHAU_API_HOST
environment variable is defined.
Default: "bootstrap.production.bacalhau.org"
.
--api-port int
:
Specify the port for RESTful communication between client and server. Gets overlooked if the BACALHAU_API_PORT
environment variable is set.
Default: 1234
.
--log-mode logging-mode
:
Choose the desired log format.
Options: 'default', 'station', 'json', 'combined', 'event'
.
Default: 'default'
.
--repo string
:
Point to the directory path of the bacalhau repository.
Default: "
$HOME/.bacalhau"`.
Retrieve the list of nodes:
Execute the command to get a list of all nodes:
Expected output:
Filter the list of nodes by labels:
Execute the command to get a list of nodes with specific labels:
Expected output:
Order the list of nodes by available memory:
Execute the command to get the list of nodes ordered by available memory:
Expected output:
Limit the number of nodes displayed and output in JSON format:
Execute the command to get a limited list of nodes in JSON format:
Expected output:
The bacalhau node approve
command offers administrators the ability to approve the cluster membership for a node using its name.
Using the approve
sub-command under the bacalhau node
umbrella, users can allow a node in the pending state to join the cluster and receive work. This feature is crucial for system administrators to manage the cluster.
[id]
:
The unique identifier of the node you wish to describe.
-h
, --help
:
Displays the help documentation for the describe
command.
-m message
:
A message to be attached to the approval action.
--api-host string
:
Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST
environment variable is set.
Default: "bootstrap.production.bacalhau.org"
--api-port int
:
Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT
environment variable is defined.
Default: 1234
--log-mode logging-mode
:
Determines the log format preference.
Options: 'default','station','json','combined','event'
Default: 'default'
--repo string
:
Points to the bacalhau repository's path.
Default: "
$HOME/.bacalhau"`
Approve a Node with ID nodeID123
:
Approve a Node with an audit message:
The bacalhau job
command provides a suite of sub-commands to submit, query, and manage jobs within Bacalhau. Users can deploy jobs, obtain job details, track execution logs, and more.
Description: Retrieves detailed information of a job using its ID.
Usage:
Description: Lists all executions associated with a job, identified by its ID.
Usage:
Description: Enumerates the historical events related to a job, identified by its ID.
Usage:
list:
Description: Provides an overview of all submitted jobs.
Usage:
logs:
Description: Fetches and streams the logs from a currently executing job.
Usage:
run:
Description: Submits a job for execution using either a JSON or YAML configuration file.
Usage:
stop:
Description: Halts a previously submitted job.
Usage:
For comprehensive details on any of the sub-commands, run:
-h
, --help
:
Description: Shows the help information for the job
command.
--api-host string
:
Description: Determines the host for RESTful communication between the client and server. This flag is overlooked if the BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Designates the port for RESTful communication. This flag is bypassed if the BACALHAU_API_PORT
environment variable is active.
Default: 1234
--log-mode logging-mode
:
Description: Chooses the preferred log format. Available choices are: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Specifies the path to the bacalhau repository.
Default: $HOME/.bacalhau
The bacalhau node reject
command offers administrators the ability to reject a compute node's request to join the cluster.
Using the reject
sub-command, administrators can reject a node in the pending state from joining the cluster and receiving work. This feature is crucial for system administrators to manage the cluster and will stop the node from taking part in the cluster until approved.
[id]
:
The unique identifier of the node you wish to describe.
-h
, --help
:
Displays the help documentation for the describe
command.
-m message
:
A message to be attached to the rejection action.
--api-host string
:
Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST
environment variable is set.
Default: "bootstrap.production.bacalhau.org"
--api-port int
:
Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT
environment variable is defined.
Default: 1234
--log-mode logging-mode
:
Determines the log format preference.
Options: 'default','station','json','combined','event'
Default: 'default'
--repo string
:
Points to the bacalhau repository's path.
Default: "
$HOME/.bacalhau"`
Reject a Node with ID nodeID123
:
Reject a Node with an audit message:
The bacalhau node describe
command offers users the ability to retrieve detailed information about a specific node using its unique identifier.
Using the describe
sub-command under the bacalhau node
umbrella, users can get comprehensive details of a node by providing its ID. This information is crucial for system administrators and network managers to understand the state, specifications, and other attributes of nodes in their infrastructure.
[id]
:
The unique identifier of the node you wish to describe.
-h
, --help
:
Displays the help documentation for the describe
command.
--output format
:
Defines the desired format for the command's output.
Options: "json"
or "yaml"
Default: "yaml"
--pretty
:
When this flag is used, the command will pretty print the output. This is applicable only for outputs in json
and yaml
formats.
--api-host string
:
Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST
environment variable is set.
Default: "bootstrap.production.bacalhau.org"
--api-port int
:
Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT
environment variable is defined.
Default: 1234
--log-mode logging-mode
:
Determines the log format preference.
Options: 'default','station','json','combined','event'
Default: 'default'
--repo string
:
Points to the bacalhau repository's path.
Default: "
$HOME/.bacalhau"`
Describing a Node with ID nodeID123
:
Describing a Node with Output in JSON Format:
Pretty Printing the Description of a Node:
The bacalhau job history
command lists the history events of a specific job based on its ID. This feature allows users to track changes, executions, and other significant milestones associated with a particular job.
--event-type string
:
Description: Specifies the type of history events to retrieve. Available options include all
, job
, and execution
.
Default: all
--execution-id string
:
Description: Filters results by a specific execution ID.
-h
, --help
:
Description: Display help for the history
command.
--hide-header
:
Description: Opts out of printing the column headers in the results.
--limit uint32
:
Description: Limits the number of results returned.
--next-token string
:
Description: Uses the provided token for pagination.
--no-style
:
Description: Strips all styling from the table output.
--node-id string
:
Description: Filters the results by a specific node ID.
--order-by string
:
Description: Organizes results based on a chosen field.
--order-reversed
:
Description: Reverses the order of the displayed results.
--output format
:
Description: Dictates the desired output format for the command. Options are table
, csv
, json
, and yaml
.
Default: table
--pretty
:
Description: Offers a more visually pleasing output for json
and yaml
formats.
--wide
:
Description: Presents full values in the table results, preventing truncation.
--api-host string
:
Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST
environment variable, if set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT
environment variable takes precedence if set.
Default: 1234
--log-mode logging-mode
:
Description: Designates the desired log format. Options include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Points to the bacalhau repository location.
Default: $HOME/.bacalhau
Retrieve the history of a specific job:
Execute the command to get the job history:
Expected output:
Filter the history by event type:
Filter the job history by the event type:
Expected output:
Filter the history by execution ID:
Filter the job history by a specific execution ID:
Expected output:
Retrieve the history in YAML format:
Get the job history in YAML format:
Expected output:
The following commands refer to bacalhau cli version v1.3.2
. For installing or upgrading a client, follow the instructions in the . Run bacalhau version
in a terminal to check what version you have.
Let’s run the bacalhau -- help
command in the terminal to find out information about available commands and flags:
Global flags
The bacalhau agent
command is a parent command that offers sub-commands to query information about the Bacalhau agent. This can be useful for debugging, monitoring, or managing the agent's behavior and health.
Usage:
Available Commands:
The bacalhau agent alive
command retrieves the agent's liveness and health information. This can be helpful to determine if the agent is running and healthy.
Let's have a look at the basic usage output:
Compare the output in JSON format:
The bacalhau agent node
command gathers the agent's node-related information. This might include details about the machine or environment where the agent is running, available resources, supported engines, etc.
To retrieve Node Information in Default Format (YAML), run:
To retrieve Node Information in JSON Format, run:
To retrieve Node Information in Pretty-printed JSON Format, run:
The bacalhau agent version
command is used to obtain the version of the bacalhau agent.
Let's have a look at the command execution in the terminal:
To retrieve the agent version in JSON format, run:
To retrieve the agent version in Pretty-printed JSON format, run:
The bacalhau cancel
command cancels a job that was previously submitted and stops it running if it has not yet completed.
Usage:
To cancel a previously submitted job, run:
To cancel a job using a short ID, run:
The bacalhau completion
command generates the autocompletion script for bacalhau for the specified shell.
Usage:
Available Commands:
The bacalhau completion bash
command generates the autocompletion script for bash.
This script depends on the 'bash-completion' package. If it is not installed already, you can install it via your OS's package manager.
The bacalhau completion fish
command generates the autocompletion script for the fish shell.
To load completions in your current shell session:
bacalhau completion fish | source
To load completions for every new session, execute once:
bacalhau completion fish > ~/.config/fish/completions/bacalhau.fish
You will need to start a new shell for this setup to take effect.
The bacalhau completion powershell
command generates the autocompletion script for powershell.
To load completions in your current shell session:
bacalhau completion powershell | Out-String | Invoke-Expression
To load completions for every new session, add the output of the above command to your powershell profile.
The bacalhau completion zsh
command generates the autocompletion script for the zsh shell.
If shell completion is not already enabled in your environment you will need to enable it. You can execute the following once:
echo "autoload -U compinit; compinit" >> ~/.zshrc
The bacalhau config
command is a parent command that offers sub-commands to modify and query information about the Bacalhau config. This can be useful for debugging, monitoring, or managing the nodes configuration.
Usage:
Available Commands:
The bacalhau config auto-resources
command automatically configures compute resource values in the bacalhau node's configuration file based on the hardware resources of the user's machine. This command streamlines the process of resource allocation for jobs, dynamically adjusting settings to align with the capabilities of the machine. It is designed to simplify the task of resource management, ensuring that the node operates efficiently and effectively within the hardware's limits.
The bacalhau config auto-resources
command intelligently adjusts resource allocation settings based on the specific hardware configuration of your machine, promoting optimal utilization for bacalhau jobs. Due to the dynamic nature of this command, the specific values set in the configuration will vary depending on the available hardware resources of the machine in use. This functionality is particularly beneficial for users who seek to optimize their node's performance without the need for manual calculations of resource limits. It is important for users to understand that these settings will directly impact the number and types of jobs their node can manage at any given time, based on the machine's resource capacity.
Ran on an Apple M1 Max with 10 Cores and 64GB RAM
Basic Usage:
Config File:
Queue 500% system resources:
Config File:
The bacalhau config default
command prints the default configuration of a bacalhau node to the standard output (stdout). This command is advantageous for users to view the baseline settings a bacalhau node will use in the absence of any user-defined configuration changes. It provides a clear view of the default operational parameters of the node, aiding users in understanding and customizing their configuration from a known baseline.
The output of this command shows the initial default settings for a new bacalhau node and is useful for understanding the foundational settings for customization. To apply these default settings, you can redirect the output to your configuration file using bacalhau config default > ~/.bacalhau/config.yaml
, which overwrites your current configuration file with the default settings. However, if you wish to always use the latest default settings, especially if the defaults are updated over time, consider deleting your existing configuration file (e.g., ~/.bacalhau/config.yaml
). This approach ensures that your bacalhau node uses the most current defaults, circumventing potential discrepancies between the latest defaults and those captured in an older configuration file created with bacalhau config default
.
This command redirects the default configuration output directly into the bacalhau configuration file at ~/.bacalhau/config.yaml
, effectively resetting it to default settings.
The bacalhau config list
command lists the configuration keys and values of the bacalhau node. This command is useful for understanding how configuration keys map to their respective values, aiding in the use of the bacalhau config set command.
Configuration values displayed by this command represent the settings that will be applied when the bacalhau node is next restarted. It is important to note that these values may not reflect the current operational configuration of an active bacalhau node. The displayed configuration is relevant and accurate for a node that is either not currently running or that has been restarted after the execution of this command.
Basic Usage:
The bacalhau config set
command sets a value in the bacalhau node's configuration file. This command is used to modify the configuration file that the bacalhau node will reference for its settings. Key names in the configuration are case-insensitive. Additionally, the command validates the value being set based on the type of the configuration key, ensuring that only appropriate and valid configurations are applied.
Changes made using this command will be applied to the configuration file, but they do not immediately affect the running configuration of an active bacalhau node. The modifications will take effect only after the node is restarted.
Configuring the Server API Port Value
Verifying that the parameter was successfully set
Configuring multiple values
Verifying that the parameters were successfully set
The bacalhau create
command is used to submit a job to the network in a declarative way by writing a jobspec instead of writing a command. JSON and YAML formats are accepted.
Usage:
To create a job using the data in job.yaml
, run:
To create a new job from an already executed job, run:
Let's have a look at a job example in YAML format:
This example shows how a YAML file can be structured to describe a job and its parameters, which can then be used in Bacalhau to perform executions.
There is no support for sharding, concurrency or minimum bidding for these jobs.
An example UCAN Invocation that runs the same job as the above example would look like:
An example UCAN Invocation that runs a WebAssembly job might look like:
Using a UCAN Invocation object allows you to customize the parameters of job execution in Bacalhau in a more flexible and detailed way.
The bacalhau describe
command provides a full description of a job in YAML format. Short form and long form of the job id are accepted.
Usage:
To describe a job with the full ID, run:
To describe a job with the shortened ID, run:
To describe a job and include all server and local events, run:
The bacalhau devstack
command is used to start a cluster of nodes and run a job on them.
Usage:
To create a devstack cluster with a single requester node and 3 compute nodes (default values), run:
To create a devstack cluster with 2 requester nodes and 10 compute nodes, run:
To create a devstack cluster with a single hybrid (requester and compute) node, run:
To run a devstack and create (or use) the config repo in a specific folder, run:
The bacalhau docker run
command runs a job using the Docker executor on the node.
Usage:
Let's run a Docker job, using the image dpokidov/imagemagick
, with a CID mounted at /input_images
and an output volume mounted at /outputs
in the container. All flags after the --
are passed directly into the container for execution:
This command allows you to start a job in a Docker container using the specified image, mount an external CID resource from IPFS inside the container to handle images, and execute a command inside the container to process files.
To check the job specification before submitting it to the bacalhau network, run:
The command does not run the job itself, but only displays information about how it would be run so you can make sure that all job parameters and commands are correctly specified before sending it to the Bacalhau network for execution
To save the job specification to a YAML file, run:
To specify an image tag (default is latest
- using a specific tag other than latest
is recommended for reproducibility), run:
To specify an image digest, run:
The command starts an Ubuntu image container using a specific version of the image identified by its SHA256 hash. This ensures the accuracy of the image source, independent of its tag and possible future changes, since the image digest remains constant for a particular version.
The bacalhau exec
command is used to execute a specific job type.
Usage:
To execute the app.py
script with Python, run:
To run a duckdb query against a CSV
file:
The bacalhau get
command is used to get the results of the job, including stdout and stderr.
Usage:
To get the results of a job, run:
To get the results of a job, using a short ID, run:
The bacalhau help
command provides help for any command in the application.
Usage:
The bacalhau id
command shows bacalhau node id info.
Usage:
The bacalhau job
command provides a suite of sub-commands to submit, query, and manage jobs within Bacalhau. Users can deploy jobs, obtain job details, track execution logs, and more.
Usage:
Available Commands:
The bacalhau job describe
command provides a detailed description of a specific job in YAML format. This description can be particularly useful when wanting to understand the attributes and current status of a specific job. To list all available jobs, the bacalhau job list
command can be used.
To describe a job with the full ID, run:
To describe a job with the shortened ID, run:
To describe a job with json output, run:
The bacalhau job executions
command retrieves a list of executions for a specific job based on its ID. This can be essential when tracking the various runs and their respective states for a particular job.
To get all executions for a specific job, run:
To get executions with YAML output, run:
The bacalhau job history
command lists the history events of a specific job based on its ID. This feature allows users to track changes, executions, and other significant milestones associated with a particular job.
To retrieve the history of a specific job, run:
To filter the history by event type, run:
To filter the history by execution ID, run:
The bacalhau job list
command provides a listing of all submitted jobs. This command offers an overview of all tasks and processes registered in the system, allowing users to monitor and manage their jobs effectively.
To list all jobs, run:
To limit the list to the last 2 jobs, run:
To order the list by creation date in descending order, run:
To filter the jobs by specific labels (region in (us-east-1, us-east-2)
and env = prod
), run:
To get the list in JSON format with pretty printing, run:
The bacalhau job logs
command allows users to retrieve logs from a job that has been previously submitted. This command is useful for tracking and debugging the progress and state of a running or completed job.
To display Logs for a Previously Submitted Job using Full Job ID, run:
To follow Logs in Real-Time, run:
The bacalhau job run
command facilitates the initiation of a job from a file or directly from the standard input (stdin). The command supports both JSON and YAML data formats. This command is particularly useful for quickly executing a job without the need for manual configurations.
A sample job used in the following examples is provided below:
This configuration describes a batch job that runs a Docker task. It utilizes the ubuntu:latest
image and executes the echo Hello Bacalhau!
command.
To run a job with a configuration provided in a job.yaml
file:
To run a Job and follow its Logs:
To run a Job Without Waiting:
To fetch Only the Job ID Upon Submission:
To fetch Only the Job ID and Wait for Completion:
To run a Job with Node Details:
To rerun a previously submitting job:
The bacalhau job stop
command allows users to terminate a previously submitted job. This is useful in scenarios where there's a need to halt a running job, perhaps due to misconfiguration or changed priorities.
To Stop a Specific Job, run:
To terminate a job without seeing any verbose feedback or messages, run:
The bacalhau list
command lists jobs on the network.
Usage:
To List jobs on the network, run:
To List jobs and output as json, run:
The bacalhau logs
command retrieves the log output (stdout, and stderr) from a job. If the job is still running it is possible to follow the logs after the previously generated logs are retrieved.
Usage:
To follow logs for a previously submitted job, run:
To retrieve the log output with a short ID, but don't follow any newly generated logs,run:
The bacalhau node
command provides a set of sub-commands to query and manage node-related information within Bacalhau. With these tools, users can access specific details about nodes, list all network nodes, and more.
Usage:
Available Commands:
The bacalhau node describe
command offers users the ability to retrieve detailed information about a specific node using its unique identifier. This information is crucial for system administrators and network managers to understand the state, specifications, and other attributes of nodes in their infrastructure.
To Describe a Node with the QmSD38wH
ID, run:
To describe a Node with Output in JSON Format, run:
The bacalhau node list
command is designed to provide users with a comprehensive list of network nodes along with details based on specified flags. It supports various filtering, ordering, and output formatting options, allowing users to tailor the output to their needs.
To retrieve the list of nodes, run:
To Filter the list of nodes by labels (Operating-System
and owner
), run:
To Order the list of nodes by available memory, run:
To get a limited list of nodes (3) in JSON format, run:
The bacalhau serve
command starts a bacalhau node.
Usage:
To Start a private bacalhau requester node, you can run either of these two commands:
To Start a private bacalhau hybrid node that acts as both compute and requester, you can run either of these two commands:
To Start a private bacalhau node with a persistent local IPFS node, run:
The command creates and starts a Bacalhau private node using the local IPFS node and specifies the path to save the IPFS data.
To Start a public bacalhau requester node, run:
To Start a public bacalhau node with the WebUI, run:
The bacalhau validate
command allows you to validate job files in JSON or YAML formats before sending them to the Bacalhau system. It is used to confirm that the structure and contents of the job description file conform to the expected format.
Usage:
To Validate the job.yaml
file, run:
The bacalhau version
command allows you to get the client and server version.
Usage:
The bacalhau wasm
command Runs and prepares WASM jobs on the network
Usage:
Available Commands:
The bacalhau wasm run
command Runs a job that was compiled to WASM.
To Run the <localfile.wasm>
module in bacalhau:
To Fetch the wasm module from <cid>
and execute it, run:
The bacalhau wasm validate
command Checks that a WASM program is runnable on the network.
Examples
Examples
Examples
Examples
Examples
Examples
Examples
Examples
You can also specify a job to run using a object in JSON format. For the fields supported by Bacalhau, see the .
Examples
Refers to example models at balcalhau repository under
Examples
Examples
Examples
Examples
Examples
Examples
Examples
Examples
Examples
Examples
Examples
Examples
Example
Examples
Examples
Examples
Examples
Example
Examples