Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
The bacalhau job
command provides a suite of sub-commands to submit, query, and manage jobs within Bacalhau. Users can deploy jobs, obtain job details, track execution logs, and more.
Description: Retrieves detailed information of a job using its ID.
Usage:
Description: Lists all executions associated with a job, identified by its ID.
Usage:
Description: Enumerates the historical events related to a job, identified by its ID.
Usage:
list:
Description: Provides an overview of all submitted jobs.
Usage:
logs:
Description: Fetches and streams the logs from a currently executing job.
Usage:
run:
Description: Submits a job for execution using either a JSON or YAML configuration file.
Usage:
stop:
Description: Halts a previously submitted job.
Usage:
For comprehensive details on any of the sub-commands, run:
-h
, --help
:
Description: Shows the help information for the job
command.
--api-host string
:
Description: Determines the host for RESTful communication between the client and server. This flag is overlooked if the BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Designates the port for RESTful communication. This flag is bypassed if the BACALHAU_API_PORT
environment variable is active.
Default: 1234
--log-mode logging-mode
:
Description: Chooses the preferred log format. Available choices are: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Specifies the path to the bacalhau repository.
Default: $HOME/.bacalhau
The bacalhau job list
command provides a listing of all submitted jobs. This command offers an overview of all tasks and processes registered in the system, allowing users to monitor and manage their jobs effectively.
-h
, --help
:
Description: Display help for the list
command.
--hide-header
:
Description: Opts out of printing the column headers in the results.
--labels string
:
Description: Filters jobs by labels. It's designed to function similar to Kubernetes label selectors.
Default: bacalhau_canary != true
--limit uint32
:
Description: Limits the number of results returned.
Default: 10
--next-token string
:
Description: Uses the provided token for pagination.
--no-style
:
Description: Strips all styling from the table output.
--order-by string
:
Description: Organizes results based on a chosen field. Valid fields are id
and created_at
.
--order-reversed
:
Description: Reverses the order of the displayed results.
--output format
:
Description: Dictates the desired output format for the command. Options are table
, csv
, json
, and yaml
.
Default: table
--pretty
:
Description: Offers a more visually pleasing output for json
and yaml
formats.
--wide
:
Description: Presents full values in the table results, preventing truncation.
--api-host string
:
Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST
environment variable, if set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT
environment variable takes precedence if set.
Default: 1234
--log-mode logging-mode
:
Description: Designates the desired log format. Options include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Points to the bacalhau repository location.
Default: $HOME/.bacalhau
List all jobs:
Execute the command to list all the jobs:
Expected output:
Limit the list to the last two jobs:
Limit the list to display only the last two jobs:
Expected output:
Order the list by creation date in descending order:
Order the jobs by their creation date in a descending manner:
Expected output:
Filter the jobs by specific labels:
Display jobs that have specific labels:
Expected output:
Display the list in JSON format with pretty printing:
Get a limited list of jobs in a formatted JSON output:
Expected output:
The bacalhau job history
command lists the history events of a specific job based on its ID. This feature allows users to track changes, executions, and other significant milestones associated with a particular job.
--event-type string
:
Description: Specifies the type of history events to retrieve. Available options include all
, job
, and execution
.
Default: all
--execution-id string
:
Description: Filters results by a specific execution ID.
-h
, --help
:
Description: Display help for the history
command.
--hide-header
:
Description: Opts out of printing the column headers in the results.
--limit uint32
:
Description: Limits the number of results returned.
--next-token string
:
Description: Uses the provided token for pagination.
--no-style
:
Description: Strips all styling from the table output.
--node-id string
:
Description: Filters the results by a specific node ID.
--order-by string
:
Description: Organizes results based on a chosen field.
--order-reversed
:
Description: Reverses the order of the displayed results.
--output format
:
Description: Dictates the desired output format for the command. Options are table
, csv
, json
, and yaml
.
Default: table
--pretty
:
Description: Offers a more visually pleasing output for json
and yaml
formats.
--wide
:
Description: Presents full values in the table results, preventing truncation.
--api-host string
:
Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST
environment variable, if set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT
environment variable takes precedence if set.
Default: 1234
--log-mode logging-mode
:
Description: Designates the desired log format. Options include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Points to the bacalhau repository location.
Default: $HOME/.bacalhau
Retrieve the history of a specific job:
Execute the command to get the job history:
Expected output:
Filter the history by event type:
Filter the job history by the event type:
Expected output:
Filter the history by execution ID:
Filter the job history by a specific execution ID:
Expected output:
Retrieve the history in YAML format:
Get the job history in YAML format:
Expected output:
The bacalhau job executions
command retrieves a list of executions for a specific job based on its ID. This can be essential when tracking the various runs and their respective states for a particular job.
-h
, --help
:
Description: Display help for the executions
command.
--hide-header
:
Description: Do not print the column headers when displaying the results.
--limit uint32
:
Description: Restricts the number of results returned.
Default: 20
--next-token string
:
Description: Uses the specified token for pagination. Useful for fetching the next set of results.
--no-style
:
Description: Removes all styling from the table output, displaying raw data.
--order-by string
:
Description: Orders results based on a specific field. Valid fields are: modify_time
, create_time
, id
, and state
.
--order-reversed
:
Description: Reverses the order of the results. Useful in conjunction with --order-by
.
--output format
:
Description: Specifies the desired output format for the command. Supported values are table
, csv
, json
, and yaml
.
Default: table
--pretty
:
Description: Pretty prints the output. This option is applicable only to json
and yaml
output formats.
--wide
:
Description: Prints full values in the table results without truncating any information.
--api-host string
:
Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST
environment variable is set, this flag will be ignored.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Specifies the desired log format. Supported values include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: $HOME/.bacalhau
List executions for a specific Job:
Expected output:
Order executions by state for a specific job:
Execute the command:
Expected output:
List executions with YAML output:
Expected output:
The bacalhau job run
command facilitates the initiation of a job from a file or directly from the standard input (stdin). The command supports both JSON and YAML data formats. This command is particularly useful for quickly executing a job without the need for manual configurations.
--dry-run
:
Description: With this flag, the job will not be submitted. Instead, it will display what would have been submitted, providing a way to preview before actual submission.
-f
, --follow
:
Description: If provided, the command will continuously display the output from the job as it runs.
--id-only
:
Description: On successful job submission, only the Job ID will be printed.
--no-template
:
Disable the templating feature. When this flag is set, the job spec will be used as-is, without any placeholder replacements
--node-details
:
Description: Displays details of all nodes. Note that this flag is overridden if --id-only
is provided.
--show-warnings
:
Description: Shows any warnings that occur during the job submission.
-E
, --template-envs
:
Specify a regular expression pattern for selecting environment variables to be included as template variables in the job spec. e.g. --template-envs ".*"
will include all environment variables.
-V
, --template-vars
:
Replace a placeholder in the job spec with a value. e.g. --template-vars foo=bar
--wait
:
Description: Waits for the job to finish execution. To set this to false, use --wait=false
Default: true
--wait-timeout-secs int
:
Description: If --wait
is provided, this flag sets the maximum time (in seconds) the command will wait for the job to finish before it terminates.
Default: 600
seconds
-h
, --help
:
Description: Displays help information for the run
command.
--api-host string
:
Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Determines the port for REST communication. If BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Selects the desired log format. Options include: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: $HOME/.bacalhau
Sample Job (job.yaml
)
A sample job used in the following examples is provided below:
This configuration describes a batch job that runs a Docker task. It utilizes the ubuntu:latest
image and executes the command echo Hello Bacalhau!
.
Running a Job using a YAML Configuration:
To run a job with a configuration provided in a job.yaml
file:
Command:
Expected Output:
Running a Job and Following its Logs:
Command:
Expected Output:
Running a Job Without Waiting:
Command:
Expected Output:
Fetching Only the Job ID Upon Submission:
Command:
Expected Output:
Fetching Only the Job ID and Wait for Completion:
Command:
Expected Output:
Running a Job with Node Details:
Command:
Expected Output:
Rerunning a previously submitting job:
Command:
Expected Output:
bacalhau job run
providing users with the ability to dynamically inject variables into their job specifications. This feature is particularly useful when running multiple jobs with varying parameters such as S3 buckets, prefixes, and time ranges without the need to edit each job specification file manually. You can find more information about templating here.
The bacalhau job describe
command provides a detailed description of a specific job in YAML format. This description can be particularly useful when wanting to understand the attributes and current status of a specific job. To list all available jobs, the bacalhau job list
command can be used.
-h
, --help
:
Description: Display help for the describe
command.
--output format
:
Description: Specifies the desired output format for the command. Supported values are json
and yaml
.
Default: yaml
--pretty
:
Description: Pretty prints the output. This option is applicable only to json
and yaml
output formats.
--api-host string
:
Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST
environment variable is set, this flag will be ignored.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Specifies the desired log format. Supported values include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: $HOME/.bacalhau
Describe a Job with Full ID:
Describe a Job with Shortened ID:
Describe a Job with JSON Output:
The bacalhau exec
command allows for the specification of jobs to be executed from the command line without the need for a job specification file (see job run).
-h
, --help
:
Description: Displays help information for the exec
sub-command.
--code
:
Includes the specified code in the job. This can be a single file or a directory containing many files. There is a limit of 10Mb on the size of the uploaded code.
-f
, --follow
:
Description: If provided, the command will continuously display the output from the job as it runs.
--wait
Description: Wait for the job to finish. Use --wait=false to return as soon as the job is submitted.
--wait-timeout-secs
Description: When using --wait, how many seconds to wait for the job to complete before giving up.
--node-details
Description: Print out details of all nodes (overridden by --id-only).
--id-only
:
Description: On successful job submission, only the Job ID will be printed.
-p
, --publisher
Description: Where to publish the result of the job.
Examples:
Publish to IPFS
-p ipfs
Publish to S3
-p s3://bucket/key
-i
, --input
Description: Mount URIs as inputs to the job. Can be specified multiple times. Format: src=URI,dst=PATH[,opt=key=value]
Examples:
Mount IPFS CID to /inputs directory
-i ipfs://QmeZRGhe4PmjctYVSVHuEiA9oSXnqmYa4kQubSHgWbjv72
Mount S3 object to a specific path
-i s3://bucket/key,dst=/my/input/path
Mount S3 object with specific endpoint and region
-i src=s3://bucket/key,dst=/my/input/path,opt=endpoint=https://s3.example.com,opt=region=us-east-1
-o
, --output
Description: name:path of the output data volumes. 'outputs:/outputs' is always added unless '/outputs' is mapped to a different name.
-e
, --env
Description: The environment variables to supply to the job (e.g. --env FOO=bar --env BAR=baz)
--timeout
Description: Job execution timeout in seconds (e.g. 300 for 5 minutes)
-l
, --labels
Description: List of labels for the job. Enter multiple in the format '-l a -l 2'. All characters not matching /a-zA-Z0-9_:|-/ and all emojis will be stripped.
-s
, --selector
Description: Selector (label query) to filter nodes on which this job can be executed, supports '=', '==', and '!='.(e.g. -s key1=value1,key2=value2). Matching objects must satisfy all of the specified label constraints.
--api-host string
:
Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Specifies the port for REST communication. If the BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Sets the desired log format. Options are: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: ``$HOME/.bacalhau`
Basic Usage:
Command:
Output:
Single file Python:
Command:
where app.py is
Output:
As red text
Basic Usage:
Command:
Output:
The bacalhau job logs
command allows users to retrieve logs from a job that has been previously submitted. This command is useful for tracking and debugging the progress and state of a running or completed job.
-f
, --follow
:
Description: This flag allows the user to follow the logs in real-time after fetching the current logs. It provides a continuous stream of log updates, similar to tail -f
in Unix-like systems.
-h
, --help
:
Description: Display help information for the logs
command.
--api-host string
:
Description: Specifies the host for the client and server to communicate through REST. This flag is disregarded if the BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Sets the port for RESTful communication between the client and server. If the BACALHAU_API_PORT
environment variable is available, this flag is ignored.
Default: 1234
--log-mode logging-mode
:
Description: Determines the desired log format. Available options include default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Specifies the path to the bacalhau repository.
Default: $HOME/.bacalhau
Display Logs for a Previously Submitted Job with Full ID:
Command:
Expected Output:
Follow Logs in Real-Time:
Command:
Expected Output:
Display Logs Using a Shortened ID:
Command:
Expected Output:
The bacalhau job stop
command allows users to terminate a previously submitted job. This is useful in scenarios where there's a need to halt a running job, perhaps due to misconfiguration or changed priorities.
--quiet
:
Description: If provided, the command will not display any output, neither to the standard output (stdout) nor to the standard error (stderr).
-h
, --help
:
Description: Displays help information for the stop
command.
--api-host string
:
Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST
environment variable is set.
Default: bootstrap.production.bacalhau.org
--api-port int
:
Description: Determines the port for REST communication. If BACALHAU_API_PORT
environment variable is set, this flag will be ignored.
Default: 1234
--log-mode logging-mode
:
Description: Selects the desired log format. Options include: default
, station
, json
, combined
, and event
.
Default: default
--repo string
:
Description: Defines the path to the bacalhau repository.
Default: $HOME/.bacalhau
Stop a Specific Job:
If you wish to halt the execution of a job, you can utilize the stop
command. Here's how you can achieve that:
Command:
Expected Output:
Silently Stop a Job:
If you prefer to terminate a job without seeing any verbose feedback or messages, the --quiet
option can be used.
Command:
Expected Output: