Description

The bacalhau job command provides a suite of sub-commands to submit, query, and manage jobs within Bacalhau. Users can deploy jobs, obtain job details, track execution logs, and more.

Usage

bacalhau job [command]

Available Commands

1. describe:

   • Description: Retrieves detailed information of a job using its ID.

   • Usage:

     Copy

     bacalhau job describe

2. executions:

   • Description: Lists all executions associated with a job, identified by its ID.

   • Usage:

     Copy

     bacalhau job executions

3. history:

   • Description: Enumerates the historical events related to a job, identified by its ID.

   • Usage:

     Copy

     bacalhau job history

4. list:

   • Description: Provides an overview of all submitted jobs.

   • Usage:

     Copy

     bacalhau job list

5. logs:

   • Description: Fetches and streams the logs from a currently executing job.

   • Usage:

     Copy

     bacalhau job logs

6. run:

   • Description: Submits a job for execution using either a JSON or YAML configuration file.

   • Usage:

     Copy

     bacalhau job run

7. stop:

   • Description: Halts a previously submitted job.

   • Usage:

     Copy

     bacalhau job stop

For comprehensive details on any of the sub-commands, run:

bacalhau job [command] --help

Flags

• -h, --help:

  • Description: Shows the help information for the job command.

Global Flags

1. --api-host string:

   • Description: Determines the host for RESTful communication between the client and server. This flag is overlooked if the BACALHAU_API_HOST environment variable is set.

   • Default: bootstrap.production.bacalhau.org

2. --api-port int:

   • Description: Designates the port for RESTful communication. This flag is bypassed if the BACALHAU_API_PORT environment variable is active.

   • Default: 1234

3. --log-mode logging-mode:

   • Description: Chooses the preferred log format. Available choices are: default, station, json, combined, and event.

   • Default: default

4. --repo string:

   • Description: Specifies the path to the bacalhau repository.

   • Default: $HOME/.bacalhau

Description

The bacalhau job history command lists the history events of a specific job based on its ID. This feature allows users to track changes, executions, and other significant milestones associated with a particular job.

Usage

bacalhau job history [id] [flags]

Flags

• --event-type string:

  • Description: Specifies the type of history events to retrieve. Available options include all, job, and execution.

  • Default: all

• --execution-id string:

  • Description: Filters results by a specific execution ID.

• -h, --help:

  • Description: Display help for the history command.

• --hide-header:

  • Description: Opts out of printing the column headers in the results.

• --limit uint32:

  • Description: Limits the number of results returned.

• --next-token string:

  • Description: Uses the provided token for pagination.

• --no-style:

  • Description: Strips all styling from the table output.

• --node-id string:

  • Description: Filters the results by a specific node ID.

• --order-by string:

  • Description: Organizes results based on a chosen field.

• --order-reversed:

  • Description: Reverses the order of the displayed results.

• --output format:

  • Description: Dictates the desired output format for the command. Options are table, csv, json, and yaml.

  • Default: table

• --pretty:

  • Description: Offers a more visually pleasing output for json and yaml formats.

• --wide:

  • Description: Presents full values in the table results, preventing truncation.

Global Flags

• --api-host string:

  • Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST environment variable, if set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT environment variable takes precedence if set.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Designates the desired log format. Options include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Points to the bacalhau repository location.

  • Default: $HOME/.bacalhau

Examples

1. Retrieve the history of a specific job:

   Execute the command to get the job history:

   Copy

   bacalhau job history j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881

   Expected output:

   Copy

   TIME      LEVEL           EXEC. ID    ...     NEW STATE          COMMENT
   ... [The output rows like the ones you've shown] ...
   16:46:04  JobLevel                              2     Pending            Completed

2. Filter the history by event type:

   Filter the job history by the event type:

   Copy

   bacalhau job history j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --event-type job

   Expected output:

   Copy

   TIME      LEVEL     EXEC. ID  NODE ID  REV.  PREVIOUS STATE  NEW STATE  COMMENT
   16:46:03  JobLevel                     1     Pending         Pending    Job created
   16:46:04  JobLevel                     2     Pending         Completed

3. Filter the history by execution ID:

   Filter the job history by a specific execution ID:

   Copy

   bacalhau job history j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --execution-id e-99362435

   Expected output:

   Copy

   TIME      LEVEL           EXEC. ID    ...     NEW STATE          COMMENT
   ... [The output rows for the specific execution ID] ...
   16:46:04  ExecutionLevel  e-99362435  QmTSJgdN  6     BidAccepted        Completed

4. Retrieve the history in YAML format:

   Get the job history in YAML format:

   Copy

   bacalhau job history j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --output yaml

   Expected output:

   Copy

   ... [The YAML formatted output] ...

Description

The bacalhau job describe command provides a detailed description of a specific job in YAML format. This description can be particularly useful when wanting to understand the attributes and current status of a specific job. To list all available jobs, the bacalhau job list command can be used.

Usage

Flags

• -h, --help:

  • Description: Display help for the describe command.

• --output format:

  • Description: Specifies the desired output format for the command. Supported values are json and yaml.

  • Default: yaml

• --pretty:

  • Description: Pretty prints the output. This option is applicable only to json and yaml output formats.

Global Flags

• --api-host string:

  • Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST environment variable is set, this flag will be ignored.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Specifies the desired log format. Supported values include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

1. Describe a Job with Full ID:
2. Describe a Job with Shortened ID:
3. Describe a Job with JSON Output:

Description

The bacalhau job list command provides a listing of all submitted jobs. This command offers an overview of all tasks and processes registered in the system, allowing users to monitor and manage their jobs effectively.

Usage

Flags

• -h, --help:

  • Description: Display help for the list command.

• --hide-header:

  • Description: Opts out of printing the column headers in the results.

• --labels string:

  • Description: Filters jobs by labels. It's designed to function similar to Kubernetes label selectors.

  • Default: bacalhau_canary != true

• --limit uint32:

  • Description: Limits the number of results returned.

  • Default: 10

• --next-token string:

  • Description: Uses the provided token for pagination.

• --no-style:

  • Description: Strips all styling from the table output.

• --order-by string:

  • Description: Organizes results based on a chosen field. Valid fields are id and created_at.

• --order-reversed:

  • Description: Reverses the order of the displayed results.

• --output format:

  • Description: Dictates the desired output format for the command. Options are table, csv, json, and yaml.

  • Default: table

• --pretty:

  • Description: Offers a more visually pleasing output for json and yaml formats.

• --wide:

  • Description: Presents full values in the table results, preventing truncation.

Global Flags

• --api-host string:

  • Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST environment variable, if set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT environment variable takes precedence if set.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Designates the desired log format. Options include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Points to the bacalhau repository location.

  • Default: $HOME/.bacalhau

Examples

1. List all jobs:

   Execute the command to list all the jobs:

   Expected output:
2. Limit the list to the last two jobs:

   Limit the list to display only the last two jobs:

   Expected output:
3. Order the list by creation date in descending order:

   Order the jobs by their creation date in a descending manner:

   Expected output:
4. Filter the jobs by specific labels:

   Display jobs that have specific labels:

   Expected output:
5. Display the list in JSON format with pretty printing:

   Get a limited list of jobs in a formatted JSON output:

   Expected output:

Description

The bacalhau exec command allows for the specification of jobs to be executed from the command line without the need for a job specification file (see ).

Usage

Flags

1. -h, --help:

   • Description: Displays help information for the exec sub-command.

2. --code:

   • Includes the specified code in the job. This can be a single file or a directory containing many files. There is a limit of 10Mb on the size of the uploaded code.

3. -f, --follow:

   • Description: If provided, the command will continuously display the output from the job as it runs.

4. --wait

   • Description: Wait for the job to finish. Use --wait=false to return as soon as the job is submitted.

5. --wait-timeout-secs

   • Description: When using --wait, how many seconds to wait for the job to complete before giving up.

6. --node-details

   • Description: Print out details of all nodes (overridden by --id-only).

7. --id-only:

   • Description: On successful job submission, only the Job ID will be printed.

8. -p, --publisher

   • Description: Where to publish the result of the job.

     Examples:

     Publish to IPFS

     -p ipfs

     Publish to S3

     -p s3://bucket/key

9. -i, --input

   • Description: Mount URIs as inputs to the job. Can be specified multiple times. Format: src=URI,dst=PATH[,opt=key=value]

     Examples:

     Mount IPFS CID to /inputs directory

     -i ipfs://QmeZRGhe4PmjctYVSVHuEiA9oSXnqmYa4kQubSHgWbjv72

     Mount S3 object to a specific path

     -i s3://bucket/key,dst=/my/input/path

     Mount S3 object with specific endpoint and region

     -i src=s3://bucket/key,dst=/my/input/path,opt=endpoint=https://s3.example.com,opt=region=us-east-1

10. -o, --output

    • Description: name:path of the output data volumes. 'outputs:/outputs' is always added unless '/outputs' is mapped to a different name.

11. -e, --env

    • Description: The environment variables to supply to the job (e.g. --env FOO=bar --env BAR=baz)

12. --timeout

    • Description: Job execution timeout in seconds (e.g. 300 for 5 minutes)

13. -l, --labels

    • Description: List of labels for the job. Enter multiple in the format '-l a -l 2'. All characters not matching /a-zA-Z0-9_:|-/ and all emojis will be stripped.

14. -s, --selector

    • Description: Selector (label query) to filter nodes on which this job can be executed, supports '=', '==', and '!='.(e.g. -s key1=value1,key2=value2). Matching objects must satisfy all of the specified label constraints.

Global Flags

1. --api-host string:

   • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST environment variable is set.

   • Default: bootstrap.production.bacalhau.org

2. --api-port int:

   • Description: Specifies the port for REST communication. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

   • Default: 1234

3. --log-mode logging-mode:

   • Description: Sets the desired log format. Options are: default, station, json, combined, and event.

   • Default: default

4. --repo string:

   • Description: Defines the path to the bacalhau repository.

   • Default: ``$HOME/.bacalhau`

Examples

Running python tasks

1. Basic Usage:

   Command:

   Output:
2. Single file Python:

   Command:

   where app.py is

   Output:

   As red text

Running duckdb queries

1. Basic Usage:

   Command:

   Output:

Description

The bacalhau job logs command allows users to retrieve logs from a job that has been previously submitted. This command is useful for tracking and debugging the progress and state of a running or completed job.

Usage

Flags

1. -f, --follow:

   • Description: This flag allows the user to follow the logs in real-time after fetching the current logs. It provides a continuous stream of log updates, similar to tail -f in Unix-like systems.

2. -h, --help:

   • Description: Display help information for the logs command.

Global Flags

1. --api-host string:

   • Description: Specifies the host for the client and server to communicate through REST. This flag is disregarded if the BACALHAU_API_HOST environment variable is set.

   • Default: bootstrap.production.bacalhau.org

2. --api-port int:

   • Description: Sets the port for RESTful communication between the client and server. If the BACALHAU_API_PORT environment variable is available, this flag is ignored.

   • Default: 1234

3. --log-mode logging-mode:

   • Description: Determines the desired log format. Available options include default, station, json, combined, and event.

   • Default: default

4. --repo string:

   • Description: Specifies the path to the bacalhau repository.

   • Default: $HOME/.bacalhau

Examples

1. Display Logs for a Previously Submitted Job with Full ID:

   Command:

   Expected Output:
2. Follow Logs in Real-Time:

   Command:

   Expected Output:
3. Display Logs Using a Shortened ID:

   Command:

   Expected Output:

Description

The bacalhau job run command facilitates the initiation of a job from a file or directly from the standard input (stdin). The command supports both JSON and YAML data formats. This command is particularly useful for quickly executing a job without the need for manual configurations.

Usage

Flags

• --dry-run:

  • Description: With this flag, the job will not be submitted. Instead, it will display what would have been submitted, providing a way to preview before actual submission.

• -f, --follow:

  • Description: If provided, the command will continuously display the output from the job as it runs.

• --id-only:

  • Description: On successful job submission, only the Job ID will be printed.

• --no-template:

  • Disable the templating feature. When this flag is set, the job spec will be used as-is, without any placeholder replacements

• --node-details:

  • Description: Displays details of all nodes. Note that this flag is overridden if --id-only is provided.

• --show-warnings:

  • Description: Shows any warnings that occur during the job submission.

• -E, --template-envs:

  • Specify a regular expression pattern for selecting environment variables to be included as template variables in the job spec. e.g. --template-envs ".*" will include all environment variables.

• -V, --template-vars:

  • Replace a placeholder in the job spec with a value. e.g. --template-vars foo=bar

• --wait:

  • Description: Waits for the job to finish execution. To set this to false, use --wait=false

  • Default: true

• --wait-timeout-secs int:

  • Description: If --wait is provided, this flag sets the maximum time (in seconds) the command will wait for the job to finish before it terminates.

  • Default: 600 seconds

• -h, --help:

  • Description: Displays help information for the run command.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for REST communication. If BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Selects the desired log format. Options include: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

Sample Job (job.yaml)

A sample job used in the following examples is provided below:

This configuration describes a batch job that runs a Docker task. It utilizes the ubuntu:latest image and executes the command echo Hello Bacalhau!.

1. Running a Job using a YAML Configuration:

   To run a job with a configuration provided in a job.yaml file:

   Command:

   Expected Output:
2. Running a Job and Following its Logs:

   Command:

   Expected Output:
3. Running a Job Without Waiting:

   Command:

   Expected Output:
4. Fetching Only the Job ID Upon Submission:

   Command:

   Expected Output:
5. Fetching Only the Job ID and Wait for Completion:

   Command:

   Expected Output:
6. Running a Job with Node Details:

   Command:

   Expected Output:
7. Rerunning a previously submitting job:

   Command:

   Expected Output:

Templating

Description

The bacalhau job stop command allows users to terminate a previously submitted job. This is useful in scenarios where there's a need to halt a running job, perhaps due to misconfiguration or changed priorities.

Usage

bacalhau job stop [id] [flags]

Flags

• --quiet:

  • Description: If provided, the command will not display any output, neither to the standard output (stdout) nor to the standard error (stderr).

• -h, --help:

  • Description: Displays help information for the stop command.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for REST communication. If BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Selects the desired log format. Options include: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

1. Stop a Specific Job:

   If you wish to halt the execution of a job, you can utilize the stop command. Here's how you can achieve that:

   Command:

   Copy

   bacalhau job stop j-10eb97de-14cd-4db4-96ec-561bb943309a

   Expected Output:

   Copy

   Checking job status
   
   	Connecting to network  ................  done ✅  0.0s
   	  Verifying job state  ................  done ✅  0.2s
   	          Stopping job ................  done ✅  0.1s
   
   Job stop successfully submitted with evaluation ID: 397fd425-8b1a-491e-952a-0632492e7ece

2. Silently Stop a Job:

   If you prefer to terminate a job without seeing any verbose feedback or messages, the --quiet option can be used.

   Command:

   Copy

   bacalhau job stop j-63b5ec0c-b5bf-4398-a152-b46c07abe52a --quiet

   Expected Output:

   Copy

   [No output displayed as the operation is run quietly.]

Description

The bacalhau job executions command retrieves a list of executions for a specific job based on its ID. This can be essential when tracking the various runs and their respective states for a particular job.

Usage

bacalhau job executions [id] [flags]

Flags

• -h, --help:

  • Description: Display help for the executions command.

• --hide-header:

  • Description: Do not print the column headers when displaying the results.

• --limit uint32:

  • Description: Restricts the number of results returned.

  • Default: 20

• --next-token string:

  • Description: Uses the specified token for pagination. Useful for fetching the next set of results.

• --no-style:

  • Description: Removes all styling from the table output, displaying raw data.

• --order-by string:

  • Description: Orders results based on a specific field. Valid fields are: modify_time, create_time, id, and state.

• --order-reversed:

  • Description: Reverses the order of the results. Useful in conjunction with --order-by.

• --output format:

  • Description: Specifies the desired output format for the command. Supported values are table, csv, json, and yaml.

  • Default: table

• --pretty:

  • Description: Pretty prints the output. This option is applicable only to json and yaml output formats.

• --wide:

  • Description: Prints full values in the table results without truncating any information.

Global Flags

• --api-host string:

  • Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST environment variable is set, this flag will be ignored.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Specifies the desired log format. Supported values include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

1. List executions for a specific Job:

   Copy

   bacalhau job executions j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881

   Expected output:

   Copy

   CREATED   MODIFIED  ID          NODE ID   REV.  COMPUTE    DESIRED  COMMENT
                                                   STATE      STATE
   16:46:03  16:46:04  e-99362435  QmTSJgdN  6     Completed  Stopped
   16:46:03  16:46:04  e-75dd20bb  QmXRdLru  6     Completed  Stopped
   16:46:03  16:46:04  e-03870df5  QmVXwmdZ  6     Completed  Stopped

2. Order executions by state for a specific job:

   Execute the command:

   Copy

   bacalhau job executions j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --order-by state

   Expected output:

   Copy

   CREATED   MODIFIED  ID          NODE ID   REV.  COMPUTE    DESIRED  COMMENT
                                                   STATE      STATE
   16:46:03  16:46:04  e-03870df5  QmVXwmdZ  6     Completed  Stopped
   16:46:03  16:46:04  e-75dd20bb  QmXRdLru  6     Completed  Stopped
   16:46:03  16:46:04  e-99362435  QmTSJgdN  6     Completed  Stopped

3. List executions with YAML output:

   Copy

   bacalhau job executions j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --output yaml

   Expected output:

   Copy

   ... [The YAML formatted output] ...