Description

The bacalhau job executions command retrieves a list of executions for a specific job based on its ID. This can be essential when tracking the various runs and their respective states for a particular job.

Usage

bacalhau job executions [id] [flags]

Flags

• -h, --help:

  • Description: Display help for the executions command.

• --hide-header:

  • Description: Do not print the column headers when displaying the results.

• --limit uint32:

  • Description: Restricts the number of results returned.

  • Default: 20

• --next-token string:

  • Description: Uses the specified token for pagination. Useful for fetching the next set of results.

• --no-style:

  • Description: Removes all styling from the table output, displaying raw data.

• --order-by string:

  • Description: Orders results based on a specific field. Valid fields are: modify_time, create_time, id, and state.

• --order-reversed:

  • Description: Reverses the order of the results. Useful in conjunction with --order-by.

• --output format:

  • Description: Specifies the desired output format for the command. Supported values are table, csv, json, and yaml.

  • Default: table

• --pretty:

  • Description: Pretty prints the output. This option is applicable only to json and yaml output formats.

• --wide:

  • Description: Prints full values in the table results without truncating any information.

Global Flags

• --api-host string:

  • Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST environment variable is set, this flag will be ignored.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Specifies the desired log format. Supported values include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

1. List executions for a specific Job:

   Copy

   bacalhau job executions j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881

   Expected output:

   Copy

   CREATED   MODIFIED  ID          NODE ID   REV.  COMPUTE    DESIRED  COMMENT
                                                   STATE      STATE
   16:46:03  16:46:04  e-99362435  QmTSJgdN  6     Completed  Stopped
   16:46:03  16:46:04  e-75dd20bb  QmXRdLru  6     Completed  Stopped
   16:46:03  16:46:04  e-03870df5  QmVXwmdZ  6     Completed  Stopped

2. Order executions by state for a specific job:

   Execute the command:

   Copy

   bacalhau job executions j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --order-by state

   Expected output:

   Copy

   CREATED   MODIFIED  ID          NODE ID   REV.  COMPUTE    DESIRED  COMMENT
                                                   STATE      STATE
   16:46:03  16:46:04  e-03870df5  QmVXwmdZ  6     Completed  Stopped
   16:46:03  16:46:04  e-75dd20bb  QmXRdLru  6     Completed  Stopped
   16:46:03  16:46:04  e-99362435  QmTSJgdN  6     Completed  Stopped

3. List executions with YAML output:

   Copy

   bacalhau job executions j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --output yaml

   Expected output:

   Copy

   ... [The YAML formatted output] ...

Description

The bacalhau job describe command provides a detailed description of a specific job in YAML format. This description can be particularly useful when wanting to understand the attributes and current status of a specific job. To list all available jobs, the bacalhau job list command can be used.

Usage

bacalhau job describe [id] [flags]

Flags

• -h, --help:

  • Description: Display help for the describe command.

• --output format:

  • Description: Specifies the desired output format for the command. Supported values are json and yaml.

  • Default: yaml

• --pretty:

  • Description: Pretty prints the output. This option is applicable only to json and yaml output formats.

Global Flags

• --api-host string:

  • Description: Specifies the host for the client and server to communicate through via REST. If the BACALHAU_API_HOST environment variable is set, this flag will be ignored.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for the client and server to communicate on using REST. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Specifies the desired log format. Supported values include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

1. Describe a Job with Full ID:

   Copy

   bacalhau job describe j-e3f8c209-d683-4a41-b840-f09b88d087b9

2. Describe a Job with Shortened ID:

   Copy

   bacalhau job describe j-47805f5c

3. Describe a Job with JSON Output:

   Copy

   bacalhau job describe --output json --pretty j-b6ad164a

Description

The bacalhau job logs command allows users to retrieve logs from a job that has been previously submitted. This command is useful for tracking and debugging the progress and state of a running or completed job.

Usage

bacalhau job logs [id] [flags]

Flags

1. -f, --follow:

   • Description: This flag allows the user to follow the logs in real-time after fetching the current logs. It provides a continuous stream of log updates, similar to tail -f in Unix-like systems.

2. -h, --help:

   • Description: Display help information for the logs command.

Global Flags

1. --api-host string:

   • Description: Specifies the host for the client and server to communicate through REST. This flag is disregarded if the BACALHAU_API_HOST environment variable is set.

   • Default: bootstrap.production.bacalhau.org

2. --api-port int:

   • Description: Sets the port for RESTful communication between the client and server. If the BACALHAU_API_PORT environment variable is available, this flag is ignored.

   • Default: 1234

3. --log-mode logging-mode:

   • Description: Determines the desired log format. Available options include default, station, json, combined, and event.

   • Default: default

4. --repo string:

   • Description: Specifies the path to the bacalhau repository.

   • Default: $HOME/.bacalhau

Examples

1. Display Logs for a Previously Submitted Job with Full ID:

   Command:

   Copy

   bacalhau job logs j-51225160-807e-48b8-88c9-28311c7899e1

   Expected Output:

   Copy

   [2023-09-24 09:01:32] INFO - Application started successfully.
   [2023-09-24 09:01:33] DEBUG - Initializing database connections.
   [2023-09-24 09:01:35] WARN - API rate limit approaching.
   [2023-09-24 09:02:01] ERROR - Failed to retrieve data from endpoint: /api/v1/data.
   [2023-09-24 09:05:00] INFO - Data sync completed with 4500 new records.

2. Follow Logs in Real-Time:

   Command:

   Copy

   bacalhau job logs --follow j-51225160-807e-48b8-88c9-28311c7899e1

   Expected Output:

   Copy

   [2023-09-24 11:30:02] INFO - User 'john_doe' logged in successfully.
   [2023-09-24 11:30:15] DEBUG - Fetching data from cache for key: userSettings_john_doe.
   [2023-09-24 11:31:05] WARN - High memory usage detected: 85% of allocated resources.
   ... [Logs continue to appear in real-time] ...

3. Display Logs Using a Shortened ID:

   Command:

   Copy

   bacalhau job logs j-ebd9bf2f

   Expected Output:

   Copy

   [2023-09-24 10:15:12] INFO - Application initialization sequence started.
   [2023-09-24 10:15:13] DEBUG - Loading configurations from /config/app.json.
   [2023-09-24 10:15:14] INFO - Connected to message broker successfully.
   [2023-09-24 10:16:00] ERROR - Failed to send email notification to user@example.com.

The bacalhau job command provides a suite of sub-commands to submit, query, and manage jobs within Bacalhau. Users can deploy jobs, obtain job details, track execution logs, and more.

Usage

bacalhau job [command]

Available Commands

1. describe:

   • Description: Retrieves detailed information of a job using its ID.

   • Usage:

     Copy

     bacalhau job describe

2. executions:

   • Description: Lists all executions associated with a job, identified by its ID.

   • Usage:

     Copy

     bacalhau job executions

3. history:

   • Description: Enumerates the historical events related to a job, identified by its ID.

   • Usage:

     Copy

     bacalhau job history

4. list:

   • Description: Provides an overview of all submitted jobs.

   • Usage:

     Copy

     bacalhau job list

5. logs:

   • Description: Fetches and streams the logs from a currently executing job.

   • Usage:

     Copy

     bacalhau job logs

6. run:

   • Description: Submits a job for execution using either a JSON or YAML configuration file.

   • Usage:

     Copy

     bacalhau job run

7. stop:

   • Description: Halts a previously submitted job.

   • Usage:

     Copy

     bacalhau job stop

For comprehensive details on any of the sub-commands, run:

bacalhau job [command] --help

Flags

• -h, --help:

  • Description: Shows the help information for the job command.

Global Flags

1. --api-host string:

   • Description: Determines the host for RESTful communication between the client and server. This flag is overlooked if the BACALHAU_API_HOST environment variable is set.

   • Default: bootstrap.production.bacalhau.org

2. --api-port int:

   • Description: Designates the port for RESTful communication. This flag is bypassed if the BACALHAU_API_PORT environment variable is active.

   • Default: 1234

3. --log-mode logging-mode:

   • Description: Chooses the preferred log format. Available choices are: default, station, json, combined, and event.

   • Default: default

4. --repo string:

   • Description: Specifies the path to the bacalhau repository.

   • Default: $HOME/.bacalhau

Description

The bacalhau job list command provides a listing of all submitted jobs. This command offers an overview of all tasks and processes registered in the system, allowing users to monitor and manage their jobs effectively.

Usage

bacalhau job list [flags]

Flags

• -h, --help:

  • Description: Display help for the list command.

• --hide-header:

  • Description: Opts out of printing the column headers in the results.

• --labels string:

  • Description: Filters jobs by labels. It's designed to function similar to Kubernetes label selectors.

  • Default: bacalhau_canary != true

• --limit uint32:

  • Description: Limits the number of results returned.

  • Default: 10

• --next-token string:

  • Description: Uses the provided token for pagination.

• --no-style:

  • Description: Strips all styling from the table output.

• --order-by string:

  • Description: Organizes results based on a chosen field. Valid fields are id and created_at.

• --order-reversed:

  • Description: Reverses the order of the displayed results.

• --output format:

  • Description: Dictates the desired output format for the command. Options are table, csv, json, and yaml.

  • Default: table

• --pretty:

  • Description: Offers a more visually pleasing output for json and yaml formats.

• --wide:

  • Description: Presents full values in the table results, preventing truncation.

Global Flags

• --api-host string:

  • Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST environment variable, if set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT environment variable takes precedence if set.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Designates the desired log format. Options include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Points to the bacalhau repository location.

  • Default: $HOME/.bacalhau

Examples

1. List all jobs:

   Execute the command to list all the jobs:

   Copy

   bacalhau job list

   Expected output:

   Copy

   CREATED   ID          JOB     TYPE   STATE
   08:19:07  d78a4cb4    docker  batch  Completed
   04:17:21  e45f31a7    docker  batch  Completed
   04:53:50  f4993f62    docker  batch  Completed
   ... (trimmed for brevity) ...

2. Limit the list to the last two jobs:

   Limit the list to display only the last two jobs:

   Copy

   bacalhau job list --limit 2

   Expected output:

   Copy

   CREATED   ID          JOB     TYPE   STATE
   03:14:16  19a26187    docker  batch  Completed
   21:47:21  2a53a13b    docker  batch  Completed

3. Order the list by creation date in descending order:

   Order the jobs by their creation date in a descending manner:

   Copy

   bacalhau job list --order-by created_at --order-reversed

   Expected output:

   Copy

   CREATED   ID          JOB     TYPE   STATE
   17:44:16  90e14efd    docker  batch  Completed
   17:44:08  8204570c    docker  batch  Completed
   17:43:50  f196521d    docker  batch  Completed
   ... (trimmed for brevity) ...

4. Filter the jobs by specific labels:

   Display jobs that have specific labels:

   Copy

   bacalhau job list --labels "region in (us-east-1, us-east-2),env = prod"

   Expected output:

   Copy

   ... (filtered jobs) ...

5. Display the list in JSON format with pretty printing:

   Get a limited list of jobs in a formatted JSON output:

   Copy

   bacalhau job list --limit 3 --output json --pretty

   Expected output:

   Copy

   ... [The JSON formatted output] ...

Description

The bacalhau job stop command allows users to terminate a previously submitted job. This is useful in scenarios where there's a need to halt a running job, perhaps due to misconfiguration or changed priorities.

Usage

bacalhau job stop [id] [flags]

Flags

• --quiet:

  • Description: If provided, the command will not display any output, neither to the standard output (stdout) nor to the standard error (stderr).

• -h, --help:

  • Description: Displays help information for the stop command.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for REST communication. If BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Selects the desired log format. Options include: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

1. Stop a Specific Job:

   If you wish to halt the execution of a job, you can utilize the stop command. Here's how you can achieve that:

   Command:

   Copy

   bacalhau job stop j-10eb97de-14cd-4db4-96ec-561bb943309a

   Expected Output:

   Copy

   Checking job status
   
   	Connecting to network  ................  done ✅  0.0s
   	  Verifying job state  ................  done ✅  0.2s
   	          Stopping job ................  done ✅  0.1s
   
   Job stop successfully submitted with evaluation ID: 397fd425-8b1a-491e-952a-0632492e7ece

2. Silently Stop a Job:

   If you prefer to terminate a job without seeing any verbose feedback or messages, the --quiet option can be used.

   Command:

   Copy

   bacalhau job stop j-63b5ec0c-b5bf-4398-a152-b46c07abe52a --quiet

   Expected Output:

   Copy

   [No output displayed as the operation is run quietly.]

Description

The bacalhau job history command lists the history events of a specific job based on its ID. This feature allows users to track changes, executions, and other significant milestones associated with a particular job.

Usage

bacalhau job history [id] [flags]

Flags

• --event-type string:

  • Description: Specifies the type of history events to retrieve. Available options include all, job, and execution.

  • Default: all

• --execution-id string:

  • Description: Filters results by a specific execution ID.

• -h, --help:

  • Description: Display help for the history command.

• --hide-header:

  • Description: Opts out of printing the column headers in the results.

• --limit uint32:

  • Description: Limits the number of results returned.

• --next-token string:

  • Description: Uses the provided token for pagination.

• --no-style:

  • Description: Strips all styling from the table output.

• --node-id string:

  • Description: Filters the results by a specific node ID.

• --order-by string:

  • Description: Organizes results based on a chosen field.

• --order-reversed:

  • Description: Reverses the order of the displayed results.

• --output format:

  • Description: Dictates the desired output format for the command. Options are table, csv, json, and yaml.

  • Default: table

• --pretty:

  • Description: Offers a more visually pleasing output for json and yaml formats.

• --wide:

  • Description: Presents full values in the table results, preventing truncation.

Global Flags

• --api-host string:

  • Description: Defines the host for client-server communication via REST. Overridden by the BACALHAU_API_HOST environment variable, if set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Sets the port for RESTful communication between the client and server. The BACALHAU_API_PORT environment variable takes precedence if set.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Designates the desired log format. Options include default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Points to the bacalhau repository location.

  • Default: $HOME/.bacalhau

Examples

1. Retrieve the history of a specific job:

   Execute the command to get the job history:

   Copy

   bacalhau job history j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881

   Expected output:

   Copy

   TIME      LEVEL           EXEC. ID    ...     NEW STATE          COMMENT
   ... [The output rows like the ones you've shown] ...
   16:46:04  JobLevel                              2     Pending            Completed

2. Filter the history by event type:

   Filter the job history by the event type:

   Copy

   bacalhau job history j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --event-type job

   Expected output:

   Copy

   TIME      LEVEL     EXEC. ID  NODE ID  REV.  PREVIOUS STATE  NEW STATE  COMMENT
   16:46:03  JobLevel                     1     Pending         Pending    Job created
   16:46:04  JobLevel                     2     Pending         Completed

3. Filter the history by execution ID:

   Filter the job history by a specific execution ID:

   Copy

   bacalhau job history j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --execution-id e-99362435

   Expected output:

   Copy

   TIME      LEVEL           EXEC. ID    ...     NEW STATE          COMMENT
   ... [The output rows for the specific execution ID] ...
   16:46:04  ExecutionLevel  e-99362435  QmTSJgdN  6     BidAccepted        Completed

4. Retrieve the history in YAML format:

   Get the job history in YAML format:

   Copy

   bacalhau job history j-6f2bf0ea-ebcd-4490-899a-9de9d8d95881 --output yaml

   Expected output:

   Copy

   ... [The YAML formatted output] ...

The bacalhau node describe command offers users the ability to retrieve detailed information about a specific node using its unique identifier.

Description:

Using the describe sub-command under the bacalhau node umbrella, users can get comprehensive details of a node by providing its ID. This information is crucial for system administrators and network managers to understand the state, specifications, and other attributes of nodes in their infrastructure.

Usage:

bacalhau node describe [id] [flags]

Flags:

• [id]:

  • The unique identifier of the node you wish to describe.

• -h, --help:

  • Displays the help documentation for the describe command.

• --output format:

  • Defines the desired format for the command's output.

  • Options: "json" or "yaml"

  • Default: "yaml"

• --pretty:

  • When this flag is used, the command will pretty print the output. This is applicable only for outputs in json and yaml formats.

Global Flags:

• --api-host string:

  • Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST environment variable is set.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT environment variable is defined.

  • Default: 1234

• --log-mode logging-mode:

  • Determines the log format preference.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Points to the bacalhau repository's path.

  • Default: "$HOME/.bacalhau"`

Examples:

1. Describing a Node with ID nodeID123:

   Copy

   bacalhau node describe nodeID123

2. Describing a Node with Output in JSON Format:

   Copy

   bacalhau node describe nodeID123 --output json

3. Pretty Printing the Description of a Node:

   Copy

   bacalhau node describe nodeID123 --pretty

The bacalhau node approve command offers administrators the ability to approve the cluster membership for a node using its name.

Description:

Using the approve sub-command under the bacalhau node umbrella, users can allow a node in the pending state to join the cluster and receive work. This feature is crucial for system administrators to manage the cluster.

Usage:

bacalhau node approve [id] [flags]

Flags:

• [id]:

  • The unique identifier of the node you wish to describe.

• -h, --help:

  • Displays the help documentation for the describe command.

• -m message:

  • A message to be attached to the approval action.

Global Flags:

• --api-host string:

  • Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST environment variable is set.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT environment variable is defined.

  • Default: 1234

• --log-mode logging-mode:

  • Determines the log format preference.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Points to the bacalhau repository's path.

  • Default: "$HOME/.bacalhau"`

Examples:

1. Approve a Node with ID nodeID123:

   Copy

   bacalhau node approve nodeID123

2. Approve a Node with an audit message:

   Copy

   bacalhau node approve nodeID123 -m "okay"

The bacalhau node delete command offers administrators the ability to remove a node from the cluster using its name.

Description:

Using the delete sub-command, administrators can remove a node from the list of available compute nodes in the cluster. This feature is necessary for the management of the infrastructure.

Usage:

bacalhau node delete [id] [flags]

Flags:

• [id]:

  • The unique identifier of the node you wish to describe.

• -h, --help:

  • Displays the help documentation for the describe command.

• -m message:

  • A message to be attached to the deletion action.

Global Flags:

• --api-host string:

  • Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST environment variable is set.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT environment variable is defined.

  • Default: 1234

• --log-mode logging-mode:

  • Determines the log format preference.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Points to the bacalhau repository's path.

  • Default: "$HOME/.bacalhau"`

Examples:

1. Delete the Node with ID nodeID123:

   Copy

   bacalhau node delete nodeID123

2. Delete a Node with an audit message:

   Copy

   bacalhau node delete nodeID123 -m "bad actor"

The bacalhau node list command is designed to provide users with a comprehensive list of network nodes along with details based on specified flags.

Description:

The list sub-command under the bacalhau node category enumerates information about nodes in the network. It supports various filtering, ordering, and output formatting options, allowing users to tailor the output to their needs.

Usage:

bacalhau node list [flags]

Flags:

• -h, --help:

  • Show the help message for the list command.

• --hide-header:

  • Do not display the column headers in the output.

• --filter-approval:

  • Only show nodes with the specified approval status. Valid values are: approved, pending, rejected.

• --filter-status:

  • Only show nodes with the specified state. Valid values are: healthy, unhealthy, unknown.

• --labels string:

  • Filter nodes based on labels. This follows the filtering format provided by Kubernetes, as shown in their documentation about labels.

• --limit uint32:

  • Restrict the number of results displayed.

• --next-token string:

  • Provide the next token for pagination.

• --no-style:

  • Output the table without any style.

• --order-by string:

  • Sort the results based on a specific field. Valid sorting fields are: id, type, available_cpu, available_memory, available_disk, available_gpu.

• --order-reversed:

  • Display the results in reverse order.

• --output format:

  • Choose the output format. Available options: table, csv, json, yaml.

  • Default: table.

• --pretty:

  • Enhance the visual appeal of the output. This is applicable only to json and yaml formats.

• --show strings:

  • Determine the column groups to be displayed. Acceptable values are: labels, version, features, capacity.

  • Default: labels, capacity.

• --wide:

  • Display full values in the output table, without truncation.

Global Flags:

• --api-host string:

  • Specify the host for client-server communication via REST. This gets ignored if the BACALHAU_API_HOST environment variable is defined.

  • Default: "bootstrap.production.bacalhau.org".

• --api-port int:

  • Specify the port for RESTful communication between client and server. Gets overlooked if the BACALHAU_API_PORT environment variable is set.

  • Default: 1234.

• --log-mode logging-mode:

  • Choose the desired log format.

  • Options: 'default', 'station', 'json', 'combined', 'event'.

  • Default: 'default'.

• --repo string:

  • Point to the directory path of the bacalhau repository.

  • Default: "$HOME/.bacalhau"`.

Examples

1. Retrieve the list of nodes:

   Execute the command to get a list of all nodes:

   Copy

   bacalhau node list

   Expected output:

   Copy

    ID        TYPE     LABELS                                              CPU     MEMORY      DISK         GPU
    QmTSJgdN  Compute  Architecture=amd64 Operating-System=linux           3.2 /   11.7 GB /   77.8 GB /    1 /
                       git-lfs=True owner=bacalhau                         3.2     11.7 GB     77.8 GB      1
    QmVXwmdZ  Compute  Architecture=amd64 Operating-System=linux           3.2 /   12.5 GB /   77.8 GB /    0 /
                       git-lfs=True owner=bacalhau                         3.2     12.5 GB     77.8 GB      0
    QmXRdLru  Compute  Architecture=amd64 Operating-System=linux           3.2 /   12.5 GB /   78.0 GB /    0 /
                       git-lfs=True owner=bacalhau                         3.2     12.5 GB     78.0 GB      0
    ... [Additional nodes information] ...

2. Filter the list of nodes by labels:

   Execute the command to get a list of nodes with specific labels:

   Copy

   bacalhau node list --labels "Operating-System=linux,owner=bacalhau"

   Expected output:

   Copy

   ID        TYPE     LABELS                                              CPU     MEMORY      DISK         GPU
   QmTSJgdN  Compute  Architecture=amd64 Operating-System=linux           3.2 /   11.7 GB /   77.8 GB /    1 /
                      git-lfs=True owner=bacalhau                         3.2     11.7 GB     77.8 GB      1
   ... [Additional nodes information] ...

3. Order the list of nodes by available memory:

   Execute the command to get the list of nodes ordered by available memory:

   Copy

   bacalhau node list --order-by available_memory

   Expected output:

   Copy

   ID        TYPE     LABELS                                              CPU     MEMORY      DISK         GPU
   QmVXwmdZ  Compute  Architecture=amd64 Operating-System=linux           3.2 /   12.5 GB /   77.8 GB /    0 /
                      git-lfs=True owner=bacalhau                         3.2     12.5 GB     77.8 GB      0
   ... [Additional nodes information] ...

4. Limit the number of nodes displayed and output in JSON format:

   Execute the command to get a limited list of nodes in JSON format:

   Copy

   bacalhau node list  --limit 3 --output json --pretty

   Expected output:

   Copy

   [
     {
       "PeerInfo": {
         "ID": "QmTSJgdN7zCPAqBCkmdsdpFbiJV8bJ6zhoxK9N5xfar1sz",
         ... [Additional node details] ...
       },
       ... [Other nodes] ...
     }
   ]

Description

The bacalhau job run command facilitates the initiation of a job from a file or directly from the standard input (stdin). The command supports both JSON and YAML data formats. This command is particularly useful for quickly executing a job without the need for manual configurations.

Usage

bacalhau job run [flags]

Flags

• --dry-run:

  • Description: With this flag, the job will not be submitted. Instead, it will display what would have been submitted, providing a way to preview before actual submission.

• -f, --follow:

  • Description: If provided, the command will continuously display the output from the job as it runs.

• --id-only:

  • Description: On successful job submission, only the Job ID will be printed.

• --node-details:

  • Description: Displays details of all nodes. Note that this flag is overridden if --id-only is provided.

• --show-warnings:

  • Description: Shows any warnings that occur during the job submission.

• --wait:

  • Description: Waits for the job to finish execution. To set this to false, use --wait=false

  • Default: true

• --wait-timeout-secs int:

  • Description: If --wait is provided, this flag sets the maximum time (in seconds) the command will wait for the job to finish before it terminates.

  • Default: 600 seconds

• -h, --help:

  • Description: Displays help information for the run command.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Determines the port for REST communication. If BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Selects the desired log format. Options include: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: $HOME/.bacalhau

Examples

Sample Job (job.yaml)

A sample job used in the following examples is provided below:

cat job.yaml

name: A Simple Docker Job
type: batch
count: 1
tasks:
  - name: My main task
    engine:
      type: docker
      params:
        Image: ubuntu:latest
        Entrypoint:
          - /bin/bash
        Parameters:
          - -c
          - echo Hello Bacalhau!

This configuration describes a batch job that runs a Docker task. It utilizes the ubuntu:latest image and executes the command echo Hello Bacalhau!.

1. Running a Job using a YAML Configuration:

   To run a job with a configuration provided in a job.yaml file:

   Command:

   Copy

   bacalhau job run job.yaml

   Expected Output:

   Copy

   Job successfully submitted. Job ID: j-2d0f513a-9eb1-49c2-8bc8-246c6fb41520
   Checking job status... (Enter Ctrl+C to exit at any time, your job will continue running):
   
    Communicating with the network  ................  done ✅  0.1s
       Creating job for submission  ................  done ✅  0.6s
   
   To get more details about the run, execute:
    bacalhau job describe j-2d0f513a-9eb1-49c2-8bc8-246c6fb41520
   
   To get more details about the run executions, execute:
    bacalhau job executions j-2d0f513a-9eb1-49c2-8bc8-246c6fb41520

2. Running a Job and Following its Logs:

   Command:

   Copy

   bacalhau job run job.yaml --follow

   Expected Output:

   Copy

   Job successfully submitted. Job ID: j-b89df816-7564-4f04-b270-e6cda89eda72
   Waiting for logs... (Enter Ctrl+C to exit at any time, your job will continue running):
   
   Hello Bacalhau!

3. Running a Job Without Waiting:

   Command:

   Copy

   bacalhau job run job.yaml --wait=false

   Expected Output:

   Copy

   j-3fd396b3-e92e-42ca-bd87-0dc9eb15e6f9

4. Fetching Only the Job ID Upon Submission:

   Command:

   Copy

   bacalhau job run job.yaml --id-only

   Expected Output:

   Copy

   j-5976ffb6-3465-4fec-8b3b-2c822cbaf417

5. Fetching Only the Job ID and Wait for Completion:

   Command:

   Copy

   bacalhau job run job.yaml --id-only --wait

   Expected Output:

   Copy

   j-293f1302-3298-4aca-b06d-33fd1e3f9d2c

6. Running a Job with Node Details:

   Command:

   Copy

   bacalhau job run job.yaml --node-details

   Expected Output:

   Copy

   Job successfully submitted. Job ID: j-05e65dd3-4e9e-4e20-a104-3c91ba934435
   Checking job status... (Enter Ctrl+C to exit at any time, your job will continue running):
   
    Communicating with the network  ................  done ✅  0.1s
       Creating job for submission  ................  done ✅  0.6s
   
   Job Results By Node:
   • Node QmVXwmdZ:
    Hello Bacalhau!
   
   To get more details about the run, execute:
    bacalhau job describe j-05e65dd3-4e9e-4e20-a104-3c91ba934435
   
   To get more details about the run executions, execute:
    bacalhau job executions j-05e65dd3-4e9e-4e20-a104-3c91ba934435

7. Rerunning a previously submitting job:

   Command:

   Copy

   bacalhau job describe j-05e65dd3-4e9e-4e20-a104-3c91ba934435 | bacalhau job run

   Expected Output:

   Copy

   Reading from /dev/stdin; send Ctrl-d to stop.Job successfully submitted. Job ID: j-d8625929-83f4-411a-b9aa-7bcfecb27a8b
   Checking job status... (Enter Ctrl+C to exit at any time, your job will continue running):
   
    Communicating with the network  ................  done ✅  0.1s
       Creating job for submission  ................  done ✅  0.6s
   
   To get more details about the run, execute:
    bacalhau job describe j-d8625929-83f4-411a-b9aa-7bcfecb27a8b
   
   To get more details about the run executions, execute:
    bacalhau job executions j-d8625929-83f4-411a-b9aa-7bcfecb27a8b

Job Templating

The bacalhau job run command also supports templating, which allows users to dynamically inject variables into their job specifications. Additional flags related to templating include:

• --no-template:

  • Description: Disable the templating feature. When this flag is set, the job spec will be used as-is, without any placeholder replacements.

• -E, --template-envs:

  • Description: Specify a regular expression pattern for selecting environment variables to be included as template variables in the job spec. e.g. --template-envs ".*" will include all environment variables.

• -V, --template-vars:

  • Description: Replace a placeholder in the job spec with a value. e.g. --template-vars foo=bar

Overview

Templating is particularly useful when running multiple jobs with varying parameters such as DuckDB query, S3 buckets, prefixes, and time ranges without the need to edit each job specification file manually.

Templating Implementation

The templating functionality in Bacalhau is built upon the Go text/template package. This powerful library offers a wide range of features for manipulating and formatting text based on template definitions and input variables.

Basic Templating Example

Sample Job Spec with Templating Variables:

Name: docker job
Type: batch
Count: 1
Tasks:
  - Name: main
    Engine:
      Type: docker
      Params:
        Image: ubuntu:latest
        Entrypoint:
          - /bin/bash
        Parameters:
          - -c
          - echo {{.greeting}} {{.name}}

Running with Templating:

bacalhau job run job.yaml --template-vars "greeting=Hello,name=World"

Defining Flag Multiple Times:

bacalhau job run job.yaml --template-vars "greeting=Hello" --template-vars "name=World"

Disabling Templating:

bacalhau job run job.yaml --no-template

Using Environment Variables for Templates

You can also use environment variables for templating:

export greeting=Hello
export name=World
bacalhau job run job.yaml --template-envs "*"

Passing A Subset of Environment Variables:

bacalhau job run job.yaml --template-envs "greeting|name"

Dry Run to Preview Templated Spec

To preview the final templated job spec without actually submitting the job, you can use the --dry-run flag:

bacalhau job run job.yaml --template-vars "greeting=Hello,name=World" --dry-run

This will output the processed job specification, showing you how the placeholders have been replaced with the provided values.

Advanced Templating Examples

Query Live Logs

Name: Live logs processing
Type: ops
Tasks:
  - Name: main
    Engine:
      Type: docker
      Params:
        Image: expanso/nginx-access-log-processor:1.0.0
        Parameters:
          - --query
          - {{.query}}
          - --start-time
          - {{or (index . "start-time") ""}}
          - --end-time
          - {{or (index . "end-time") ""}}
    InputSources:
      - Target: /logs
        Source:
          Type: localDirectory
          Params:
            SourcePath: /data/log-orchestration/logs

This is an ops job that runs on all nodes that match the job selection criteria. It accepts duckdb query variable, and two optional start-time and end-time variables to define the time range for the query.

To run this job, you can use the following command:

bacalhau job run job.yaml \
  -V "query=SELECT status FROM logs WHERE status LIKE '5__'" \
  -V "start-time=-5m"

Query S3 Logs

Name: S3 logs processing
Type: batch
Count: 1
Tasks:
  - Name: main
    Engine:
      Type: docker
      Params:
        Image: expanso/nginx-access-log-processor:1.0.0
        Parameters:
          - --query
          - {{.query}}
    InputSources:
      - Target: /logs
        Source:
          Type: s3
          Params:
            Bucket: {{.AccessLogBucket}}
            Key: {{.AccessLogPrefix}}
            Filter: {{or (index . "AccessLogPattern") ".*"}}
            Region: {{.AWSRegion}}

This is a batch job that runs on a single node. It accepts the duckdb query variable, and four other variables to define the S3 bucket, prefix, and pattern for the logs and the AWS region.

To run this job, you can use the following command:

bacalhau job run job.yaml  \
    -V "AccessLogBucket=my-bucket" \
    -V "AWSRegion=us-east-1" \
    -V "AccessLogPrefix=2023-11-19-*"  \
    -V "AccessLogPattern=^[10-12].*"

The bacalhau node command provides a set of sub-commands to query and manage node-related information within Bacalhau. With these tools, users can access specific details about nodes, list all network nodes, and more.

Usage

bacalhau node [command]

Available Commands

1. :

   • Description: Approves a single node to join the cluster.

   • Usage:

     Copy

     bacalhau node approve

2. :

   • Description: Deletes a node from the cluster using its ID.

   • Usage:

     Copy

     bacalhau node delete

3. :

   • Description: Retrieves detailed information of a node using its ID.

   • Usage:

     Copy

     bacalhau node describe

4. :

   • Description: Lists the details of all nodes present in the network.

   • Usage:

     Copy

     bacalhau node list

5. :

• Description: Reject a specific node's request to join the cluster.

• Usage:

  Copy

  bacalhau node reject

For comprehensive details on any of the sub-commands, run:

bacalhau node [command] --help

Flags

• -h, --help:

  • Description: Shows the help information for the node command.

Global Flags

• --api-host string:

  • Description: Specifies the host for RESTful communication between the client and server. The flag will be ignored if the BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Designates the port for RESTful communication. The flag will be bypassed if the BACALHAU_API_PORT environment variable is active.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Chooses the preferred log format. Available choices are: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Specifies the path to the bacalhau repository.

  • Default: /Users/walid/.bacalhau

The bacalhau node reject command offers administrators the ability to reject a compute node's request to join the cluster.

Description:

Using the reject sub-command, administrators can reject a node in the pending state from joining the cluster and receiving work. This feature is crucial for system administrators to manage the cluster and will stop the node from taking part in the cluster until approved.

Usage:

bacalhau node rejected [id] [flags]

Flags:

• [id]:

  • The unique identifier of the node you wish to describe.

• -h, --help:

  • Displays the help documentation for the describe command.

• -m message:

  • A message to be attached to the rejection action.

Global Flags:

• --api-host string:

  • Specifies the host for client-server communication through REST. This flag is overridden if the BACALHAU_API_HOST environment variable is set.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Designates the port for REST-based communication between client and server. This flag is overlooked if the BACALHAU_API_PORT environment variable is defined.

  • Default: 1234

• --log-mode logging-mode:

  • Determines the log format preference.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Points to the bacalhau repository's path.

  • Default: "$HOME/.bacalhau"`

Examples:

1. Reject a Node with ID nodeID123:

   Copy

   bacalhau node reject nodeID123

2. Reject a Node with an audit message:

   Copy

   bacalhau node reject nodeID123 -m "potentially bad"

Description

The bacalhau agent node command retrieves information about the agent's node, providing insights into the agent's environment and aiding in debugging.

Usage

bacalhau agent node [flags]

Flags

• -h, --help:

  • Displays help information for the node sub-command.

• --output format:

  • Defines the output format (either JSON or YAML).

  • Options: json, yaml

  • Default: yaml

• --pretty:

  • Beautifies the output when using JSON or YAML formats.

Global Flags

• --api-host string:

  • The host for REST communication. Overrides the BACALHAU_API_HOST environment variable.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • The port for REST communication. Overridden if BACALHAU_API_PORT environment variable is set.

  • Default: 1234

• --log-mode logging-mode:

  • Specifies the log format. Choices are: default, station, json, combined, event.

  • Default: default

• --repo string:

  • Path to the bacalhau repository.

  • Default: ``$HOME/.bacalhau`

Examples

1. Retrieve Node Information in Default Format (YAML)

   Copy

   bacalhau agent node

2. Retrieve Node Information in JSON Format

   Copy

   bacalhau agent node --output json

3. Retrieve Node Information in Pretty-printed JSON Format

   Copy

   bacalhau agent node --output json --pretty

Description

The bacalhau agent alive command provides information about the agent's liveness and health. This is essential for monitoring and ensuring that the agent is active and functioning correctly.

Usage

bacalhau agent alive [flags]

Flags

• -h, --help:

  • Description: Displays help information for the alive sub-command.

• --output format:

  • Description: Determines the format in which the output is displayed. Available formats include JSON and YAML.

  • Options: json, yaml

  • Default: yaml

• --pretty:

  • Description: Formats the output for enhanced readability. This flag is relevant only when using JSON or YAML output formats.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Specifies the port for REST communication. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Sets the desired log format. Options are: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: ``$HOME/.bacalhau`

Examples

Checking the Agent's Liveness and Health Info

1. Basic Usage:

   Command:

   Copy

   → bacalhau agent alive

   Output:

   Copy

   status: OK

2. Output in JSON format:

   Command:

   Copy

   → bacalhau agent alive --output json --pretty

   Output:

   Copy

   {
     "Status": "OK"
   }

The bacalhau agent command is a parent command that offers sub-commands to query information about the Bacalhau agent. This can be useful for debugging, monitoring, or managing the agent's behavior and health.

Usage

bacalhau agent [command]

Available Commands

1. alive:

   • Description: Retrieves the agent's liveness and health information. This can be helpful to determine if the agent is running and healthy.

   • Usage:

     Copy

     bacalhau agent alive

2. node:

   • Description: Gathers the agent's node-related information. This might include details about the machine or environment where the agent is running, available resources, supported engines, etc.

   • Usage:

     Copy

     bacalhau agent node

3. version:

   • Description: Retrieves the Bacalhau version of the agent. This can be beneficial for ensuring compatibility or checking for updates.

   • Usage:

     Copy

     bacalhau agent version

For more detailed information on any of the sub-commands, you can use the command:

bacalhau agent [command] --help

Flags

• -h, --help:

  • Description: Displays help information for the agent command.

Global Flags

• --api-host string:

  • Description: Specifies the host used for RESTful communication between the client and server. The flag is disregarded if the BACALHAU_API_HOST environment variable is set.

  • Default: bootstrap.production.bacalhau.org

• --api-port int:

  • Description: Specifies the port for REST communication. If the BACALHAU_API_PORT environment variable is set, this flag will be ignored.

  • Default: 1234

• --log-mode logging-mode:

  • Description: Sets the desired log format. Options are: default, station, json, combined, and event.

  • Default: default

• --repo string:

  • Description: Defines the path to the bacalhau repository.

  • Default: ``$HOME/.bacalhau`

The bacalhau config command is a parent command that offers sub-commands to modify and query information about the Bacalhau config. This can be useful for debugging, monitoring, or managing the nodes configuration.

Usage

bacalhau config [command]

Available Commands

1. list:

   • Description: Lists the configuration keys and values of the bacalhau node. This command is useful for understanding how configuration keys map to their respective values, aiding in the use of the bacalhau config set command.

   • Usage:

     Copy

     bacalhau config list

2. set:

   • Description: Sets a value in the bacalhau node's configuration file. This command is used to modify the configuration file that the bacalhau node will reference for its settings.

   • Usage:

     Copy

     bacalhau config set <key> <value>

The bacalhau agent version command is used to obtain the version of the bacalhau agent.

Description:

Using this command, users can quickly retrieve the version of the agent, allowing them to confirm the specific release of the software they are using.

Usage:

bacalhau agent version [flags]

Flags:

• -h, --help:

  • Show help for the version command.

• --output format:

  • Defines the output format of the command's results. Accepted formats include "json" and "yaml".

• --pretty:

  • Used for pretty printing the output, enhancing readability. This flag is applicable only for the "json" and "yaml" output formats.

Global Flags:

• --api-host string:

  • Designates the host for client-server communication via REST. If the BACALHAU_API_HOST environment variable is present, this flag will be disregarded.

  • Default: "bootstrap.production.bacalhau.org"

• --api-port int:

  • Defines the port for client-server communication through REST. This flag becomes irrelevant if the BACALHAU_API_PORT environment variable is specified.

  • Default: 1234

• --log-mode logging-mode:

  • Specifies the desired logging format.

  • Options: 'default','station','json','combined','event'

  • Default: 'default'

• --repo string:

  • Indicates the path to the bacalhau repository.

  • Default: "$HOME/.bacalhau"`

Examples

1. Retrieve the agent version:

   Execute the command to get the agent version:

   Copy

   bacalhau agent version

   Expected output:

   Copy

   Bacalhau v0.0.0-xxxxxxx
   BuildDate 2023-09-22 16:03:44 +0000 UTC
   GitCommit 0fe81cb488f666845ac72c73a4b804aaa658e511

2. Retrieve the agent version in JSON format:

   Copy

   bacalhau agent version --output json

   Expected output:

   Copy

   {"major":"0","minor":"0","gitversion":"v0.0.0-xxxxxxx","gitcommit":"0fe81cb488f666845ac72c73a4b804aaa658e511","builddate":"2023-09-22T16:03:44Z","goos":"linux","goarch":"amd64"}

3. Retrieve the agent version in Pretty-printed JSON format:

   Copy

   bacalhau agent version --output json --pretty

   Expected output:

   Copy

   {
     "major": "0",
     "minor": "0",
     "gitversion": "v0.0.0-xxxxxxx",
     "gitcommit": "0fe81cb488f666845ac72c73a4b804aaa658e511",
     "builddate": "2023-09-22T16:03:44Z",
     "goos": "linux",
     "goarch": "amd64"
   }

Welcome to the official API documentation for Bacalhau. This guide provides a detailed insight into Bacalhau's RESTful HTTP APIs and demonstrates how to make the most out of them.

Overview

Bacalhau prioritizes an "API-first" design, enabling users to interact with their deployed systems programmatically. In the v1.4.0 the API model was changed to include only two endpoints, focused on orchestrating, querying and managing your network nodes and jobs. Each endpoint has a clear, separate environment and goal, allowing to manage coordination between nodes, jobs, and executions more effectively.

• Endpoint Prefix: All APIs are versioned and prefixed with /api/v1.

• Default Port: By default, Bacalhau listens on port 1234.

API endpoints

Orchestrator

The Majority of Bacalhau’s functionality is channeled through the Orchestrator endpoint and its operations. It handles user requests and schedules and it is critical for creating, managing, monitoring, and analyzing jobs within Bacalhau. It also provides mechanisms to query information about the nodes in the cluster.

api/v1/orchestrator/

Here’s the job submission format, where you can tag a YAML file with the job specifications or input the commands with your CLI

# Submit a job
curl -X PUT \
     -H "Content-Type: application/json" \
     -d '{
          "Job": {
            "Name": "test-job",
            "Namespace": "default",
            "Type": "batch",
            "Count": 1,
            "Labels": {
              "foo": "bar",
              "env": "dev"
            },
            "Tasks": [
              {
                "Name": "task1",
                "Engine": {
                  "Type": "docker",
                  "Params": {
                    "Image": "ubuntu:latest",
                    "Entrypoint": [
                      "echo",
                      "hello"
                    ]
                  }
                },
                "Publisher": {
                  "Type": "noop",
                  "Params": {}
                },
                "ResourcesConfig": {
                  "CPU": "0.1",
                  "Memory": "10mb"
                },
                "Network": {
                  "Type": "None"
                },
                "Timeouts": {
                  "ExecutionTimeout": 30
                }
              }
            ]
          }
        }' \
     http://0.0.0.0:20000/api/v1/orchestrator/jobs

{"JobID":"28c08f7f-6fb0-48ed-912d-a2cb6c3a4f3a","EvaluationID":"996b12e4-bcc5-4d74-ac21-0c421dafb7de"}

Agent

This endpoint offers a convenient route to collate detailed information about the Bacalhau node you're interacting with, whether it's acting as the orchestrator or a compute node. It provides you with insights into the node's health, capabilities, and the deployed Bacalhau version.

api/v1/agent/node

Here’s the command structure for querying your current node. You can check on its status and collate information on its health and capabilities:

# Is alive
curl 0.0.0.0:20000/api/v1/agent/alive

Features

Pagination

To handle large datasets, Bacalhau supports pagination. Users can define the limit in their request and then utilize the next_token from the response to fetch subsequent data chunks.

Ordering

To sort the results of list-based queries, use the order_by parameter. By default, the list will be sorted in ascending order. If you want to reverse it, use the reverse parameter. Note that the fields available for sorting might vary depending on the specific API endpoint.

Pretty JSON Output

By default, Bacalhau's APIs provide a minimized JSON response. If you want to view the output in a more readable format, append pretty to the query string.

HTTP Methods

Being RESTful in nature, Bacalhau's API endpoints rely on standard HTTP methods to perform various actions:

1. GET: Fetch data.

2. PUT: Update or create data.

3. DELETE: Remove data.

The behavior of an API depends on its HTTP method. For example, /api/v1/orchestrator/jobs:

1. GET: Lists all jobs.

2. PUT: Submits a new job.

3. DELETE: Stops a job.

HTTP Response Codes

Understanding HTTP response codes is crucial. A 2xx series indicates a successful operation, 4xx indicates client-side errors, and 5xx points to server-side issues. Always refer to the message accompanying the code for more information.

Description

The bacalhau config list command lists the configuration keys and values of the bacalhau node. This command is useful for understanding how configuration keys map to their respective values, aiding in the use of the bacalhau config set command.

Note: Configuration values displayed by this command represent the settings that will be applied when the bacalhau node is next restarted. It is important to note that these values may not reflect the current operational configuration of an active bacalhau node. The displayed configuration is relevant and accurate for a node that is either not currently running or that has been restarted after the execution of this command.

Usage

bacalhau config list [flags]

Flags

• -h, --help:

  • Description: Displays help information for the list sub-command.

• --hide-header:

  • Description: Do not print the column headers when displaying the results.

  • Default: false

• --no-style:

  • Description: Removes all styling from the table output, displaying raw data.

  • Default: false

• --output format:

  • Description: Determines the format in which the output is displayed. Available formats include Table, JSON, and YAML.

  • Options: json, yaml, table

  • Default: table

• --pretty:

  • Description: Formats the output for enhanced readability. This flag is relevant only when using JSON or YAML output formats.

  • Default: true

• --wide:

  • Description: Prints full values in the table results without truncating any information.

  • Default: false

Examples

Listing the Bacalhau nodes configuration settings

1. Basic Usage:

   Command:

   Copy

   bacalhau config list

   Output:

   Copy

    KEY          VALUE
    <key_name> <key_value>
   ...

2. Output in JSON format:

   Command:

   Copy

   bacalhau config list --output json --pretty

   Output:

   Copy

   [
     {
       "Key": "<key_name>",
       "Value": <key_value>
     },
     ...
   ]

Description:

The bacalhau config set command sets a value in the bacalhau node's configuration file. This command is used to modify the configuration file that the bacalhau node will reference for its settings. Key names in the configuration are case-insensitive. Additionally, the command validates the value being set based on the type of the configuration key, ensuring that only appropriate and valid configurations are applied.

Note: Changes made using this command will be applied to the configuration file, but they do not immediately affect the running configuration of an active bacalhau node. The modifications will take effect only after the node is restarted.

Usage

bacalhau config set <key> <value>

Flags

• -h, --help:

  • Description: Displays help information for the set sub-command.

Examples

Configuring the Server API Port Value

bacalhau config set node.serverapi.port 9999

bacalhau config list | grep serverapi.port
 node.serverapi.port                                             9999

cat ~/.bacalhau/config.yaml
node:
    serverapi:
        port: 9999

Configuring the Logging Mode Value

bacalhau config set node.loggingmode json

bacalhau config list | grep loggingmode
 node.loggingmode                                                json

cat ~/.bacalhau/config.yaml
node:
    loggingmode: json

Multiple Set commands append to the file

bacalhau config set node.serverapi.port 9999
bacalhau config set node.serverapi.host 0.0.0.0
bacalhau config set node.loggingmode json

cat ~/.bacalhau/config.yaml
node:
    loggingmode: json
    serverapi:
        host: 0.0.0.0
        port: 9999

Set command value validation

Example of invalid logging mode value

bacalhau config set node.loggingmode some-invalid-value

Error: setting "node.loggingmode": "some-invalid-value" is an invalid log-mode (valid modes: ["default" "station" "json" "combined" "event"])

Example of invalid time duration value

bacalhau config set node.volumesizerequesttimeout 10days

Error: setting "node.volumesizerequesttimeout": time: unknown unit "days" in duration "10days"

Describe Node

Endpoint: GET /api/v1/orchestrator/nodes/:nodeID

Retrieve information about a specific node.

Parameters:

1. :nodeID: Identifier of the node to describe. (e.g. QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT)

Response:

• Node: Detailed information about the requested node.

Example:

curl 127.0.0.1:1234/api/v1/orchestrator/nodes/QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT
{
  "Node": {
    "PeerInfo": {
      "ID": "QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT",
      "Addrs": [
        "/ip4/34.34.247.247/tcp/1235"
      ]
    },
    "NodeType": "Compute",
    "Labels": {
      "Architecture": "amd64",
      "Operating-System": "linux",
      "git-lfs": "True",
      "owner": "bacalhau"
    },
    "ComputeNodeInfo": {
      "ExecutionEngines": [
        "docker",
        "wasm"
      ],
      "Publishers": [
        "s3",
        "noop",
        "ipfs"
      ],
      "StorageSources": [
        "urldownload",
        "inline",
        "repoclone",
        "repoclonelfs",
        "s3",
        "ipfs"
      ],
      "MaxCapacity": {
        "CPU": 3.2,
        "Memory": 12561049190,
        "Disk": 582010404864,
        "GPU": 1
      },
      "AvailableCapacity": {
        "CPU": 3.2,
        "Memory": 12561049190,
        "Disk": 582010404864,
        "GPU": 1
      },
      "MaxJobRequirements": {
        "CPU": 3.2,
        "Memory": 12561049190,
        "Disk": 582010404864,
        "GPU": 1
      },
      "RunningExecutions": 0,
      "EnqueuedExecutions": 0
    },
    "BacalhauVersion": {
      "Major": "1",
      "Minor": "1",
      "GitVersion": "v1.1.0",
      "GitCommit": "970e1a0f23c7eb739a097aa8212f7964434bcd97",
      "BuildDate": "2023-09-25T07:59:00Z",
      "GOOS": "linux",
      "GOARCH": "amd64"
    }
  }
}

List Nodes

Endpoint: GET /api/v1/orchestrator/nodes

Retrieve a list of nodes.

Parameters:

2. limit: Set the maximum number of jobs to return. Default is set to 10.

3. next_token: Utilize this parameter for pagination continuation.

4. order_by: Determine the ordering of jobs. Choose between id, type, available_cpu, available_memory, available_disk or available_gpu. (default is id).

5. reverse: Opt to reverse the default order of displayed jobs.

Response:

• Nodes: List of matching nodes.

• NextToken (string): Pagination token.

Example:

Find two linux nodes with most available Memory

curl --get  "127.0.0.1:1234/api/v1/orchestrator/nodes?limit=2&order_by=available_memory" --data-urlencode 'labels=Operating-System=linux'
{
  "NextToken": "",
  "Nodes": [
    {
      "PeerInfo": {
        "ID": "QmcC3xifiiCuGGQ9rpvefUoary9tY65x2HaNxSdeMTvM9U",
        "Addrs": [
          "/ip4/212.248.248.248/tcp/1235"
        ]
      },
      "NodeType": "Compute",
      "Labels": {
        "Architecture": "amd64",
        "Operating-System": "linux",
        "env": "prod",
        "git-lfs": "False",
        "name": "saturnia_len20"
      },
      "ComputeNodeInfo": {
        "ExecutionEngines": [
          "wasm",
          "docker"
        ],
        "Publishers": [
          "noop",
          "ipfs"
        ],
        "StorageSources": [
          "urldownload",
          "inline",
          "ipfs"
        ],
        "MaxCapacity": {
          "CPU": 102,
          "Memory": 858993459200,
          "Disk": 562967789568,
          "GPU": 2
        },
        "AvailableCapacity": {
          "CPU": 102,
          "Memory": 858993459200,
          "Disk": 562967789568,
          "GPU": 2
        },
        "MaxJobRequirements": {
          "CPU": 96,
          "Memory": 858993459200,
          "Disk": 562967789568,
          "GPU": 2
        },
        "RunningExecutions": 0,
        "EnqueuedExecutions": 0
      },
      "BacalhauVersion": {
        "Major": "1",
        "Minor": "1",
        "GitVersion": "v1.1.0",
        "GitCommit": "970e1a0f23c7eb739a097aa8212f7964434bcd97",
        "BuildDate": "2023-09-25T07:59:00Z",
        "GOOS": "linux",
        "GOARCH": "amd64"
      }
    },
    {
      "PeerInfo": {
        "ID": "QmXaXu9N5GNetatsvwnTfQqNtSeKAD6uCmarbh3LMRYAcF",
        "Addrs": [
          "/ip4/35.245.245.245/tcp/1235"
        ]
      },
      "NodeType": "Compute",
      "Labels": {
        "Architecture": "amd64",
        "Operating-System": "linux",
        "git-lfs": "True",
        "owner": "bacalhau"
      },
      "ComputeNodeInfo": {
        "ExecutionEngines": [
          "docker",
          "wasm"
        ],
        "Publishers": [
          "noop",
          "ipfs",
          "s3"
        ],
        "StorageSources": [
          "s3",
          "ipfs",
          "urldownload",
          "inline",
          "repoclone",
          "repoclonelfs"
        ],
        "MaxCapacity": {
          "CPU": 12.8,
          "Memory": 53931124326,
          "Disk": 718749414195,
          "GPU": 0
        },
        "AvailableCapacity": {
          "CPU": 12.8,
          "Memory": 53931124326,
          "Disk": 718749414195,
          "GPU": 0
        },
        "MaxJobRequirements": {
          "CPU": 12.8,
          "Memory": 53931124326,
          "Disk": 718749414195,
          "GPU": 0
        },
        "RunningExecutions": 0,
        "EnqueuedExecutions": 0
      },
      "BacalhauVersion": {
        "Major": "1",
        "Minor": "1",
        "GitVersion": "v1.1.0",
        "GitCommit": "970e1a0f23c7eb739a097aa8212f7964434bcd97",
        "BuildDate": "2023-09-25T07:59:00Z",
        "GOOS": "linux",
        "GOARCH": "amd64"
      }
    }
  ]
}

Describe Job

Endpoint: GET /api/v1/orchestrator/jobs/:jobID

Retrieve the specification and current status of a particular job.

Parameters:

1. jobID: Identifier of the job to describe. This can be full ID of the job (e.g. j-28c08f7f-6fb0-48ed-912d-a2cb6c3a4f3a) or just the short format (e.g. j-28c08f7f) if it's unique.

Response:

Job: Specification for the requested .

Example:

curl 127.0.0.1:1234/api/v1/orchestrator/jobs/j-d586d2cc-6fc9-42c4-9dd9-a78df1d7cd01
{
  "Job": {
    "ID": "j-d586d2cc-6fc9-42c4-9dd9-a78df1d7cd01",
    "Name": "A sample job",
    "Namespace": "default",
    "Type": "batch",
    "Priority": 0,
    "Count": 1,
    "Constraints": [],
    "Meta": {
      "bacalhau.org/requester.id": "QmdZQ7ZbhnvWY1J12XYKGHApJ6aufKyLNSvf8jZBrBaAVL",
      "bacalhau.org/requester.publicKey": "CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDVRKPgCfY2fgfrkHkFjeWcqno+MDpmp8DgVaY672BqJl/dZFNU9lBg2P8Znh8OTtHPPBUBk566vU3KchjW7m3uK4OudXrYEfSfEPnCGmL6GuLiZjLf+eXGEez7qPaoYqo06gD8ROdD8VVse27E96LlrpD1xKshHhqQTxKoq1y6Rx4DpbkSt966BumovWJ70w+Nt9ZkPPydRCxVnyWS1khECFQxp5Ep3NbbKtxHNX5HeULzXN5q0EQO39UN6iBhiI34eZkH7PoAm3Vk5xns//FjTAvQw6wZUu8LwvZTaihs+upx2zZysq6CEBKoeNZqed9+Tf+qHow0P5pxmiu+or+DAgMBAAE="
    },
    "Labels": {
      "env": "prod",
      "name": "demo"
    },
    "Tasks": [
      {
        "Name": "main",
        "Engine": {
          "Type": "docker",
          "Params": {
            "Entrypoint": [
              "/bin/bash"
            ],
            "Image": "ubuntu:latest",
            "Parameters": [
              "-c",
              "echo hello world"
            ]
          }
        },
        "Publisher": {
          "Type": "",
          "Params": {}
        },
        "Env": {},
        "Meta": {},
        "InputSources": [],
        "ResultPaths": [],
        "Resources": {
          "CPU": "",
          "Memory": "",
          "Disk": "",
          "GPU": ""
        },
        "Network": {
          "Type": "None"
        },
        "Timeouts": {
          "ExecutionTimeout": 1800
        }
      }
    ],
    "State": {
      "StateType": "Completed",
      "Message": ""
    },
    "Version": 0,
    "Revision": 2,
    "CreateTime": 1695883778909107178,
    "ModifyTime": 1695883779369191994
  }
}

List Jobs

Endpoint: GET /api/v1/orchestrator/jobs

Retrieve a list of jobs.

Parameters:

1. namespace: Specify a namespace to filter the jobs. Use * to display jobs from all namespaces.

3. limit: Set the maximum number of jobs to return. Default is set to 10.

4. next_token: Utilize this parameter for pagination continuation.

5. order_by: Determine the ordering of jobs. Choose between id or create_time (default is create_time).

6. reverse: Opt to reverse the default order of displayed jobs.

Response:

• NextToken (string): Pagination token.

Example:

List jobs with limit set to 3:

curl 127.0.0.1:1234/api/v1/orchestrator/jobs?limit=3
{
  "Jobs": [
    {
      "ID": "j-f6331e9a-727d-4175-8350-095b6b372408",
      # ...
    },
    {
      "ID": "j-f7853204-a553-4991-a1a3-816b88fdbfc7",
      # ...
    },
    {
      "ID": "j-f791ad14-af5b-4c26-8c93-15cc23dca811",
      # ...
    }
  ],
  "NextToken": ""
}

List with label filtering

curl --get 127.0.0.1:1234/api/v1/orchestrator/jobs --data-urlencode 'labels=env in (prod,dev)'

Create Job

Endpoint: PUT /api/v1/orchestrator/jobs

Submit a new job for execution.

Request Body:

Response:

1. JobID (string): Identifier for the new job.

2. EvaluationID (string): Identifier for the evaluation to schedule the job.

3. Warnings (string[]): Any warnings during job submission.

Example:

curl -X PUT \
     -H "Content-Type: application/json" \
     -d '{
          "Job": {
            "Name": "test-job",
            "Type": "batch",
            "Count": 1,
            "Labels": {
              "foo": "bar",
              "env": "dev"
            },
            "Tasks": [
              {
                "Name": "task1",
                "Engine": {
                  "Type": "docker",
                  "Params": {
                    "Image": "ubuntu:latest",
                    "Entrypoint": [
                      "echo",
                      "hello"
                    ]
                  }
                },
                "Publisher": {
                  "Type": "noop"
                }
              }
            ],
            "CreateTime": 1234
          }
        }' \
     127.0.0.1:1234/api/v1/orchestrator/jobs

 {
  "JobID": "j-9809ae4b-d4fa-47c6-823b-86c924e60604",
  "EvaluationID": "5dac9fe0-2358-4ec7-bec9-6747dfa2b33e",
  "Warnings": [
    "job create time is ignored when submitting a job"
  ]
}

Stop Job

Endpoint: DELETE /api/v1/orchestrator/jobs/:jobID

Terminate a specific job asynchronously.

Parameters:

1. :jobID: Identifier of the job to describe. This can be full ID of the job (e.g. j-28c08f7f-6fb0-48ed-912d-a2cb6c3a4f3a) or just the short format (e.g. j-28c08f7f) if it's unique.

2. reason: A message for debugging and traceability.

Response:

• EvaluationID (string): Identifier for the evaluation to stop the job.

Example:

curl -X DELETE 127.0.0.1:1234/api/v1/orchestrator/jobs/j-50ee38d5-2812-4365-aceb-7b47b8f3858e
{
  "EvaluationID": "1316fdfe-97c4-43bc-8e0b-50a7f02f18bb"
}

Job History

Endpoint: GET /api/v1/orchestrator/jobs/:jobID/history

Retrieve historical events for a specific job.

Parameters:

1. since: Timestamp to start (default: 0).

2. event_type: Filter by event type: job, execution, or all (default).

3. execution_id: Filter by execution ID.

4. node_id: Filter by node ID.

5. limit: Maximum events to return.

6. next_token: For pagination.

Response:

• History: List of matching historical events.

• NextToken (string): Pagination token.

Example:

List events for a specific execution

curl 127.0.0.1:1234/api/v1/orchestrator/jobs/j-4cd1566f-84cb-4830-a96b-1349f5b54b1b/history\?execution_id=e-82f7813f-58da-4323-8261-886af35284c4
{
  "NextToken": "",
  "History": [
    {
      "Type": "ExecutionLevel",
      "JobID": "j-4cd1566f-84cb-4830-a96b-1349f5b54b1b",
      "NodeID": "QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT",
      "ExecutionID": "e-82f7813f-58da-4323-8261-886af35284c4",
      "JobState": null,
      "ExecutionState": {
        "Previous": 1,
        "New": 1
      },
      "NewRevision": 1,
      "Comment": "",
      "Time": "2023-09-28T07:23:01.352803607Z"
    },
    {
      "Type": "ExecutionLevel",
      "JobID": "j-4cd1566f-84cb-4830-a96b-1349f5b54b1b",
      "NodeID": "QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT",
      "ExecutionID": "e-82f7813f-58da-4323-8261-886af35284c4",
      "JobState": null,
      "ExecutionState": {
        "Previous": 1,
        "New": 2
      },
      "NewRevision": 2,
      "Comment": "",
      "Time": "2023-09-28T07:23:01.446196661Z"
    },
    {
      "Type": "ExecutionLevel",
      "JobID": "j-4cd1566f-84cb-4830-a96b-1349f5b54b1b",
      "NodeID": "QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT",
      "ExecutionID": "e-82f7813f-58da-4323-8261-886af35284c4",
      "JobState": null,
      "ExecutionState": {
        "Previous": 2,
        "New": 3
      },
      "NewRevision": 3,
      "Comment": "",
      "Time": "2023-09-28T07:23:01.604862596Z"
    },
    {
      "Type": "ExecutionLevel",
      "JobID": "j-4cd1566f-84cb-4830-a96b-1349f5b54b1b",
      "NodeID": "QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT",
      "ExecutionID": "e-82f7813f-58da-4323-8261-886af35284c4",
      "JobState": null,
      "ExecutionState": {
        "Previous": 3,
        "New": 3
      },
      "NewRevision": 4,
      "Comment": "",
      "Time": "2023-09-28T07:23:01.611816334Z"
    },
    {
      "Type": "ExecutionLevel",
      "JobID": "j-4cd1566f-84cb-4830-a96b-1349f5b54b1b",
      "NodeID": "QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT",
      "ExecutionID": "e-82f7813f-58da-4323-8261-886af35284c4",
      "JobState": null,
      "ExecutionState": {
        "Previous": 3,
        "New": 5
      },
      "NewRevision": 5,
      "Comment": "",
      "Time": "2023-09-28T07:23:01.705013737Z"
    },
    {
      "Type": "ExecutionLevel",
      "JobID": "j-4cd1566f-84cb-4830-a96b-1349f5b54b1b",
      "NodeID": "QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT",
      "ExecutionID": "e-82f7813f-58da-4323-8261-886af35284c4",
      "JobState": null,
      "ExecutionState": {
        "Previous": 5,
        "New": 7
      },
      "NewRevision": 6,
      "Comment": "",
      "Time": "2023-09-28T07:23:02.483265228Z"
    }
  ]
}

Job Executions

Endpoint: GET /api/v1/orchestrator/jobs/:jobID/executions

Retrieve all executions for a particular job.

Parameters:

1. limit: Maximum executions to return.

2. next_token: For pagination.

3. order_by: Order by modify_time (default), create_time, id, state.

4. reverse: Reverse the order.

Response:

• Executions: List of relevant executions.

• NextToken (string): Pagination token.

Example

List executions for a batch job with 3 executions (i.e. count=3)

curl 127.0.0.1:1234/api/v1/orchestrator/jobs/j-412c34b4-da77-4a46-886c-76e03615a04e/executions
{
  "NextToken": "",
  "Executions": [
    {
      "ID": "e-cdd9fb3e-3183-4069-8bc9-679b6bcce4db",
      "Namespace": "default",
      "EvalID": "",
      "Name": "",
      "NodeID": "QmYgxZiySj3MRkwLSL4X2MF5F9f2PMhAE3LV49XkfNL1o3",
      "JobID": "j-412c34b4-da77-4a46-886c-76e03615a04e",
      "AllocatedResources": {
        "Tasks": {}
      },
      "DesiredState": {
        "StateType": 2,
        "Message": "execution completed"
      },
      "ComputeState": {
        "StateType": 7,
        "Message": ""
      },
      "PublishedResult": {
        "Type": "",
        "Params": null
      },
      "RunOutput": {
        "stdout": "hello world\n",
        "stdouttruncated": false,
        "stderr": "",
        "stderrtruncated": false,
        "exitCode": 0,
        "runnerError": ""
      },
      "PreviousExecution": "",
      "NextExecution": "",
      "FollowupEvalID": "",
      "Revision": 6,
      "CreateTime": 1695886565851709698,
      "ModifyTime": 1695886566370340241
    },
    {
      "ID": "e-836a4a50-f6cd-479f-a20d-2a12ff7fea64",
      "Namespace": "default",
      "EvalID": "",
      "Name": "",
      "NodeID": "QmXaXu9N5GNetatsvwnTfQqNtSeKAD6uCmarbh3LMRYAcF",
      "JobID": "j-412c34b4-da77-4a46-886c-76e03615a04e",
      "AllocatedResources": {
        "Tasks": {}
      },
      "DesiredState": {
        "StateType": 2,
        "Message": "execution completed"
      },
      "ComputeState": {
        "StateType": 7,
        "Message": ""
      },
      "PublishedResult": {
        "Type": "",
        "Params": null
      },
      "RunOutput": {
        "stdout": "hello world\n",
        "stdouttruncated": false,
        "stderr": "",
        "stderrtruncated": false,
        "exitCode": 0,
        "runnerError": ""
      },
      "PreviousExecution": "",
      "NextExecution": "",
      "FollowupEvalID": "",
      "Revision": 6,
      "CreateTime": 1695886565855906980,
      "ModifyTime": 1695886566505560693
    },
    {
      "ID": "e-b7e7adc7-b28c-4af0-9002-a7fdce303634",
      "Namespace": "default",
      "EvalID": "",
      "Name": "",
      "NodeID": "QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT",
      "JobID": "j-412c34b4-da77-4a46-886c-76e03615a04e",
      "AllocatedResources": {
        "Tasks": {}
      },
      "DesiredState": {
        "StateType": 2,
        "Message": "execution completed"
      },
      "ComputeState": {
        "StateType": 7,
        "Message": ""
      },
      "PublishedResult": {
        "Type": "",
        "Params": null
      },
      "RunOutput": {
        "stdout": "hello world\n",
        "stdouttruncated": false,
        "stderr": "",
        "stderrtruncated": false,
        "exitCode": 0,
        "runnerError": ""
      },
      "PreviousExecution": "",
      "NextExecution": "",
      "FollowupEvalID": "",
      "Revision": 6,
      "CreateTime": 1695886565853878926,
      "ModifyTime": 1695886566583711985
    }
  ]
}

Job Results

Endpoint: GET /api/v1/orchestrator/jobs/:jobID/results

Fetch results published by all executions for the defined job. Applicable only for batch and ops jobs.

Response:

• Results: List of all published results.

• NextToken (string): Pagination token.

Example:

curl 127.0.0.1:1234/api/v1/orchestrator/jobs/j-479d160f-f9ab-4e32-aec9-a45554126450/results
{
  "NextToken": "",
  "Results": [
    {
      "Type": "s3",
      "Params": {
        "Bucket": "bacalhau-test-datasets",
        "Key": "my-prefix/my-result-file.tar.gz",
        "Region": "eu-west-1",
        "ChecksumSHA256": "qKAFvkLvSc+QqHE4hFiy4qVEmXhr423lQaRBfJecsgo=",
        "VersionID": "bNS92VdFudVI7NPsXF51Qn.RPw31TKNG"
      }
    }
  ]
}

The Labels block within a Job specification plays a crucial role in Bacalhau, serving as a mechanism for filtering jobs. By attaching specific labels to jobs, users can quickly and effectively filter and manage jobs via both the Command Line Interface (CLI) and Application Programming Interface (API) based on various criteria.

Labels Parameters

Labels are essentially key-value pairs attached to jobs, allowing for detailed categorizations and filtrations. Each label consists of a Key and a Value. These labels can be filtered using operators to pinpoint specific jobs fitting certain criteria.

Filtering Operators

Jobs can be filtered using the following operators:

1. in: Checks if the key's value matches any within a specified list of values.

2. notin: Validates that the key's value isn’t within a provided list of values.

3. exists: Checks for the presence of a specified key, regardless of its value.

4. !: Validates the absence of a specified key. (i.e., DoesNotExist)

5. gt: Checks if the key's value is greater than a specified value.

6. lt: Checks if the key's value is less than a specified value.

7. = & ==: Used for exact match comparisons between the key’s value and a specified value.

8. !=: Validates that the key’s value doesn't match a specified value.

Example Usage

Filter jobs with a label whose key is "environment" and value is "development":

bacalhau job list --labels 'environment=development'

Filter jobs with a label whose key is "version" and value is greater than "2.0":

bacalhau job list --labels 'version gt 2.0'

Filter jobs with a label "project" existing:

bacalhau job list --labels 'project'

Filter jobs without a "project" label:

bacalhau job list --labels '!project'

Practical Applications

• Job Management: Enables efficient management of jobs by categorizing them based on distinct attributes or criteria.

• Automation: Facilitates the automation of job deployment and management processes by allowing scripts and tools to target specific categories of jobs.

• Monitoring & Analytics: Enhances monitoring and analytics by grouping jobs into meaningful categories, allowing for detailed insights and analysis.