The release v.1.4.0 introduced changes to the API and CLI. For more info check out this page and the release notes.
From v.1.3.2 onwards the HTTP API has been updated and the following endpoints have been migrated:
/api/v1/requester/list
GET /api/v1/orchestrator/jobs
/api/v1/requester/nodes
GET /api/v1/orchestrator/nodes
/api/v1/requester/states
GET /api/v1/orchestrator/jobs/:jobID
/api/v1/requester/results
GET /api/v1/orchestrator/jobs/:jobID
/api/v1/requester/events
GET /api/v1/orchestrator/jobs/:jobID/history
/api/v1/requester/submit
PUT /api/v1/orchestrator/jobs
/api/v1/requester/cancel
DELETE /api/v1/orchestrator/jobs/:jobID
/api/v1/requester/debug
GET /api/v1/orchestrator/nodes/:nodeID
/api/v1/requester/websocket/events
GET /api/v1/orchestrator/jobs/:jobID/history
Orchestrator endpoint handles user requests and schedules and it is critical for creating, managing, monitoring, and analyzing jobs within Bacalhau. It also provides mechanisms to query information about the nodes in the cluster.
This page describes the resources and activities available via the Orchestrator endpoint
Endpoint: GET /api/v1/orchestrator/jobs/:jobID
Retrieve the specification and current status of a particular job.
Parameters:
jobID: Identifier of the job to describe. This can be full ID of the job (e.g. j-28c08f7f-6fb0-48ed-912d-a2cb6c3a4f3a) or just the short format (e.g. j-28c08f7f) if it's unique.
Response:
Job: Specification for the requested job.
Example:
Endpoint: GET /api/v1/orchestrator/jobs
Retrieve a list of jobs.
Parameters:
namespace: Specify a namespace to filter the jobs. Use * to display jobs from all namespaces.
labels: Use label-based criteria to filter jobs. See Label Filtering for usage details.
limit: Set the maximum number of jobs to return. Default is set to 10.
next_token: Utilize this parameter for pagination continuation.
order_by: Determine the ordering of jobs. Choose between id or create_time (default is create_time).
reverse: Opt to reverse the default order of displayed jobs.
Response:
Jobs: List of matching jobs.
NextToken (string): Pagination token.
Example:
List jobs with limit set to 3:
List with label filtering
Endpoint: PUT /api/v1/orchestrator/jobs
Submit a new job for execution.
Request Body:
Job: JSON definition of the job.
Response:
JobID (string): Identifier for the new job.
EvaluationID (string): Identifier for the evaluation to schedule the job.
Warnings (string[]): Any warnings during job submission.
Example:
Endpoint: DELETE /api/v1/orchestrator/jobs/:jobID
Terminate a specific job asynchronously.
Parameters:
:jobID: Identifier of the job to describe. This can be full ID of the job (e.g. j-28c08f7f-6fb0-48ed-912d-a2cb6c3a4f3a) or just the short format (e.g. j-28c08f7f) if it's unique.
reason: A message for debugging and traceability.
Response:
EvaluationID (string): Identifier for the evaluation to stop the job.
Example:
Endpoint: GET /api/v1/orchestrator/jobs/:jobID/history
Retrieve historical events for a specific job.
Parameters:
since: Timestamp to start (default: 0).
event_type: Filter by event type: job, execution, or all (default).
execution_id: Filter by execution ID.
node_id: Filter by node ID.
limit: Maximum events to return.
next_token: For pagination.
Response:
History: List of matching historical events.
NextToken (string): Pagination token.
Example:
List events for a specific execution
Endpoint: GET /api/v1/orchestrator/jobs/:jobID/executions
Retrieve all executions for a particular job.
Parameters:
limit: Maximum executions to return.
next_token: For pagination.
order_by: Order by modify_time (default), create_time, id, state.
reverse: Reverse the order.
Response:
Executions: List of relevant executions.
NextToken (string): Pagination token.
Example
List executions for a batch job with 3 executions (i.e. count=3)
Endpoint: GET /api/v1/orchestrator/jobs/:jobID/results
Fetch results published by all executions for the defined job. Applicable only for batch and ops jobs.
Response:
Results: List of all published results.
NextToken (string): Pagination token.
Example:
Result of a job that used the S3 Publisher:
Endpoint: GET /api/v1/orchestrator/nodes/:nodeID
Retrieve information about a specific node.
Parameters:
:nodeID: Identifier of the node to describe. (e.g. QmUDAXvv31WPZ8U9CzuRTMn9iFGiopGE7rHiah1X8a6PkT)
Response:
Node: Detailed information about the requested node.
Example:
Endpoint: GET /api/v1/orchestrator/nodes
Retrieve a list of nodes.
Parameters:
labels: Use label-based criteria to filter nodes. See Label Filtering for usage details.
limit: Set the maximum number of jobs to return. Default is set to 10.
next_token: Utilize this parameter for pagination continuation.
order_by: Determine the ordering of jobs. Choose between id, type, available_cpu, available_memory, available_disk or available_gpu. (default is id).
reverse: Opt to reverse the default order of displayed jobs.
Response:
Nodes: List of matching nodes.
NextToken (string): Pagination token.
Example:
Find two linux nodes with most available Memory
The API changed in Bacalhau v.1.4.0, which can be seen in the and the .
We’re proud to say that we consider ourselves to be an “API-first” organization, that is, every decision we take to make something new in Bacalhau, we build it out from the API first. Everything else, from the SDK to the CLI derives its interface and operations from it. So, let’s take it for a spin!
When working with Bacalhau Networks, it’s good to have an idea of how many nodes are on the network, and what they’re capable of running. We can get this information from the `/api/v1/orchestrator/nodes` endpoint at the IP address/URL of our Requester node.
This should output something like the following:
But JSON structures aren’t always the best way to visualize complex systems. Fortunately, we’re working with Python! So, we can use Matplotlib to render out our Bacalhau network structure for us!
Run the following install commands in your terminal:
And then append the following code to our previous code:
We’ll get a handy graph showing us all of our nodes, and which Requester Nodes our Compute Nodes are connected to!
Now that we know what nodes we have, what they’re capable of, and how they’re connected to each other, we can start to think about scheduling Jobs.
The simplest Job type that can be executed on a Bacalhau network is a “batch” Job. This is a Job that’s run on the first instance that becomes available to execute it, so no need to worry too much about concurrency or parallelism at this point. To create a Job and execute it via the API, you can run the following code:
Once that request completes, the createJobRespData variable will have a value something like the following:
Now that we’ve submitted a Job, it would probably be helpful to get the results of that execution. And it’s super simple to do so! All we need is to pass the value of the JobID key that we received once we created our job to the `/api/v1/orchestrator/jobs/{job_id}/results` endpoint.
Add the following code to the end of the last block:
You should get something like this:
But wait! Where are our results? Well, when we created our job, we didn’t specify a publisher to send our results to. This doesn’t doesn’t mean that we span one up for nothing, though. The output of each Job is still stored in our network, and we can retrieve those by accessing our Job `executions`.
Retrieving our Job executions is very similar to retrieving our Job results. This time, we hit the `/api/v1/orchestrator/jobs/{job_id}/executions` endpoint instead.
Append the following code to the last block we executed:
And when you run the code again, you should receive something like the following:
This time, we can see that our `Items` key has an array of objects which tells us when our Job was executed, where, and what the output of that Job was.
The code also prints out the results of each execution of the Job along with it’s execution ID:
If we had executed our Job on more than one Node (for instance, if the Job type was an “Ops” or “Service” Job which run on all available Nodes), our code would have output the results for each execution in the same `Items` array.
The Bacalhau Agent APIs provide a convenient means to retrieve information about the Bacalhau node you are communicating with, whether it serves as the orchestrator or functions as a compute node. These APIs offer valuable insights into the node's health, capabilities, and deployed Bacalhau version.
Endpoint: GET /api/v1/agent/alive
This API can be used to determine if the agent is operational and responding as expected.
Response:
Endpoint: GET /api/v1/agent/version
This API provides details about the Bacalhau version, including major and minor version numbers, Git version, Git commit, build date, and platform information.
Response:
Endpoint: GET /api/v1/agent/node
This API provides detailed information about the node, including its peer ID and network addresses, node type (e.g., Compute), labels, compute node capabilities, and the deployed Bacalhau version.
Response:
Note that in version 1.4.0 API logic and endpoints have changed. Check out the release notes and updated API description in the API documentation section.
Welcome to the official API documentation for Bacalhau. This guide provides a detailed insight into Bacalhau's RESTful HTTP APIs and demonstrates how to make the most out of them.
Bacalhau prioritizes an "API-first" design, enabling users to interact with their deployed systems programmatically. In the v1.4.0 the API model was changed to include only two endpoints, focused on orchestrating, querying and managing your network nodes and jobs. Each endpoint has a clear, separate environment and goal, allowing to manage coordination between nodes, jobs, and executions more effectively.
Endpoint Prefix: All APIs are versioned and prefixed with /api/v1.
Default Port: By default, Bacalhau listens on port 1234.
The Majority of Bacalhau’s functionality is channeled through the Orchestrator endpoint and its operations. It handles user requests and schedules and it is critical for creating, managing, monitoring, and analyzing jobs within Bacalhau. It also provides mechanisms to query information about the nodes in the cluster.
Here’s the job submission format, where you can tag a YAML file with the job specifications or input the commands with your CLI
This endpoint offers a convenient route to collate detailed information about the Bacalhau node you're interacting with, whether it's acting as the orchestrator or a compute node. It provides you with insights into the node's health, capabilities, and the deployed Bacalhau version.
Here’s the command structure for querying your current node. You can check on its status and collate information on its health and capabilities:
To handle large datasets, Bacalhau supports pagination. Users can define the limit in their request and then utilize the next_token from the response to fetch subsequent data chunks.
To sort the results of list-based queries, use the order_by parameter. By default, the list will be sorted in ascending order. If you want to reverse it, use the reverse parameter. Note that the fields available for sorting might vary depending on the specific API endpoint.
By default, Bacalhau's APIs provide a minimized JSON response. If you want to view the output in a more readable format, append pretty to the query string.
Being RESTful in nature, Bacalhau's API endpoints rely on standard HTTP methods to perform various actions:
GET: Fetch data.
PUT: Update or create data.
DELETE: Remove data.
The behavior of an API depends on its HTTP method. For example, /api/v1/orchestrator/jobs:
GET: Lists all jobs.
PUT: Submits a new job.
DELETE: Stops a job.
Understanding HTTP response codes is crucial. A 2xx series indicates a successful operation, 4xx indicates client-side errors, and 5xx points to server-side issues. Always refer to the message accompanying the code for more information.
Since /api/v1/requester/* was changed to /api/v1/orchestrator/ in v1.4.0, all /api/v1/requester/* requests will result in 410 error.
Find more information about the or see the .