Runs REST API - Wherobots

This document provides a comprehensive overview of the Wherobots Runs REST API, a programming interface for managing job runs. If you are orchestrating jobs with Apache Airflow, an open-source platform for scheduling and managing complex data workflows, we recommend using the WherobotsRunOperator. For more information on this constructor, see the documentation for the WherobotsRunOperator

Track your Job Runs in Wherobots CloudUsage History in Wherobots Cloud provides insights into job progress, resource consumption, execution timelines, and configuration settings. For more information, see Usage History.

Before you start

Before using the Runs REST API, ensure that you have the following required resources:

A Wherobots account. For more information, see Create a Wherobots Account.
A Wherobots API key. For more information on creating and accessing API keys see API Keys.

HTTP Request

Example

curl -X 'POST' \
 'https://api.cloud.wherobots.com/runs' \
  -H 'accept: application/json' \
  -H 'X-API-Key: YOUR-API-KEY' \
  -H 'Content-Type: application/json' \
  -d 'input json'

In line 4, replace YOUR-API-KEY with a Wherobots API key.

Response codes

The following table details a comprehensive listing of response codes encountered during a run’s execution.

Response code	Definition
200	Successful response
201	Run created (Create Run)
400	Bad Request
401	Unauthorized
403	Forbidden
404	Not found
409	Conflict
422	Validation Error
500	Internal Server Error

Run statuses

A run’s status field has one of the following values: PENDING, RUNNING, COMPLETED, FAILED, CANCELLED, TIMED_OUT

Create Run

The Create Run endpoint allows you to create a new run in Wherobots. A request needs a name and exactly one entrypoint: runPython to run a Python file, or runJar to run a JAR file. A request with neither entrypoint, or with both, returns 422. Optional parameters set the runtime, the timeout, and the runtime environment. Route: POST https://api.cloud.wherobots.com/runs To choose the compute region, add the region query parameter, for example POST https://api.cloud.wherobots.com/runs?region=aws-us-west-2. region is a query parameter; don’t put it in the request body.

Request parameters

The following parameters are associated with the Create run endpoint. Query parameters:

Parameter	Type	Required or Optional	Parameter description	Rules for Accepted Values
`region`	`str`	Optional	Compute region where the Run should be created. When omitted, your Organization’s default region is used.	See Cloud and Cloud Region Availability for valid values.

Request body schema:

Parameter	Type	Required or Optional	Parameter description	Rules for Accepted Values
`name`	`str`	Required	The user-assigned name of the run.	8 to 255 characters. Letters, digits, `_`, `-`, and `.` are allowed.
`runtime`	`str`	Optional	Wherobots runtime. When omitted, a default runtime is assigned.	`micro`, `tiny`, `small`, `medium`, `large`, `x-large`, `2x-large`, `4x-large`, `medium-himem`, `large-himem`, `x-large-himem`, `2x-large-himem`, `4x-large-himem`, `x-large-hicpu`, `2x-large-hicpu` Which runtimes you can use depends on your Organization’s edition and the region. See Runtimes.
`version`	`str`	Optional	String literals that map to a specific Wherobots version. `“latest”` refers to the most recent, stable production version available. `”preview”` refers to the most recent pre-release or testing version available. Defaults to `“latest”` You can find additional available versions in Environment Presets.	`latest`, `preview`
`runPython`	`dict`	Required unless `runJar` is set	Used to run a Python file.	An S3 path to Python file. Wherobots Managed Storage and Storage Integration S3 paths are supported. Takes the following keys: `uri:str` specifying the file path to your script and `args:list[str]` an optional list of arguments to pass to the script. You can use either `runPython` or `runJar`, not both.
`runJar`	`dict`	Required unless `runPython` is set	Used to run a JAR file.	An S3 path to Java Archive (JAR) file. Wherobots Managed Storage and Storage Integration S3 paths are supported. Takes the following keys: `uri:str` specifying the file path to your script and `args:list[str]` an optional list of arguments to pass to the script. You can use either `runPython` or `runJar`, not both.
`timeoutSeconds`	`int`	Optional	The timeout of the run, in seconds. Defaults to `3600`.	If the value is greater than Max Workload Alive Hours multiplied by `3600`, the API server will change this value to the maximum value allowed.
`environment`	`dict`	Optional	The model for runtime environment configs, include spark cluster configs, dependencies. Defaults to .	See Environment keys.

Environment keys

Environment parameter	Type	Required or Optional	Parameter description	Rules for Accepted Values
`sparkDriverDiskGB`	`int`	Optional	The driver disk size of the Spark cluster.
`sparkExecutorDiskGB`	`int`	Optional	The executor disk size of the Spark cluster.
`sparkConfigs`	`dict{str:str}`	Optional	The user specified spark configs.
`dependencies`	`list[dict[str, Any]]`	Optional	Indicates the 3rd party dependencies need to add to the runtime. Defaults to an empty list.	Must be an array (list) of objects (dictionaries). Each object must represent either a Python Package Index or a File Dependency.

Run Python

The following code snippet details a Job Run request that runs a Python file. In practice, replace S3-PATH-TO-FILE with the S3 path of your Python file in Wherobots. Wherobots Managed Storage and Storage Integration S3 paths are supported.

args returns an empty list, [], if uri: "YOUR-S3-PATH" is used in the runPython request without args: list[str].

{
  "runtime": "tiny",
  "name": "test_run",
  "runPython": {
    "uri": "S3-PATH-TO-FILE"
  },
  "timeoutSeconds": 3600
}

Run JAR

The following code snippet details a Jobs run request that runs a JAR file. In practice, replace S3-PATH-TO-FILE with the S3 path of your JAR file in Wherobots. Wherobots Managed Storage and Storage Integration S3 paths are supported.

{
  "runtime": "tiny",
  "name": "test_run",
  "runJar": {
    "uri": "S3-PATH-TO-FILE",
    "mainClass": "main.class"
  },
  "timeoutSeconds": 3600
}

Dependencies

The following details information on the third-party dependencies that can be added to the runtime environment.

Parameter	Type	Details	Rules for Accepted Values
`sourceType`	`str`	Enum for the source type dependency.	Valid values: `PYPI` or `FILE`
`filePath`	`str`	File path to dependency file.	Must be a valid file path accessible within the runtime environment. The file extension must be one of the following: `.jar`, `.whl`, `.zip`, or `.json`.

Python Package Index (PyPI) dependency object

Python Package Index dependency objects are used to represent third-party Python packages that need to be added to the runtime environment.

Parameter	Type	Details	Rules for Accepted Values
`sourceType`	`str`	Source type of the dependency.	Must be set to `PYPI`.
`libraryName`	`str`	Name of the Python library to be installed from PyPI.	Must be a valid PyPI package name (e.g., `"pandas"`, `"requests"`).
`libraryVersion`	`str`	The specific version of the PyPI library to install.	Must be a valid version specifier compatible with PyPI.

File dependency object

The file dependency object allows you to specify a file to be added to the runtime environment.

Parameter	Parameter Type	Details	Rules for Accepted Values
`sourceType`	`str`	Indicates the source type of the dependency.	Must be set to `FILE`.
`filePath`	`str`	The file path to the dependency file.	Must be a valid file path accessible within the runtime environment. The file extension must be one of: `.jar`, `.whl`, `.zip`, or `.json`. JSON files can be used to place a configuration file at a predetermined file-system location on all nodes of the compute cluster (`/opt/wherobots/<file name>.json`).

Dependencies example

{
  "dependencies": [
    {
      "sourceType": "PYPI",
      "libraryName": "pandas",
      "libraryVersion": "1.5.3"
    },
    {
      "sourceType": "FILE",
      "filePath": "/path/to/my_custom_library.jar"
    }
  ]
}

Request example

This is an example of a Jobs run request that runs a Python file.

POST /runs?region=aws-us-west-2

{
  "runtime": "tiny",
  "name": "string",
  "runPython": {
    "uri": "string",
    "args": []
  },
  "timeoutSeconds": 3600,
  "environment": {
    "sparkConfigs": {},
    "dependencies": []
  }
}

This is an example of a Jobs run request that runs a JAR file.

POST /runs?region=aws-us-west-2

{
  "runtime": "tiny",
  "name": "string",
  "runJar": {
    "uri": "string",
    "args": [],
    "mainClass": "string"
  },
  "timeoutSeconds": 3600,
  "environment": {
    "sparkConfigs": {},
    "dependencies": []
  }
}

Successful response example

The following is an example of a successful response. A successful response returns the 201 status code.

{
  "id": "string",
  "createTime": "2024-09-25T02:23:34.646Z",
  "updateTime": "2024-09-25T02:23:34.646Z",
  "name": "string",
  "status": "PENDING",
  "triggeredBy": {
    "id": "string",
    "createTime": "2024-09-25T02:23:34.646Z",
    "updateTime": "2024-09-25T02:23:34.646Z",
    "email": "string",
    "name": "string",
    "roles": [ ... ]
  },
  "startTime": "2024-09-25T02:23:34.646Z",
  "completeTime": "2024-09-25T02:23:34.646Z",
  "kubeApp": {
    "id": "string",
    "createTime": "2024-09-25T02:23:34.646Z",
    "updateTime": "2024-09-25T02:23:34.646Z",
    "instanceId": "string",
    "appType": "string",
    "status": "PENDING",
    "createdById": "string",
    "apiPayload": {},
    "appMeta": {},
    "events": [],
    "message": "string"
  },
  "payload": {
    "runtime": "tiny",
    "name": "string",
    "runPython": {
      "uri": "string",
      "args": []
    },
    "runJar": {
      "uri": "string",
      "args": [],
      "mainClass": "string"
    },
    "timeoutSeconds": 3600,
    "environment": {
      "sparkConfigs": {},
      "dependencies": []
    }
  }
}

Get Run

The Get Run endpoint allows you to get the details of a Wherobots Job Run. In practice, replace <RUN_ID>, with the ID of an existing run, which can be found by querying List Runs. Route: GET https://api.cloud.wherobots.com/runs/<RUN_ID>

Successful response example

{
  "id": "string",
  "createTime": "2024-09-25T18:23:01.538Z",
  "updateTime": "2024-09-25T18:23:01.538Z",
  "name": "string",
  "status": "PENDING",
  "triggeredBy": {
    "id": "string",
    "createTime": "2024-09-25T18:23:01.538Z",
    "updateTime": "2024-09-25T18:23:01.538Z",
    "email": "string",
    "name": "string",
    "roles": [ ... ]
  },
  "startTime": "2024-09-25T18:23:01.538Z",
  "completeTime": "2024-09-25T18:23:01.538Z",
  "kubeApp": {
    "id": "string",
    "createTime": "2024-09-25T18:23:01.538Z",
    "updateTime": "2024-09-25T18:23:01.538Z",
    "instanceId": "string",
    "appType": "string",
    "status": "PENDING",
    "createdById": "string",
    "apiPayload": {},
    "appMeta": {},
    "events": [],
    "message": "string"
  },
  "payload": {
    "runtime": "tiny",
    "name": "string",
    "runPython": {
      "uri": "string",
      "args": []
    },
    "runJar": {
      "uri": "string",
      "args": [],
      "mainClass": "string"
    },
    "timeoutSeconds": 3600,
    "environment": {
      "sparkConfigs": {},
      "dependencies": []
    }
  }
}

Cancel Run

The Cancel Run endpoint allows you to cancel a Wherobots Job Run. In practice, replace <RUN_ID>, with the ID of an existing run, which can be found by querying List Runs. Route: POST https://api.cloud.wherobots.com/runs/<RUN_ID>/cancel

Successful response

Returns 200 when request cancels successfully. It can take a minute before the status changes to CANCELLED. Query Get Run to check a Job Run’s status.

Get Run Logs

The Get Run Logs endpoint allows you to see the logs for a Wherobots Job Run. In practice, replace <RUN_ID>, with the ID of an existing run, which can be found by querying List Runs. Route: GET https://api.cloud.wherobots.com/runs/<RUN_ID>/logs

Request query parameters schema

Example: https://api.cloud.wherobots.com/runs/<RUN_ID>/logs?size=100

Parameter	Type	Required or Optional	Parameter description	Rules for Accepted Values
`size`	`int`	Optional	The maximum number of log items fetched for the current page.	Defaults to `100`
`cursor`	`str`	Optional	The position to continue from. Pass the `next_page` value from the previous response to fetch the next page.

Successful response example

{
  "items": [
    {
      "timestamp": "string",
      "raw": "string"
    }
  ],
  "current_page": "string",
  "next_page": "string"
}

List Runs

The List Runs endpoint allows you to see all of the Wherobots runs in your Organization. Route: GET https://api.cloud.wherobots.com/runs

Request parameters

The following query parameters can be provided to refine the search:

Parameter	Type	Required or Optional	Parameter Description	Rules for Accepted Values
`name`	`str`	Optional	Filter the runs by name pattern.	Max length: 255 Can contain any number of letters, numbers, underscores, hyphens, or periods, and can include a single asterisk.
`created_after`	`str`	Optional	Filter for runs that have been created after this date and time.	ISO 8601 date-time, for example `2024-09-25T00:00:00Z`
`status`	`array[str]`	Optional	Filter the runs by status.	Valid values: `PENDING`, `RUNNING`, `COMPLETED`, `FAILED`, `CANCELLED`, `TIMED_OUT`
`region`	`str`	Optional	Filter the runs by the compute region in which they executed.	See Cloud and Cloud Region Availability for valid values.
`cursor`	`str`	Optional	Cursor for the page to fetch. Pass the `next_page` or `previous_page` value from a previous response.
`size`	`int`	Optional	The maximum number of items fetched for the current page.	Defaults to `50`. Maximum: `100` Minimum: `0`

Response fields

Field	Description
`items`	The runs on this page. Each item has the same fields as a Get Run response.
`total`	The total number of runs that match the filters.
`current_page`	Cursor to refetch the current page.
`next_page`	Cursor for the next page.
`previous_page`	Cursor for the previous page.
`current_page_backwards`	Cursor to refetch the current page starting from the last item.

Successful response example

{
  "items": [
    {
      "id": "string",
      "createTime": "2024-09-25T20:47:25.280Z",
      "updateTime": "2024-09-25T20:47:25.280Z",
      "name": "string",
      "status": "PENDING",
      "triggeredBy": {
        "id": "string",
        "createTime": "2024-09-25T20:47:25.280Z",
        "updateTime": "2024-09-25T20:47:25.280Z",
        "email": "string",
        "name": "string",
        "roles": [ ... ]
      },
      "startTime": "2024-09-25T20:47:25.280Z",
      "completeTime": "2024-09-25T20:47:25.280Z",
      "kubeApp": {
        "id": "string",
        "createTime": "2024-09-25T20:47:25.280Z",
        "updateTime": "2024-09-25T20:47:25.280Z",
        "instanceId": "string",
        "appType": "string",
        "status": "PENDING",
        "createdById": "string",
        "apiPayload": {},
        "appMeta": {},
        "events": [],
        "message": "string"
      },
      "payload": {
        "runtime": "tiny",
        "name": "string",
        "runPython": {
          "uri": "string",
          "args": []
        },
        "timeoutSeconds": 3600,
        "environment": {
          "sparkConfigs": {},
          "dependencies": []
        }
      }
    }
  ],
  "total": 0,
  "current_page": "string",
  "current_page_backwards": "string",
  "previous_page": "string",
  "next_page": "string"
}

​Before you start

​HTTP Request

​Response codes

​Run statuses

​Create Run

​Request parameters

​Environment keys

​Run Python

​Run JAR

​Dependencies

Python Package Index (PyPI) dependency object

File dependency object

Dependencies example

​Request example

​Successful response example

​Get Run

​Successful response example

​Cancel Run

​Successful response

​Get Run Logs

​Request query parameters schema

​Successful response example

​List Runs

​Request parameters

​Response fields

​Successful response example

Before you start

HTTP Request

Response codes

Run statuses

Create Run

Request parameters

Environment keys

Run Python

Run JAR

Dependencies

Request example

Successful response example

Get Run

Successful response example

Cancel Run

Successful response

Get Run Logs

Request query parameters schema

Successful response example

List Runs

Request parameters

Response fields

Successful response example