Job Runs

This document provides a comprehensive overview of the Runs REST API for Wherobots. For a detailed tutorial about using the WherobotsRunOperator constructor, see WherobotsRunOperator.

Track your Job Runs in Wherobots Cloud

In Wherobots Cloud, Job Runs lets monitor the progress of jobs runs, understand resource consumption, review the execution timeline, and verify configuration settings.

Before you start¶

Before using the Runs REST API, ensure that you have the following required resources:

An account within a Professional or Enterprise Edition Organization. For more information, see Create a Wherobots Account.
A Wherobots API key. For more information on creating and accessing API keys see API Keys.

HTTP Request¶

Example
curl -X 'POST' \
 'https://api.cloud.wherobots.com/runs' \
  -H 'accept: application/json' \
  -H 'X-API-Key: YOUR-API-KEY' \
  -H 'Content-Type: application/json' \
  -d 'input json'

In line 4, replace YOUR-API-KEY with a Wherobots API key.

Response codes¶

The following table details a comprehensive listing of response codes encountered during a run's execution.

Response code	Definition
200	Successful response
400	Bad Request
401	Unauthorized
403	Forbidden
404	Not found
422	Validation Error
500	Internal Server Error

Create Run¶

The Create Run endpoint allows you to create a new run in Wherobots, specifying the runtime, name, and optional parameters for running Python or JAR files.

Route: POST http://api.cloud.wherobots.com/runs?region=<region>

Request parameters¶

The following parameters are associated with the Create run endpoint.

Query parameters:

Parameter	Type	Required or Optional	Parameter description	Rules for Accepted Values
`region`	`str`	Required	Compute region where the Run should be created.	Valid values: `aws-eu-west-1`, `aws-us-west-2`

Request body schema:

Parameter	Type	Required or Optional	Parameter description	Rules for Accepted Values
`runtime`	`str`	Optional	Wherobots runtime.	`tiny`, `small`, `medium`, `large`, `x-large`, `2x-large`, `4x-large`, `medium-himem`, `large-himem`, `x-large-himem`, `2x-large-himem`, `4x-large-himem`, `tiny-a10-gpu`, `small-a10-gpu`, `medium-a10-gpu` You can see the runtimes available to your Organization in the Start a Notebook dropdown in Wherobots Cloud.
`name`	`str`	Optional	The user-assigned name of the run. If not specified, a default name will be generated.	8 to 255 character matching `^[a-zA-Z0-9_-.]+$`
`runPython`	`dict`	Optional	Used to run a Python file.	An S3 path to Python file. Wherobots Managed Storage and Storage Integration S3 paths are supported. Takes the following keys: `uri:str` specifying the file path to your script and `args:list[str]` an optional list of arguments to pass to the script. You can use either `runPython` or `runJar`, not both.
`runJar`	`dict`	Optional	Used to run a JAR file.	An S3 path to Java Archive (JAR) file. Wherobots Managed Storage and Storage Integration S3 paths are supported. Takes the following keys: `uri:str` specifying the file path to your script and `args:list[str]` an optional list of arguments to pass to the script. You can use either `runPython` or `runJar`, not both.
`timeoutSeconds`	`int`	Optional	The timeout of the run. Defaults to `3600`.	If the value is greater than Max Workload Alive Hours multiplied by `3600`, the API server will change this value to the maximum value allowed.
`environment`	`dict`	Optional	The model for runtime environment configs, include spark cluster configs, dependencies. Defaults to `{}`.	See Environment keys.

Environment keys¶

Environment parameter	Type	Required or Optional	Parameter description	Rules for Accepted Values
`sparkDriverDiskGB`	`int`	Optional	The driver disk size of the Spark cluster.
`sparkExecutorDiskGB`	`int`	Optional	The executor disk size of the Spark cluster.
`sparkConfigs`	`dict{str:str}`	Optional	The user specified spark configs.
`dependencies`	`list[dict[str, Any]]`	Optional	Indicates the 3^rd party dependencies need to add to the runtime. Required if adding sparkConfigs.	Must be an array (list) of objects (dictionaries). Each object must represent either a Python Package Index or a File Dependency.

Run Python¶

The following code snippet details a Job Run request that runs a Python file. In practice, replace S3-PATH-TO-FILE with the S3 path of your Python file in Wherobots. Wherobots Managed Storage and Storage Integration S3 paths are supported.

Note

args returns an empty list, [], if uri: "YOUR-S3-PATH" is used in the runPython request without args: list[str].

{
  "runtime": "tiny",
  "name": "test_run",
  "runPython": {
    "uri": "S3-PATH-TO-FILE"
  },
  "timeoutSeconds": 3600
}

Run JAR¶

The following code snippet details a Jobs run request that runs a JAR file. In practice, replace S3-PATH-TO-FILE with the S3 path of your JAR file in Wherobots. Wherobots Managed Storage and Storage Integration S3 paths are supported.

{
  "runtime": "tiny",
  "name": "test_run",
  "runJar": {
    "uri": "S3-PATH-TO-FILE",
    "mainClass": "main.class"
  },
  "timeoutSeconds": 3600
}

Dependencies¶

The following details information on the third-party dependencies that can be added to the runtime environment.

Parameter	Type	Details	Rules for Accepted Values
`sourceType`	`str`	Enum for the source type dependency.	Valid values: `PYPI` or `FILE`
`filePath`	`str`	File path to dependency file.	Must be a valid file path accessible within the runtime environment. The file extension must be one of the following: `.jar`, `.whl`, or `.zip`.

Python Package Index (PyPI) dependency object¶

Python Package Index dependency objects are used to represent third-party Python packages that need to be added to the runtime environment.

Parameter	Type	Details	Rules for Accepted Values
`sourceType`	`str`	Source type of the dependency.	Must be set to `PYPI`.
`libraryName`	`str`	Name of the Python library to be installed from PyPI.	Must be a valid PyPI package name (e.g., `"pandas"`, `"requests"`).
`libraryVersion`	`str`	The specific version of the PyPI library to install.	Must be a valid version specifier compatible with PyPI.

File dependency object¶

The file dependency object allows you to specify a file to be added to the runtime environment.

Parameter	Parameter Type	Details	Rules for Accepted Values
`sourceType`	`str`	Indicates the source type of the dependency.	Must be set to `FILE`.
`filePath`	`str`	The file path to the dependency file.	Must be a valid file path accessible within the runtime environment. The file extension must be one of: `.jar`, `.whl`, or `.zip`.

Dependencies example¶

{
  "dependencies": [
    {
      "sourceType": "PYPI",
      "libraryName": "pandas",
      "libraryVersion": "1.5.3"
    },
    {
      "sourceType": "FILE",
      "filePath": "/path/to/my_custom_library.jar"
    }
  ]
}

Request example¶

This is an example of a Jobs run request that runs a Python file.

POST /runs?request=aws-us-west-2

{
  "runtime": "tiny",
  "name": "string",
  "runPython": {
    "uri": "string",
    "args": []
  },
  "timeoutSeconds": 3600,
  "environment": {
    "sparkConfigs": {},
    "dependencies": []
  }
}

This is an example of a Jobs run request that runs a JAR file.

POST /runs?region=aws-us-west-2

{
  "runtime": "tiny",
  "name": "string",
  "runJar": {
    "uri": "string",
    "args": [],
    "mainClass": "string"
  },
  "timeoutSeconds": 3600,
  "environment": {
    "sparkConfigs": {},
    "dependencies": []
  }
}

Successful response example¶

The following is an example of a successful response. A successful response returns the 200 status code.

{
  "id": "string",
  "createTime": "2024-09-25T02:23:34.646Z",
  "updateTime": "2024-09-25T02:23:34.646Z",
  "name": "string",
  "status": "PENDING",
  "triggeredBy": {
    "id": "string",
    "createTime": "2024-09-25T02:23:34.646Z",
    "updateTime": "2024-09-25T02:23:34.646Z",
    "email": "string",
    "name": "string",
    "roles": [ ... ]
  },
  "startTime": "2024-09-25T02:23:34.646Z",
  "completeTime": "2024-09-25T02:23:34.646Z",
  "kubeApp": {
    "id": "string",
    "createTime": "2024-09-25T02:23:34.646Z",
    "updateTime": "2024-09-25T02:23:34.646Z",
    "instanceId": "string",
    "appType": "string",
    "status": "PENDING",
    "createdById": "string",
    "apiPayload": {},
    "appMeta": {},
    "events": [],
    "message": "string"
  },
  "payload": {
    "runtime": "tiny",
    "name": "string",
    "runPython": {
      "uri": "string",
      "args": []
    },
    "runJar": {
      "uri": "string",
      "args": [],
      "mainClass": "string"
    },
    "timeoutSeconds": 3600,
    "environment": {
      "sparkConfigs": {},
      "dependencies": []
    }
  }
}

Get Run¶

The Get Run endpoint allows you to get the details of a Wherobots Job Run. In practice, replace <RUN_ID>, with the ID of an existing run, which can be found by querying List Runs.

Route: GET http://api.cloud.wherobots.com/runs/<RUN_ID>

Successful response example¶

{
  "id": "string",
  "createTime": "2024-09-25T18:23:01.538Z",
  "updateTime": "2024-09-25T18:23:01.538Z",
  "name": "string",
  "status": "PENDING",
  "triggeredBy": {
    "id": "string",
    "createTime": "2024-09-25T18:23:01.538Z",
    "updateTime": "2024-09-25T18:23:01.538Z",
    "email": "string",
    "name": "string",
    "roles": [ ... ]
  },
  "startTime": "2024-09-25T18:23:01.538Z",
  "completeTime": "2024-09-25T18:23:01.538Z",
  "kubeApp": {
    "id": "string",
    "createTime": "2024-09-25T18:23:01.538Z",
    "updateTime": "2024-09-25T18:23:01.538Z",
    "instanceId": "string",
    "appType": "string",
    "status": "PENDING",
    "createdById": "string",
    "apiPayload": {},
    "appMeta": {},
    "events": [],
    "message": "string"
  },
  "payload": {
    "runtime": "tiny",
    "name": "string",
    "runPython": {
      "uri": "string",
      "args": []
    },
    "runJar": {
      "uri": "string",
      "args": [],
      "mainClass": "string"
    },
    "timeoutSeconds": 3600,
    "environment": {
      "sparkConfigs": {},
      "dependencies": []
    }
  }
}

Cancel Run¶

The Cancel Run endpoint allows you to cancel a Wherobots Job Run. In practice, replace <RUN_ID>, with the ID of an existing run, which can be found by querying List Runs.

Route: POST http://api.cloud.wherobots.com/runs/<RUN_ID>/cancel

Successful response¶

Returns 200 when request cancels successfully. It can take a minute before the status changes to CANCELLED. Query Get Run to check a Job Run's status.

Get Run Logs¶

The Get Run Logs endpoint allows you to see the logs for a Wherobots Job Run. In practice, replace <RUN_ID>, with the ID of an existing run, which can be found by querying List Runs.

Route: GET http://api.cloud.wherobots.com/runs/<RUN_ID>/logs

Request query parameters schema¶

Example: http://api.cloud.wherobots.com/runs/<RUN_ID>/logs?cursor=0&size=100

Parameter	Type	Required or Optional	Parameter description	Rules for Accepted Values
`size`	`int`	Optional	This parameter indicates the position of the last item on the previous page.	Defaults to `100`
`cursor`	`int`	Optional	This parameter indicates the maximum number of items fetched for the current page.	Defaults to `0`

Successful response example¶

{
  "items": [
    {
      "timestamp": 0,
      "raw": "string"
    }
  ],
  "current_page": 0,
  "next_page": 0
}

List Runs¶

The List Runs endpoint allows you to see all of the Wherobots runs in your Organization.

Route: GET http://api.cloud.wherobots.com/runs

Request parameters¶

The following query parameters can be provided to refine the search:

Parameter	Type	Required or Optional	Parameter Description	Rules for Accepted Values
`region`	`str`	Optional	Filter the runs by the compute region in which they executed.	Valid values: `aws-eu-west-1`, `aws-us-west-2`
`name`	`str`	Optional	Filter the runs by name pattern.	Max length: 255 Can contain any number of letters, numbers, underscores, hyphens, or periods, and can include a single asterisk.
`created_after`	`str`	Optional	Filter for runs that have been created after this date.	$YYYY-MM-DD -Hours:Minutes:Seconds UTC
`status`	`array[str]`)	Optional	Filter the runs by status.	Valid values: `PENDING`, `RUNNING`, `FAILED`, `COMPLETED`, `CANCELLED`
`cursor`	`str`	Optional	Indicates the position of the last item on the previous page for pagination.
`size`	`int`	Optional	Indicates the maximum number of items fetched for the current page.	Defaults to `50`. Maximum: `100` Minimum: `0`

Successful response example¶

{
  "items": [
    {
      "id": "string",
      "createTime": "2024-09-25T20:47:25.280Z",
      "updateTime": "2024-09-25T20:47:25.280Z",
      "name": "string",
      "status": "PENDING",
      "triggeredBy": {
        "id": "string",
        "createTime": "2024-09-25T20:47:25.280Z",
        "updateTime": "2024-09-25T20:47:25.280Z",
        "email": "string",
        "name": "string",
        "roles": [ ... ]
      },
      "startTime": "2024-09-25T20:47:25.280Z",
      "completeTime": "2024-09-25T20:47:25.280Z",
      "kubeApp": {
        "id": "string",
        "createTime": "2024-09-25T20:47:25.280Z",
        "updateTime": "2024-09-25T20:47:25.280Z",
        "instanceId": "string",
        "appType": "string",
        "status": "PENDING",
        "createdById": "string",
        "apiPayload": {},
        "appMeta": {},
        "events": [],
        "message": "string"
      },
      "payload": {
        "runtime": "tiny",
        "name": "string",
        "runPython": {
          "uri": "string",
          "args": []
        },
        "timeoutSeconds": 3600,
        "environment": {
          "sparkConfigs": {},
          "dependencies": []
        }
      }
    }
  ],
  "total": 0,
  "current_page": "string",
  "current_page_backwards": "string",
  "previous_page": "string",
  "next_page": "string"
}