> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wherobots.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Bring Your Own Spark (BYOS)

> Run the WherobotsDB engine on a Spark cluster you own and operate, on Amazon EMR or Databricks. Requirements, versions, Spark properties, and authentication shared by both platforms.

**Bring Your Own Spark (BYOS)** lets you run WherobotsDB on a Spark cluster you own and operate. WherobotsDB is a [drop-in replacement for Apache Sedona](https://wherobots.com/apache-sedona/wherobots-vs-apache-sedona/): you install its JAR on your existing cluster, and your existing code, SQL, and configuration continue to work unchanged.

Your data never leaves your cloud account or workspace. Wherobots does not run, manage, or connect to your cluster — the only outbound traffic is a lightweight authenticated call from the JAR to Wherobots Cloud for license metering.

This page covers what the BYOS platforms share. For the installation steps, pick your platform:

> **[Amazon EMR](https://docs.wherobots.com/develop/deployment-options/byos-aws-emr.md)**
>
> Install WherobotsDB on an EMR on EC2 cluster with a bootstrap action, and test it in EMR Studio.

> **[Databricks](https://docs.wherobots.com/develop/deployment-options/byos-databricks.md)**
>
> Install WherobotsDB on a Databricks cluster with an init script, including Photon-enabled clusters.

## Deployment options

| Deployment | Where workloads run | Who operates the compute |
| - | - | - |
| Hosted region | Serverless compute in Wherobots' AWS account | Wherobots |
| [Bring Your Own Cloud (BYOC)](https://docs.wherobots.com/develop/storage-management/byoc.md) region | Infrastructure that Wherobots manages inside a VPC in your AWS account | Wherobots |
| Bring Your Own Spark (BYOS) | Your own Spark cluster on Amazon EMR or Databricks | You. Wherobots supplies only the engine |

## Wherobots requirements

Both platforms need the following in your Wherobots Organization:

* An account in a **Professional**, **Innovation**, or **Enterprise** Edition Organization.
  > [!NOTE]
  >
  > Admins and Users can both create the personal API key this deployment uses. Only an Admin can add an API key to a service principal. See [Organization Roles](https://docs.wherobots.com/get-started/organization-info/organization-roles.md).
* The **BYOS feature enabled on your Organization**. Contact Wherobots to enable it. BYOS unlocks both halves of this deployment: read access to the WherobotsDB release artifacts, and the runtime authentication and usage-reporting endpoints the JAR calls. Without it, those endpoints return `403` and cluster initialization fails.
* A **Wherobots API key**. See [Create an API key](#create-an-api-key).
* The WherobotsDB release artifacts from Wherobots: the platform's shaded WherobotsDB JAR, the `geotools-wrapper` JAR, and the `wherobots-sedona` Python wheels. These are not published to Maven Central or PyPI.

> [!NOTE]
>
> Once BYOS is enabled for your Organization, a read-only **WherobotsDB Releases** storage source appears automatically in your Wherobots Cloud storage list. Go to **Storage**, open that source, and download the JARs and wheel from the release folder. If the source does not appear, BYOS is not yet enabled — contact Wherobots.

The cluster also needs **outbound HTTPS (port 443) to Wherobots Cloud** for license metering. Without it, `SedonaContext.create` fails to authenticate.

## Versions

WherobotsDB JARs are compiled against **JDK 17**. Each platform page lists the JAR artifact for each supported Spark and Scala version. A Spark or Scala mismatch surfaces at runtime as `NoSuchMethodError` or `ClassNotFoundException`.

WherobotsDB is versioned independently of Apache Sedona. Each WherobotsDB release is API-compatible with a specific Apache Sedona version, and the GeoTools wrapper version follows the Sedona version — so `VERSION` and `GEOTOOLS_VERSION` are different values. For example:

| WherobotsDB (`VERSION`) | Compatible Apache Sedona | GeoTools wrapper (`GEOTOOLS_VERSION`) |
| - | - | - |
| `1.28.0` | `1.9.0` | `1.9.0-33.5` |

Always use the exact filenames Wherobots delivered to you.

## Spark properties

Both platforms set these Spark properties on the cluster. The platform pages show where to set them, along with the properties each platform needs to load the JARs.

| Property | Value | Purpose |
| - | - | - |
| `spark.serializer` | `org.apache.spark.serializer.KryoSerializer` | Configures Kryo serialization for geometry types |
| `spark.kryo.registrator` | `org.apache.sedona.core.serde.SedonaKryoRegistrator` | Registers the geometry types with Kryo |
| `spark.sedona.enableParserExtension` | `false` | Must remain `false` |
| `spark.sql.extensions` | `org.apache.sedona.viz.sql.SedonaVizExtensions,org.apache.sedona.sql.SedonaSqlExtensions` | Registers the `ST_*` SQL functions |

> [!WARNING]
>
> `spark.sql.extensions` is a comma-separated list, and the value you set **replaces** any value already in effect. If your cluster registers other extensions, such as Apache Iceberg's, append the two Sedona entries to your existing list.

For how to check the configuration with `SedonaConf` and for the optional spatial join parameters, see [WherobotsDB Spark Parameters](https://docs.wherobots.com/reference/wherobots-db/parameters.md).

## How Wherobots authentication works

WherobotsDB validates your API key at startup and uses it for usage-based billing. Both operations call Wherobots Cloud endpoints that are part of the BYOS feature:

1. At startup, the JAR authenticates against `POST /apps/authenticate`, which returns a workload ID.
2. While the workload runs, the JAR periodically reports usage heartbeats to `POST /apps/report/{workload_id}`.

If BYOS is not enabled for your Organization, both endpoints return `403` and initialization fails. The same feature grants access to the release artifacts.

The cluster reads the key from the `spark.wherobots.api.key` Spark property or the `WHEROBOTS_API_KEY` environment variable. Each platform page shows how to supply it from a secret store: AWS Secrets Manager on Amazon EMR, and Databricks secrets on Databricks.

### Create an API key

**1. Open the API Keys page**

Log in to [Wherobots Cloud](https://cloud.wherobots.com) and go to **Settings → API Keys**.

**2. Create the key**

Select **Create new key**, name it, and choose an expiration date. The default is one year. Select **Submit**.

**3. Copy it immediately**

> [!WARNING]
>
> The key is shown only once. Copy it before leaving the page.

For details on managing and rotating keys, see the [Wherobots API Keys documentation](https://docs.wherobots.com/get-started/wherobots-cloud/api-keys.md).

## Feature differences from Wherobots Cloud

BYOS provides the WherobotsDB engine on your cluster. Features of the managed Wherobots Cloud platform are not included:

* The **Wherobots managed catalogs** — `wherobots`, `wherobots_open_data`, `wherobots_pro_data`, and the tutorial `examples_temp` catalog — are not available. Notebooks or jobs that reference them must be pointed at your own catalogs or read public data directly.
* Runtime conveniences that Wherobots Cloud preconfigures — the Spark checkpoint directory, visualization packages such as SedonaKepler, and pre-installed Python dependencies — must be set up on your cluster yourself.

## Support

For issues or questions, contact Wherobots support.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.