> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wherobots.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Query Unity Catalog Tables

> Read and write Databricks Unity Catalog Delta and Iceberg tables from a Wherobots Notebook using SedonaContext.

You can access your Unity Catalog Tables in a Wherobots Notebook, Job Run, or SQL Session. The following sections
detail how to work with your Unity Catalog tables in a Wherobots Notebook.

> [!NOTE]
>
> Before you can read and write Unity Catalog tables, you need to [connect Wherobots to Databricks Unity Catalog](https://docs.wherobots.com/get-started/initial-storage/connect-to-unity-catalog.md).

## Before you start

* A Databricks workspace with Unity Catalog enabled and a catalog created.
* A Wherobots account with access to your Iceberg or Delta Table in Databricks Unity Catalog.
  * In order to read and write Unity Catalog tables, ensure that your Databricks workspace is configured correctly by completing the steps in [Connect to Databricks Unity Catalog](https://docs.wherobots.com/get-started/initial-storage/connect-to-unity-catalog.md).

## Set the SedonaContext

In a Wherobots Notebook, create the `SedonaContext` and import any other necessary libraries for your analysis.

The following imports the necessary modules from the Sedona library, creates a `SedonaContext` object, and
imports [`expr`](https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.expr.html).

```python theme={"system"}
from sedona.spark import *
from pyspark.sql.functions import expr
config = SedonaContext.builder().getOrCreate()
sedona = SedonaContext.create(config)
```

## Set your Databricks Resource Variables

Define the resources that point to your Databricks resources:

```python theme={"system"}
CATALOG = "YOUR-CATALOG" # Change this to your catalog
SCHEMA  = "YOUR-SCHEMA" # Change this to your schema name
SOURCE_TABLE = "YOUR-SOURCE-TABLE" # Change this to the table you're reading into Wherobots
OUTPUT_TABLE = "YOUR-OUTPUT-TABLE" # Change this to the table you're writing to from Wherobots
SOURCE_TABLE_FQN = f"`{CATALOG}`.`{SCHEMA}`.`{SOURCE_TABLE}`"
OUTPUT_TABLE_FQN = f"`{CATALOG}`.`{SCHEMA}`.`{OUTPUT_TABLE}`"
```

## Reading from a Delta Table

> [tab] Writing to a Managed Catalog in Wherobots
>
> ```python theme={"system"}
> # Read from a Unity Catalog Databricks Managed Delta table
> df = sedona.read.table(SOURCE_TABLE_FQN)
>
> # Assuming the table has a column named "geom_wkb" that stores geometries in WKB format,
> # use Wherobots to convert those to an equivalent GEOMETRY column.
> df_parsed = df.withColumn("geom", expr("ST_GeomFromWKB(geom_wkb)"))
>
> # Perform spatial analysis in Wherobots, which creates a new GEOMETRY column.
> # For example, create a 100-meter buffer around an existing geometry.
> # useSpheroid = true measures the distance in meters on lon/lat geometries.
> # If geom_wkb holds projected coordinates in meters, use ST_Buffer(geom, 100) instead.
> df_analyzed = df_parsed.withColumn("buffered_geom", expr("ST_Buffer(geom, 100, true)"))
>
> # Write the enriched table, preserving the new GEOMETRY column,
> # to your Managed Catalog.
> sedona.sql("CREATE SCHEMA IF NOT EXISTS org_catalog.default")
> df_analyzed.writeTo(f"org_catalog.default.`{OUTPUT_TABLE}`") \
>     .createOrReplace()
> ```

> [tab] Writing to an External Delta Table in Unity Catalog
>
> To write to an external Delta table, you must specify an **external location** in a Wherobots Notebook.
>
> **What's an external location?**
>
> An external location is a Unity Catalog object that links a cloud storage path to a **storage credential** to manage data access. You can manage them in the **Catalog Explorer**.
>
> Finding an External Location
>
> 1. In the **Catalog Explorer**, navigate to **External Data** > **External Locations**.
> 2. A list of registered locations will appear. Click on a location to view its details.
>
> Creating an External Location
>
> Creation is a two-step process: first create a **storage credential** that grants Databricks access to your cloud storage, then create the external location itself.
>
> 1. Go to **Catalog Explorer** > **External Data** > **External Locations** and click **Create location**.
> 2. Enter a name, provide the cloud storage URL, and select the storage credential you created.
>
> For detailed instructions, see [Manage external locations and storage credentials](https://docs.databricks.com/en/data-governance/unity-catalog/manage-external-locations-and-credentials.html).
>
> ```python theme={"system"}
> # Define the external location for the output Delta table.
> # Replace this with the actual path in your cloud storage.
> OUTPUT_TABLE_EXTERNAL_LOCATION = 's3://your-bucket-name/path/to/external/location/'
>
> # Read the source table using the fully qualified name variable.
> df = sedona.read.table(SOURCE_TABLE_FQN)
>
> # Assuming the table has a column named "geom_wkb" that stores geometries in WKB format,
> # use Wherobots to convert those to an equivalent GEOMETRY column.
> df_parsed = df.withColumn("geom", expr("ST_GeomFromWKB(geom_wkb)"))
>
> # Perform spatial analysis in Wherobots, which creates a new GEOMETRY column.
> # For example, create a 100-meter buffer around an existing geometry.
> # useSpheroid = true measures the distance in meters on lon/lat geometries.
> # If geom_wkb holds projected coordinates in meters, use ST_Buffer(geom, 100) instead.
> df_analyzed = df_parsed.withColumn("buffered_geom", expr("ST_Buffer(geom, 100, true)"))
>
> # To write back to a standard Databricks Delta table, convert any GEOMETRY
> # columns to a binary format like Well-Known Binary (WKB).
> # Here, we convert both the original and the new buffered geometry columns.
> df_for_databricks = df_analyzed.withColumn(
>     "geom_wkb", expr("ST_AsBinary(geom)")
> ).withColumn(
>     "buffered_geom_wkb", expr("ST_AsBinary(buffered_geom)")
> ).drop("geom", "buffered_geom")
>
> # Create a temporary view to reference in the final SQL command.
> df_for_databricks.createOrReplaceTempView("temp_final_df_view")
>
> # Use a SQL command to create the external table in Unity Catalog.
> # The LOCATION keyword ensures the data is written to your specified cloud storage path.
> sedona.sql(f"""
> CREATE OR REPLACE TABLE {OUTPUT_TABLE_FQN}
> USING delta
> LOCATION '{OUTPUT_TABLE_EXTERNAL_LOCATION}'
> AS SELECT * FROM temp_final_df_view
> """)
> ```

## Reading from an Iceberg Table

> [tab] Writing to a Managed Catalog in Wherobots
>
> ```python theme={"system"}
> # Read an Iceberg table from your Databricks catalog
> df = sedona.read.table(SOURCE_TABLE_FQN)
>
> # Assuming the table has a column named "geom_wkb" that stores geometries in WKB format,
> # use Wherobots to convert those to an equivalent GEOMETRY column.
> df_parsed = df.withColumn("geom", expr("ST_GeomFromWKB(geom_wkb)"))
>
> # Perform spatial analysis, which creates a new GEOMETRY column.
> # For example, create a 100-meter buffer around an existing geometry.
> # useSpheroid = true measures the distance in meters on lon/lat geometries.
> # If geom_wkb holds projected coordinates in meters, use ST_Buffer(geom, 100) instead.
> df_analyzed = df_parsed.withColumn("buffered_geom", expr("ST_Buffer(geom, 100, true)"))
>
> # Write the enriched table, preserving the new GEOMETRY column,
> # to your Managed Catalog.
> sedona.sql("CREATE SCHEMA IF NOT EXISTS org_catalog.default")
> df_analyzed.writeTo(f"org_catalog.default.`{OUTPUT_TABLE}`") \
>     .createOrReplace()
> ```

> [tab] Writing to a new Managed Iceberg Table in Unity Catalog
>
> ```python theme={"system"}
> # Read the source table using the fully qualified name variable.
> df = sedona.read.table(SOURCE_TABLE_FQN)
>
> # Assuming the table has a column named "geom_wkb" that stores geometries in WKB format,
> # use Wherobots to convert those to an equivalent GEOMETRY column.
> df_parsed = df.withColumn("geom", expr("ST_GeomFromWKB(geom_wkb)"))
>
> # Perform spatial analysis in Wherobots, which creates a new GEOMETRY column.
> # For example, create a 100-meter buffer around an existing geometry.
> # useSpheroid = true measures the distance in meters on lon/lat geometries.
> # If geom_wkb holds projected coordinates in meters, use ST_Buffer(geom, 100) instead.
> df_analyzed = df_parsed.withColumn("buffered_geom", expr("ST_Buffer(geom, 100, true)"))
>
> # To write back to a new Managed Iceberg table in Databricks, convert any GEOMETRY
> # columns to a binary format like Well-Known Binary (WKB).
> # Here, we convert both the original and the new buffered geometry columns.
> df_for_databricks = df_analyzed.withColumn(
>     "geom_wkb", expr("ST_AsBinary(geom)")
> ).withColumn(
>     "buffered_geom_wkb", expr("ST_AsBinary(buffered_geom)")
> ).drop("geom", "buffered_geom")
>
> # Write the results back to a new Managed Iceberg table in Databricks
> df_for_databricks.writeTo(OUTPUT_TABLE_FQN) \
>     .createOrReplace()
> ```

## Next steps

> [card] [Try it in a notebook](https://docs.wherobots.com/tutorials/example-notebooks/unity-catalog-delta-tables.md)
>
> Walk through a hands-on tutorial reading Unity Catalog Delta tables in Wherobots.

> [card] [Usage and limitations](https://docs.wherobots.com/get-started/initial-storage/connect-to-unity-catalog.md#usage-and-limitations)
>
> Review workflow use cases, catalog naming rules, and the foreign-catalog limit.

> [card] [Federate AWS Glue Catalog](https://docs.wherobots.com/get-started/initial-storage/aws/connect-to-glue-catalog.md)
>
> Connect WherobotsDB to another external catalog.

> [card] [Connect WherobotsDB to external catalogs](https://docs.wherobots.com/get-started/initial-storage/data-federation-overview.md)
>
> Learn how Wherobots queries external data sources in place.

## Usage and limitations

* **Catalog Naming:** You cannot use a local alias for a catalog. If you have a pre-existing catalog in your Wherobots Organization named `wherobots`, trying to connect a Databricks catalog with the name `wherobots` will cause a permanent naming conflict and must be avoided.
* **Catalog Limit:** The integration supports a limit of 10 foreign catalogs per Organization.
* **UniForm:** If you use Databricks' Universal Format (UniForm) to enable Iceberg reads on a Delta table, that table will be **read-only**.

### Workflows explained

The following table provides a detailed summary of each workflow and its intended use case.

| Use Case | Read Source (Unity Catalog) | Write Destination |
| :- | :- | :- |
| **Preserve `GEOMETRY` columns** for continued complex spatial analysis and visualization within the Wherobots environment. | Managed **Delta** Table | Managed Catalog in Wherobots |
| **Generate spatial features for AI and BI in Databricks.** Complete complex spatial analysis in Wherobots and write spatially-enriched feature columns back to Unity Catalog for use in Databricks' ML models and BI dashboards. | Managed **Delta** Table | External **Delta** Table (in Unity Catalog) |
| **Preserve `GEOMETRY` columns** for continued complex spatial analysis and visualization within the Wherobots environment. | Managed **Iceberg** Table | Managed Catalog in Wherobots |
| **Generate spatial features for AI and BI in Databricks.** Complete complex spatial analysis in Wherobots and write spatially-enriched feature columns back to Unity Catalog for use in Databricks' ML models and BI dashboards. | Managed **Iceberg** Table | Managed **Iceberg** Table (in Unity Catalog) |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.