Query Unity Catalog Tables

You can access your Unity Catalog Tables in a Wherobots Notebook, Job Run, or SQL Session. The following sections detail how to work with your Unity Catalog tables in a Wherobots Notebook.

Before you can read and write Unity Catalog tables, you need to connect Wherobots to Databricks Unity Catalog.

Before you start

A Databricks workspace with Unity Catalog enabled and a catalog created.
A Wherobots account with access to your Iceberg or Delta Table in Databricks Unity Catalog.
- In order to read and write Unity Catalog tables, ensure that your Databricks workspace is configured correctly by completing the steps in Connect to Databricks Unity Catalog.

Set the SedonaContext

In a Wherobots Notebook, create the SedonaContext and import any other necessary libraries for your analysis. The following imports the necessary modules from the Sedona library, creates a SedonaContext object, and imports expr.

from sedona.spark import *
from pyspark.sql.functions import expr
config = SedonaContext.builder().getOrCreate()
sedona = SedonaContext.create(config)

Set your Databricks Resource Variables

Define the resources that point to your Databricks resources:

CATALOG = "YOUR-CATALOG" # Change this to your catalog
SCHEMA  = "YOUR-SCHEMA" # Change this to your schema name
SOURCE_TABLE = "YOUR-SOURCE-TABLE" # Change this to the table you're reading into Wherobots
OUTPUT_TABLE = "YOUR-OUTPUT-TABLE" # Change this to the table you're writing to from Wherobots
SOURCE_TABLE_FQN = f"`{CATALOG}`.`{SCHEMA}`.`{SOURCE_TABLE}`"
OUTPUT_TABLE_FQN = f"`{CATALOG}`.`{SCHEMA}`.`{OUTPUT_TABLE}`"

Reading from a Delta Table

Writing to a Wherobots-managed catalog
Writing to an External Delta Table in Unity Catalog

# Read from a Unity Catalog Databricks Managed Delta table
df = sedona.read.table(SOURCE_TABLE_FQN)

# Assuming the table has a column named "geom_wkb" that stores geometries in WKB format,
# use Wherobots to convert those to an equivalent GEOMETRY column.
df_parsed = df.withColumn("geom", expr("ST_GeomFromWKB(geom_wkb)"))

# Perform spatial analysis in Wherobots, which creates a new GEOMETRY column.
# For example, create a 100-meter buffer around an existing geometry.
df_analyzed = df_parsed.withColumn("buffered_geom", expr("ST_Buffer(geom, 100)"))

# Write the enriched table, preserving the new GEOMETRY column,
# to your Wherobots-managed catalog.
sedona.sql("CREATE SCHEMA IF NOT EXISTS org_catalog.default")
df_analyzed.writeTo(f"org_catalog.default.`{OUTPUT_TABLE}`") \
    .createOrReplace()

To write to an external Delta table, you must specify an external location in a Wherobots Notebook.

An external location is a Unity Catalog object that links a cloud storage path to a storage credential to manage data access. You can manage them in the Catalog Explorer.

Finding an External Location

In the Catalog Explorer, navigate to External Data > External Locations.
A list of registered locations will appear. Click on a location to view its details.

Creating an External LocationCreation is a two-step process: first create a storage credential that grants Databricks access to your cloud storage, then create the external location itself.

Go to Catalog Explorer > External Data > External Locations and click Create location.
Enter a name, provide the cloud storage URL, and select the storage credential you created.

For detailed instructions, see Manage external locations and storage credentials.

# Define the external location for the output Delta table.
# Replace this with the actual path in your cloud storage.
OUTPUT_TABLE_EXTERNAL_LOCATION = 's3://your-bucket-name/path/to/external/location/'

# Read the source table using the fully qualified name variable.
df = sedona.read.table(SOURCE_TABLE_FQN)

# Assuming the table has a column named "geom_wkb" that stores geometries in WKB format,
# use Wherobots to convert those to an equivalent GEOMETRY column.
df_parsed = df.withColumn("geom", expr("ST_GeomFromWKB(geom_wkb)"))

# Perform spatial analysis in Wherobots, which creates a new GEOMETRY column.
# For example, create a 100-meter buffer around an existing geometry.
df_analyzed = df_parsed.withColumn("buffered_geom", expr("ST_Buffer(geom, 100)"))

# To write back to a standard Databricks Delta table, convert any GEOMETRY
# columns to a binary format like Well-Known Binary (WKB).
# Here, we convert both the original and the new buffered geometry columns.
df_for_databricks = df_analyzed.withColumn(
    "geom_wkb", expr("ST_AsBinary(geom)")
).withColumn(
    "buffered_geom_wkb", expr("ST_AsBinary(buffered_geom)")
).drop("geom", "buffered_geom")

# Create a temporary view to reference in the final SQL command.
df_for_databricks.createOrReplaceTempView("temp_final_df_view")

# Use a SQL command to create the external table in Unity Catalog.
# The LOCATION keyword ensures the data is written to your specified cloud storage path.
sedona.sql(f"""
CREATE OR REPLACE TABLE {OUTPUT_TABLE_FQN}
USING delta
LOCATION '{OUTPUT_TABLE_EXTERNAL_LOCATION}'
AS SELECT * FROM temp_final_df_view
""")

Reading from an Iceberg Table

Writing to a Wherobots-managed Catalog
Writing to a new Managed Iceberg Table in Unity Catalog

# Read an Iceberg table from your Databricks catalog
df = sedona.read.table(SOURCE_TABLE_FQN)

# Assuming the table has a column named "geom_wkb" that stores geometries in WKB format,
# use Wherobots to convert those to an equivalent GEOMETRY column.
df_parsed = df.withColumn("geom", expr("ST_GeomFromWKB(geom_wkb)"))

# Perform spatial analysis, which creates a new GEOMETRY column.
# For example, create a 100-meter buffer around an existing geometry.
df_analyzed = df_parsed.withColumn("buffered_geom", expr("ST_Buffer(geom, 100)"))

# Write the enriched table, preserving the new GEOMETRY column,
# to your Wherobots-managed catalog.
sedona.sql("CREATE SCHEMA IF NOT EXISTS org_catalog.default")
df_analyzed.writeTo(f"org_catalog.default.`{OUTPUT_TABLE}`") \
    .createOrReplace()

# Read the source table using the fully qualified name variable.
df = sedona.read.table(SOURCE_TABLE_FQN)

# Assuming the table has a column named "geom_wkb" that stores geometries in WKB format,
# use Wherobots to convert those to an equivalent GEOMETRY column.
df_parsed = df.withColumn("geom", expr("ST_GeomFromWKB(geom_wkb)"))

# Perform spatial analysis in Wherobots, which creates a new GEOMETRY column.
# For example, create a 100-meter buffer around an existing geometry.
df_analyzed = df_parsed.withColumn("buffered_geom", expr("ST_Buffer(geom, 100)"))

# To write back to a new Managed Iceberg table in Databricks, convert any GEOMETRY
# columns to a binary format like Well-Known Binary (WKB).
# Here, we convert both the original and the new buffered geometry columns.
df_for_databricks = df_analyzed.withColumn(
    "geom_wkb", expr("ST_AsBinary(geom)")
).withColumn(
    "buffered_geom_wkb", expr("ST_AsBinary(buffered_geom)")
).drop("geom", "buffered_geom")

# Write the results back to a new Managed Iceberg table in Databricks
df_for_databricks.writeTo(OUTPUT_TABLE_FQN) \
    .createOrReplace()

Next steps

Try it in a notebook

Walk through a hands-on tutorial reading Unity Catalog Delta tables in Wherobots.

Usage and limitations

Review workflow use cases, catalog naming rules, and the foreign-catalog limit.

Federate AWS Glue Catalog

Connect another external data source with Wherobots Data Federation.

Data Federation overview

Learn how Wherobots queries external data sources in place.

Usage and limitations

Catalog Naming: You cannot use a local alias for a catalog. If you have a pre-existing catalog in your Wherobots Organization named wherobots, trying to connect a Databricks catalog with the name wherobots will cause a permanent naming conflict and must be avoided.
Catalog Limit: The integration supports a limit of 10 foreign catalogs per Organization.
UniForm: If you use Databricks’ Universal Format (UniForm) to enable Iceberg reads on a Delta table, that table will be read-only.

Workflows explained

The following table provides a detailed summary of each workflow and its intended use case.

Use Case	Read Source (Unity Catalog)	Write Destination
Preserve `GEOMETRY` columns for continued complex spatial analysis and visualization within the Wherobots environment.	Managed Delta Table	Wherobots-Managed Catalog
Generate spatial features for AI and BI in Databricks. Complete complex spatial analysis in Wherobots and write spatially-enriched feature columns back to Unity Catalog for use in Databricks’ ML models and BI dashboards.	Managed Delta Table	External Delta Table (in Unity Catalog)
Preserve `GEOMETRY` columns for continued complex spatial analysis and visualization within the Wherobots environment.	Managed Iceberg Table	Wherobots-Managed Catalog
Generate spatial features for AI and BI in Databricks. Complete complex spatial analysis in Wherobots and write spatially-enriched feature columns back to Unity Catalog for use in Databricks’ ML models and BI dashboards.	Managed Iceberg Table	Managed Iceberg Table (in Unity Catalog)

Wherobots Fundamentals

Getting Started

Data Hub

Team & Access

Plans & Billing

Query Unity Catalog Tables

Before you start

Set the SedonaContext

Set your Databricks Resource Variables

Reading from a Delta Table

Reading from an Iceberg Table

Next steps

Try it in a notebook

Usage and limitations

Federate AWS Glue Catalog

Data Federation overview

Usage and limitations

Workflows explained

​Before you start

​Set the SedonaContext

​Set your Databricks Resource Variables

​Reading from a Delta Table

​Reading from an Iceberg Table

​Next steps

Try it in a notebook

Usage and limitations

Federate AWS Glue Catalog

Data Federation overview

​Usage and limitations

​Workflows explained

Before you start

Set the SedonaContext

Set your Databricks Resource Variables

Reading from a Delta Table

Reading from an Iceberg Table

Next steps

Usage and limitations

Workflows explained