Connect SedonaDB to Wherobots catalogs

SedonaDB is a single-node spatial database engine from the Apache Sedona project. It runs inside your Python process, on a laptop or a server. The wherobots-catalog package connects SedonaDB to Wherobots-managed catalogs, so you can query their tables without starting a Wherobots runtime. Queries run on your machine, and SedonaDB reads table files from cloud storage over the network. Wherobots issues the storage credentials for each table, so you don’t configure storage keys. The integration supports:

Reading Iceberg v2 and v3 tables, including tables with deleted rows.
Reading geometry columns, with or without a declared coordinate reference system (CRS). See Work with geometry.

The integration doesn’t support:

Writing tables. INSERT, CREATE TABLE, DELETE, and DROP TABLE fail with a “not implemented” error. Write tables from WherobotsDB instead.
Tables with raster columns. Any query on these tables fails with Havasu raster columns (rast) cannot be read yet, even if it doesn’t select the raster column. Query them in WherobotsDB instead.
Foreign catalogs. list_catalogs() includes catalogs such as AWS Glue and Databricks Unity Catalog, but reading them fails with namespace operations are only valid for MANAGED catalogs. To query them, connect WherobotsDB to external catalogs.

For the scope of this workflow and compatibility considerations, see Connect external engines to Wherobots catalogs.

Before you start

Ensure that you have the following:

Python 3.10 or later.
A Wherobots API key, created for your user or for a service principal.
Access to the Wherobots-managed catalog you want to read. The samples on this page use the wherobots_open_data catalog.

Connect SedonaDB to a Wherobots catalog

Install the packages

pip install "apache-sedona[db]" wherobots-catalog

Set your API key

export WHEROBOTS_API_KEY="YOUR_API_KEY_HERE"

To pass the key from your own code instead, call wherobots.catalog.login(api_key=...).

Connect to your catalogs

import sedona.db
import wherobots.catalog

sd = sedona.db.connect()
wbc = wherobots.catalog.login()
sd.register(wbc)

login() returns the catalogs that your API key can access. sd.register(wbc) makes them available to SQL in your sd session.

Verify the connection

print(wbc.wherobots_open_data.overture_maps_foundation.list_tables())

The output lists the tables in the overture_maps_foundation namespace, also known as a database:

['addresses_address', 'base_bathymetry', 'base_infrastructure', 'base_land', 'base_land_cover', 'base_land_use', 'base_water', 'buildings_building', 'buildings_building_part', 'divisions_division', 'divisions_division_area', 'divisions_division_boundary', 'geocodes', 'geocodes_index', 'places_place', 'transportation_connector', 'transportation_segment']

In a Job Run, SedonaDB is preinstalled, so install only wherobots-catalog. login() finds the Job Run’s credentials, so you don’t need an API key.

Browse catalogs

Use attributes or indexing to move from a catalog to a namespace to a table. Use indexing for names that aren’t valid Python identifiers.

print(wbc.list_catalogs())
print(wbc.wherobots_open_data.list_namespaces())

places = wbc.wherobots_open_data.overture_maps_foundation.places_place
places = wbc["wherobots_open_data.overture_maps_foundation.places_place"]
print(places.schema)

Both places assignments return the same table.

Query a table

Query a table with SQL or with the DataFrame API. Both samples find the libraries in a bounding box around Boston. Neither reads data until you call a method such as show().

SQL
DataFrame

libraries = sd.sql("""
    SELECT names['primary'] AS name, geometry
    FROM wherobots_open_data.overture_maps_foundation.places_place
    WHERE contains(lower(names['primary']), 'library')
      AND ST_Intersects(
          geometry,
          ST_MakeEnvelope(-71.19, 42.22, -70.92, 42.40, 4326)
      )
    ORDER BY name
""")
libraries.show(5)

places = wbc.wherobots_open_data.overture_maps_foundation.places_place
boston_bounds = sd.funcs.st_makeenvelope(-71.19, 42.22, -70.92, 42.40, 4326)

libraries = places.filter(
    places.names["primary"].funcs.lower().funcs.contains("library"),
    places.geometry.geo.intersects(boston_bounds),
).select(name=places.names["primary"], geometry="geometry").sort("name")
libraries.show(5)

Both samples print the following:

┌────────────────────────────────────────────────────┬─────────────────────────────────────────────┐
│                        name                        ┆                   geometry                  │
│                        utf8                        ┆                   geometry                  │
╞════════════════════════════════════════════════════╪═════════════════════════════════════════════╡
│ Academic Library                                   ┆ POINT(-71.095681 42.348524)                 │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ African Studies Library                            ┆ POINT(-71.10827048 42.35052328)             │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Albert Alphin Library                              ┆ POINT(-71.0903056363387 42.346227142268916) │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Annenberg Library at Pine Manor College            ┆ POINT(-71.15659546 42.32076095)             │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ Archbishop Iakovos Library and Learning Resource … ┆ POINT(-71.12908543 42.31708149)             │
└────────────────────────────────────────────────────┴─────────────────────────────────────────────┘

Save query results

to_pandas() loads the result into memory on your machine and returns a GeoPandas GeoDataFrame when the result has a geometry column. to_parquet() writes the result to a local file.

libraries_gdf = libraries.to_pandas()
print(type(libraries_gdf).__name__, len(libraries_gdf))

libraries.to_parquet("boston_libraries.parquet")

GeoDataFrame 189

Join a table with local data

SedonaDB uses a spatial filter on a catalog table to skip files that can’t match it. A join condition doesn’t skip files, so a join on its own reads the whole table. Filter the catalog table first, then join the result. This sample registers a local GeoDataFrame as a view and counts the places in each area:

import geopandas as gpd
from shapely.geometry import box

areas = gpd.GeoDataFrame(
    {"area": ["north", "south"]},
    geometry=[
        box(-71.09, 42.345, -71.07, 42.355),
        box(-71.08, 42.335, -71.065, 42.345),
    ],
    crs="EPSG:4326",
)
sd.create_data_frame(areas).to_view("areas", overwrite=True)

sd.sql("""
    SELECT a.area, count(*) AS places
    FROM (
        SELECT geometry
        FROM wherobots_open_data.overture_maps_foundation.places_place
        WHERE ST_Intersects(
            geometry,
            ST_MakeEnvelope(-71.19, 42.22, -70.92, 42.40, 4326)
        )
    ) p
    JOIN areas a ON ST_Intersects(p.geometry, a.geometry)
    GROUP BY a.area
    ORDER BY a.area
""").show()

┌───────┬────────┐
│  area ┆ places │
│  utf8 ┆  int64 │
╞═══════╪════════╡
│ north ┆   5414 │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌┤
│ south ┆   1364 │
└───────┴────────┘

To check how much of a table a query reads, run it with EXPLAIN and find WherobotsTableExec in the plan. In the run behind this sample, the plan shows groups=2 with the filter and groups=206 for the join on its own.

Work with geometry

The CRS that SedonaDB uses for a geometry column depends on what the column declares. Run SHOW CREATE TABLE in WherobotsDB to see the declaration:

Column in `SHOW CREATE TABLE`	CRS in SedonaDB	`ST_SRID` in SedonaDB
`GEOMETRY(4326)`	`ogc:crs84`	`4326`
`GEOMETRY(3857)`	`epsg:3857`	`3857`
`GEOMETRY`, with no declared CRS	`ogc:crs84`	`4326`

To see the CRS that SedonaDB uses, print the column from the table’s schema:

print(places.schema.field("geometry"))

SedonaField geometry: geometry<Wkb(ogc:crs84)>

Troubleshooting

Authentication is not configured

WHEROBOTS_API_KEY isn’t set in the environment that runs Python. Set it, or pass the key with wherobots.catalog.login(api_key=...).If the error is Authentication failed (HTTP 401) instead, the key isn’t valid. Create a new API key.

Mismatched CRS arguments: ogc:crs84 vs None

The geometry in your query has no CRS, or a different CRS from the column. Pass an SRID to the function that creates the geometry, such as ST_MakeEnvelope(-71.19, 42.22, -70.92, 42.40, 4326). If the two CRSs differ, convert one with ST_Transform.

Cannot create a view from a Wherobots Catalog DataFrame

to_view() isn’t available on a table object. Create the view from SQL in your own session instead:

sd.sql(
    "SELECT * FROM wherobots_open_data.overture_maps_foundation.places_place"
).to_view("places", overwrite=True)

Next steps

Wherobots open data

Find the datasets in the wherobots_open_data catalog.

Geometry tables

Create tables with geometry columns that declare a CRS.

SedonaDB documentation

Learn the SedonaDB SQL functions and DataFrame API.

Start here

Havasu data catalogs

Vector data

Raster data

Iceberg tables

Maps and visualization

Connect SedonaDB to Wherobots catalogs

Before you start

Connect SedonaDB to a Wherobots catalog

Browse catalogs

Query a table

Save query results

Join a table with local data

Work with geometry

Troubleshooting

Next steps

Wherobots open data

Geometry tables

SedonaDB documentation

​Before you start

​Connect SedonaDB to a Wherobots catalog

​Browse catalogs

​Query a table

​Save query results

​Join a table with local data

​Work with geometry

​Troubleshooting

​Next steps

Wherobots open data

Geometry tables

SedonaDB documentation

Before you start

Connect SedonaDB to a Wherobots catalog

Browse catalogs

Query a table

Save query results

Join a table with local data

Work with geometry

Troubleshooting

Next steps