Transform the Coordinate Reference System
WherobotsDB doesn’t control the coordinate unit (degree-based or meter-based) of all geometries in a Geometry column. The unit of all related distances in Spatial SQL is same as the unit of all geometries in a Geometry column. For more details, please read theST_Transform section in WherobotsDB API References.
To convert Coordinate Reference System of the Geometry column created before, use the following code:
ST_Transform is the source CRS of the geometries. It is WGS84, the most common degree-based CRS.
The second EPSG code EPSG:3857 in ST_Transform is the target CRS of the geometries. It is the most common meter-based CRS.
This ST_Transform transform the CRS of these geometries from EPSG:4326 to EPSG:3857. The details CRS information can be found on EPSG.io
The coordinates of polygons have been changed. The output will be like this:
Range query
UseST_Contains, ST_Intersects, ST_Within to run a range query over a single column.
The following example finds all counties that are within the given polygon:
Read Spatial SQL API to learn how to create a Geometry type query window
KNN query
UseST_Distance to calculate the distance and rank the distance.
The following code returns the 5 nearest neighbor of the given polygon.
Range join
A range join query is to find geometries from A and geometries from B such that each geometry pair satisfies a certain predicate. Most predicates supported by Spatial SQL can trigger a range join. SQL example:Distance join
Introduction: Find geometries from A and geometries from B such that the distance of each geometry pair is less or equal than a certain distance. It supports the planar Euclidean distance calculatorsST_Distance, ST_HausdorffDistance, ST_FrechetDistance and the meter-based geodesic distance calculators ST_DistanceSpheroid and ST_DistanceSphere.
Below are SQL examples for planar Euclidean distance predicates.
Fully within a certain distance:
ST_DistanceSpheroid (works for ST_DistanceSphere too):
Less than a certain distance:
Broadcast index join
Introduction: Perform a range join or distance join but broadcast one of the sides of the join. This maintains the partitioning of the non-broadcast side and doesn’t require a shuffle. WherobotsDB will create a spatial index on the broadcasted table. WherobotsDB uses broadcast join only if the correct side has a broadcast hint. The supported join type - broadcast side combinations are:- Inner - either side, preferring to broadcast left if both sides have the hint
- Left semi - broadcast right
- Left anti - broadcast right
- Left outer - broadcast right
- Right outer - broadcast left
ST_Distance, ST_DistanceSpheroid, ST_DistanceSphere, ST_HausdorffDistance or ST_FrechetDistance:
pointDf1 above).
Automatic broadcast index join
When one table involved a spatial join query is smaller than a threshold, WherobotsDB will automatically choose broadcast index join instead of WherobotsDB optimized join. The current threshold is controlled by sedona.join.autoBroadcastJoinThreshold and set to the same asspark.sql.autoBroadcastJoinThreshold.
K Nearest-Neighbor join / KNN join
Introduction: WherobotsDB supports nearest-neighbor searching on geospatial data through the K Nearest-Neighbor (KNN) join method. This join operation identifies the k-nearest neighbors of a point or region in one dataset relative to another dataset, based on geographic proximity. The distance metric used can be either Euclidean or great-circle distance, depending on the specific use case. When either the queries or objects data contain non-point geometries and the sphere distance model is selected (use_sphere = true), WherobotsDB calculates the centroid of each geometry to perform the KNN join. With the planar model (use_sphere = false), distances are the true minimum distance between the two geometries, with no centroid approximation. This function returns the k-nearest geometries from the object side table for each geometry in the queries side table.
SQL Example:
- Partitioning: Increase the number of partitions with df.repartition(n) to ensure better distribution of data across nodes.
- Broadcasting: For smaller datasets, WherobotsDB will use a broadcast join to avoid shuffling large amounts of data.
Create a User-Defined Function (UDF)
User-Defined Functions (UDFs) are user-created procedures that can perform operations on a single row of information. To cover almost all use cases, we will showcase 4 types of UDFs for a better understanding of how to use geometry with UDFs. Sedona’s serializer deserializes the SQL geometry type to JTS Geometry (Scala/Java) or Shapely Geometry (Python). You can implement any custom logic using the rich ecosystem around these two libraries.Geometry to primitive
This UDF example takes a geometry type input and returns a primitive type output:- Scala
- Java
- Python
Geometry to Geometry
This UDF example takes a geometry type input and returns a geometry type output:- Scala
- Java
- Python
Geometry, primitive to geometry
This UDF example takes a geometry type input and a primitive type input and returns a geometry type output:- Scala
- Java
- Python
Geometry, primitive to Geometry, primitive
This UDF example takes a geometry type input and a primitive type input and returns a geometry type and a primitive type output:- Scala
- Java
- Python
Spatial vectorized UDFs (Python only)
By default, when you create a user defined function (UDF) in Python, that function isn’t vectorized. Processing User-Defined Functions (UDFs) one row at a time often leads to slower performance. A faster alternative is to use vectorized UDFs, which leverage Apache Arrow to handle data in batches. To create a vectorized UDF, use the@sedona_vectorized_udf decorator.
Currently, @sedona_vectorized_udf supports scalar UDFs.
Vectorized UDFs offer significantly improved performance, potentially running up to 2x faster than standard UDFs.
When you use geometry as an input type, please include the BaseGeometry type,
like Point from shapely or geopandas GeoSeries, when you use GEO_SERIES vectorized UDF.
That’s how Sedona infers the type and knows if the data should be cast.
udf_type is the type of the UDF function. Supported UDF types are:
SHAPELY_SCALARGEO_SERIES
Shapely scalar UDF
GeoSeries UDF
S2/H3 based approximate equi-join
When the performance of the WherobotsDB-optimized join doesn’t suit your workflow needs, possibly because of intricate and overlapping geometries, WherobotsDB also provides Google S2 or Uber H3-based approximate equi-joins as alternatives. The equi-joins leverage Spark’s internal equi-join algorithm and might be more performant given that you can choose to skip the refinement step. However, skipping this step can sacrifice query accuracy. Please use the following steps:1. Generate cell ids for both tables
Use ST_S2CellIds or ST_H3CellIds to generate cell IDs. Each geometry may produce one or more IDs.2. Perform equi-join
Join the two tables by their cellId3. Optional: Refine the result
Due to the nature of S2/H3 Cellid, the equi-join results might have a few false-positives depending on the S2/H3 level you choose. A smaller level indicates bigger cells, less exploded rows, but more false positives. To ensure the correctness, you can use one of the Spatial Predicates to filter out them. Use this query instead of the query in Step 2.ST_Contains, to remove false positives. You can also use ST_Intersects and so on.
4. Optional: De-duplicate
Due to the explode function used when we generate S2/H3 Cell Ids, the resulting DataFrame may have several duplicate<lcs_geom, rcs_geom> matches. You can remove them by performing a GroupBy query.
first function is to take the first value from a number of duplicate values.
If you don’t have a unique id for each geometry, you can also group by geometry itself. See below:
If you are doing point-in-polygon join, this is not a problem and you can safely discard this issue. This issue only happens when you do polygon-polygon, polygon-linestring, linestring-linestring join.
S2/H3 for distance join
This also works for distance join. You first need to useST_Buffer(geometry, distance) to wrap one of your original geometry column. If your original geometry column contains points, this ST_Buffer will make them become circles with a radius of distance.
For example. run this query first on the left table before Step 1.
distance should be degree instead of meter or mile. You will have to estimate the corresponding degrees based on your meter values. Please use this calculator.

