Skip to main content

​
create_spatial_dataframe

Create a Spark DataFrame from a GeoPandas GeoDataFrame, transferring the data with Arrow. This is faster than spark.createDataFrame(gdf) for large GeoDataFrames. Geometry columns become Sedona geometry columns, and the other columns get the Spark types of their Arrow types.

​
Parameters

​
SparkSession
required
The Spark session.
​
gpd.GeoDataFrame
required
The GeoPandas GeoDataFrame to convert. It must have at least one geometry column.

​
Returns

​
DataFrame
A Spark DataFrame with the columns of gdf.

​
Raises

  • ValueError: If gdf has no geometry column.

​
dataframe_to_arrow

Collect a DataFrame as a PyArrow Table In the output Table, geometry will be encoded as a GeoArrow extension type. The resulting output is compatible with lonboard.viz(), geopandas.GeoDataFrame.from_arrow(), or any library compatible with GeoArrow extension types.

​
Parameters

​
required
A Spark DataFrame
​
default:"None"
A CRS-like object (e.g., pyproj.CRS or string interpretable by pyproj.CRS). If provided, this will override any CRS present in the output geometries. If omitted, the CRS will be inferred from the values present in the output if exactly one CRS is present in the output.

​
Returns

​
A pyarrow.Table.