Introducing the STAC Reader for Wherobots

​

Benefits

• Direct STAC Integration: Easily access and load data from the growing ecosystem of STAC-compliant catalogs directly into your Wherobots workflows.
• Unified Geospatial Analysis: Leverage Sedona’s powerful spatial and temporal data processing capabilities on data loaded from STAC sources.
• Performance Optimization: The reader features Spatial and Temporal Filter Pushdown, allowing you to apply spatial (e.g., st_contains, st_intersects) or temporal (e.g., datetime BETWEEN) filters in your queries.
  • These filters are pushed down to the STAC API level where they’re supported by the API.
  • This significantly reduces the amount of data that needs to be transferred and processed, leading to faster query execution.
• Flexible Data Access: Connect to STAC collections through an HTTP/HTTPS endpoint, an S3-compatible object store, or a local JSON file.

​

Key Considerations

• Configuration Options: You can fine-tune the reader’s behavior using Sedona configuration properties (e.g., spark.sedona.stac.load.maxPartitionItemFiles, spark.sedona.stac.load.numPartitions) and reader options (e.g., itemsLimitMax, itemsLoadProcessReportThreshold, itemsLimitPerRequest).
  • These configurations allow you to control partitioning, limit the number of items loaded, manage API request sizes, and monitor loading progress.
• Resulting Schema: Data loaded via the STAC reader results in a Sedona DataFrame with a predefined schema reflecting STAC item properties, including metadata, geometry, timestamps, links, and assets.
• API Limits: When accessing public or private STAC APIs, be mindful of potential rate limits or query constraints imposed by the API provider.
  • The itemsLimitPerRequest option can help manage this.

​

Get Started

1. Use the Spark data source reader:
   1. Load the STAC data
   2. Apply optional configuration
   3. Query the loaded data.
2. Use the dedicated Python STAC Client.

​

1. Load STAC Data into a DataFrame

df = sedona.read.format("stac").load("https://earth-search.aws.element84.com/v1/collections/sentinel-2-pre-c1-l2a")
df.printSchema()
df.show()

​

2. Apply Configuration Options

# Example applying reader options
df = sedona.read \
    .format("stac") \
    .option("itemsLimitMax", "1000") \
    .option("itemsLimitPerRequest", "50") \
    .load("https://earth-search.aws.element84.com/v1/collections/sentinel-2-pre-c1-l2a")

​

3. Query Loaded Data (Leveraging Filter Pushdown)

SELECT id, geometry
FROM STAC_TABLE
WHERE st_contains(ST_GeomFromText('POLYGON((...))'), geometry)

​

Use the Python STAC Client

from sedona.spark import *
import datetime

# Initialize the client
client = Client.open("https://planetarycomputer.microsoft.com/api/stac/v1")

# Search for items within a date range and get a DataFrame
df_items = client.search(
    collection_id="sentinel-2-l2a",
    datetime=["2023-01-01T00:00:00Z", "2023-01-05T00:00:00Z"],
    bbox=[-74, 40, -73, 41], # Example bbox for New York area
    return_dataframe=True,
    max_items=10
)

df_items.show()

# Save results to GeoParquet
client.get_collection("sentinel-2-l2a").save_to_geoparquet(
    output_path="/path/to/output/s2_data",
    bbox=[-74, 40, -73, 41],
    datetime="2023-01"
)

​

Read the Documentation

​

Additional Resources

• STAC Specification: https://stacspec.org/
• STAC Browser (Example Tool): https://github.com/radiantearth/stac-browser