Changelog

​

Spark 4.0 & Hadoop 3.4 Upgrade

• ANSI SQL Enabled by Default: The default SQL model has changed to ANSI SQL.
  • Action Required: Review and migrate existing SQL queries to ensure compliance with ANSI standards.
• Third-Party JAR Updates: Existing data source JARs may no longer be compatible.
  • Action Required: Replace current jars with versions that support Spark 4.

• Hadoop JAR Updates: Third-party Hadoop libraries require updates.
  • Action Required: Replace current jars with versions that support Hadoop 3.4.

​

Enhanced SSO Methods and Passkeys

• Google account login
• GitHub account login
• Passkeys: Register with passkeys for faster and more secure sign-in using device biometrics or screen lock

​

Multi-Organization Support

​

AWS ap-south-1 (Mumbai) Region Support

• Lower latency: Deploy closer to users, data sources, or endpoints in India and South Asia
• Architectural flexibility: Build geographically distributed applications for South Asia

​

Data Federation for Databricks Unity Catalog

• Zero-Copy architecture: Read tables managed by Unity Catalog without moving or duplicating data
• Maintained governance: Databricks Workspace Admins retain catalog- and table-level access control
• Secure federation: Connect securely using Databricks authentication credentials
• Accelerated innovation: Use Wherobots’ 300+ spatial functions, raster inference, and compute on Unity Catalog data

• Direct Unity Catalog connection from Wherobots Data Hub
• Full read and write operations on Managed Iceberg tables
• Read access to Delta Lake tables
• Advanced Spatial analytics with 300+ spatial functions

​

Enhanced Accuracy for ST_Transform

SELECT ST_Transform(
  geom,
  'EPSG:4326',
  'EPSG:26910',
  '{"area_of_interest": "POLYGON((-122.5 37.7, -122.3 37.7, -122.3 37.8, -122.5 37.8, -122.5 37.7))"}'
)
FROM my_table;

​

GeoTools Upgrade, Python Path Standardization, and WKB Writer Update

• GeoTools Library Upgrade: The GeoTools library has been upgraded from v28.5 to v33.1. This could introduce changes in behavior for ST_Transform and various raster functions.
• Python Import Path Standardization: Python import paths under sedona.XXX are now deprecated in favor of sedona.spark.XXX.
• WKB Writer Default Byte Order: The default WKB writer byte order has changed to little-endian. This change might affect GeoParquet files, Havasu-Iceberg tables, and ST_AsWKB functions.

• Community Organizations: Share your feedback in the #vendor-wherobots channel on the Apache Sedona Discord.
• Professional & Enterprise Organizations: Send feedback to [email protected].

​

WherobotsDB 2.0 Preview

​

Dekart Integration

• Go from Spatial SQL to an interactive map in seconds
• Create up-to-date visualizations from large, dynamic datasets
• Avoid proprietary lock-in with open-source architecture

• Direct authentication with Wherobots API keys
• Query with Spatial SQL from Dekart interface
• Instant visualization with Kepler.gl maps
• Bring Your Own Data from AWS S3

​

Enhancing Isochrone and Reverse Geocoding

​

New Address Parsing and Standardization Functions

• ExpandAddress: Normalizes and expands address strings into their canonical forms (e.g., “st” becomes “street,” “apt” becomes “apartment”)
• ParseAddress: Breaks down a single address string into its individual components (house number, street, city, state, postcode)

​

Hadoop Dependency Update

​

Security Updates and Scala 2.13 Migration

• Customers who compile and deploy custom Scala applications or UDFs as .jar files
• Users of the Scala kernel for Scala notebooks

• Update your build environment to use a Scala 2.13.x compiler
• Update all dependencies from their _2.12 versions to their _2.13 versions
• Recompile your project to produce a new .jar file compatible with Scala 2.13

• Bug fixes for improved performance and reliability
• Dependency security updates with latest patches

​

Stability and Security Enhancements

​

Vectorized Python UDFs and OSM Data Support

• New Feature: Added support for vectorized User Defined Functions (UDFs) in Python
• New Feature: Introduced a parser for OpenStreetMap (OSM) nodes data within the OSM PBF reader
• Improvement: Enhanced error messaging for invalid geometries during tile generation

​

Java 17, GDAL 3.8.4, Spark 3.5.5 & Ubuntu 24.04

• Operating System: Upgraded from Ubuntu 22.04 to 24.04 LTS
• GDAL: Upgraded from version 3.4.1 to 3.8.4
• Java: Standardized on Java 17 (LTS), moving from Java 19
• Spark: Updated from 3.5.4 to 3.5.5

• Java 17 (LTS): Custom code, UDFs, or JAR files compiled for Java 19 will need recompilation using Java 17 SDK
• GDAL 3.8.4: May alter behavior of certain geospatial functions. Review GDAL release notes for versions 3.5-3.8

​

AWS us-east-1 (N. Virginia) Region Support

• Lower Latency: Deploy closer to US East Coast users and data sources
• Optimized AWS Ecosystem Integration: Seamless integration with existing AWS resources in us-east-1
• Data Residency & Compliance: Meet data residency requirements on the US East Coast
• Architectural Flexibility: Build geographically distributed applications

​

EU Region Support (AWS eu-west-1)

• Compliance: Run notebook compute resources within the EU to aid in meeting GDPR and data sovereignty requirements
• Latency Reduction: Potentially increase performance for data sources located in Europe
• Workflow Efficiency: Set a default EU region for new notebooks

• Notebooks, Job Runs, and SQL Sessions can run in eu-west-1
• Default Region Preference for Admins
• Wherobots File System storage located within your selected region

​

SAM2 Support in WherobotsAI Raster Inference

• RS_Text_to_BBoxes: Text-prompted object detection returning bounding boxes, confidence scores, and labels
• RS_Text_to_Segments: Text-prompted instance segmentation returning segmentation polygons, confidence scores, and labels

​

Audit Logs

• Enhanced Security: Monitor key activities to identify potential security risks
• Improved Accountability: Track user actions and system events
• Operational Transparency: Gain insights into resource usage
• Data Retention: Use the API to preserve logs beyond Wherobots’ 90-day retention period

• Comprehensive activity tracking across security, files, jobs, notebooks, organization, data, and runtimes
• Intuitive UI with flexible filtering
• On-demand CSV export from UI
• Programmatic export via REST API
• Admin-only access

​

Travel Isochrones Functionality and Dataset

• ST_Isochrone: Generate a single polygon representing the reachable area within a specified time
• ST_Isochrones: Generate an array of polygons for multiple time limits
• Dataset Enrichment: Overture Places enriched with 4 new isochrone geometry columns

-- ST_Isochrone Example
ST_Isochrone(geometry, 1, 'car', false)

-- ST_Isochrones Example
ST_Isochrones(geometry, array(1, 1.5, 2), 'car', false, true)

​

Job Run Monitoring

• Job Runs Summary View with status, trigger, duration, and details
• Detailed information including consumption metrics (Spatial Units, CPU, Memory)
• Filtering capabilities by name
• Job Run cancellation directly from the UI
• Configuration information display

​

Idle Timeout for Notebooks

• Customizable Timeout: Select 15 minutes, 45 minutes (default), or 120 minutes
• Persistent User Settings: Your chosen setting is saved across sessions
• Automatic Kernel Termination: Jupyter kernel terminates after idle period
• Notebook Instance Specific: Only applies to Notebook Instances, not Job Runs

​

STAC Reader for Wherobots

• Direct STAC Integration: Access STAC-compliant catalogs directly in Wherobots workflows
• Performance Optimization: Spatial and Temporal Filter Pushdown reduces data transfer and processing
• Flexible Data Access: Connect via HTTP/HTTPS endpoint, S3-compatible object store, or local JSON file

df = sedona.read.format("stac").load("https://earth-search.aws.element84.com/v1/collections/sentinel-2-pre-c1-l2a")
df.printSchema()
df.show()

​

1.4 - A Major Leap Forward for Wherobots

​

Wherobots 1.3

• Spatial query engine: compatible with Apache Sedona
• Havasu Iceberg data lake storage layer: compatible with Apache Iceberg
• Vtiles: a scalable PMTiles-style vector tile generator

• Raster inference: a scalable raster inference engine in SQL and Python
• Map matching: a scalable map matcher for snapping massive GPS traces to the road network

​

Wherobots 1.2

• Spatial query engine: compatible with Apache Sedona
• Havasu Iceberg data lake storage layer: compatible with Apache Iceberg