Skip to content

SAM2 Support in WherobotsAI Raster Inference 🆕

Wherobots is excited to announce enhancements to WherobotsAI Raster Inference, including the introduction of Text-Based Raster Detection & Segmentation.

To achieve this, we've integrated support for Meta AI's Segment Anything Model 2 (SAM2) and Google DeepMind's OWLv2 models for text-prompted inference.

Additionally, we've created 2 new Raster Inference functions (RS_Text_to_BBoxes and RS_Text_to_Segments) for converting text prompts to segmentation and object detection results.

Finally, we're releasing a new example notebook to showcase these capabilities. Our new example notebook demonstrates using the simple text prompt "airplanes" with RS_Text_to_BBoxes to detect georeferenced bounding boxes for airplanes and RS_Text_to_Segments to outline detected airplanes with georeferenced polygon segments.

segment-example

Visualization of Raster Inference Results

WherobotsAI Raster Inference Requires GPU Access

Access to WherobotsAI Raster Inference features, including the new SQL functions (RS_Text_to_BBoxes, RS_Text_to_Segments) and associated models (e.g., owlv2, sam2), requires execution on GPU-Optimized runtimes within Wherobots Cloud.

By default, access to GPU-Optimized runtimes is limited to Professional and Enterprise Edition Organizations who have specifically requested and been granted access. Community Edition Organizations do not have access to GPU runtimes.

Professional and Enterprise Organizations needing GPU access for Raster Inference should submit a compute request to support@wherobots.com.

Refer to Wherobots Pricing for details on paid tiers.

Paid Organization Setup:

Benefits

  • Text-Prompted Analysis: Utilize natural language prompts with RS_Text_to_BBoxes (OWLv2) and RS_Text_to_Segments (SAM2) for flexible feature identification.
  • Scalable Processing: Apply AI models, including the newly integrated OWLv2 and SAM2, to large-scale raster datasets leveraging Wherobots' distributed computing environment.
  • Integrated Geospatial Workflow: Perform data loading, preprocessing, AI inference, and visualization within a single platform.
  • Efficient Feature Extraction: Provides tools to identify, classify, and delineate features from raster data based on pre-trained models or text descriptions.

Key Features

  • Text-Prompted Inference Functions:
    • RS_Text_to_BBoxes: Performs text-prompted object detection on raster images using specified models (owlv2). Returns bounding boxes (WKT), confidence scores, and labels.
    • RS_Text_to_Segments: Performs text-prompted instance segmentation using specified models (sam2). Returns segmentation polygons (WKT), confidence scores, and labels.
  • Integrated Model Support:
    • WherobotsAI's Raster Inference product has integrated the OWLv2 and SAM2 models, allowing for text-based object detection and instance segmentation.

New Example Notebook: Text-Based Object Detection and Segmentation

We're also releasing a new example notebook that demonstrates the usage of RS_Text_to_BBoxes (with OWLv2) and RS_Text_to_Segments (with SAM2).

  • Goal: Detect and segment airplanes in NAIP imagery of Miami airport using the text prompt "airplanes".
  • Content: Includes steps for loading raster data, applying the text-prompted SQL functions, processing results (filtering, exploding arrays), and visualizing detections/segments using SedonaKepler and show_detections.

For information on accessing this example notebook, review the Get Started section.

Important Considerations

  • The performance of text-prompted functions (RS_Text_to_BBoxes, RS_Text_to_Segments) is dependent on feature contrast, clarity of the text prompt, and image resolution/quality.
  • Confidence scores provide a relative measure of certainty, but results from AI models should be validated, especially for critical applications. Human review is recommended.
  • The suitability of specific pre-trained models (landcover-eurosat-sentinel2, solar-satlas-sentinel2, marine-satlas-sentinel2, owlv2, sam2) varies based on input imagery characteristics and the target features.

Get Started

Access Wherobots Cloud to utilize these new Raster Inference features.

Access Wherobots Cloud

To execute the cells in this Jupyter Notebook, do the following:

  1. Log in to Wherobots Cloud.
  2. Start a GPU-Optimized runtime instance.
  3. Open a notebook. We recommend using a Tiny GPU-Optimized runtime.
  4. Click File > Open from Path....
  5. Enter the examples/Analyzing_Data/Raster_Text_To_Segments_Airplanes.ipynb path.

Read the Documentation

Refer to the Wherobots documentation for detailed information: