Integrate Amazon S3 with Wherobots using a guided CloudFormation workflow, so you can use Wherobots as the spatial engine on data that stays in your own S3 buckets.
Wherobots’ integration with Amazon Simple Storage Service (S3) lets you use Wherobots as the spatial engine that operates on your data while the data stays in your own Amazon S3 buckets.Accelerate your creation of spatial data products by working with data directly from Amazon S3 public or private buckets, bypassing the need for time-consuming data transfers.A guided workflow in Wherobots Cloud creates the integration for you: it generates pre-filled AWS CloudFormation templates that provision the AWS resources Wherobots needs, so you never have to hand-write IAM policies or trust relationships.
The S3 storage integration provides the following benefits:
Ease of access to data
Integrating your S3 buckets lets Wherobots Organization members work with data stored in Amazon S3 without manually transferring or duplicating it.
Guided, self-service setup
A wizard walks Admins through the integration and hands you pre-filled CloudFormation templates — no manual IAM policy editing required.
Reusable Cloud Connections
A single Cloud Connection (the IAM role Wherobots assumes to reach your account) can back multiple storage integrations.
Secure authentication
Wherobots accesses your account through cross-account IAM role assumption with attribute-based access control (ABAC) — no long-lived access keys are shared.
Control data access
Choose Read or Read & Write access per integration.
An S3 storage integration is built from two pieces, each provisioned by its own pre-filled CloudFormation stack in your AWS account:
Cloud Connection (the trust relationship)
A Cloud Connection is the reusable IAM role Wherobots assumes to reach your AWS account — one connection can back multiple storage integrations and Glue catalogs. It’s created by the connection stack.See Cloud Connections for the full definition and when to create one.
Storage integration (the bucket grant)
The storage integration grants a Cloud Connection access to a specific S3 bucket (or a prefix within it). A second CloudFormation stack attaches a policy to the connection role.A Read & Write grant includes the multipart-upload support required by Spark and Iceberg, which enables Managed Catalogs.
Both CloudFormation stacks in the AWS console after a successful setup: the connection (trust relationship) stack and the storage (bucket grant) stack.
Public buckets skip the Cloud Connection. When you point the wizard at a publicly accessible bucket, Wherobots can read it without assuming a role, so the connection step is skipped. A Cloud Connection is required for both private bucket and requester pays buckets.
A Cloud Connection is a single, reusable trust relationship between your Wherobots Organization and a cloud provider account. For AWS, it’s a first-class object that stores your AWS account ID and an IAM role (with an external ID) that Wherobots assumes on your behalf to access resources in your account.
Create one Cloud Connection per AWS account and reuse it across all of your S3 storage integrations and AWS Glue catalogs in that account, instead of configuring credentials separately for each one.
Trust is established with AWS CloudFormation. Create the connection directly in the Wherobots UI, or download the CloudFormation template (YAML) so your security team can review it before deploying.
Cloud Connections live in Organization Settings under Cloud Connections, where Admins can create, list, verify, and delete them.
Verify: Confirms Wherobots can assume your IAM role through the two-hop AWS STS AssumeRole chain.
Delete: Blocked while any storage integration or Glue catalog is still bound to the connection. You must remove any resources bound to that Cloud Connection prior to its deletion.
The following requirements must be met within both your Wherobots Organization and AWS account before creating an S3 storage integration:
Wherobots Requirements
An Admin account within a Professional, Innovation, or Enterprise Edition Organization.
Wherobots Organization members with the User role can use existing integrations set up by Admins but cannot create new ones. See Organization Roles.
AWS Requirements
An AWS account.
Permission to create CloudFormation stacks that provision IAM resources in that account (typically AdministratorAccess). CloudFormation creates the IAM role and policies for you, so you don’t edit them by hand.
What CloudFormation provisions
The pre-filled templates create or modify IAM resources on your behalf. The actions they perform typically require AdministratorAccess:
IAM Action
Description
CreateRole
Creates the Cloud Connection IAM role
UpdateAssumeRolePolicy
Sets the trust policy that lets Wherobots assume the role
PutRolePolicy
Attaches the inline policy granting bucket access
AttachRolePolicy
Attaches a managed policy to the role
DeleteRolePolicy
Removes an inline policy (on stack update or delete)
DetachRolePolicy
Detaches a managed policy (on stack update or delete)
The following Amazon S3 bucket types can be integrated with Wherobots:
Public Bucket
A public bucket on Amazon S3 is a bucket that has turned off Amazon S3’s default Block all public access option. Public buckets can be read without a Cloud Connection.
Granting external write access to a public S3 bucket is strongly discouraged. Use a private bucket for Managed Catalogs.
Private Bucket
A private bucket on Amazon S3 is a bucket that keeps the default Block all public access option enabled. Private buckets require a Cloud Connection. This is the recommended option for Managed Catalogs.
Requester Pays Bucket
In Amazon S3, a Requester Pays bucket shifts the responsibility for the cost of the request and the data download from the bucket owner to the person accessing the data.
Additional fees applyAccessing data from Requester Pays buckets will result in additional fees charged to you, not the bucket owner.
For more information on Amazon S3 buckets, see Creating a bucket in the Amazon S3 documentation.
A storage integration grants Wherobots server-side access to your bucket: compute runtimes assume the Cloud Connection role and read objects through the S3 API. Visualizing that data in Wherobots Cloud takes a different path — the browser reads objects directly from your bucket over HTTPS range requests.Those browser reads are subject to Cross-Origin Resource Sharing (CORS), and the CloudFormation stacks in this workflow create IAM resources only — they never touch your bucket’s CORS configuration. If you plan to visualize data stored in your own bucket, allow-list the Wherobots Cloud origins in the bucket’s CORS policy as part of creating the integration.
Setting up CORS
The CORS policy Wherobots recommends, why it’s needed, and how to apply it to your bucket.
On the Wherobots Hosted platform, follow the wizard below as-is — the integration is available to Wherobots Cloud compute automatically.
If your Organization uses Bring Your Own Cloud (BYOC), the wizard shows an additional Allowed compute option while you create the integration. Use it to choose which compute environments — Wherobots Cloud and/or your BYOC region — may access the integration.
BYOC integration wizard with Allowed compute option
To keep a bucket reachable only from your BYOC region, scope Allowed compute to that region and exclude the Wherobots Hosted platform by unchecking it. See Data isolation & access boundaries.
This workflow switches between Wherobots Cloud and the AWS Console. Each time you launch a CloudFormation stack, Wherobots opens the AWS Console in a new tab with every parameter pre-filled — you only need to acknowledge and create the stack, then return to Wherobots.
S3 Path Restriction: Bucket paths cannot contain periods. For example, s3://my.bucket.name is not allowed. Acceptable paths can consist of alphanumeric characters, underscores, equal signs, and dashes.
You can also start this wizard from the Storage Integration card on the Data Hub empty state.
The Storage page, with the Create Storage Integration button in the top right.
2
Enter the integration details
On the Details step, enter:
Name — a name for this integration (must include at least one letter).
S3 Path — your bucket path, optionally with a prefix, prefaced by s3:// (for example, s3://my-bucket or s3://my-bucket/optional/prefix).
Click Check Bucket & Continue. Wherobots checks whether the bucket is publicly accessible or requires a Cloud Connection.
Step 1, Details: name the integration and enter the S3 path.
If the bucket is public, the wizard skips the Connection step and takes you straight to Deploy.
3
Choose or create a Cloud Connection
For a private bucket, the Connection step asks you to pick an existing Cloud Connection or create a new one.
Only Admins can create Cloud Connections. If you don’t have permission, ask your Admin to create one for you.
Reuse a connection: Select an existing connection from the dropdown, then continue to the next step.
Create a connection: Select Create new connection and follow the steps below.
Create a new Cloud Connection
Creating a connection takes two steps inside the Create Cloud Connection dialog.
1
Enter the connection details
Connection Name — a label that identifies the trust relationship in Wherobots.
AWS Account ID — the 12-digit AWS account you’re connecting to.
Click Create Connection.
Connection details: name the connection and enter your AWS Account ID.
2
Grant access in AWS
Wherobots generates a pre-filled CloudFormation stack that creates the connection’s IAM role. Click Open in AWS Console to launch it in a new tab, or Download Template to run it yourself.
Grant access in AWS: launch the pre-filled CloudFormation stack that creates the Cloud Connection role.
On the AWS Quick create stack page, everything is pre-filled. Scroll to the bottom, select I acknowledge that AWS CloudFormation might create IAM resources, and click Create stack.
On the AWS Quick create stack page, acknowledge the IAM capability and click Create stack.
Back in Wherobots, click Done to finish creating the connection.
4
Choose an access level
With a connection selected, choose how much access Wherobots should have to this path:
Read: Wherobots can read objects from this path.
Read & Write: Wherobots can read and write objects, enabling Managed Catalogs.
Click Continue.
5
Deploy the storage stack
On the Deploy step, launch the second CloudFormation stack — the storage grant stack. This grants the connection access to your bucket. The parameters are pre-filled.
Click Open in AWS Console (or Download Template).
On the AWS Quick create stack page, select I acknowledge that AWS CloudFormation might create IAM resources, and click Create stack.
Return to Wherobots and click Create integration.
Step 3, Deploy: launch the pre-filled CloudFormation stack that grants the connection access to your bucket.
6
Verify access
On the Done step, Wherobots confirms the integration was created. After your CloudFormation stack finishes, click Verify Access.Wherobots checks read access — and, for a Read & Write integration, write access. When the checks pass, click Done.
A successful verification confirms read and (for Read & Write) write access.
IAM role policies can take a few minutes to propagate across AWS. If verification fails immediately after creating the stack, wait a moment and click Retry.
The Done step also links to where you can use and manage the new integration:
Browse your files in Storage opens the Storage browser with the new integration selected.
Organization Settings > Storage is where Admins manage the integration later.
7
Add a CORS policy for browser visualization
The stacks you just deployed grant Wherobots server-side access to the bucket. If you plan to visualize that data in Wherobots Cloud — on a Wherobots-GL map in a notebook, or in the hosted PMTiles viewer — allow-list the Wherobots Cloud origins in the bucket’s CORS configuration. Setting up CORS has the recommended policy and how to apply it in the S3 console.This is because the requests come from the browser rather than the server. Browsers enforce CORS as a security mechanism for the user.
A storage integration is not a catalog. Creating a storage integration connects Wherobots to your S3 bucket, but it doesn’t expose your data as a queryable catalog. To query the data, create a catalog from the bucket in the Data Hub: click Add Catalog, then choose your storage under Catalog in. See Create a Managed Catalog.
After creating a Managed Catalog from your S3 storage integration, access your data using the catalog reference format:
CATALOG_NAME.DATABASE_NAME.TABLE_NAME
To use new storage integrations or catalogs in your notebooks, you must start a new runtime. Notebooks can only access integrations created before the runtime started.
# Replace `CATALOG_NAME`, `DATABASE_NAME`, and `TABLE_NAME` with your specific names.from sedona.spark import *config = SedonaContext.builder().getOrCreate()sedona = SedonaContext.create(config)# Access data using catalog.database.table formatdf = sedona.table("CATALOG_NAME.DATABASE_NAME.TABLE_NAME")df.printSchema()df.show()
# Replace `CATALOG_NAME`, `DATABASE_NAME`, and `TABLE_NAME` with your specific names.from sedona.spark import *config = SedonaContext.builder().getOrCreate()sedona = SedonaContext.create(config)# Access data using catalog.database.table formatdf = sedona.sql("SELECT * FROM CATALOG_NAME.DATABASE_NAME.TABLE_NAME")df.show()
A Managed Catalog can be created from a Read & Write private bucket storage integration at any time, allowing for multiple catalogs per integration.
Use private buckets for Managed CatalogsGranting external write access to a public S3 bucket is strongly discouraged. Use a private S3 bucket for your Managed Catalog.
A Managed Catalog is a metadata repository that is created, owned, and controlled directly within your Wherobots Organization.When you connect a data source like an S3 bucket and register it as a managed catalog, Wherobots takes on the following responsibilities:
Source of Truth
Wherobots becomes the authoritative source for all metadata, including schemas, table definitions, file locations, and partition information.
Data Discovery
Wherobots actively scans the underlying storage (e.g., S3) to discover new data and automatically update the catalog.
Lifecycle Management
Wherobots handles all metadata operations, such as creating, updating, and deleting tables. Changes in the underlying data are automatically synced to the catalog.
Optimization
Because Wherobots has full control, it can build and manage advanced spatial indexes and perform other performance optimizations directly on the metadata.
You typically use a managed catalog when your raw spatial data files reside in an AWS S3 private
bucket and you want Wherobots to handle all aspects of data management, query optimization, and spatial ETL.
Name: Alphanumeric characters, spaces, special characters, or underscores (must include at least one letter). No dashes or periods allowed.
Path (Optional): Enter the sub-folder where you’d like to store this Managed Catalog
Runtime Restart Required After Data IntegrationTo use new storage integrations or catalogs in your notebooks, you must start a new runtime.
Notebooks can only access storage integrations or catalogs that were created before the runtime started.
The following limitations apply to S3 storage integrations:
Current limitations
Bucket paths cannot contain periods (e.g., s3://my.bucket.name is not allowed)
A bucket can only be configured with a single storage integration
Public buckets are read-only and cannot be used for Managed Catalogs due to the write access requirements
Creating an integration requires permission to deploy CloudFormation stacks that provision IAM resources in your AWS account
The CloudFormation stacks grant IAM access only; they do not set the bucket’s CORS configuration. Visualizing data from your own bucket requires a CORS policy on the bucket
Was this page helpful?
Assistant
Responses are generated using AI and may contain mistakes.
