Notebook and data storage¶
This page provides detailed information about the Wherobots storage management.
Wherobots notebook and data storage guidance¶
When creating geospatial solutions, we recommend that you store all of your code, configuration files, and notebook files with in the Wherobots File System.
You can use other locations inside your Jupyter notebook environment for temporary data storage. For long-term persistence and storage of your spatial datasets, we recommend using S3 or other cloud storage options Wherobots integrates with or your own S3 bucket with cross-account S3 access.
You can use Wherobots Managed Storage or integrate external Amazon S3 storage with Wherobots.
Wherobots File System¶
Jupyter's top-level directory points to the /home/wherobots directory in your
Jupyter notebook environment's file system. This directory is backed by Wherobots File System
and automatically persisted across your notebook instances, and is only visible and accessible
to your Wherobots user.
To share notebooks with other members of your organization, you can download your
notebook files from your Jupyter notebook environment and exchange them with your
team members. You can also use Wherobots Managed Storage (see below) and upload
your notebook files into the S3 data folder /data/shared. You can then upload new
notebook files into Jupyter notebook environment with Jupyter's built-in upload function.
Wherobots Managed Storage¶
Wherobots provides out-of-the-box, managed storage for your geospatial datasets and Havasu tables, accessible from your notebooks and jobs with WherobotsDB. You can browse and manage the contents of your Wherobots Managed Storage with the Managed Storage page in Wherobots Cloud in Wherobots Cloud.
Wherobots Managed Storage allows you to see the file and directory structure of your data
warehouse on Wherobots Cloud.
Note
You can also use the Wherobots Managed Storage to upload files and data to be used by your spatial analytics tasks.
Folder permissions¶
The following top-level folders are read-only, indicating that it is not possible to create new folders, upload files, or delete any existing folders:
/(root)/data/
The following folders are where the user can create new folders, upload files, or delete any file within them:
/data/customer-XXXX/data/shared/spark-logs
Note
The /spark-logs folder allows read and write permissions, but it is recommended that users refrain from tampering with the logs, as it may affect their experience.
User specific folders¶
The folders with the name of customer-XXXX/ are unique to each user.
The files and folders within them can only be accessed by the user.
Organization-wide shared folders¶
Folders named shared/ grant access to everyone within your
organization. Any member of the organization can create new folders,
delete existing ones, and have the privileges to read, write, upload,
and delete files within this shared folder.
Copy a file or folder's S3 Path¶
If needed, you can copy the direct S3 path for your directory and files by clicking on the "copy" icon on the right-hand side of each displayed folder or file.
S3 Storage integration¶
Wherobots’ integration with Amazon Simple Storage Service (S3) allows S3 customers to utilize Wherobots as the spatial engine that operates on their data while still using Amazon S3 for their data storage.
For more information, see S3 Storage integration