> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wherobots.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Jupyter Notebook Management

> Work with JupyterLab in Wherobots Cloud, including executing notebooks, importing custom Python modules, monitoring Spark jobs, and exporting notebooks for job submission.

After starting a notebook instance, Wherobots Cloud opens a pre-configured JupyterLab environment where you can write and execute spatial queries, visualize geospatial data, and build analytics workflows.

For information on starting a runtime and creating a notebook instance, see [Notebook Instance Management](/develop/notebook-management/notebook-instance-management).

<Note>
  The introductory notebook appears when you first open a Wherobots Cloud Notebook. After that, you will see the JupyterLab **Launcher**.
</Note>

<Frame caption="JupyterLab Launcher">
  <img src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/jupyter-dashboard.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=6771fcb818a1ed1508dc9b78a2aa6ab5" alt="JupyterLab Launcher" width="3038" height="1780" data-path="images/develop/notebook-management/jupyter-notebook-management/jupyter-dashboard.png" />
</Frame>

## Before you start

The following is required to manage a Wherobots Notebook:

* An account within a Community, Professional, or Enterprise Edition Organization. For more information, see [Create an Account](/get-started/wherobots-cloud/create-account).

## Available kernels

Wherobots Cloud provides two kernels for your Jupyter Notebooks:

* **Python kernel (ipykernel)** -- for Python-based spatial workflows using Apache Sedona and WherobotsDB.
* **Scala kernel (Scala)** -- for Scala-based spatial workflows.

To create a new notebook with either kernel, click **File > New Launcher** in JupyterLab and select the kernel.

## Execute notebook cells

### Run all cells

To execute all code cells in a notebook, do the following:

1. Click **Run** in the JupyterLab toolbar.
2. Click **Run All Cells**.

<Frame caption="Run all cells from the toolbar">
  <img alt="Execute all cells" src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/execute-all.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=b955193b2015046f3137265c77325f6a" width="400" data-path="images/develop/notebook-management/jupyter-notebook-management/execute-all.png" />
</Frame>

<Note>
  When you first execute a WherobotsDB code cell, you may see the following warning:

  ```
  WARN TaskSchedulerImpl: Initial job has not accepted any resources;
  check your cluster UI to ensure that workers are registered and have sufficient resources
  ```

  This is expected behavior. Executors take 1-5 minutes to start depending on the runtime size.
</Note>

### Monitor jobs with the Spark Web UI

The Spark Web UI helps you monitor running jobs, analyze query performance, and identify bottlenecks. To access it, click **Sedona Spark** in JupyterLab and select the correct port number.

<Frame caption="Spark Web UI">
  <img src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/spark-ui.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=ede8c1aaa8555fa03accc3277be50e44" alt="Spark UI" width="973" height="551" data-path="images/develop/notebook-management/jupyter-notebook-management/spark-ui.png" />
</Frame>

To retrieve the port number programmatically, run the following in a notebook cell:

```python theme={"system"}
# Get the Spark Web UI port number
spark_ui_port = sedona.sparkContext.uiWebUrl.split(":")[-1]
print(spark_ui_port)
```

For more information, see [Web UI](https://spark.apache.org/docs/latest/web-ui.html) in the Apache Spark documentation.

## Import custom Python modules

You can package custom Python modules as a zip file and import them into your notebook. This is useful for reusing shared utility functions across notebooks and job submissions.

### Create and upload a module

1. On your local machine, create a directory for your module:

   ```shell theme={"system"}
   mkdir zipmoduletest
   ```

2. Create a Python file with your module code. For example, create `zipmoduletest/hellosedona.py`:

   ```python theme={"system"}
   def hello(input):
       return 'hello ' + str(input)
   ```

3. Add an empty `__init__.py` file to make the directory a valid Python package:

   ```shell theme={"system"}
   touch zipmoduletest/__init__.py
   ```

4. Verify the directory structure:

   ```shell theme={"system"}
   ls zipmoduletest
   ```

   The output should show:

   ```
   __init__.py       hellosedona.py
   ```

5. Zip the module files:

   ```shell theme={"system"}
   cd zipmoduletest && zip -r9 ../zipmoduletest.zip *
   ```

6. Upload `zipmoduletest.zip` to Wherobots Managed Storage or your integrated Amazon S3 bucket. For more information, see [Notebook and Data Storage](../storage-management/storage).

### Use the module in a notebook

After uploading, add the zip file to the Spark context and import your module:

```python theme={"system"}
# Add the zip file to the Spark context
sedona.sparkContext.addPyFile('s3://<your-bucket>/path-to-file/zipmoduletest.zip')

# Import and use the custom module
from zipmoduletest.hellosedona import hello

result = hello("Sedona")
print(result)
```

Output:

```
hello Sedona
```

<Tip>
  You can also use `addPyFile` to import custom Python modules in job submissions, not just notebooks.
</Tip>

## Access data from Amazon S3

For information on reading and writing data from an integrated Amazon S3 bucket, see [Access Integrated Storage in a Notebook](../storage-management/s3-storage-integration#access-integrated-storage-in-a-notebook).

<Note>
  To use new storage integrations or catalogs in your notebooks, you must start a new runtime. Notebooks can only access storage integrations or catalogs that were created before the runtime started.
</Note>

## Open a notebook by path

To open a specific notebook file, do the following:

1. Click **File > Open from path**.
2. Enter the notebook path.
3. Click **Open**.

## Export notebooks

You can export notebooks as executable scripts for use with [WherobotsRunOperator](../run-operator) job submissions.

### Export a Python notebook

To export a Python notebook as an executable script, do the following:

1. Click **File** in the JupyterLab toolbar.

2. Hover over **Save and Export Notebook As...**

   <Frame caption="Export notebook menu">
     <img alt="Save and export notebook" src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/create-executable.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=4fd5028a55df64b33200a2ecc3f0c6a5" width="300" data-path="images/develop/notebook-management/jupyter-notebook-management/create-executable.png" />
   </Frame>

3. Select **Executable Script**.

   <Frame caption="Select Executable Script">
     <img alt="Select executable script" src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/create-executable-continued.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=bf5012983aeb1ba7dfc27a155fea0a89" width="300" data-path="images/develop/notebook-management/jupyter-notebook-management/create-executable-continued.png" />
   </Frame>

   The `.py` file will download to your machine.

### Export a Scala notebook

To export a Scala notebook as an executable script, do the following:

1. Click **File** in the JupyterLab toolbar.

2. Hover over **Save and Export Notebook As...**

   <Frame caption="Export notebook menu">
     <img alt="Save and export notebook" src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/create-executable.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=4fd5028a55df64b33200a2ecc3f0c6a5" width="300" data-path="images/develop/notebook-management/jupyter-notebook-management/create-executable.png" />
   </Frame>

3. Select **Executable Script**.

   <Frame caption="Select Executable Script">
     <img alt="Select executable script" src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/create-executable-continued.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=bf5012983aeb1ba7dfc27a155fea0a89" width="300" data-path="images/develop/notebook-management/jupyter-notebook-management/create-executable-continued.png" />
   </Frame>

   The `.scala` file will download to your machine.

<Note>
  The exported Scala file does not include a `main` class. To make it runnable, wrap your code (excluding `import` statements) in an `App` object:

  ```scala theme={"system"}
  object MyApp extends App {
    // Your exported code here (excluding import statements)
  }
  ```
</Note>

<Note>
  You cannot execute `.scala` files in a Python kernel. Use the Scala kernel to run Scala code.
</Note>

You can import the Scala executable file into `sedona-maven-example/src/main/scala/com/wherobots/sedona/` for job submission.

### Build a JAR file from a Scala notebook

To package your Scala notebook as a JAR file for job submission, do the following:

1. Click **File > New Launcher** in JupyterLab.

   <Frame caption="Open launcher">
     <img alt="Open launcher" src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/open-launcher.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=7c3ab168ec0c06982fca2be2c36f2245" width="300" data-path="images/develop/notebook-management/jupyter-notebook-management/open-launcher.png" />
   </Frame>

2. Open **Terminal**.

   <Frame caption="Open terminal">
     <img alt="Open terminal" src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/open-terminal.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=f043a71cb64b40340fca8192d00f91cf" width="550" data-path="images/develop/notebook-management/jupyter-notebook-management/open-terminal.png" />
   </Frame>

3. Navigate to the `sedona-maven-example` directory and build the project:

   ```shell theme={"system"}
   cd sedona-maven-example
   mvn clean package
   ```

4. In the JupyterLab file browser, locate the `target` folder.

   <Frame caption="Target folder with built JAR">
     <img alt="Target folder" src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/target-folder.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=46639128cef80728aaa1da8f2bbb6b8a" width="300" data-path="images/develop/notebook-management/jupyter-notebook-management/target-folder.png" />
   </Frame>

5. Right-click `sedonadb-example-0.0.1.jar` and select **Download**.

   <Frame caption="Download the JAR file">
     <img alt="Download jar file" src="https://mintcdn.com/wherobots/ZUrkIWfbiuyJbzDN/images/develop/notebook-management/jupyter-notebook-management/download-jar-file.png?fit=max&auto=format&n=ZUrkIWfbiuyJbzDN&q=85&s=250ffb826dfb10cc22ba472b3ccec78c" width="300" data-path="images/develop/notebook-management/jupyter-notebook-management/download-jar-file.png" />
   </Frame>

<Tip>
  You can add custom dependencies to the `pom.xml` file located at `notebook-example/scala/sedona-maven-example`.
</Tip>

Once you have the JAR file, refer to [WherobotsRunOperator](../run-operator) to create a job.

## Session expiration

A warning about a lost server connection may appear after extended use. Wherobots uses cookies for authentication that expire after one hour for security. Refresh the page to be redirected to the login screen. After logging back in, you will return to your previous page.
