Jupyter Notebook on HPC platforms at Mines

The following is only for advanced users needing a special environment using your own Jupyter lab version, custom build widgets, etc.
 
IMPORTANT: Most functionality that users will ever need from Jupyter Lab is immediately available at wendian-ondemand.mines.edu and mio-ondemand.mines.edu under “Interactive Apps.”
 
DO NOT follow this guide unless you have tried using the above links, and have opened a ticket for help!
Starting a Jupyter notebook server for your personal use during simple development or editing is acceptable when CPU core use and memory usage is low.
Login to the server with a SSH terminal session by making a tunnel from port 8788 open on your machine to the servers localhost and port 8888 on mio.mines.edu. These port numbers can be identical, for illustration purposes here they will clearly show which computers they belong too.
Connecting to a Jupyter Notebook running on a computer node requires starting an interactive job and a tunnel to the compute node. See below for an outline. For more details visit: https://jupyter.org

Required Modules

Wendian

[joeuser@wendian ~$] module load apps/python3

Mio

[joeuser@mio ~$] module load apps/python3

Wendian Compute Node Jupyter Notebook

Log into Wendian and load the module for python:
[joeuser@wendian001 ~]$ ml apps/python3
Start an interactive job with a single task to run the Jupyter Notebook server. This job will possible run on a shared node resource as it is requesting only a single core.
[joeuser@wendian001 ~]$ srun -n1 --pty bash -i
Start the Jupyter Notebook with no browser and looking for a connection on port 8888, and broadcast the ip hostname of the node name. Here, this is “c022”
[joeuser@c022 ~]$ jupyter-notebook --no-browser --port=8888 --ip=c022
[I 15:25:35.527 NotebookApp] JupyterLab extension loaded from /sw/apps/python3/anaconda-2020.02/lib/python3.7/site-packages/jupyterlab
[I 15:25:35.527 NotebookApp] JupyterLab application directory is /sw/apps/python3/anaconda-2020.02/share/jupyter/lab
[I 15:25:35.535 NotebookApp] Serving notebooks from local directory: /beegfs/home/aa/bb/joeuser
[I 15:25:35.535 NotebookApp] The Jupyter Notebook is running at:
[I 15:25:35.535 NotebookApp] http://c022:8888/?token=d69a5ca8a56a8f73b8d7f5a2fc286d54f61da15682a16c28
[I 15:25:35.536 NotebookApp]  or http://127.0.0.1:8888/?token=d69a5ca8a56a8f73b8d7f5a2fc286d54f61da15682a16c28
[I 15:25:35.536 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 15:25:35.656 NotebookApp]  
    To access the notebook, open this file in a browser:
        file:///beegfs/home/aa/bb/joeuser/.local/share/jupyter/runtime/nbserver-100087-open.html
    Or copy and paste one of these URLs:
        http://c022:8888/?token=d69a5ca8a56a8f73b8d7f5a2fc286d54f61da15682a16c28
     or http://127.0.0.1:8888/?token=d69a5ca8a56a8f73b8d7f5a2fc286d54f61da15682a16c28

Starting a Tunnel in a new terminal

Open a second terminal to start an ssh tunnel to the compute node through the Wendian login node.
joeuser@MyComputer ~ % ssh -L 8888:c022:8888 wendian.mines.edu

Open your web browser and connect to http://localhost:8888 or http://127.0.0.1:8888 and enter the token from the output when you started the notebook. Because the port numbers in the tunnel are the same you can copy the link with the token all-in-one to connect to the running jupyter notebook server.

 

Mio Compute Node Jupyter Notebook

Log into Mio and load the module for python:
[joeuser@mio001 ~]$ ml apps/python3
Start an interactive job with a single task to run the Jupyter Notebook server. This job will possible run on a shared node resource as it is requesting only a single core.
[joeuser@mio001 ~]$ srun -n1 --pty bash -i
Start the Jupyter Notebook with no browser and looking for a connection on port 8888, and broadcast the ip hostname of the node name. Here, this is “compute022”
[joeuser@compute022 ~]$ jupyter-notebook --no-browser --port=8888 --ip=compute022
[I 15:25:35.527 NotebookApp] JupyterLab extension loaded from /opt/python/anaconda3-2020.02/lib/python3.7/site-packages/jupyterlab
[I 15:25:35.527 NotebookApp] JupyterLab application directory is /opt/python/anaconda3-2020.02/share/jupyter/lab
[I 15:25:35.535 NotebookApp] Serving notebooks from local directory: /mioHome/u/aa/bb/joeuser
[I 15:25:35.535 NotebookApp] The Jupyter Notebook is running at: 
[I 15:25:35.535 NotebookApp] http://c022:8888/?token=d69a5ca8a56a8f73b8d7f5a2fc286d54f61da15682a16c28
[I 15:25:35.536 NotebookApp]  or http://127.0.0.1:8888/?token=d69a5ca8a56a8f73b8d7f5a2fc286d54f61da15682a16c28
[I 15:25:35.536 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). 
[C 15:25:35.656 NotebookApp]   

To access the notebook, open this file in a browser:   
  file:///beegfs/home/aa/bb/joeuser/.local/share/jupyter/runtime/nbserver-100087-open.html   
Or copy and paste one of these URLs:       
  http://compute022:8888/?token=d69a5ca8a56a8f73b8d7f5a2fc286d54f61da15682a16c28      
 or http://127.0.0.1:8888/?token=d69a5ca8a56a8f73b8d7f5a2fc286d54f61da15682a16c28

Starting a Tunnel in a new terminal

Open a second terminal to start an ssh tunnel to the compute node through the Wendian login node.
joeuser@MyComputer ~ % ssh -L 8888:compute022:8888 mio.mines.edu

Open your web browser and connect to http://localhost:8888 or http://127.0.0.1:8888 and enter the token from the output when you started the notebook. Because the port numbers in the tunnel are the same you can copy the link with the token all-in-one to connect to the running jupyter notebook server.

 

Tutorial on Mio (login node)

Strictly for light weight testing and application use only. See HPC policy for login node use. 

Login into Mio with a tunnel using the following command:

joeuser@MyComputer % ssh -L 8788:localhost:8888 mio.mines.edu  
# 8788 is used as an illustration, but could be 8888 
joeuser@mio.mines.edu's Password:
      
[joeuser@mio ~$] ml apps/python3
# Alias command name of "module load" is shorten to "ml"
[joeuser@mio ~$] jupyter-notebook --no-browser --port=8888
[I 14:51:43.994 NotebookApp] JupyterLab extension loaded from /opt/python/anaconda3-2020.02/lib/python3.7/site-packages/jupyterlab
[I 14:51:43.994 NotebookApp] JupyterLab application directory is /opt/python/anaconda3-2020.02/share/jupyter/lab
[I 14:51:43.999 NotebookApp] Serving notebooks from local directory: /mioHome/u/aa/bb/joeuser
[I 14:51:43.999 NotebookApp] The Jupyter Notebook is running at:
[I 14:51:43.999 NotebookApp] http://localhost:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74
[I 14:51:43.999 NotebookApp]  or http://127.0.0.1:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74
[I 14:51:43.999 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 14:51:44.006 NotebookApp] 
    To access the notebook, open this file in a browser:
        file:///mioHome/u/aa/bb/joeuser/.local/share/jupyter/runtime/nbserver-23724-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74
     or http://127.0.0.1:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74
Connect by using a web browser, such as Chrome, by typing in the address bar: localhost:8788. This shift in port numbers is because this is your computer port number not the one listed above. This will connect to the port on your computer which is tunneled through the SSH connection on port 8788 to the server’s localhost at port 8888. If these port numbers are the same you can copy and paste either one of the links in the output at the start of the jupyter notebook. This will connect you to the login page of the jupyter notebook server. Copy and paste the value of the token shown in the terminal, from the example output, this would be f31f16637096193b500a0f1aa02a2344541c3023f35fef74 (red text above).

Tutorial on Wendian (Login node)

Strictly for light weight testing and application use only. See HPC policy for login node use.

Log into Wendian by starting a tunnel at your ssh shell with this command:

joeuser@MyComputer % ssh -L 8788:localhost:8888 wendian.mines.edu
# 8788 is used as an illustration, but could be 8888
[joeuser@wendian ~$] module load apps/python3
# Alias command name of "module load" is shorten to "ml"
[joeuser@wendian ~$] juypter-notebook --no-browser --port=8888
[I 14:51:43.994 NotebookApp] JupyterLab extension loaded from /sw/apps/python3/anaconda-2020.02/lib/python3.7/site-packages/jupyterlab 
[I 14:51:43.994 NotebookApp] JupyterLab application directory is /sw/apps/python3/anaconda-2020.02/share/jupyter/lab
[I 14:51:43.999 NotebookApp] Serving notebooks from local directory: /beegfs/home/aa/bb/joeuser
[I 14:51:43.999 NotebookApp] The Jupyter Notebook is running at: 
[I 14:51:43.999 NotebookApp] http://localhost:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74 
[I 14:51:43.999 NotebookApp]  or http://127.0.0.1:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74 
[I 14:51:43.999 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). 
[C 14:51:44.006 NotebookApp]   

To access the notebook, open this file in a browser:       
  file:///mioHome/u/aa/bb/joeuser/.local/share/jupyter/runtime/nbserver-23724-open.html   
Or copy and paste one of these URLs:       
  http://localhost:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74      
 or http://127.0.0.1:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74

Connect by using a web browser, such as Chrome, by typing in the address bar: localhost:8788. This shift in port numbers is because this is your computer port number not the one listed above. This will connect to the port on your computer which is tunneled through the SSH connection on port 8788 to the server’s localhost at port 8888. If these port numbers are the same you can copy and paste either one of the links in the output at the start of the Jupyter notebook. This will connect you to the login page of the Jupyter notebook server. Copy and paste the value of the token shown in the terminal, from the example output, this would be f31f16637096193b500a0f1aa02a2344541c3023f35fef74 (red text above).

Connecting to the Jupyter Notebook

Login page of Jupyter Notebook
 
 
After login is complete you will be shown the directory tree of files located on the server. You can open any browser supported file types for example, image files: .png, .jpg; embedded 3d objects in .html webpages, text files, and of course jupyter notebook files: .ipynb
 
 
Directory tree showing in a Jupyter Notebook 

Stopping a Jupyter Notebook

Stopping notebook server can either be done in the terminal where you started it by pressing Crtl+C and confirming to shut the server down.
 
^C[I 15:08:03.885 NotebookApp] interrupted
Serving notebooks from local directory: /mioHome/u/aa/bb/joeuser
1 active kernel
The Jupyter Notebook is running at:
http://localhost:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74
 or http://127.0.0.1:8888/?token=f31f16637096193b500a0f1aa02a2344541c3023f35fef74
Shutdown this notebook server (y/[n])? y
[C 15:08:05.414 NotebookApp] Shutdown confirmed
[I 15:08:05.491 NotebookApp] Shutting down 1 kernel
[I 15:08:05.993 NotebookApp] Kernel shutdown: 7e5e024b-f812-4f3d-a221-f778a8b4c81e
 
Or by selecting “Quit” on the directory tree tab in the upper right.

Advanced Stopping a Jupyter Notebook

If you have a Jupyter notebook running either started with a “nohup” or running in the background of a terminal sometime you may have to investigate where it’s running. If you have the Module loaded you can try listing the notebooks running.

[joeuser@HPCsystem ~$] jupyter-notebook list
Currently running servers:
http://localhost:8889/?token=7be1a092f6af2461cbeaeb666c73216b87ff7435c59280d1 :: /mioHome/u/aa/bb/joeuser
http://localhost:8888/?token=b407c83176ddb33197e4d250287460042f3df129ca4b3f79 :: /mioHome/u/aa/bb/joeuser
http://localhost:8890/?token=e5ef8cfb1bbd40adbb414b00be8a3dbf2034f7c3b994cf53 :: /mioHome/u/aa/bb/joeuser
 
This shows me that I have three jupyter notebook server running all on different port numbers and with different access tokens. If I start a tunnel with the any of these port number I could connect and quit each one, or to stop one of these servers use this command with the corresponding port number:
 
[joeuser@HPCsystem ~]$ jupyter-notebook stop 8889
Shutting down server on port 8889 ...

Alternatively, you can search for process ID of all the running jupyter notebooks:

[joeuse@HPCsystem ~]$ ps -aux | grep jupyter-notebook
joeuser 24775  0.1  0.3 179404 50888 pts/11   S    17:14   0:00 /opt/python/anaconda3-2020.02/bin/python /opt/python/anaconda3-2020.02/bin/jupyter-notebook --no-browser --port=8888
joeuser 24858  0.0  0.3 331600 56788 ?        Sl   15:39   0:02 /opt/python/anaconda3-2020.02/bin/python /opt/python/anaconda3-2020.02/bin/jupyter-notebook --no-browser --port=8888
joeuser 28875  0.0  0.3 177384 52144 ?        S    15:46   0:00 /opt/python/anaconda3-2020.02/bin/python /opt/python/anaconda3-2020.02/bin/jupyter-notebook --no-browser --port=8888

Some of these Jupyter-notebook process are not connect to any terminal and will need to be stopped or kill with they process IDs (Second number, in red above). To kill the second process listed:

[joeuse@HPCsystem ~]$ kill 24858

 

Advanced Startup of a Jupyter Notebook

Starting a Jupyter Notebook and leave it running to connect to later or encase you lose connection. Requires using the “nohup” command when starting the Jupyter notebook, and will need to be monitored to note accidentally leaving jobs running, or worst holding a node that you forgot about.