Jupyter on Kebnekaise¶

Since the JupyterLab will only be accessible from within HPC2N’s domain, it is by far easiest to do this from inside ThinLinc, so this is highly recommended. You can find information about using ThinLinc at HPC2N in the ThinLinc section.

At HPC2N, you currently need to start JupyterLab on a specific compute node. To do that you need a submit file and inside that you load the JupyterLab module and its prerequisites (and possibly other Python modules if you need them - more about that later).

Thus, the steps to start JupyterLab at HPC2N is:

1. Load JupyterLab and its prerequisites.
2. Create a batch submit file for running a compute node as server for JupyterLab.
3. Wait for the job to start.
4. Check the slurm-JOBID.out file for the url where JupyterLab is being served.
5. Open the url in a browser inside the HPC2N domain (ThinLinc).
6. You can now work in the JupyterLab for as long as the job is running (the walltime you asked for in the batch submit file).

This way it is possible to run a JupyterLab with own-installed Python packages, modified kernel, Julia, R, etc. See examples later in this tutorial.

To see the currently available versions of JupyterLab, do:

module spider JupyterLab

You then do this to check how to load a specific version:

module spider JupyterLab/<version>

which will give you a list of any prerequisites that should be loaded first.

module load GCC/10.3.0 JupyterLab/3.2.8

Useful flags to Jupyter¶

You can use Jupyter --help and Jupyter lab --help to see extra options.

• lab: This launches JupyterLab computational environment for Jupyter.
• --no-browser: Prevent the opening of the default url in the browser.
• --ip=<IP address>: The IP address the JupyterLab server will listen on. Default is ‘localhost’. It is recommended to use $(hostname) to get the content of the environment variable for the hostname for the node you are allocated by the batch job.

Simple example¶

This is a simple example of starting JupyterLab version 3.2.8 at HPC2N.

JupyterLab with extra Python packages¶

Site-installed Python packages¶

If you need extra Python packages and these packages are already installed at HPC2N as modules, you can just load them. It is easiest to do so before you start the JupyterLab, inside the submit file.

#!/bin/bash
# Here you should put your own project id
#SBATCH -A hpc2nXXXX-YYY
# This example use 1 core
#SBATCH -n 1
# Ask for a suitable amount of time. Remember, this is the time the Jupyter notebook will be available! HHH:MM:SS.
#SBATCH --time=05:00:00

# Clear the environment from any previously loaded modules
module purge > /dev/null 2>&1

# Load the module environment suitable for the job
module load GCC/10.3.0 JupyterLab/3.2.8 OpenMPI/4.1.1 SciPy-bundle/2021.05 matplotlib/3.4.2

# Start JupyterLab
jupyter lab --no-browser --ip $(hostname)

You then submit the batch script, get the url, and open it in a browser, like described in the “simple example” above.

Own-installed Python packages¶

Running JupyterLab with some own-installed Python packages requires you to use a virtual environment and your own Jupyter kernel. This is not as difficult as it may sound.

We will illustrate this with an example.

JupyterLab with Julia¶

1. In order to use Julia kernels in a Jupyter notebook, you need to install the IJulia package in Julia. To do so, first load the modules for JupyterLab and Julia.
   Example: Loading Jupyterlab/3.2.8 and Julia v. 1.8.5

   module load GCC/10.3.0 JupyterLab/3.2.8
   module load Julia/1.8.5-linux-x86_64
   

2. Then, start a Julia session and add the IJulia package. This can be done either in Julian mode:

   $julia 
   using Pkg 
   Pkg.add("IJulia") 
   Pkg.build("IJulia") 
   

   or in package mode:

   $julia 
   julia>] 
   (v1.8) pkg>add IJulia
   (v1.8) pkg&gt;build IJulia</pre>
   

3. Once this package is installed, write a batch submit file to start a JupyterLab (in batch mode) using this template:

   #!/bin/bash
   # Here you should put your own project id
   #SBATCH -A hpc2nXXXX-YYY
   # This example use 1 core
   #SBATCH -n 1
   # Ask for a suitable amount of time. Remember, this is the time the Jupyter notebook will be available! HHH:MM:SS.
   #SBATCH --time=05:00:00
   
   # Clear the environment from any previously loaded modules
   module purge > /dev/null 2>&1
   
   # Load the module for JupyterLab - here version 3.2.8 
   module load GCC/10.3.0 JupyterLab/3.2.8 
   
   # Load the Julia module - here version 1.8.5 
   ml Julia/1.8.5-linux-x86_64 
   
   # Start JupyterLab
   jupyter lab --no-browser --ip $(hostname)
   

4. Submit the above submit file (here I named it MyJupvenv.sh).

   sbatch MyJupvenv.sh
   

5. You get the <job-id> when you do the above command.

6. Check the SLURM output file (slurm-<job.id>.out); grab the URL with the hostname as described in the first part of this document, since the localhost one requires you to login to the compute node.

7. Start a web browser within HPC2N (ThinLinc interface). Put in the URL you grabbed, including the token.

8. Inside JupyterLab, start the new Julia kernel. Just click the launcher for that one if no other kernel is running. Notice that besides the Julia version 1.8.5, other options are provided for the kernel of Julia but only the one that matches the loaded module in step 1 will be available.

9. If a kernel is running (shown under kernels), then shut down that kernel and click “Kernel” in the menu, and then “Change kernel”. Pick your kernel from the drop-down menu.

10. You can now run your files etc. with the Julia kernel.

11. NOTE! Sometimes it is still running on the default kernel. If so, Click the 3 little dots in the right side of the editor-window for the program and pick your kernel. Then rerun your files.

JupyterLab with R¶

1. In order to use R kernels in a Jupyter notebook you need to install a suitable kernel, for instance the IRkernel.
2. First load the modules for JupyterLab and a compatible R version with the same tool chain, at this moment (spring 2024) the following combo works:
   

   module load GCC/12.3.0 R/4.3.2 
   module load OpenMPI/4.1.5 JupyterLab/4.0.5
   

3. Then, start an R session on the command line and install the IRkernel (choose a location, CRAN, for instance Sweden):

   $R
   install.packages('IRkernel')
   IRkernel::installspec() 
   #give a name to the kernel in this case I chose ir432
   IRkernel::installspec(name = 'ir432', displayname = 'R 4.3.2')
   

4. Once this package is installed, write a submit file to start a JupyterLab (in batch mode) using this template:
   Example, loading JupyterLab/4.0.5, its prerequisites, and the R version 4.3.2:
   

   #!/bin/bash
   # Here you should put your own project id
   #SBATCH -A hpc2nXXXX-YYY
   # This example use 1 core
   #SBATCH -n 1
   # Ask for a suitable amount of time. Remember, this is the time the Jupyter notebook will be available! HHH:MM:SS.
   #SBATCH --time=05:00:00
   
   # Clear the environment from any previously loaded modules
   module purge > /dev/null 2>&1
   
   # Load the modules for the R and JupyterLab
   module load GCC/12.3.0 R/4.3.2 
   module load OpenMPI/4.1.5 JupyterLab/4.0.5
   
   # Start JupyterLab with the installed kernel
   jupyter lab --no-browser --ip $(hostname)
   

5. Submit the above submit file (here I named it MyJupvenv.sh).
   

   sbatch MyJupvenv.sh
   

6. You get the <job-id> when you do the above command.
7. Check the SLURM output file (slurm-<job.id>.out); grab the URL with the hostname as described in the first part of this document, since the localhost one requires you to login to the compute node.
8. Start a webbrowser within HPC2N (ThinLinc interface). Put in the URL you grabbed, including the token.
9. Inside JupyterLab, start the new R kernel. Just click the launcher for that one if no other kernel is running.
10. If a kernel is running (shown under kernels), then shut down that kernel and click “Kernel” in the menu, and then “Change kernel”. Pick your kernel from the drop-down menu.
11. You can now run your files etc. with the R kernel.
12. NOTE! Sometimes it is still running on the default kernel. If so, Click the 3 little dots in the right side of the editor-window for the program and pick your kernel. Then rerun your files.