Environment

The systems at HPC2N predominantly run various versions of Linux. If you have no experience with that, there are several good online sources that can be used to familiarize yourself with the basic commands.

The default shell at HPC2N is bash (Bourne Again SHell), and it is recommended to use that, since it is the only one that is fully supported by the batch system. You can find some information about bash here:

Your home directory at HPC2N is backed up regularly.

You can read more about the files system at HPC2N in general.

The majority of software on HPC2N are loaded with modules. You can read more about modules on our Using modules (Lmod) page.

Running the command ‘env’ give you a list of the paths and environment variables that are currently set in your shell.

Note

On this page, a dollar-sign ($) prefacing a line is used to illustrate the bash-shell prompt on the terminal. If you are using a shell from the c-shell family, the symbol would instead be a percentage-symbol (%).

Do NOT copy the prompt if you are copying a command to use it.

Environment Variables

There are many environment variables. Some will be set automatically at login and some you can change if needed. Many of the environment variables will be related to paths and storage locations, but there are also some which are set for specific applications (like compilers). These environment variables will be set when you load the ‘module’ for the application in question. (Read more about modules in the section about them.)

You should use environment variables instead of actual paths whenever possible to avoid problems if the specific paths to any of these change. Some of the environment variables you should have are:

  • $HOME: path to your home directory
  • $PATH: all directories searched for commands/applications
  • $SHELL: your current shell (bash, tcsh, csh, ksh)
  • $KRB5CCNAME: path to your Kerberos ticket

You can find more about environment variables here.

Note

All environment variables begin with the dollar sign ($) and are all uppercase. They may be used on the command line or in any scripts in place of and in combination with hard-coded values.

Example

$ ls $HOME
...

You may find the value of any environment variable by using the echo command:

$ echo $SHELL
/bin/bash

You may list the values of all environment variables using the env command:

$ env
USER=myusername
HOME=/home/b/myusername
SHELL=/bin/bash
...

You may create or overwrite an environment variable using either export or setenv, depending on your shell:

  • (for bash and sh)
$ export VARIABLE=value
  • (for tcsh and csh)
% setenv VARIABLE value

Environment variables prefaced with “SNIC_”

These are variables that are named and used the same way across most of Sweden’s HPC centres (former SNIC centres).

They are set as default for all users at HPC2N. They can be used in e.g. job scripts, and it is preferred that the variables are used instead of the absolute value.

Here follows a short description of the variables:

  • SNIC_BACKUP is the user’s primary directory at the centre (the part of the centre storage that is backed up at HPC2N is your home directory - everything under $HOME)
  • SNIC_NOBACKUP is the recommended user directory for project storage without backup in batch jobs (the part of the centre storage that is NOT backed up)
  • SNIC_TMP is the recommended directory for best performance during job runs. At HPC2N it is local scratchdisk on nodes.
  • SNIC_RESOURCE contains information about which compute resource you are running on. At HPC2N it can have the value ‘kebnekaise’
  • SNIC_SITE contains information about which HPC centre (former SNIC site) you are running at. At HPC2N, this variable has the value ‘hpc2n’

SNIC_BACKUP and SNIC_NOBACKUP are located on site-wide shared file systems, while SNIC_TMP is node-local scratch space.

You can see which “SNIC” variables are set with this command:

$ env|grep SNIC|sort

Here is an example of what you get when you run the above command (for the user user123, on the cluster Kebnekaise):

$ env|grep SNIC|sort
SNIC_BACKUP=/home/u/user123
SNIC_NOBACKUP=/proj/nobackup/<your-proj-storage>
SNIC_RESOURCE=kebnekaise
SNIC_SITE=hpc2n
SNIC_TMP=/scratch

Using Environment Variables

This page is assuming you use bash or another shell in the Bourne family of shells. Most of the information will be correct for other shells as well, but the examples cannot be assumed to be.

HPC2N recommends using bash, since it is the only shell that is fully supported by the batch system.

Environment variables are a set of dynamically named values, generally meant to make your life easier. Some are built-in, some are set by loading various modules, and you can also define extra to help yourself.

An example could be an environment variable with a standard name that stores the location that a particular computer system uses to store temporary files.

You can see your current environment variables by invoking the command ‘env’ or list a specific variable with ‘echo $ENVIRONMENT_VARIABLE’. It is important to remember the dollar sign.

Example: listing all your environment variables

$ env
MODULE_VERSION_STACK=3.2.7
TERM=xterm
SHELL=/bin/bash
RANDFILE=/dev/urandom
SSH_TTY=/dev/pts/22
COLUMNS=80
EDITOR=/usr/bin/vim
LANG=en_US.UTF-8
...

The above is shortened, as it tends to be long.

In shells, like bash (which is the recommended shell at HPC2N), you can change the environment variables by using 

$ export ENVIRONMENT_VARIABLE=VALUE

If you do not use ‘export’ when setting or creating a variable, it will only have meaning in the current shell.

Some common environment variables and their meaning:

  • HOME contains the user’s home directory
  • PATH lists the directories the shell searches for the commands the user can type without having to provide the full path
  • HOSTNAME contains the name of the host you are currently connected to
  • PWD (print working directory) is used to output the path of the current working directory.
  • KRB5CCNAME points at a file that contains your Kerberos credentials
  • SHELL lists your current shell (bash, tcsh, csh, ksh)
  • SLURM_SUBMIT_DIR contains the directory you submitted your job script from (only usable from within a job)

Example, creating your own shell variables

Assume you have the binary of your favourite program in the directory projects/coolness/bin in your home directory ($HOME). You would like a variable for this directory, both to be able to change to it quickly, and to be able to easily set the path to it when you want to use it. Notice that you are using the environment variable $HOME for your home directory here.

First issue the command (for whatever name you give it): 

$ VARIABLE_NAME=$HOME/projects/coolness/bin

and then, regardless of what directory you are in, you can issue

$ cd $VARIABLE_NAME

to go quickly to the directory containing your cool program.

A variable assignment like this will work just fine, but its scope (visibility) is limited to the current shell. If you launch a program or enter another shell, that child task will not know about your environment variables unless you export them first.

Unless you know for sure that an environment variable will have meaning only in the current shell, it is a good idea to always use export when creating variables to ensure they will be global in scope.

Example

$ export VARIABLE_NAME=$HOME/projects/coolness/bin

To not have to retype them every time you need them (after logging in again), you can add them to your .profile or .bashrc (at HPC2N you should make the changes to the file $HOME/.bashrc).

Example, adding a directory to your path

Say you have written and compiled a new program and wants to be able to run it without typing out the whole path every time. You can achieve this by adding the directory to your path.

See your path with the command echo $PATH.

Example

$ echo $PATH
/home/u/user123/Public/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/puppetlabs/bin:/usr/local/sbin:/usr/heimdal/sbin:/sbin:/usr/sbin

To add your directory /home/u/user123/coolness to the path, open the file $HOME/.bashrc with your favourite editor and add a colon and then the directory after what is already in the (customized) path. If no customizations have been done, and there is no entry for the path, add PATH=$PATH and then the colon and the directory. Like this:

PATH=$PATH:/home/u/user123/coolness