Here you are going to learn how to use default kernels and how to enable your customized environment
based on e.g conda or virtualenv.
On Mistral, we already provide Python environments that are managed by us.
Python 3 / Unstable: the new Python modules concept on Mistral (see details)
Python 3 bleeding edge (using the module anaconda3/bleeding_edge)
Python 3 (using the module Python/3.5.2)
Python 2 bleeding_edge (using the module anaconda2/bleeding_edge)
Python 2 (using the module python/2.7.12)
It is not possible for users to install or update packages.
Installing new packages into these system modules/kernels depends on the number of users using them. If its only for one user, we suggest that you use your own kernel.
Requests for new packages are made through our system department and might take a time.
The following steps will show you how to create and enable your own environment.
In order to use conda or virtualenv you need to load any available Python interpreter available. Recommanded is the python3/unstable (module python3/unstable).
You need to create a conda environment in your home ($HOME):
% mkdir $HOME/kernels
% conda create --prefix $HOME/kernels/your-kernel ipykernel python=3.x
% source activate $HOME/kernels/your-kernel
% python -m ipykernel install --user --name new-kernel --display-name="new kernel"
% conda deactivate
Restart the server and the new kernel should be available.
You can find your new kernels in ~/.local/share/jupyter/kernels/.
More details on kernels can be found here
If virtualenv is not available, you have to install it first before trying the following steps.
The best way to install virtualenv is with pip:
% python -m pip install --user virtualenv
% python -m virtualenv --system-site-packages /path/to/new-kernel
% source /path/to/new-kernel/bin/activate
% pip install ipykernel
% python -m ipykernel install --user --name new-kernel --display-name="new-kernel"
You can now add/install additional packages that you need in your new environment and then:
(new-kernel) % deactivate
You can even go further with the configuration of your new kernel by updating the kernel.json.
The content looks like this:
It is possible to specify additional environment variables:
Where to install the new environment?
Depending on the number and size of the python packages installed in your new enviroment, disk usage in your $HOME can easily exceed the limit.
Therefore, it is preferable to create conda/virtualenv environments in /work (in the corresponding project).
In that case, creating a conda env for example in work:
% conda create --prefix /work/project_id/$USER ...
Helper script for kernel.json
You can create a start-kernel.sh shell script and make it executable (chmod +x start-kernel.sh). Inside the script you can put all configuration you want in your new kernel.
An example can be for example loading system modules. The sctructure of the script can be like this:
module load netcdf_c/4.3.2-gcc48
module load python/3.5.2
python -m ipykernel_launcher -f "$1"
And the kernel.json: