Hyak, the High Performance Computing Cluster at the UW, allows software to be installed in loadable ‘modules.’ Modules can be loaded, unloaded and shared across groups, and they are a flexible way of maintaining different software packages and even different versions of the same software package.
For the basics on using Modules under Hyak see: https://hyak.uw.edu/docs/tools/modules/
NOTE: Hyak users in the College of the Environment who wish to request assistance with the installation of a specific module or software package should fill out the eSITS Help Request. Please include the software package’s name, version and a URL to its home page.
Module Installation
Software Modules on Klone Hyak consist of two parts:
- The installer directory that includes the software packages files and executables.
- A ‘modulefile’ that describes how to load the module
The modulefile will include the full path to the software’s install directory and will describe any prerequisites that should be loaded before the software is loaded. The modulefile is either a ‘TCL’ or a ‘LUA’ script file. A typical LUA modulefile looks like this:
help([[
sratoolkit
]])
local name = "sratoolkit"
local version = "3.0.6"
local base = pathJoin("/sw/contrib/coenv-src",name,version,"bin")
prepend_path("PATH", base)
Installing a Shared Module: Summary
Shared Modules are installed under the /sw/contrib filesystem in Hyak. Modules installed here are made available for any user of hyak to load a utilize. The College of the Environment has modules installed under the path /sw/contrib/coenv-src with modulefiles in /sw/contrib/modulefiles/coenv
NOTE: Installing Modules requires a basic level of understanding of how software is usually compiled and installed under a UNIX environment.
Shared Modules are installed just like other compiled-from-source UNIX software, except that it will be installed into a specific location so it can be loaded/unloaded by LMOD:
$ wget https://path.to/software-version.tgz $ tar xzf software-version.tgz $ cd software-version $ ./configure --prefix=/sw/contrib/coenv-src/software/version $ make –j8 $ make install
Once the software is installed in /sw/contrib/coenv-src/software/version, create a new modulefile and put it in /sw/contrib/modulefiles/coenv/software/version.lua:
help([[
software
]])
local name = "software"
local version = "version"
local base = pathJoin("/sw/contrib/coenv-src",name,version,"bin")
prepend_path("PATH", base)
Replace ‘software’ with the software name and ‘version’ with the numeric version number matching the install path above. After a small delay while the cluster indexes update, you should be able to:
$ module load coenv/software
Installing a Shared Module: Details
The following steps should be completed on a Hyak compute node. To launch a new node use the command:
salloc -A coenv -p compute -N 1 -c 4 --mem=8G --time=4:00:00
To install a module, you will first need to locate and download the software. Open source software is commonly distributed as source code that is compressed and archived into a single file. These files can usually be found on the software’s website or in the ‘Releases’ section of the Github page. For example, to install the ‘samtools’ software package, visit the github: https://github.com/samtools/samtools, click ‘Releases’ and right click on the file “samtools-1.17.tar.bz2” and click ‘Copy link location’
Navigate to your the /sw/contrib/coenv directory and use the ‘wget’ command to download the file:
Extract the software with the command ‘tar xjf samtools-1.17.tar.bz2’
You will be left with a folder that contains the tool’s source code:
Navigate to the directory: ‘cd samtools-1.17’
A typical source directory will include some common files, including a script called ‘configure.’ the configure script will prepare the code for compilation on your computer, while the command ‘make’ will compile it. In our case, we will be installing into a non-default install directory. We specify the install directory using the –prefix option to the configure command:
./configure --prefix=/sw/contrib/coenv-src/<packagename>/<version>
Note that you will need to replace the <packagename> and <version> with the appropriate values:
Once the configuration is complete, copile the software:
This step can take some time, once it completes successfully, you should be able to see the install location at /sw/contrib/coenv-src/samtools/1.17:
Typically, the program will be installed in the ‘bin’ subfolder, so we will create a modulefile with the appropriate program name and version:
help([[
samtools
]])
local name = "samtools"
local version = "1.17"
local base = pathJoin("/sw/contrib/coenv-src",name,version,"bin")
prepend_path("PATH", base)
And save it to: /sw/contrib/modulefiles/coenv/samtools/1.17.lua
The Hyak cluster takes a few minutes to index new modulefiles. Once it is ready, you can now load the module file with:
$ module load coenv/samtools
Or by its version explicitly:
$ module load coenv/samtools/1.17