[SOLVED] Bundle for Environment Modules

Developer Discussion
1

Hi to all,

I am setting up a small cluster with high performance computing tools. By now I already have Munge and Slurm, compilers and libraries installed. I am starting to compile the codes for calculations.

Usually, Environment Modules are used to load and/or unload this calculation programs with command lines as:

module load Program

However, I could not find a bundle to install that provides this capability. Is there any? In RH based systems, there is the rpm environment-modules, I believe.

How can I get this working in clearlinux?

Thanks in Advance.

2

maybe @juro.bystricky can see what is needed here, he’s been slowly adding a few components in this space.

3

Nice coincidence. We have all components to release Slurm/Munge/Pdsh bundle as well, it is currently being tested. As for the Environment Modules, I agree that they are useful and should be included with Clear Linux. I only had a cursory look at the repository a while ago, seemed fairly straightforward (at least compared to building Munge/Slurm). It is possible to live without the Environment Modules, but you need to set the environment yourself (typically via a script file). Having said that, I will revisit the issue.

4

Thank you very much for your answer. I will mark this as solved and I will keep tracking if this bundles are added to the repository.

I believe it shouldn’t be dificult to compile the Environment Modules package in my system also.

Cheers

5

Dear @juro.bystricky and @ahkok . Sorry I couldn’t post this before. I was kind of AFK.

The best option that I could find was to set up the environment variables in the slurm scripts for jobs. I will only run jobs with the sbatch command so it isn’t a problem for now.

Furthermore. I kind that have a how-to documentation on how to set up slurm in clear linux from a fresh installation. If You wish, I can translate to english to be added somewhere in clear linux docs.

6

Hello @rfkspada, there has been some recent development in Clear Linux regarding cluster tools.
CLR now has a new bundle “cluster-tools” which contains slurm/munge/pdsh/openmpi
https://github.com/clearlinux/clr-bundles/blob/1d28029ea849d2322ff33b643bae01bdf1386733/bundles/cluster-tools
There is also a new bundle for environment modules:
https://github.com/clearlinux/clr-bundles/blob/1d28029ea849d2322ff33b643bae01bdf1386733/bundles/modules

Sometimes within the next few days the “modules” bundle will be included in the cluster-tools bundle, but currently you need to install the bundle separately via

$ sudo swupd bundle-add modules

(After you install the bundle, you need to log out and log in again)
The “modules” bundle provides only some basic “modulefiles”, it is assumed packages provide their own “modulefiles” or the user edits/creates needed “modulefiles” by himself/herself. If you use a CLR package that could benefit from a “modulfile”, please let me know. (I plan on providing one for OpenMPI)

As for the documentation: I am not sure your documentation describes compiling/installing slurm/munge from sources or installing slurm using the cluster-tools bundle. In any case, documentation is always desired.
A good place would be, for example, here:

(I plan to post there a simple step-by-step document on cluster-tool bundle setup up as well.)