Getting started example: SLURM on Longleaf

This guide will provide you with enough basic information to run straight-forward jobs on Longleaf. Please use the table of contents to access more detailed information, or email research@unc.edu with questions.

 

Log in to longleaf.unc.edu
Log into longleaf.unc.edu using UNC credentials (UNC onyen and onyen password).

ssh <onyen>@longleaf.unc.edu

If you are using Linux or OS X, simply use the terminal emulator packaged with your distribution.  Windows users may want to use SecureCRT or SSH Secure Shel, a secure network connection tool available to Faculty, Staff, and Students through UNC ITS Software Acquisition.

Notable Directories
Your home directory will be: /nas/longleaf/home/<onyen>.
Your scratch space will be: /pine/scr/o/n/onyen.

 

Hello SLURM
Longleaf uses SLURM to schedule and submit jobs. Below are the most common methods to submit jobs to Longleaf. Note: always submit your compute jobs via SLURM.

Method 1: The Submission Script
Create a bash script using your favorite editor. If you don’t have a favorite editor, use nano (for now).

nano example.sh

The script contains job submission options followed by application commands. Please enter the following into your script (Note: that each SBATCH switch below has two ‘-‘ characters, not one):

 

#!/bin/bash
#SBATCH --job-name=first_slurm_job
#SBATCH --ntasks=1
#SBATCH --time=1:00
#SBATCH --mem=100

echo "Hello SLURM"


Save your file and exit nano. Submit your job using the sbatch command:

sbatch example.sh

You have created a script, example.shthat will ask for 1 core and 100MB of memory for 1 minute.  It will name the job “first_slurm_job” and run the echocommand.

 

Method 2: Inline Submission
If you would like to submit your job at the command line without creating a script, please try the following:

$ sbatch -t 1 -n 1 -o hello.out --wrap="echo hello SLURM"

We left out the --job-name and --mem options, and used shorthand options for time and ntasks. We also specified our own output file.

If no Slurm options are specified then the defaults are:

-n 1 = 1 CPU processor

–mem-per-cpu=4800 = memory per CPU in MB

-t 60 or -t 1:00:00 or -t 0-01:00 = 60 minutes or 1 hour run time.

 

If you see the following error message then you used the sbatch command without the --wrap option or a valid shell script.   Use the Script submission method or Inline submission method as shown above.

/var/spool/slurmd/job1535767/slurm_script: line 445: /var/spool/slurmd/bin/util/arch.sh: No such file or directory

 

 

Submitting Interactive Jobs
Submitting interactive jobs requires the srun option --pty as in the “srun” command shown below:

srun --x11=first --pty [my_interactive_job]

The --x11-first option allows XWindows to operate between the login and compute nodes.

For example, the command below will give you an interactive bash shell session:

srun --x11=first --pty /bin/bash


Partition (Queue) Information

sinfo

squeue
squeue -u <onyen>
squeue -u <onyen> -l

Job details

scontrol show jobid <job_id_number>

Cancel Job

scancel <job_id_number>

Details of Completed Job

Note that -jbelow has a single hyphen ‘-‘, and --formathas two hyphens ‘–‘.

sacct -j <jobid> --format=JobID,JobName,MaxRSS,Elapsed,State

See man sacctfor details.

Add Modules

module avail
module add matlab
module list matlab

To save your software additions for future login sessions

module save matlab

See man moduleor module --help for more details.

See also Modules Approach to Software Management