There are 6 types of queues on Pinnacles, and each type of queue has its own configurations and policy.
|
Queue types |
Maximum wall-clock time |
Default time |
Max nodes |
Job limits |
|
test |
1 hr |
5 min |
2 |
1 |
|
bigmem |
3 d |
1 hr |
2 |
2 |
|
gpu |
3 d |
1 hr |
2 |
2 |
|
short |
6 hr |
1 hr |
4 |
12 |
|
medium |
1 d |
6 hr |
4 |
6 |
|
long |
3 d |
1 d |
4 |
3 |
1: test queue has access to all node types. In order to test jobs on specific node types, please use Slurm “constraints” flag.
For example, in order to test jobs on GPU nodes, use the following line in Slurm submission script
#SBATCH --constraint=gpu
Note: Access to GPUs also requires #SBATCH --gres=gpu:X, X represents how many resources are required. For example:
#SBATCH--gres=gpu:2
In order to test job on bigmem node:
#SBATCH --constraint=bigmem
2: Short queue is the default queue. If user does not specify the partition or the queue type in the job script, the job will be submitted to the short queue
3: Default time means that If users do not specify the wall-clock time, the default time will be used.
4: Max node means maximum number of nodes per job, if exceed this limit, user will see “PartitionNodeLimit” on the queue for the job, and the specific job will not be picked up.
5: Job limits: for each type of queues, only a limited number of jobs per user is allowed to submit, if exceed the limit, user will not be able to submit more and a message of “sbatch: error: Batch job submission failed: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits)” will appear.
Sample job script:
|
#!/bin/bash #SBATCH -N 1 # request only 1 node #SBATCH -p test # this job will be submitted to test queue #SBATCH -t 00:10:00 # this job requests to run for 10mins #SBATCH --ntasks-per-node=56 # this job requests for 56 cores on a node #SBATCH --output=my_%j.stdout # standard output will be redirected to this file # #SBATCH --constraint=bigmem #uncomment this line if you need the access to the bigmem node # #SBATCH --constraint=gpu. #uncomment this line if you need the access to GPU # #SBATCH --gres=gpu:2 #uncomment this line if you need GPU access (2 GPUs)
#SBATCH --job-name=my_job # this is your job’s name #SBATCH --export=ALL
# type 'man sbatch' for more information and options # this job will ask for 1 full CPU node (56 cores) for 10 min
# run your job |
In order to submit a job, such as submit a script called “my_job.sh” as a job:
|
sbatch my_job.sh |
List your queued and running jobs:
|
squeue --me |
Cancel a queued job or kill a running job, e.g. a job with ID 32145:
|
scancel 32145 |
Check status of a job, e.g. a job with ID 32145:
|
sacct -j 32145 |
