Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PBS executor not working for task.config.getCpus() == 1 if no default set cluster #5922

Open
fbartusch opened this issue Mar 25, 2025 · 2 comments

Comments

@fbartusch
Copy link

Dear Nextflow team,

Due to licensing issues we had to go back from MOAB/Torque to plain Torque on our cluster.
Because of the migration we lost the default resource configuration for each queue, thus processes with cpus = { 1} in the configuration do not run anymore. The -l nodes=1:ppn=1 line is missing in the jobscript and our lovely scheduler refuses to run the job.

I think the problem could be solved if you don't assume that there is a default values for nodes and cores set for the queues in PBS, e.g. also consider the case task.config.getCpus() == 1 here

@bentsherman
Copy link
Member

Both the PBS and MOAB executors have this condition, so I'm hesitant to remove it. If it's always been that way, does that mean your cluster is behaving in a non-standard way?

You can also use clusterOptions to set the cpus manually

@fbartusch
Copy link
Author

Our cluster does not behave in a non-standard way, it was just not optimally configured. Due to this I found this "issue".

When a Nextflow task required only one CPU, the PBS executor did not add the appropriate -l nodes=1:ppn=1 to the job script. As a consequence, these tasks did not start on our cluster. We have now set a proper default in our configuration, and the tasks are running correctly.

My point is that you cannot assume this default is set everywhere. I don’t know every detail of the codebase, but I don’t see any issues with changing this line

if( task.config.getCpus() > 1 ) {

to

if( task.config.getCpus() >= 1 ) {

This ensures that -l nodes=1:ppn=1 is always included in the job script, preventing reliance on a scheduler default that may or may not be set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants