Page MenuHomePhabricator

Make exec hosts as submit hosts too
Closed, ResolvedPublic

Description

So tasks running on SGE can submit (sub-)tasks themselves, to avoid using direct subprocesses.


Version: unspecified
Severity: normal

Details

Reference
bz54786

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 2:27 AM
bzimport added a project: Toolforge.
bzimport set Reference to bz54786.

Can we have progress on that? Subprocess works poorly for multiple cpu bound sub-task.

Despite some hesitation about possible runaway jobs, this has been done.

Note that the jsub/jstart tools are not available on the exec nodes by design; submitting jobs recursively should be considered an advanced operation and careful consideration of parameters to qsub should be a requirement.