Back to the main page.

Bug 1937 - implement job dependencies in qsubfeval

Status CLOSED FIXED
Reported 2013-01-17 23:31:00 +0100
Modified 2014-03-26 16:51:42 +0100
Product: FieldTrip
Component: qsub
Version: unspecified
Hardware: PC
Operating System: Mac OS
Importance: P3 major
Assigned to: Robert Oostenveld
URL:
Tags:
Depends on:
Blocks:
See also:

Robert Oostenveld - 2013-01-17 23:31:06 +0100

this should look like jobid1 = qsubfeval(@function1, {arg1}); jobid2 = qsubfeval(@function2, {arg2}, 'waitfor', jobid1); Both jobs would be immediately scheduled, but would use qsub -depend=afterok: to have job2 wait until job1 is done. That allows for more efficient scheduling of larger pipelines consisting of multiple rows (subjects) and columns (analysis steps). </p>

Robert Oostenveld - 2013-01-17 23:47:48 +0100

I studied the code. It needs some modifications to qsublist, which should allow for pbsid = qsublist('pbsid', jobid) to translate from matlab jobid to pbsid. Then it needs some change to qsubfeval to add -depend=afterok: to the qsub command line. See http://www.clusterresources.com/torquedocs/commands/qsub.shtml#W </p>

Robert Oostenveld - 2014-02-18 18:32:48 +0100

thanks to Alexander we now have this implemented. He suggested some code changes, which I incorporated. There is also a (FT specific) test script. roboos@mentat001> svn commit Sending qsub/qsubfeval.m Sending qsub/qsublist.m Adding test/inspect_bug1937.m Transmitting file data ... Committed revision 9211. Some differences to what Alexander suggested: - I did not change qsubget, it is possible to use output=cell to get the options - I did not fix the pre/postambles, but rather call ft_freqanalysis_mvar instead of fr_freqanalysis.


Robert Oostenveld - 2014-02-24 10:56:24 +0100

I closed several bugs at once that all have been resolved for some time. If you disagree, please reopen.


Robert Oostenveld - 2014-03-26 16:51:42 +0100

On 19 Mar 2014, at 11:56, Alexander Backus wrote: Ik heb qsubfeval.m aangepast zodat er nu een cell array van jobs kan worden opgegeven bij waitfor. Op deze manier wordt de betreffende job op Hold gehouden totdat alle jobs in de cell array Completed zijn. Dit is bijvoorbeeld erg handig voor een secondlevel script zoals ft_freqgrandaverage, waarbij gewacht moet worden totdat alle individuele subjects klaar zijn. Zie inspect_bug1937b.m voor een implementatie hiervan (gebaseerd op ons eerdere script). ---- 189-109> svn commit qsub/ test/ Sending qsub/qsubfeval.m Sending test/inspect_bug1937.m Transmitting file data .. Committed revision 9313.