Back to the main page.
Bug 1361 - add backend='inprocess' option to qsubfeval
| Status | CLOSED FIXED |
| Reported | 2012-03-07 19:10:00 +0100 |
| Modified | 2012-03-14 10:00:45 +0100 |
| Product: | FieldTrip |
| Component: | qsub |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Windows |
| Importance: | P3 normal |
| Assigned to: | Eelke Spaak |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Eelke Spaak - 2012-03-07 19:10:17 +0100
This would be very helpful when you have a script that is designed to be run on the torque cluster, but you occasionally want to run on some small test data locally. You would not place load on the cluster that way, and also you can still run your script in case the entire cluster is experiencing max. load. There already is the backend='local' option, but that involves spawning a new matlab process for each job, which is good for testing the cluster functionality itself, but not really for quickly running a batch-type script locally (lot of overhead involved in spawning new matlabs).
Robert Oostenveld - 2012-03-07 22:50:28 +0100
do I understand it correctly that you are suggesting a sort of feval in the qsubfeval function? That would be very little work in qsubfeval, i.e. write the input mat file, call qsubexec, which writes the output mat file, create the e and o files, and you are done!
Eelke Spaak - 2012-03-08 08:21:50 +0100
(In reply to comment #1) Yes, that's exactly my idea. Best would even be to skip the writing/reading of the .mat-files, but probably that would require a lot more coding.
Robert Oostenveld - 2012-03-08 09:14:05 +0100
I suggest to use backend=local for the new feval and backend=system for the existing system call. Furthermore I would indeed not want to change the communication through files. That is part of the testing which should be possible with the local execution. Furthermore, in the future there will be a fieldtrip.exe for distributed computing which will also work with these input/output job files. I think that memreq and timreq should also be implemented, so that those features can also be tested. There would be no limits in case of timreq or memreq=inf.
Robert Oostenveld - 2012-03-11 11:33:33 +0100
committed three improvements to qsub in one go: enhancement - this addresses bug #1254, bug #1331 and bug #1361. Allow the user to specify toolbox names when compiling. Allow the user to specify a custom batchid instead of user_host_pid_bM. Implemented backend=local using a call to qqubexec inside qsubfeval (i.e. really local execution), renamed the existing implementation to backend-system. mbp> svn commit Sending qsub/private/generatejobid.m Sending qsub/qsubcellfun.m Sending qsub/qsubcompile.m Sending qsub/qsubfeval.m Transmitting file data .... Committed revision 5435.