Back to the main page.
Bug 2601 - allow user specification of where qsubcellfun maintains it input/output files
| Status | CLOSED WONTFIX |
| Reported | 2014-06-04 21:04:00 +0200 |
| Modified | 2016-06-14 16:18:23 +0200 |
| Product: | FieldTrip |
| Component: | qsub |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Linux |
| Importance: | P5 enhancement |
| Assigned to: | Robert Oostenveld |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Marcel Zwiers - 2014-06-04 21:04:15 +0200
Created attachment 633 suggested modifications of getcustompwd.m Matlab can be slowed down very much if the current directory contains a lot of files (e.g. > 10^3-10^4; see also http://www.mathworks.com/matlabcentral/answers/112086-matlab-slow-when-too-many-files-in-directory). In case of large batch jobs, qsubcellfun writes a significant number of files to the current directory and can therefore run into this issue. Suggested solution: adapt getcustompwd (e.g. as in attachment) and in qsubfeval and qsubget change this line: curPwd = getcustompwd(); into this one: curPwd = getcustompwd('qsub_iofiles'); In addition, the qsub submitoptions should also be extended with a "-o qsub_iofiles' option. This requires a simple modification of qsubfeval, but maybe of some other functions too?
Robert Oostenveld - 2015-11-18 16:59:29 +0100
this requires changing the overall directory handling in the code. There are now multiple locations where the current pwd is assumed to hold the input and output, it would have to be changed everywhere and the working directory would have to be passed around constantly (or stored in the qsublist). I consider this too much hassle and won't work on it.