Back to the main page.
Bug 668 - ft_artifact_zvalue causes a steadily increasing memory use
| Status | CLOSED FIXED |
| Reported | 2011-05-13 21:55:00 +0200 |
| Modified | 2011-09-14 14:33:30 +0200 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Mac OS |
| Importance: | P1 minor |
| Assigned to: | Jörn M. Horschig |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Robert Oostenveld - 2011-05-13 21:55:54 +0200
... which causes it to run out of memory on a ctf151 datasetr on my 4GB laptop! Once upon a time, it used to work so efficient memory wise, but this problem suggests that we cannot even process a 275 channel dataset on a 8GB computer any more. This needs discussion in the FT meeting.
Robert Oostenveld - 2011-05-13 21:57:14 +0200
changed status from multiple bugs from NEW into ASSIGNED
Robert Oostenveld - 2011-05-13 21:58:09 +0200
changed status from multiple bugs from NEW to ASSIGNED
Jan-Mathijs Schoffelen - 2011-05-15 20:06:00 +0200
Are you passing a data structure as second input argument? In other words: is the function required to fetch or read the data?
Robert Oostenveld - 2011-05-15 20:59:29 +0200
reading from disk
Jörn M. Horschig - 2011-07-26 12:04:04 +0200
Might the problem be caused just by storing the data? In my case, 137 trials, this is already ~1.5gigs: Name Size Bytes Class Attributes dat 1x137 1408594544 cell The rest of the code looks quite memory-efficient. Btw, in case you give data as an input argument, the data is stored again in a dat-matrix, which also is extremely inefficient. If we do not want to store the data in memory, then we need to apply the preprocessing algorithm twice to the data (thus the function will be twice as slow), once for calculating the mean and standard devitation, and once for transforming the amplitude-values to z-values. Should I add a cfg-option which allows this? I think it might be preferable to do on low-memory machines.
Jörn M. Horschig - 2011-08-29 17:06:36 +0200
As discussed some weeks ago, I fixed this so that ft_artifact_zvalue can be called with cfg.memory='low', meaning that it will require way less memory (but more computation time). If strcmp(cfg.memory, 'low') then it will not store all the preprocessed data, but in return do the desired preprocessing two times, in two different loops