Back to the main page.
Bug 595 - reading a few channel from an sqd file is very slow and takes too much memory
| Status | CLOSED FIXED |
| Reported | 2011-04-22 11:42:00 +0200 |
| Modified | 2011-05-05 21:24:25 +0200 |
| Product: | FieldTrip |
| Component: | yokogawa |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Mac OS |
| Importance: | P1 normal |
| Assigned to: | Robert Oostenveld |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Lesley Sand - 2011-04-22 11:42:01 +0200
[[Robert wrote this from Lesley's account]] ... this is due to the low-level function reading all data into memory and only then selecting the channel of interest. It is especially obvious when you read the trigger channel, which won't fit in memory on a 2GB/4GB laptop. This can be solved by 1) reading chunks and selecting the channel for each chunk 2) using other ("sqdread") reading functions
Robert Oostenveld - 2011-04-22 14:47:08 +0200
this is now much faster, by using the sqdproject code from maryland. The header reading still uses the p-files, the data reading uses the sqdread function if it is available. If not available, the yokogawa p-files will still be used. manzana> svn commit Sending fileio/ft_read_data.m Sending fileio/ft_read_event.m Sending fileio/private/read_trigger.m Sending fileio/private/read_yokogawa_event.m Transmitting file data .... Committed revision 3376.
Tilmann Sander-Thommes - 2011-04-26 11:16:28 +0200
This is a question for the people at CNL / KIT MEG lab at Maryland: I have a copy of sqdproject2.1.1beta obtained from people at Maryland years ago. Is there a newer version ? Is it somewhere available on the web ? Is Shantanu Ray still the maintainer ? It is good to have a faster and easier to use library for Yokogawa MEG data and we might aim to make it more mainstream and test / adapt it to other Yokogawa systems !?
Robert Oostenveld - 2011-04-26 13:28:03 +0200
(In reply to comment #2) I am not sure who the maintainer is, but there is a beta3 version at http://www.isr.umd.edu/Labs/CSSL/simonlab. I had to make 3 fixes to the code to make it work, and I'll forward those to Jonathan Simon. I will also ask him whether I can redistribute the code in fieldtrip/external/sqdproject. If sqdproject is on your path (e.g. in fieldtrip/external, where ft_hastoolbox will look for it)), ft_read_data will use the sqdread function instead of the p-file. The sqdread is much faster.
Tilmann Sander-Thommes - 2011-04-26 13:36:50 +0200
Can you send me the fixes or a fixed version ? I would test sqdread on data from other machines. I remember that the Maryland code was specific for the machine at Maryland.