Back to the main page.
Bug 3027 - improve support for NIRS data from Homer
| Status | CLOSED FIXED |
| Reported | 2015-12-15 12:26:00 +0100 |
| Modified | 2019-08-10 12:41:48 +0200 |
| Product: | FieldTrip |
| Component: | fileio |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Mac OS |
| Importance: | P5 normal |
| Assigned to: | Robert Oostenveld |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: |
Robert Oostenveld - 2015-12-15 12:26:03 +0100
Homer is used by various people that process NIRS data. It would be good to better interface Homer to FT. See http://www.nmr.mgh.harvard.edu/PMI/resources/homer2/home.htm
Robert Oostenveld - 2015-12-15 12:29:28 +0100
first stage is to gather information. I have example data from Janny ac011> ll bug3027/janny/ total 7624 -rw-r--r--@ 1 roboos staff 584 Dec 15 11:35 being_imitated.sd -rw-r--r--@ 1 roboos staff 927 Dec 15 11:39 conversie_script_matlab -rw-r--r--@ 1 roboos staff 3893100 Dec 15 11:39 data_janny4juni.txt and example data from Homer mac011> ll bug3027/homer_SampleData/ total 8 drwxr-xr-x@ 4 roboos staff 136 Dec 15 12:07 Example1_Simple_Probe drwxr-xr-x@ 6 roboos staff 204 Dec 15 11:58 Example2_tw1 drwxr-xr-x@ 5 roboos staff 170 Dec 15 11:58 Example3_111tap drwxr-xr-x@ 13 roboos staff 442 Dec 15 11:58 Example4_twNI drwxr-xr-x@ 8 roboos staff 272 Dec 15 11:58 Example5_tapTest drwxr-xr-x@ 8 roboos staff 272 Dec 15 11:58 Example6 drwxr-xr-x@ 10 roboos staff 340 Dec 15 11:58 Example7_Zim -rwxr-xr-x@ 1 roboos staff 491 Nov 30 2012 processOpt.cfg and historical data plus documentation from Jan-Mathijs mac011> ll bug3027/NIRS_DCC/ total 31184 -rwxr-xr-x 1 roboos staff 31744 Dec 15 12:26 Header_NIRS.doc -rwxr-xr-x 1 roboos staff 222208 Dec 15 12:26 NIRS arrangement Jan 2013.ppt -rwxr-xr-x 1 roboos staff 677376 Dec 15 12:27 NIRS arrangement Jan 2013_2.ppt -rwxr-xr-x 1 roboos staff 4019 Dec 15 12:27 NIRS_DCC_hdr.m -rwxr-xr-x 1 roboos staff 2063362 Dec 15 12:27 NIRSdata_try_triggers.txt -rwxr-xr-x 1 roboos staff 6058880 Dec 15 12:27 NIRSdata_try_triggers_from_Presentation.txt -rwxr-xr-x 1 roboos staff 872914 Dec 15 12:27 Protocol for matlab NIRS analyses programs_FINAL.pdf -rwxr-xr-x 1 roboos staff 24064 Dec 15 12:27 channels_sources_detectors.xls -rwxr-xr-x 1 roboos staff 894 Dec 15 12:27 conversie_script.m -rwxr-xr-x 1 roboos staff 3893100 Dec 15 12:27 data_janny4juni.txt -rwxr-xr-x 1 roboos staff 1018416 Dec 15 12:27 edited_data_770.txt -rwxr-xr-x 1 roboos staff 1018416 Dec 15 12:27 edited_data_850.txt -rwxr-xr-x 1 roboos staff 20335 Dec 15 12:27 header.h -rwxr-xr-x 1 roboos staff 7470 Dec 15 12:27 oxy_and_deoxy_2_conditions770.m -rwxr-xr-x 1 roboos staff 2401 Dec 15 12:27 oxy_and_deoxy_deltaC770.m -rwxr-xr-x 1 roboos staff 1374 Dec 15 12:27 proc_ev_mark.m -rwxr-xr-x 1 roboos staff 3622 Dec 15 12:27 process_raw_data_16Dfrontal_INCY.m -rwxr-xr-x 1 roboos staff 608 Dec 15 12:27 read_dcc_nirsdata.m -rwxr-xr-x 1 roboos staff 993 Dec 15 12:27 read_nirsdcc_data.m -rwxr-xr-x 1 roboos staff 4844 Dec 15 12:27 read_nirsdcc_header.m
Robert Oostenveld - 2015-12-15 12:31:47 +0100
@Janny: the "protocol for NIRS analysis" pdf file has Sarah Fox as author. Is she involved in the DCC NIRS projects, or is this a protocol that was downloaded somewhere from the internet?
Robert Oostenveld - 2015-12-15 12:36:01 +0100
@Janny: the "Header NIRS.doc" contains the Donders institute array version 1 november 2012. Who write that? Who is U415123 that has been commenting on it? It is a Dutch person, as the comments are in Dutch. As long as we cannot get clear what is involved with the lab and with the data processing, it will not be possible to provide a clear solution to get the data processed in FT.
Robert Oostenveld - 2015-12-15 12:36:58 +0100
@Janny, is the NIRS system that you use the same (or similar) to the one here http://www.cbcd.bbk.ac.uk/research/nirs-cbcd-headgear-plugin-1/nirs-cbcd-headgear-plugin
Robert Oostenveld - 2015-12-15 12:41:26 +0100
(In reply to Robert Oostenveld from comment #4) I found this on our intranet https://intranet.donders.ru.nl/index.php?id=4368 which links to this http://www.ucl.ac.uk/medphys/research/borl/imaging/topography which answers the previous question (yes, it is the same).
Robert Oostenveld - 2015-12-15 12:43:49 +0100
Our intranet links to https://intranet.donders.ru.nl/uploads/media/nts4_instructions_01.pdf which contains this ---- There are two other versions of the control and acquisition software installed on the PC. They both have shortcuts on the desktop. ‘nts4_v2.0.exe’ is a new version of the UCL code, that writes data to file for all source-detector combinations, regardless of whether they are being used or not. This allows the data file to be used with the HOMER analysis software. A further conversion process is needed first though. This is provided by the ‘nts2nirs.m’ matlab script. This is in the folder ‘Matlab code/code for data conversion to HOMER format’. ‘nts_lscp.exe’ is the software that has been written by the Paris group. This has more functionality than the UCL code. Unfortunately though, it does not have an instruction manual yet. The Matlab scripts that we currently use for data analysis are contained in the folder ‘Matlab code/NTS data analysis code september 2012’. ---- The "Paris group" was also mentioned by Jan-Mathijs, I suppose they are an important user of the system that have contributed their developments back to the UCL group who has subsequently been sharing it with DCC. Or do we have a direct relation with the "Paris group"? Does anyone have a link to the "Paris group"?
Jörn M. Horschig - 2015-12-16 10:11:15 +0100
Hi, interfacing with Homer should be really simple, as it relies on a well-defined data format (.nirs). The .nirs file is a .mat file in disguise (i.e. saved .mat with another extension). All details on the .nirs files can be found here: http://homer-fnirs.org/faq/#format I'm gonna paste it here: 2. NIRS data format The most basic unit of a HOMER2 data set is a matlab file in the NIRS format that is the result of data acquisition. An example of a device that generates files in this format is the continuous wave techen device called cw6. The NIRS format has the following fields as a minimum requirement: t - This variable is the data time points array. It contains the data acquisition times. The dimensions of this array are
Jörn M. Horschig - 2015-12-16 10:13:02 +0100
Created attachment 762 an example .nirs file
Janny Stapel - 2016-01-08 17:44:39 +0100
(In reply to Robert Oostenveld from comment #2) @Robert: that pdf has indeed been written by Sarah Lloyd-Fox, who is a leading person in the infant nirs community, working in London. She is not involved in our newer nirs projects, but Ricarda Braukmann works with her in a collaborative infant-at-risk-for-autism EU consortium. Ricarda has spent several months in London to learn how to apply nirs and has been familiarized with their analysis pipeline. I have used the pdf as source for inspiration (Ricarda brought it to Nijmegen), to see how others perform their analyses. However, we decided not to use their (Matlab) scripts for our analyses, as they contained a couple of mistakes, and were tailor-made for their research design (with an in-built trial length differing from ours, a fixed number of conditions and triggers etcetera).
Janny Stapel - 2016-01-08 17:46:15 +0100
(In reply to Robert Oostenveld from comment #4) @Robert: Yes and no. The acquisition machine is the same. Ricarda has used the same headgear as the people at Birkbeck (including Sarah Lloyd-Fox, she works at CBCD at Birkbeck) in studies for the EU consortium on autism (caled EU AIMS). However, together with the "instrumentenmakerij", we have worked on new head gear / caps, as the ones used in London do not allow testing central sites, and we are interested in motor regions. We came up with an EEG cap that Norbert adjusted for us. I can send you pictures if that is useful?
Janny Stapel - 2016-01-08 17:47:26 +0100
(In reply to Robert Oostenveld from comment #6) @Robert: there has been contact in the past with this "Paris group", as Alex Cristia worked for a couple of months at the MPI in Nijmegen, and she is part of the babylab in Paris (http://www.lscp.net/BabyWeb/equipe/chercheurs.php). The nirs system (same type as we now have at the DCCN) was shipped from Paris to the DCCN. Initially, the idea was to collaborate within the Donders during the period the system was here (6 months) on novel projects, also for us to try out the system. Unfortunately, this didn't work out. Alex has provided us with a demo, and as far as we could understand, the special aspect of the acquisition software developed in Paris, is that it synchs all the computers, like tv-crews can do to make sure that if they film a tv-screen, you don't see a flickering screen (at least, that is how I understood it). To our minds, this was far more sophisticated than we would need, an idea that was confirmed by the developer of our nirs acquisition machine when we discussed it with him.
Janny Stapel - 2016-01-08 17:48:45 +0100
(In reply to Robert Oostenveld from comment #3) @Robert, just to clarify, that is my U-number, so me commenting.
Janny Stapel - 2016-01-08 18:02:41 +0100
(In reply to Jörn M. Horschig from comment #7) Hi, Short question about this: we have indeed a script converting the data files from the nts nirs system (which is located at DCCN, the .txt files) into .nirs files (the type used by Homer). For us, it would be okay to first change our files into .nirs files and then convert them into files readable for Fieldtrip. If we would proceed like this, should I then still make a documentation sheet about how our raw data files are built up? I started writing an overview, so it is no problem for me to send it to you in a couple of days from now, but I am not sure what the most logical approach is? (nts --> Fieldtrip, or nts --> Homer --> Fieldtrip) Other users of the NTS nirs system will have to build their own structure for explaining which detectors where placed where and should be connected in the data analyses to which sources (so which source-detector pairs form a channel), regardless of the route we now choose. Homer has a GUI (called SD_GUI) for creating a file specifying these details (called .SD files, another file format which is a Matlab file in disguise). I've tried using the GUI and failed, but it did provide me insight in how to build an SD file in Matlab from scratch. What I could do, if we opt for going for the longer path (with .nirs files as an in-between step), is create some documentation about how to build/adjust an SD file given data from the NTS machine. The longer path is to some extent logical, as there will be many more people knowing how to convert their data into .nirs files, than people using exactly the same type of nirs acquisition machine, the nts, like we do in the Nijmegen babylab. Best, Janny
Jörn M. Horschig - 2016-04-26 16:14:20 +0200
(In reply to Janny Stapel from comment #13) Hi Janny, sorry for the slow reply. I was not sure if Robert wanted to comment on this or not. But, given the time passed, I guess not ;) I'd be pragmatic if I were you and do whatever seems less work for you. Intuitively, I would say Homer->FT would rather easy to do, but there are some inherent limitation that we also currently face.For example optodes are usually placed in an arbitrary coordinate system relative to each other. This does not allow for a topographic plot on a head. Other than that I am not sure what data fields FT needs that Homer does not have. I would propose to make a comparison of the two data representations side-by-side. Maybe start a page on the wiki for this?
Janny Stapel - 2016-10-14 12:29:48 +0200
Created attachment 811 Documentation on .nirs files
Janny Stapel - 2016-10-14 12:48:33 +0200
Created attachment 812 Matlab script to convert nts-system nirs data (.txt files) into .nirs files
Janny Stapel - 2016-10-14 12:49:19 +0200
Created attachment 813 Matlab script used to implement conversion nts to .nirs (homer2 format)
Janny Stapel - 2016-10-14 12:50:26 +0200
Created attachment 814 Example SD file (homer2 type) that Matlab script uses to convert from nts text files to .nirs
Janny Stapel - 2016-10-14 14:27:56 +0200
Dear Robert, Apologies for the long delay, I plan to pursue the nirs analyses on a higher tempo coming months, now that I am back at Donders. I would be delighted if you could create a script that converts Homer2 data (.nirs files) into files readable by Fieldtrip. Homer2 does not provide the option of inspecting the data trial-by-trial, and this is particularly important when trying get a grip on what the data looks like, before we start implementing artifact protocols. The babyBRAIN group is very happy with the artifact rejection options of Fieldtrip (e.g., the option of looking at variability between trials and channels etcetera), which another reason for me to prefer Fieldtrip. To shortly comment on Jörn suggestion about visualization: Although visualization of the topography of effects over the head would be fancy, it is not essential and can be safely left out. I have attached a word document containing a description of a .nirs file in general. Furthermore you find attached two Matlab scripts: one for converting nts-system (what we have at DCCN) data into a .nirs file, and one script I used to concretely implement that conversion on an example data set that you can download here: https://www.wetransfer.com/downloads/c54d4be48c4cc4cb95fbe8e1394f25ee20161014105209/98815ebbdc70f05a704b73aa1cec244220161014105209/9551f3 The link contains a .nirs file (which is the one I hope you can convert to an FT structure) and the raw data from the nts-system, just for in case you are curious how we got from our data to the .nirs file (no dataprocessing steps involved, just shuffling of numbers into different matrices). Best, Janny
Jörn M. Horschig - 2016-10-18 17:06:08 +0200
Hi Janny, great to see that there is progress! It would indeed be good to have a nirs-to-fieldtrip wrapper. It is not super simple though, as the nirs-files in general are more condensed and a lot of information that fieldtrip would like has been thrown out, but to a certain degree the conversion should be possible (possibly some ft-functions need to be adapted checking for empty fields, but that should not be a big issue). We have just been at the sfNIRS conference in Paris and I remember that I talked to someone who wrote a .nirs-ft wrapper and I hoped to have convinced him to contribute this to the community. I will try to contact him again and ask for the code. I'll let you know when I know more. Best, Jörn
Robert Oostenveld - 2017-01-09 14:28:15 +0100
Hi Jorn, Janny and I are working on this bug. Can you send us a nice and rich example Oxymon file that we can look at (with FT) and include in the test script? thanks Robert
Robert Oostenveld - 2017-01-17 11:50:42 +0100
roboos@mac011> git commit [bug3027-homer 433c13e] ENH - added support for NIRS data that was processed with Homer, or converted to Homer format. See http://bugzilla.fieldtriptoolbox.org/show_bug.cgi?id=3027 6 files changed, 253 insertions(+), 30 deletions(-) create mode 100644 fileio/private/homer2opto.m create mode 100644 test/test_bug3027.m This has been merged in the master branch at https://github.com/fieldtrip/fieldtrip/pull/296 There is a test script that works. If any issues are detected, please open a new bug here on bugzilla or give a suggestion for improvement on github.
Jörn M. Horschig - 2017-01-18 10:12:00 +0100
Created attachment 824 oxy3-file Here an example of our oxy3-files. Attached is a 24-channel example, measured on the motor cortex during a finger tapping experiment. Trigger value A indicates onset of the movement, trigger value B indicates offset of the movement. (btw, I plan to use this dataset as one of two datasets for a FieldTrip-NIRS tutorial. I hope to have time for writing such a tutorial after the summer.)
Jörn M. Horschig - 2017-01-18 10:15:00 +0100
Created attachment 825 Optodetemplates.xml I also included the (required) optodetemplates.xml, which determines the (relative) positions of the optodes for the oxy3-file. The oxy3-file has a reference to the optodetemplates.xml, thus it is required to have such a file in order to read in the oxy3-file properly.
Robert Oostenveld - 2017-01-18 11:00:28 +0100
Comment on attachment 824 oxy3-file I copied the file to /home/common/matlab/fieldtrip/data/test/original/nirs/artinis and tried this filename = dccnpath('/home/common/matlab/fieldtrip/data/test/original/nirs/artinis/1102 24 channel_2.oxy3'); hdr = ft_read_header(filename); dat = ft_read_data(filename); evt = ft_read_event(filename); cfg = []; cfg.dataset = filename; cfg.viewmode = 'vertical'; cfg.preproc.demean = 'yes'; ft_databrowser(cfg); but get stuck on the interactive selection of an optode template (which happens in a p-file that I cannot debug any further).
Robert Oostenveld - 2017-01-18 11:12:54 +0100
(In reply to Jörn M. Horschig from comment #24) Sorry, I missed this upload. I'll try with this.
Robert Oostenveld - 2019-08-10 12:35:35 +0200
This closes a whole series of bugs that have been resolved (either FIXED/WONTFIX/INVALID) for quite some time. If you disagree, please file a new issue on https://github.com/fieldtrip/fieldtrip/issues.