Back to the main page.

Bug 2796 - do not use source.avg.xxx in any FieldTrip function and use logical inside

Status ASSIGNED
Reported 2014-12-24 22:48:00 +0100
Modified 2016-12-13 17:51:16 +0100
Product: FieldTrip
Component: core
Version: unspecified
Hardware: PC
Operating System: Mac OS
Importance: P5 normal
Assigned to: Robert Oostenveld
URL:
Tags:
Depends on:
Blocks:
See also:

Robert Oostenveld - 2014-12-24 22:48:19 +0100

although old-style data structures might contain source.avg.xxx, all fieldtrip code should internally work with the new-style source representation


Robert Oostenveld - 2015-01-05 14:05:52 +0100

oboos@mentat001> git commit -a [bug2796-sourceavg 67dbb8c] restructuring - replace all code that uses source.avg with the new (flat) source structure. This is a challenging refactoring of the code, and the current commit status is intermediate and not very well defined. See http://bugzilla.fcdonders.nl/show_bug.cgi?id=2796 29 files changed, 585 insertions(+), 196 deletions(-)


Robert Oostenveld - 2015-02-04 09:47:40 +0100

mac011> svn commit Sending besa2fieldtrip.m Sending fileio/private/getdimord.m Sending fileio/private/getdimsiz.m Sending ft_annotate.m Sending ft_anonimizedata.m Sending ft_detect_movement.m Sending ft_dipolefitting.m Sending ft_examplefunction.m Sending ft_freqstatistics.m Sending ft_globalmeanfield.m Sending ft_interpolatenan.m Sending ft_math.m Sending ft_prepare_leadfield.m Sending ft_prepare_sourcemodel.m Sending ft_regressconfound.m Sending ft_removetmsartifact.m Sending ft_sourceanalysis.m Sending ft_sourcegrandaverage.m Sending ft_sourcemovie.m Sending ft_sourceplot.m Sending ft_sourcestatistics.m Sending ft_statistics_montecarlo.m Sending ft_timelockstatistics.m Sending loreta2fieldtrip.m Sending private/getdimord.m Sending private/getdimsiz.m Sending spass2fieldtrip.m Sending test/test_bug1150.m Sending test/test_bug1775.m Sending test/test_bug2185.m Sending test/test_ft_statistics_montecarlo.m Sending test/test_tutorial_plotting.m Sending utilities/ft_checkconfig.m Sending utilities/ft_checkdata.m Sending utilities/ft_datatype_source.m Sending utilities/ft_datatype_volume.m Sending utilities/ft_selectdata.m Sending utilities/private/getdimord.m Sending utilities/private/getdimsiz.m Transmitting file data ....................................... Committed revision 10156. ------------------------------------------------------------------------ r10156 | roboos | 2015-02-04 09:45:19 +0100 (Wed, 04 Feb 2015) | 2 lines restructuring - large merge of various changes that I made in https://github.com/fieldtrip/fieldtrip/pull/53. The relevant changes have to do with the way that source.avg is handled (now always removed at the beginning) and with source.inside (now always logical). This commit needs to be tested before it can be released. ------------------------------------------------------------------------ I merged the changes in 2795 and the changes in the master into bug2796-sourceavg branch. Subsequently I made a patch with all differences between the branch and master and applied that patch to the SVN version.


Robert Oostenveld - 2015-02-04 09:48:11 +0100

the automatic update to home/common is disabled for the moment


Robert Oostenveld - 2015-02-05 15:56:49 +0100

JM already found some of the issues. I did further testing, which shows that the inside handling was actually still quite in a rough state. I updated all functions in fieldtrip/inverse to do the inside handling correctly. it would be good to write a test script that deals with the source.avg and source.trial representation, but also with the various inverse methods. E.g. I guess that nobody has ever tried whether the eloreta implementation works on single trial data. ------------------------------------------------------------------------ r10170 | roboos | 2015-02-05 13:59:15 +0100 (Thu, 05 Feb 2015) | 2 lines bugfix - removed three more ocurrences of dip.outside. The handling of inside and outside should now be (almost) consistent between all inverse functions. Needs to be checked... ------------------------------------------------------------------------ r10169 | roboos | 2015-02-05 13:56:47 +0100 (Thu, 05 Feb 2015) | 2 lines restructuring and bugfixes - this continuous on r10156 where the handling of source.avg and source.trial was changes, but in which also the inside/outside handling was changed. All code now assumes (or should assume) that the inside specification in the cfg and data structures is a boolean vector rather than an indexed vector. The outside field is deprecated alltogether.


Robert Oostenveld - 2015-02-05 17:07:54 +0100

the automatic update to home/common and to the ftp (every evening) are enabled again. roboos@mentat002> crontab -e crontab: installing new crontab


Robert Oostenveld - 2015-02-05 17:17:15 +0100

the following functions seem to be using grid.inside as an indexed representation mac011> grep -l length.*inside *.m ft_dipolefitting.m ft_prepare_leadfield.m ft_sourceanalysis.m ft_sourcedescriptives.m


Robert Oostenveld - 2015-02-05 17:43:57 +0100

(In reply to Robert Oostenveld from comment #6) of these, only ft_sourcedescriptives was affected (but also seriously). mac011> svn commit Sending ft_sourcedescriptives.m Sending inverse/minimumnormestimate.m Sending test/test_bug1901.m Transmitting file data ... Committed revision 10173.


Robert Oostenveld - 2015-02-05 17:47:01 +0100

*** Bug 2826 has been marked as a duplicate of this bug. ***


Robert Oostenveld - 2015-02-06 09:02:26 +0100

there are a number of test scripts failing because inside the test script it is assumed that inside is represented as list of indices. Furthermore, some lower-level functions (in subdirectories) are still coded for a indexed representation. Potentially these ones mac011> grep -l length.*inside */*.m forward/ft_headmodel_interpolate.m private/data2raw.m private/estimate_fwhm1.m private/lbex.m private/mollify.m private/patchsvd.m private/sel50p.m private/univariate2bivariate.m test/test_bug1232.m test/test_bug2355.m utilities/ft_source2full.m utilities/ft_source2sparse.m


Robert Oostenveld - 2015-02-06 15:02:50 +0100

the following test scripts were recently working fine, but are now failing test_bug1443.txt test_bug1530.txt test_bug1563.txt test_bug1764.txt test_bug1984_2187.txt test_bug2069.txt test_bug2265.txt test_bug2355.txt test_bug2359.txt test_bug2381.txt test_bug2511.txt test_bug2606.txt test_bug2647.txt test_bug798.txt test_example_simulate_forward_beamforming.txt test_ft_componentanalysis.txt test_ft_freqstatistics.txt test_ft_sourceinterpolate.txt test_ft_statistics_montecarlo.txt test_ft_timelockstatistics.txt test_tutorial_beamformer.txt test_tutorial_beamformer20120321.txt test_tutorial_beamformer20131122.txt test_tutorial_beamformingextended.txt test_tutorial_beamformingextended20131122.txt test_tutorial_clusterpermutationfreq.txt test_tutorial_clusterpermutationtimelock.txt test_tutorial_coherence.txt test_tutorial_natmeg_beamforming.txt test_tutorial_natmeg_statistics.txt


Robert Oostenveld - 2015-02-06 15:41:29 +0100

(In reply to Robert Oostenveld from comment #10) I fixed the fieldtrip code to get the first few working again, but have not yet checked all. mac011> svn commit Deleting fileio/private/data2raw.m Sending forward/ft_headmodel_interpolate.m Sending ft_sourceinterpolate.m Sending ft_sourceplot.m Deleting private/data2raw.m Sending test/test_bug1443.m Sending utilities/ft_checkdata.m Sending utilities/ft_datatype_source.m Sending utilities/ft_datatype_volume.m Deleting utilities/private/data2raw.m Transmitting file data ....... Committed revision 10174.


Jens Klinzing - 2016-12-13 17:51:16 +0100

I am currently working on making ft_sourcedescriptives work for me (concerning the deprecated .avg, .trial, .method etc. for PCC input) and I'd like to do it in a way that makes later merging it with the main branch one day most likely. Where is the average over trials, as calculated by ft_sourcedescriptives, supposed to be represented under the new convention? Is it supposed to replace the trials represented in cell arrays under source.pow? In that way a structure can never have both, trials and their average?