Back to the main page.

Bug 3448 - dft filter (freq) increases the spectral power (freq) rather than decrease it

Status	CLOSED FIXED
Reported	2018-09-04 15:05:00 +0200
Modified	2018-10-08 10:26:26 +0200
Product:	FieldTrip
Component:	preproc
Version:	unspecified
Hardware:	PC
Operating System:	Linux
Importance:	P5 normal
Assigned to:	Jan-Mathijs Schoffelen
URL:
Tags:
Depends on:
Blocks:
See also:

Georgios Michalareas - 2018-09-04 15:05:58 +0200

I have a single channel dataset. The data comes from a photodiode on a backprojection screen on which a 60 Hz projector is displaying stimuli. Naturally in the spectrum of the data thee is a small 60 hz peak. I applied a dft filter through ft_preprocessing and computed the spectrum again . I was expecting the peak to be gone at 60 Hz but instead the peak had been dramatically increased. It is not clear to me at all how this could happen. data is in variable 'data' in attached file 'datafile4dft.mat' The code to reproduce the problem is the following: %----------------------------------------------------- load datafile4dft data cfg=[]; cfg.dftfilter='yes'; cfg.dftfreq=[60 120 ]; datadft=ft_preprocessing(cfg,data); freqCfg = []; freqCfg.method = 'mtmfft'; freqCfg.taper = 'hanning'; freqCfg.foi = [1:200]; freqCfg.tapsmofrq = 1; freqCfg.keeptrials = 'yes'; freqCfg.feedback = 'text'; freq=ft_freqanalysis(freqCfg, data); freqdft=ft_freqanalysis(freqCfg, datadft); figure;hold on; plot(freq.freq,squeeze(freq.powspctrm)); plot(freqdft.freq,squeeze(freqdft.powspctrm)); legend('before dft','after dft');

Georgios Michalareas - 2018-09-04 15:10:45 +0200

datafile4dft.mat is more than 1MB so attachement here failed. I uploaded in my gdrive and you can find it in this link: https://drive.google.com/file/d/1Y9gmd3MLKAzM5y_OTdg87ZFtTlESNVKi/view?usp=sharing

Jan-Mathijs Schoffelen - 2018-09-24 20:45:07 +0200

Hehe, I can only think of one person who does an fft on more than 800000 samples... I think what you observe is the consequence of the fact that the frequency of the artifact is at 60.01, rather than at 60 Hz. I noticed this when I trimmed the data to be 1200*600 samples long, and repeated the ft_freqanalysis, focussing on freqCfg.foi = [58:1./600:62]; Please confirm. I don't think this is a failure of the dft-filter as such.

Jan-Mathijs Schoffelen - 2018-09-24 20:54:03 +0200

In addition, the freq and freqdft are somewhat different outside the frequency notches, because ft_preproc_dftfilter trims the data to an integer number of cycles of the to-be-notched frequencies before estimating the dft-coefficients. This causes the 'outside' data to contribute to the difference. If I do something like data.trial{1} = data.trial{1}(1:600*1200); data.time{1} = data.time{1}(600*1200); and then freqCfg.foi = 58:1./600:62; I observe the peak frequency to be slightly off 60 Hz. If I do cfg.dftfreq = 60+[-10:10]./600; datadft2 = ft_preprocessing(cfg, data); and then freqdft2 = ft_freqanalysis(freqCfg,datadft2); it notches nicely.

Jan-Mathijs Schoffelen - 2018-10-08 09:41:12 +0200

can this be closed?

Georgios Michalareas - 2018-10-08 09:59:32 +0200

You are the best man, thanks a lot! Yes close it