Overlearning in marginal distribution-based ICA: analysis and solutions

Särelä, Mr Jaakko (2003) Overlearning in marginal distribution-based ICA: analysis and solutions. [Journal (Paginated)]

Full text available as:



The present paper is written as a word of caution, with users of independent component analysis (ICA) in mind, to overlearning phenomena that are often observed.\\ We consider two types of overlearning, typical to high-order statistics based ICA. These algorithms can be seen to maximise the negentropy of the source estimates. The first kind of overlearning results in the generation of spike-like signals, if there are not enough samples in the data or there is a considerable amount of noise present. It is argued that, if the data has power spectrum characterised by $1/f$ curve, we face a more severe problem, which cannot be solved inside the strict ICA model. This overlearning is better characterised by bumps instead of spikes. Both overlearning types are demonstrated in the case of artificial signals as well as magnetoencephalograms (MEG). Several methods are suggested to circumvent both types, either by making the estimation of the ICA model more robust or by including further modelling of the data.

Item Type:Journal (Paginated)
Keywords:independent component analysis, blind source separation, overlearning, overfitting, spikes, bumps, high-order ICA
Subjects:Computer Science > Statistical Models
Computer Science > Machine Learning
Computer Science > Neural Nets
ID Code:3638
Deposited By: Särelä, Dr Jaakko
Deposited On:24 May 2004
Last Modified:11 Mar 2011 08:55


Repository Staff Only: item control page