---
abstract: |2
The present paper is written as a word of caution, with users of
independent component analysis (ICA) in mind, to overlearning
phenomena that are often observed.\\
We consider two types of overlearning, typical to high-order
statistics based ICA. These algorithms can be seen to maximise the
negentropy of the source estimates. The first kind of overlearning
results in the generation of spike-like signals, if there are not
enough samples in the data or there is a considerable amount of
noise present. It is argued that, if the data has power spectrum
characterised by $1/f$ curve, we face a more severe problem, which
cannot be solved inside the strict ICA model. This overlearning is
better characterised by bumps instead of spikes. Both overlearning
types are demonstrated in the case of artificial signals as well as
magnetoencephalograms (MEG). Several methods are suggested to
circumvent both types, either by making the estimation of the ICA
model more robust or by including further modelling of the data.
altloc:
- http://www.cis.hut.fi/jaakkos/papers/sarela03.pdf
- http://www.jmlr.org/papers/volume4/sarela03a/sarela03a.pdf
chapter: ~
commentary: ~
commref: ~
confdates: ~
conference: ~
confloc: ~
contact_email: ~
creators_id:
- 4715
creators_name:
- family: Särelä
given: Jaakko
honourific: Mr
lineage: ''
date: 2003-12
date_type: published
datestamp: 2004-05-24
department: ~
dir: disk0/00/00/36/38
edit_lock_since: ~
edit_lock_until: ~
edit_lock_user: ~
editors_id: []
editors_name:
- family: Lee
given: Te-Won
honourific: prof.
lineage: ''
- family: Cardoso
given: Jean-Francois
honourific: prof.
lineage: ''
- family: Oja
given: Erkki
honourific: prof.
lineage: ''
- family: Amari
given: Shun-Ichi
honourific: prof.
lineage: ''
eprint_status: archive
eprintid: 3638
fileinfo: /style/images/fileicons/application_pdf.png;/3638/1/sarela03.pdf
full_text_status: public
importid: ~
institution: ~
isbn: ~
ispublished: pub
issn: ~
item_issues_comment: []
item_issues_count: 0
item_issues_description: []
item_issues_id: []
item_issues_reported_by: []
item_issues_resolved_by: []
item_issues_status: []
item_issues_timestamp: []
item_issues_type: []
keywords: 'independent component analysis, blind source separation, overlearning, overfitting, spikes, bumps, high-order ICA'
lastmod: 2011-03-11 08:55:36
latitude: ~
longitude: ~
metadata_visibility: show
note: ~
number: ~
pagerange: 1447-1469
pubdom: FALSE
publication: Journal of machine learning research
publisher: MIT press
refereed: TRUE
referencetext: ~
relation_type: []
relation_uri: []
reportno: ~
rev_number: 12
series: ~
source: ~
status_changed: 2007-09-12 16:52:31
subjects:
- comp-sci-stat-model
- comp-sci-mach-learn
- comp-sci-neural-nets
succeeds: ~
suggestions: ~
sword_depositor: ~
sword_slug: ~
thesistype: ~
title: ' Overlearning in marginal distribution-based ICA: analysis and solutions'
type: journalp
userid: 4715
volume: 4