--- abstract: | Linear Independent Components Analysis (ICA) has become an important signal processing and data analysis technique, the typical application being blind source separation in a wide range of signals, such as biomedical, acoustical and astrophysical ones. Nonlinear ICA is less developed, but has the potential to become at least as powerful. This paper presents MISEP, an ICA technique for linear and nonlinear mixtures, which is based on the minimization of the mutual information of the estimated components. MISEP is a generalization of the popular INFOMAX technique, which is extended in two ways: (1) to deal with nonlinear mixtures, and (2) to be able to adapt to the actual statistical distributions of the sources, by dynamically estimating the nonlinearities to be used at the outputs. The resulting MISEP method optimizes a network with a specialized architecture, with a single objective function: the output entropy. Examples of both linear and nonlinear ICA performed by MISEP are presented in the paper. altloc: [] chapter: ~ commentary: ~ commref: ~ confdates: ~ conference: ~ confloc: ~ contact_email: ~ creators_id: [] creators_name: - family: Almeida given: Luis B. honourific: '' lineage: '' date: 2002-12 date_type: published datestamp: 2003-01-05 department: ~ dir: disk0/00/00/26/87 edit_lock_since: ~ edit_lock_until: ~ edit_lock_user: ~ editors_id: [] editors_name: [] eprint_status: archive eprintid: 2687 fileinfo: /style/images/fileicons/application_pdf.png;/2687/1/PAPER.pdf full_text_status: public importid: ~ institution: ~ isbn: ~ ispublished: unpub issn: ~ item_issues_comment: [] item_issues_count: 0 item_issues_description: [] item_issues_id: [] item_issues_reported_by: [] item_issues_resolved_by: [] item_issues_status: [] item_issues_timestamp: [] item_issues_type: [] keywords: 'Independent components analysis, nonlinear, blind source separation, ICA, BSS' lastmod: 2011-03-11 08:55:07 latitude: ~ longitude: ~ metadata_visibility: show note: Submitted to the JOurnal of Machine Learning Research number: ~ pagerange: ~ pubdom: FALSE publication: Submitted to the Journal of Machine Learning Research publisher: ~ refereed: FALSE referencetext: |+ L. B. Almeida. Multilayer perceptrons. In E. Fiesler and R. Beale, editors, Handbook of Neural Computation. Institute of Physics, Oxford University Press, 1997. available at http://www.oup-usa.org/acadref/ncc1_2.pdf . L. B. Almeida. Linear and nonlinear ICA based on mutual information. In Proc. Symp. 2000 on Adapt. Sys. for Sig. Proc., Commun. and Control, Lake Louise, Alberta, Canada, 2000. L. B. Almeida. Simultaneous MI-based estimation of independent components and of their distributions. In Proc. Second Int. Worksh. Independent Component Analysis and Blind Signal Separation, pages 169-174, Helsinki, Finland, 2000. L. B. Almeida. ICA of linear and nonlinear mixtures based on mutual information. In Proc. 2001 Int. Joint Conf. on Neural Networks, Washington, D.C., 2001. L. B. Almeida. MISEP - an ICA method for linear and nonlinear mixtures, based on mutual information. In Proc. 2002 Int. Joint Conf. on Neural Networks, Honolulu, Hawaii, 2002. L. B. Almeida. MISEP - linear and nonlinear ICA based on mutual information. Signal Processing, 2003. To be submitted for publication. S. Amari, A. Cichocki, and H. H. Yang. A new learning algorithm for blind signal separation. In NIPS 95, pages 882-893. MIT Press, 1996. A. Bell and T. Sejnowski. An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7:1129-1159, 1995. G. Burel. Blind separation of sources: A nonlinear neural algorithm. Neural Networks, 5 (6):937-947, 1992. J.-F. Cardoso. Infomax and maximum likelihood for source separation. IEEE Letters on Signal Processing, 4:112-114, 1997. J.-F. Cardoso and A. Souloumiac. Jacobi angles for simultaneous diagonalization. SIAM Journal of Matrix Analysis and Applications, 17(1), 1996. P. Comon. Independent component analysis - a new concept? Signal Processing, 36: 287-314, 1994. G. Darmois. Analyse generale des liaisons stochastiques. Rev. Inst. Internat. Stat., 21:2-8, 1953. G. Deco and W. Brauer. Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures. Neural Networks, 8:525-535, 1995. S. Haykin and P. Gupta. A new activation function for blind signal separation. ASL Technical Report 1, McMaster University, Hamilton, Ontario, Canada, 1999. A. Hyvarinen and E. Oja. A fast fixed-point algorithm for independent component analysis. Neural Computation, 9(7):1483-1492, 1997. A. Hyvarinen and P. Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks, 12(3):429-439, 1999. T.-W. Lee, M. Girolami, A. Bell, and T. Sejnowski. An unifying information-theoretic framework for independent component analysis. International Journal on Mathematical and Computer Modeling, 1998. T.-W. Lee, M. Girolami, and T. Sejnowski. Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and super-gaussian sources. Neural Computation, 11:417-441, 1999. G. C. Marques and L. B. Almeida. An objective function for independence. In Proc. International Conference on Neural Networks, pages 453-457, Washington DC, 1996. G. C. Marques and L. B. Almeida. Separation of nonlinear mixtures using pattern repulsion. In J. F. Cardoso, C. Jutten, and P. Loubaton, editors, Proc. First Int. Worksh. Independent Component Analysis and Signal Separation, pages 277-282, Aussois, France, 1999. A. Taleb and C. Jutten. Entropy optimization - application to blind separation of sources. In Proc. ICANN'97, Lausanne, Switzerland, 1997. H. Valpola. Nonlinear independent component analysis using ensemble learning: Theory. In Proc. Second Int. Worksh. Independent Component Analysis and Blind Signal Separation, pages 251-256, Helsinki, Finland, 2000. relation_type: [] relation_uri: [] reportno: ~ rev_number: 12 series: ~ source: ~ status_changed: 2007-09-12 16:46:18 subjects: - comp-sci-stat-model - comp-sci-neural-nets succeeds: ~ suggestions: |- I've filled in the "Publication" field because the system forces me to do so, but what I'm trying to do is to submit a manuscript that was submitted for publication and is currently being refereed (I assume it's possible to deposit this kind of preprints in this archive). -- Thanks -- sword_depositor: ~ sword_slug: ~ thesistype: ~ title: 'MISEP - Linear and Nonlinear ICA Based on Mutual Information' type: journalp userid: 3730 volume: ~