---
abstract: 'Minimisation methods for training feed-forward networks with back-propagation are compared. Feed-forward neural network training is a special case of function minimisation, where no explicit model of the data is assumed. Therefore, and due to the high dimensionality of the data, linearisation of the training problem through use of orthogonal basis functions is not desirable. The focus is on function minimisation on any basis. Quasi-Newton and conjugate gradient methods are reviewed, and the latter are shown to be a special case of error back-propagation with momentum term. Three feed-forward learning problems are tested with five methods. It is shown that, due to the fixed stepsize, standard error back-propagation performs well in avoiding local minima. However, by using not only the local gradient but also the second derivative of the error function a much shorter training time is required. Conjugate gradient with Powell restarts shows to be the superior method.'
altloc:
  - http://www.robotic.dlr.de/Smagt/thesis/chapter2.ps.gz
chapter: ~
commentary: ~
commref: ~
confdates: ~
conference: ~
confloc: ~
contact_email: ~
creators_id: []
creators_name:
  - family: Smagt
    given: P. van der
    honourific: ''
    lineage: ''
date: 1994
date_type: published
datestamp: 1998-07-03
department: ~
dir: disk0/00/00/04/97
edit_lock_since: ~
edit_lock_until: ~
edit_lock_user: ~
editors_id: []
editors_name: []
eprint_status: archive
eprintid: 497
fileinfo: /style/images/fileicons/application_postscript.png;/497/2/chapter2.ps
full_text_status: public
importid: ~
institution: ~
isbn: ~
ispublished: pub
issn: ~
item_issues_comment: []
item_issues_count: 0
item_issues_description: []
item_issues_id: []
item_issues_reported_by: []
item_issues_resolved_by: []
item_issues_status: []
item_issues_timestamp: []
item_issues_type: []
keywords: 'feed-forward neural network training, numerical optimisation techniques, neural function approximation, error back-propagation, conjugate gradient, quasi-Newton'
lastmod: 2011-03-11 08:54:00
latitude: ~
longitude: ~
metadata_visibility: show
note: ~
number: 1
pagerange: 1-11
pubdom: FALSE
publication: Neural Networks
publisher: ~
refereed: TRUE
referencetext: ~
relation_type: []
relation_uri: []
reportno: ~
rev_number: 10
series: ~
source: ~
status_changed: 2007-09-12 16:29:09
subjects:
  - comp-sci-neural-nets
succeeds: ~
suggestions: ~
sword_depositor: ~
sword_slug: ~
thesistype: ~
title: Minimisation methods for training feed-forward networks
type: journalp
userid: 462
volume: 7
<script src="//archive-bar.soton.ac.uk/archive-bar.js"></script>