This site has been permanently archived. This is a static copy provided by the University of Southampton.
--- abstract: |- This paper presents a theory of error in cross-validation testing of algorithms for predicting real-valued attributes. The theory justifies the claim that predicting real-valued attributes requires balancing the conflicting demands of simplicity and accuracy. Furthermore, the theory indicates precisely how these conflicting demands must be balanced, in order to minimize cross-validation error. A general theory is presented, then it is developed in detail for linear regression and instance-based learning. altloc: - http://extractor.iit.nrc.ca/publications/NRC-35072.pdf chapter: ~ commentary: ~ commref: ~ confdates: ~ conference: ~ confloc: ~ contact_email: ~ creators_id: [] creators_name: - family: Turney given: Peter D. honourific: '' lineage: '' date: 1994 date_type: published datestamp: 2001-10-13 department: ~ dir: disk0/00/00/18/20 edit_lock_since: ~ edit_lock_until: ~ edit_lock_user: ~ editors_id: [] editors_name: [] eprint_status: archive eprintid: 1820 fileinfo: /style/images/fileicons/application_pdf.png;/1820/3/NRC%2D35072.pdf full_text_status: public importid: ~ institution: ~ isbn: ~ ispublished: pub issn: ~ item_issues_comment: [] item_issues_count: 0 item_issues_description: [] item_issues_id: [] item_issues_reported_by: [] item_issues_resolved_by: [] item_issues_status: [] item_issues_timestamp: [] item_issues_type: [] keywords: ~ lastmod: 2011-03-11 08:54:48 latitude: ~ longitude: ~ metadata_visibility: show note: ~ number: ~ pagerange: 361-391 pubdom: FALSE publication: Journal of Experimental and Theoretical Artificial Intelligence publisher: ~ refereed: TRUE referencetext: | Aha, D.W., Kibler, D. (1989) Noise-tolerant instance-based learning algorithms, Proceed-ings of the Eleventh International Joint Conference on Artificial Intelligence, 794- 799. Aha, D.W., Kibler, D., & Albert, M.K. (1991) Instance-based learning algorithms, Machine Learning, 6:37-66. Akaike, H. (1970) Statistical predictor identification, Annals of the Institute of Statistical Mathematics, 22:203-217. Akaike, H. (1973) Information theory and an extension of the maximum likelihood principle, Second International Symposium on Information Theory, edited by B.N. Petrov and F. Csaki (Budapest: Akademia Kiado). Akaike, H. (1974) A new look at the statistical model identification, IEEE Transactions on Automatic Control, AC-19: 716-723. Barron, A.R. (1984) Predicted squared error: a criterion for automatic model selection, in Self-organizing Methods in Modeling: GMDH Type Algorithms, edited by S.J. Farlow (New York: Marcel Dekker). Dasarathy, B.V. (1991) Nearest Neighbor Pattern Classification Techniques, Edited col-lection (California: IEEE Press). Draper, N.R. & Smith, H. (1981) Applied Regression Analysis, Second Edition (New York: John Wiley & Sons). Ein-Dor, P. & Feldmesser, J. (1987) Attributes of the performance of central processing units: a relative performance prediction model, Communications of the ACM, 30:308-317. Eubank, R.L. (1988) Spline Smoothing and Nonparametric Regression (New York: Marcel Dekker). Fraser, D.A.S. (1976) Probability and Statistics: Theory and Applications (Massachusetts: Duxbury Press). Geman, S., Bienenstock, E., & Doursat, R. (1992) Neural networks and the bias/variance dilemma, Neural Computation, 4:1-58. Kibler, D., Aha, D.W., & Albert, M.K. (1989) Instance-based prediction of real-valued attributes, Computational Intelligence, 5:51-57. Moody, J.E. (1991) Note on generalization, regularization, and architecture selection in nonlinear learning systems, First IEEE-SP Workshop on Neural Networks for Signal Processing (California: IEEE Press). Moody, J.E. (1992) The effective number of parameters: an analysis of generalization and regularization in nonlinear learning systems, in Advances in Neural Information Processing Systems 4, edited by J.E. Moody, S.J. Hanson, and R.P. Lippmann (Cali-fornia: Morgan Kaufmann). Sakamoto, Y., Ishiguro, M., & Kitagawa, G. (1986) Akaike Information Criterion Statis-tics (Dordrecht, Holland: Kluwer Academic Publishers). Strang, G. (1976) Linear Algebra and Its Applications (New York: Academic Press). Turney, P.D. (1990) The curve fitting problem: a solution, British Journal for the Philoso-phy of Science, 41:509-530. relation_type: [] relation_uri: [] reportno: ~ rev_number: 12 series: ~ source: ~ status_changed: 2007-09-12 16:41:12 subjects: - comp-sci-art-intel - comp-sci-mach-learn - comp-sci-stat-model succeeds: ~ suggestions: ~ sword_depositor: ~ sword_slug: ~ thesistype: ~ title: A theory of cross-validation error type: journalp userid: 2175 volume: 6