creators_name: Edmonds, Bruce
editors_name: Aerts, Diederik
type: journalp
datestamp: 2001-08-30
lastmod: 2011-03-11 08:54:46
metadata_visibility: show
title: Complexity and Scientific Modelling
ispublished: pub
subjects: comp-sci-mach-learn
subjects: phil-sci
full_text_status: public
keywords: complexity, modelling, representation, specificity, noise, error rate, simplicity, language, order, disorder, randomness
abstract: There have been many attempts at formulating measures of complexity of physical processes. Here we reject this direct approach and attribute complexity only to models of these processes in a given language, to reflect its "difficulty". A framework for modelling is outlined which includes the language of modelling, the complexity of models in that language, the error in the model's predictions and the specificity of the model. Many previous formulations of complexity can be seen as either: a special case of this framework; attempts to "objectify" complexity by considering only minimally complex models or its asymptotic behaviour; relativising it to a fixed mathematical structure in the absence of noise; misnamed in that they capture the specificity rather than the complexity. Such a framework makes sense of a number of aspects of scientific modelling. Complexity does not necessarily correspond to a lack of simplicity or lie between order and disorder. When modelling is done by agents with severe resource limitations, the acceptable trade-offs between complexity, error and specificity can determine the effective relations between these. The characterisation of noise will emerge from this. Simpler theories are not a priori more likely to be correct but sometimes preferring the simpler theory at the expense of accuracy can be a useful heuristic.
date: 2000
date_type: published
publication: Foundations of Science
volume: 5
number: 3
publisher: Kluwer Academic
pagerange: 379-390
refereed: TRUE
referencetext: Cambridge.

[2] Chaitin, G.J. 1966. On the Length of Programs for Computing Finite Binary Sequences, Journal of the Association of
Computing Machinery, 13, 547-569.

[3] Crutchfield, J.P. 1994. The Calculi of Emergence: Computation, Dynamics and Induction. Physica D, 75, 11-54.

[4] Edmonds, B. 1995. A Hypertext Bibliography of Measures of Complexity. 

[5] Edmonds, B. (forthcoming). What is Complexity?: the philosophy of Complexity per se with application to some
examples in evolution. In F. Heylighen & D. Aerts (eds.): The Evolution of Complexity, Kluwer, Dordrecht.

[6] Grassberger, P. 1986. Towards a Quantitative Theory of Self-Generated Complexity. International Journal of
Theoretical Physics, 25, 907-938.

[7] Kauffman, S.A. 1993. The Origins of Order. Oxford University Press, New York.

[8] Kolmogorov, A.N. 1965. Three Approaches to the Quantitative Definition of Information, Problems of Information
Transmission, 1, 1-17.

[9] Murphy, P.M. and Pazzani, M.J. 1994. Exploring the Decision Forest: An Empirical Investigation of Occam's Razor in
Decision Tree Induction, Journal of Artificial Intelligence Research, 1, 257-275. 

[10] Pearl, J.P. 1978. On the Connection Between the Complexity and Credibility of Inferred Models, International Journal
of General Systems, 4, 255-264.

[11] Popper, K.R. 1968. Logic of Scientific Discovery, Hutchinson, London.

[12] Quine, W.V.O. 1960. Simple Theories of a Complex World, in The Ways of Paradox, Eds., Random House, New
York, pages 242-246.

[13] Rissanen, J. 1990. Complexity of Models. In Zurek,W.H. (ed.). Complexity, Entropy and the Physics of Information.
Addison-Wesley, Redwood City, California, 117-125.

[14] Sober, E. 1975. Simplicity. Clarendon Press, Oxford.

[15] Solomonoff, R.J. 1964. A Formal theory of Inductive Inference. Information and Control, 7, 1-22, 224-254.

citation:   Edmonds, Bruce  (2000) Complexity and Scientific Modelling.  [Journal (Paginated)]     
document_url: http://cogprints.org/1773/1/compsciA4.ps
document_url: http://cogprints.org/1773/5/compsci.pdf