---
abstract: |-
Research on bias in machine learning algorithms has generally been concerned with the
impact of bias on predictive accuracy. We believe that there are other factors that should
also play a role in the evaluation of bias. One such factor is the stability of the algorithm;
in other words, the repeatability of the results. If we obtain two sets of data from the same
phenomenon, with the same underlying probability distribution, then we would like our
learning algorithm to induce approximately the same concepts from both sets of data. This
paper introduces a method for quantifying stability, based on a measure of the agreement
between concepts. We also discuss the relationships among stability, predictive accuracy,
and bias.
altloc:
- http://extractor.iit.nrc.ca/publications/NRC-38313.pdf
chapter: ~
commentary: ~
commref: ~
confdates: ~
conference: ~
confloc: ~
contact_email: ~
creators_id: []
creators_name:
- family: Turney
given: Peter D.
honourific: ''
lineage: ''
date: 1995
date_type: published
datestamp: 2001-10-13
department: ~
dir: disk0/00/00/18/19
edit_lock_since: ~
edit_lock_until: ~
edit_lock_user: ~
editors_id: []
editors_name: []
eprint_status: archive
eprintid: 1819
fileinfo: /style/images/fileicons/application_pdf.png;/1819/3/NRC%2D38313.pdf
full_text_status: public
importid: ~
institution: ~
isbn: ~
ispublished: pub
issn: ~
item_issues_comment: []
item_issues_count: 0
item_issues_description: []
item_issues_id: []
item_issues_reported_by: []
item_issues_resolved_by: []
item_issues_status: []
item_issues_timestamp: []
item_issues_type: []
keywords: 'stability, bias, accuracy, repeatability, agreement, similarity.'
lastmod: 2011-03-11 08:54:48
latitude: ~
longitude: ~
metadata_visibility: show
note: ~
number: ~
pagerange: 23-33
pubdom: FALSE
publication: Machine Learning
publisher: Kluwer
refereed: TRUE
referencetext: |
Carnap, R. (1947). Meaning and necessity: A study in semantics and modal logic.
Chicago: University of Chicago Press.
Famili, A., & Turney, P. (1991). Intelligently helping the human planner in industrial
process planning. Artificial Intelligence for Engineering Design, Analysis and Man-ufacturing,
5, 109-124.
Fraser, D.A.S. (1976). Probability and statistics: Theory and applications. Massachusetts:
Duxbury Press.
Haussler, D. (1988) Quantifying inductive bias: AI learning systems and Valiant’s learning
framework. Artificial Intelligence, 36, 177-221.
Honavar, V. (1992). Inductive learning using generalized distance measures. Proceedings
of the 1992 SPIE Conference on Adaptive and Learning Systems. Orlando, Florida.
Levenshtein, A. (1966). Binary codes capable of correcting deletions, insertions, and
reversals. Soviet Physics, 10, 703-710.
Murphy, P.M. & Pazzani, M.J. (1994). Exploring the decision forest: an empirical investi-gation
of Occam’s razor in decision tree induction. Journal for AI Research, ftp
p.gp.cs.cmu.edu, cd /usr/jair/pub, 1, 257-275.
Quinlan, J.R. (1992). C4.5: Programs for machine learning. California: Morgan
Kaufmann.
Rendell, L. (1986). A general framework for induction and a study of selective induction.
Machine Learning, 1, 177-226.
Schaffer, C. (1992). An empirical technique for quantifying preferential bias in inductive
concept learners. Unpublished manuscript. Department of Computer Science,
CUNY/Hunter College, New York.
Schaffer, C. (1993). Overfitting avoidance as bias. Machine Learning, 10, 153-178.
Utgoff, P.E. (1986). Shift of bias for inductive concept learning. In J.G. Carbonell, R.S.
Michalski, and T.M. Mitchell (eds) Machine Learning: An Artificial Intelligence
Approach, Volume II. California: Morgan Kaufmann.
Vapnik, V.N. (1982). Estimation of dependencies based on empirical data. New York:
Springer.
relation_type: []
relation_uri: []
reportno: ~
rev_number: 12
series: ~
source: ~
status_changed: 2007-09-12 16:41:11
subjects:
- comp-sci-art-intel
- comp-sci-mach-learn
- comp-sci-stat-model
succeeds: ~
suggestions: ~
sword_depositor: ~
sword_slug: ~
thesistype: ~
title: 'Technical note: Bias and the quantification of stability'
type: journalp
userid: 2175
volume: 20