Cogprints

Perspectives on problem solving in educational assessment: analytical, interactive, and collaborative problem solving

Greiff, Samuel and Holt, Daniel V. and Funke, Joachim (2013) Perspectives on problem solving in educational assessment: analytical, interactive, and collaborative problem solving. [Journal (On-line/Unpaginated)]

Full text available as:

[img]
Preview
 PDF
1475Kb

Abstract

Problem solving has received broad public interest as an important competency in modern societies. In educational large-scale assessments paper-pencil based analytical problem solving was included first (e.g., Programme for International Student Assessment, PISA 2003). With growing interest in more complex situations, the focus has shifted to interactive problem solving (e.g., PISA 2012) requiring identification and control of complex systems. In the future, collaborative problem solving represents the next step in assessing problem solving ability (e.g., PISA 2015). This paper describes these different approaches to assessing problem solving ability in large-scale assessments considering theoretical questions as well as assessment issues. For each of the three types of problem solving, the definition and understanding of the construct is explained, items examples are shown together with some empirical results, and limitations of the respective approach are discussed. A final discussion centers on the connection of cognitive and differential psychology within educational research and assessment.

Item Type:Journal (On-line/Unpaginated)
Keywords:complex problem solving; collaborative problem solving; PISA study
Subjects:Psychology > Cognitive Psychology
ID Code:9041
Deposited By: Funke, Dr. Joachim
Deposited On:17 Sep 2013 14:33
Last Modified:17 Sep 2013 14:33

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

Autor, D. H., Levy, F., & Murnane, R. J. (2003). The skill content of recent technological change: An empirical exploration. Quarterly Journal of Economics, 118 (4), 1279-1333.

Baumert, J., Evans, R. H., & Geiser, H. (1998). Technical problem solving among 10-year-old students as related to science achievement, out-of-school experience, domain-specific control beliefs, and attribution patterns. Journal of Research in Science Teaching, 35 (9), 987-1013.

Brannick, M. T. & Prince, C. (1997). An overview of team performance measurement. In M. T. Brannick, E. Salas, & C. Prince (Eds.), Team performance assessment and measurement: theory methods and applications (pp. 3-16). Mahwah, NJ: Lawrence Erlbaum.

Cooke, N. J., Kiekel, P. A., Salas, E., Stout, R., Bowers, C., & Cannon-Bowers, J. (2003). Measuring team knowledge: A window to the cognitive underpinnings of team performance. Group Dynamics: Theory, Research and Practice, 7, 179–219.

Daniel, R. C. & Embretson, S. E. (2010). Designing cognitive complexity in mathematical problem-solving items. Applied Psychological Measurement, 35 (5), 348-364.

Danner, D., Hagemann, D., Holt, D. V., Hager, M., Schankin, A., Wüstenberg, S., & Funke, J. (2011). Measuring performance in a complex problem solving task: Reliability and validity of the Tailorshop simulation. Journal of Individual Differences, 32, 225–233.

Dunbar, K. & Fugelsang, J. (2005). Scientific thinking and reasoning. In K. L. Holyoak & R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 705-725). New York, NY: Cambridge University Press.

Dörner, D. (1986). Diagnostik der operativen Intelligenz [Assessment of operative intelligence]. Diagnostica, 32 (4), 290-308.

Embretson, S. E. & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.

Engelmann, T., Tergan, S. O., & Hesse, F. W. (2010). Evoking knowledge and information awareness for enhancing computer-supported collaborative problem solving. The Journal of Experimental Education, 78, 268-290.

Fischer, A., Greiff, S., & Funke, J. (2012). The process of solving complex problems. Journal of Problem Solving, 4 (1), 19-42.

Frensch, P. A. & Funke, J. (Eds.). (1995). Complex problem solving: The European perspective. Hillsdale, NJ: Lawrence Erlbaum.

Frensch, P. & Buchner, A. (1999). Domain-generality versus domain-specificity: A binary concept and its impact on psychological theory and research. In R. J. Sternberg (Ed.), The nature of cognition (pp. 137-172). Cambridge, MA: MIT Press.

Funke, J. (2001). Dynamic systems as tools for analysing human judgment. Thinking & Reasoning, 7, 69-89.

Funke, J. (2003). Problemlösendes Denken [Problem Solving and Funke, J. (2010). Complex problem solving: A case for complex cognition? Cognitive Processing, 11, 133-142.

Greiff, S. (2012). Assessment and theory in Complex Problem Solving. A continuing contradiction? Journal of Educational and Developmental Psychology, 2 (1), 49-56.

Greiff, S. & Fischer, A. (2013). Der Nutzen einer Komplexen Problemlösekompetenz: Theoretische Überlegungen und empirische Befunde [Usefulness of Complex Problem Solving competency: Theoretical considerations and empirical results]. Zeitschrift für Pädagogische Psychologie, 27 (1), 1-13.

Greiff, S. & Funke, J. (2009). Measuring complex problem solving: The MicroDYN approach. In F. Scheuermann (Ed.), The Transition to computer-based assessment - Lessons learned from large-scale surveys and implications for testing (pp. 157-163). Luxembourg: Office for Official Publications of the European Communities.

Greiff, S. , Wüstenberg, S., & Funke, J. (2012). Dynamic Problem Solving: A new measurement perspective. Applied Psychological Measurement, 36 (3), 189- 213.

Griffin, P., McGaw, B., & Care, E. (2011). Assessment and teaching 21st century skills. Heidelberg: Springer.

Holyoak, K. J. (1985). The pragmatics of analogical transfer. In G. H. Bower (Ed.), The psychology of learning and motivation (pp. 59-87). New York, NY: Academic Press.

Jonassen, D. H. (Ed.) (2007). Learning to solve complex scientific problems. New York: Lawrence Erlbaum.

Klahr, D. & Dunbar, K. (1988). Dual space search during scientific reasoning. Cognitive Science, 12, 1-48.

Klahr, D. , Triona, L. M., & Williams, C. (2007). Hands on what? The relative effectiveness of physical versus virtual materials in an engineering desing project by middle school children. Journal of Research in Science Teaching, 44, 183–203.

Lee, C. D. & Smagorinsky, P. (Eds.) (2000). Vygotskian perspectives on literacy research: constructing meaning through collaborative inquiry. Cambridge: University Press.

Leutner, D., Klieme, E., Meyer, K., & Wirth, J. (2004). Problemlösen [Problem Solving]. In PISA-Konsortium Deutschland (Eds.), PISA 2003: Der Bildungsstand der Jugendlichen in Deutschland - Ergebnisse des zweiten internationalen Vergleichs (S. Leutner, D., Fleischer, J., Wirth, J., Greiff, S., & Funke, J. (2012). Analytische und dynamische Problemlösekompetenz im Lichte internationaler Schulleistungsvergleichsstudien [Analytical and dynamic problem solving from an international educational studies perspective]. Psychologische Rundschau, 63, 34-42.

Lovett, M. C. (2002). Problem solving. In D. Medin (Ed.), Stevens’ handbook of experimental psychology: Memory and cognitive processes (pp. 317-362). New York: Wiley.

Mayer, R. E. (2003). Learning and instruction. Upper Saddle River, NJ: Prentice Hall.

Mayer, R. E. & Wittrock, M. C. (2006). Problem solving. In P. A. Alexander & P. H. Winne (Eds.), Handbook of Educational Psychology (pp. 287-303). Mahwah, NJ: Lawrence Erlbaum.

McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86, 730-740.

Morgan, B., Salas, E., & Glickman, A. S. (1993). An analysis of team evaluation and maturation. Journal of General Psychology, 120 (3), 277-291.

Novick, L. R. & Bassok, M. (2005). Problem solving. In K. J. Holyoak & R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 321-349). New York, NY: Cambridge University Press.

Novick, L. R. , Hurley, S. M., & Francis, M. (1999). Evidence for abstract, schematic knowledge of three spatial diagram representation. Memory and Cognition, 27, 288-308.

O’Neil, H. F., Chuang, S., & Chung, G. K. W. K. (2003). Issues in the computer- based assessment of collaborative problem solving. Assessment in Education, 10, 361-373.

O’Neil, H. F., , Chung, G., & Brown, R. (1997). Use of networked simulations as a context to measure team competencies. In H. F. O’Neil (Ed.), Workforce readiness: competencies and assessment (pp. 411-452). Mahwah, NJ: Lawrence Erlbaum.

OECD (2003). The PISA 2003 assessment framework. Accessed at: http://www.oecd.org/edu/preschoolandschool/programmeforinternationalstu dentassessmentpisa/33694881.pdf.

OECD (2009). Chapter 1: Programme for International Student Assessment: An overivew. Accessed at: http://www.oecd.org/dataoecd/13/30/48578439.pdf.

OECD (2010). PISA 2012 problem solving framework. Paris: OECD.

Popper, K. R. (1999). All life is problem solving. London: Routledge.

Rupp, A. A., Nugent, R., & Nelson, B. (2012). Evidence-centered design for diagnostic assessment: evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4, 1-10.

Schulte-Mecklenbeck, M., Kühberger, A., & Ranyard, R. (Eds.) (2010). A handbook of process tracing methods for decision making. New York: Taylor & Francis.

Schweizer, F., Wüstenberg, S., & Greiff, S. (2013). Validity of the MicroDYN approach: Complex problem solving predicts school grades beyond working memory capacity. Learning and Individual Differences, 24, 42–52.

Sternberg, R. J. (1995). Expertise in complex problem solving: A comparison of alternative concepts. In P. A. Frensch & J. Funke (Eds.), Complex problem solving. The European perspective (pp. 295-321). Hillsdale, NJ: Lawrence Erlbaum.

Sugrue, B. (1995). A theory-based framework for assessing domain-specific problem-solving ability. Educational Measurement Issues and Practice, 14 (3), 29-36.

Wirth, J. & Klieme, E. (2003). Computer-based assessment of problem solving competence. Assessment in Education. Principles, Policy, & Practice, 10, 329–345.

Wittmann, W. W. & Süß, H.-M. (1999). Investigating the paths between working memory, intelligence, knowledge, and complex problem solving performances via Brunswik symmetry. In P. L. Ackerman, P. C. Kyllonen, & R. D. Roberts (Eds.), Learning and individual differences: Process, trait and content (pp. 77-108). Washington: American Psychological Association.

Wüstenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solving: More than reasoning? Intelligence, 40, 1-14.

Zoanetti, N. (2010). Interactive computer based assessment tasks: How problem solving process data can inform instruction. Australasian Journal of Educational Technology, 26, 585-606.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.