Dunne, T., Long, C., Craig, T.S., Venter, E.J. (2012). Meeting the requirements of both classroom-based and systemic assessment: The potential of Rasch measurement theory. Pythagoras 33(3), Art.#19

Please download to get full document.

View again

of 37
23 views
PDF
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Document Description
The challenges inherent in assessing mathematical proficiency depend on a number of factors, among which are an explicit view of what constitutes mathematical proficiency, an understanding of how children learn and the purpose and function of
Document Share
Document Transcript
  This is a pre-publication draft. The final version of this paper is published in Pythagoras 33(3), Art.#19. http://dx.doi.org/10.4102/pythagorasv33i3.19  Any citations should refer to that version. Meeting the requirements of both classroom-based and systemic assessment of mathematics proficiency: The potential of Rasch measurement theory Tim Dunne 1 , Caroline Long 2 , Tracy Craig 3 , Elsie Venter  4   1 Department of Statistical Sciences, University of Cape Town, South Africa 2 Centre for Evaluation and Assessment, University of Pretoria, South Africa 3 Academic Support Programme for Engineering in Cape Town, University of Cape Town, South Africa 4 Independent researcher Abstract The challenges inherent in assessing mathematical proficiency depend on a number of factors, among which are an explicit view of what constitutes mathematical proficiency, an understanding of how children learn and the purpose and function of teaching. All of these factors impact on an approach to assessment. In this article we distinguish between two broad types of assessment, classroom-based and systemic assessment. We argue that the process of assessment informed by Rasch measurement theory (RMT) can  potentially support the demands of both classroom-based and systemic assessment, particularly if a developmental approach to learning is adopted, and an underlying model of developing mathematical proficiency is explicit in the assessment instruments and their supporting material. An example of a mathematics instrument and its analysis which illustrates this approach, is  presented. We note that the role of assessment in the 21 st  century is potentially powerful. This influential role can only be justified if the assessments are of high quality and can be selected to match suitable moments in learning progress and the teaching process. Users of assessment data must have sufficient knowledge and insight to interpret the resulting numbers validly, and have sufficient discernment to make considered educational inferences from the data for teaching and learning responses. Introduction Assessing mathematical proficiency is a complex task. The challenges inherent in this process depend on a number of factors; a view regarding what constitutes mathematical proficiency, an understanding of how children learn and the approach adopted as to the purpose and function of teaching. The particular attitude brought to the assessment of mathematical proficiency will depend in part on these factors. However, besides these central questions in mathematics  This is a pre-publication draft. The final version of this paper is published in Pythagoras 33(3), Art.#19. http://dx.doi.org/10.4102/pythagorasv33i3.19  Any citations should refer to that version. education, there are important questions to consider about the relationship between classroom- based assessment and systemic assessment types. While there is potential for positive information exchange between these two types of assessment, more often there is an unnecessary conflict or simply lack of constructive communication. Classroom teachers are at times perplexed at the outcomes of systemic assessment, in other cases confused about what actions to take as a result of the reported outcomes, and in the worst case scenario, demoralised. The potential for positive information exchange demands that questions about quality at both classroom and systemic sites are addressed (see also Wyatt-Smith & Gunn, 2009, p. 83). In this article, we differentiate explicitly between the two broad types of assessment represented  by classroom-based and systemic or external assessment 1 . The rationale for assessing, the demands of the stakeholders, the forms of the assessment instruments and the types of data  produced differ, or can differ, substantially. Having briefly explored the differences between these two types, we discuss the broad distinction between two different approaches to learning and teaching, an approach that may be termed a developmental approach  and an approach that may be termed a deficit approach  (Griffin, 2009). The particular approach adopted within a context will inevitably impact on the purposes associated with assessment. If systemic assessment is to be useful within the classroom the results need to be interpreted by teachers and found applicable to the classroom context. Underlying this requirement of applicability is the presence of a model of developing mathematical proficiency that includes  both a plausible conceptual development (from the mathematical perspective) and cognitive development (from the learner perspective). A model such as envisaged here should be somewhat loosely configured and attend to common ground so that it does not exclude different approaches to mathematical teaching and learning (see Usiskin, 2007). This degree of coherence, from broad consensus towards a developmental model, to a working curriculum document that outlines the  broad ideas, to a more specified curriculum at school level and a school programme providing more detail 2 , is at present a legitimate dream to work towards. Also envisaged in the dream is the idea that  professional development, accountability testing and  formative classroom experience  are integrated around core aspects (see Bennett & Gitomer, 2009). The theoretical insights informing a developmental model and the elaborated assessment programme are not the 1  For detailed descriptions of assessment types and a coherent framework see Black (1998). 2  See Thijs & Van den Akker (2009), for descriptions of curricula at the macro, meso and micro levels.  This is a pre-publication draft. The final version of this paper is published in Pythagoras 33(3), Art.#19. http://dx.doi.org/10.4102/pythagorasv33i3.19  Any citations should refer to that version. immediate concern of this paper. We propose merely to show how applying Rasch measurement theory (RMT) may support such a project. An essential part of that support is the facility of the Rasch model to yield measurement-like differences and changes. These quantities can enrich the evidence accessible from classroom- based assessment and satisfy the expectations of external stakeholders, in particular if one takes a developmental approach to learning (Griffin, 2009; Van Wyk & Andrich, 2006). An example is presented which illustrates the intervention potential of an assessment programme which adheres to RMT and within which the Rasch model is applied. We advocate that this model should be seriously considered for inclusion in the approach to national systemic and external assessment programmes, in particular for mathematics. In essence we explore the question, ‘ What model of assessment may support teaching and learning in the classroom, and in addition enable broad-based monitoring of learning progression within districts and provinces? ’  Reciprocally, ‘ How may systemic assessments serve not only their intrinsic purposes to inform decision-makers about performance levels in broad strokes, but simultaneously inform and enrich teaching and learning within the variety of classroom level challenges into which these single instruments intrude? ’   Classroom and systemic assessment The important distinctions between and commonalities across classroom-based assessment and systemic assessment types are discussed below. In addition the complexities involved in reporting results at an individual level and monitoring change over time are noted. Classroom-based assessment The teacher in the classroom is concerned with the learning processes and development of the learners in her class. Successful assessment is often of a formative nature and can emerge as continuous assessment, hence directing learning and teaching while the summative component, recording marks for the purpose of reporting, also plays a role. 3  The rationale for a teacher to run assessment exercises is to determine whole-class and, particularly, individual levels of current development, to diagnose current obstacles to learning progress, and to provide subsequent targeted scaffolding to appropriate classroom segments. In the best scenario, the forms of 3  We consider the terms formative and summative assessment not as discrete entities, but as depicting points on a continuum. Assessment moments may have elements of both kinds.  This is a pre-publication draft. The final version of this paper is published in Pythagoras 33(3), Art.#19. http://dx.doi.org/10.4102/pythagorasv33i3.19  Any citations should refer to that version. evidence used in classroom assessment may vary from projects requiring extended planning, to quick quizzes. Such variety embraces different learning styles and different facets of mathematical proficiency and adheres to cognitive science principles (Bennett & Gitomer, 2009,  p. 49). The stakeholders in classroom-based assessment are the teacher and the learners. The data sets  produced by the classroom assessment exercises are not necessarily designed to be expressly meaningful to anyone outside the classroom, though inevitably and importantly teachers within a school community may share ideas and discuss assessments and their results. The particularity and the immediacy of a test or assessment give it currency in the classroom context and for the classroom processes, at a specific period in time. We may note that in any classroom test or assessment, the teacher is generally concerned with a current spectrum of learner skills and needs in the class, which invariably differs from the spectrum that confronts the educational decision-maker at district or provincial level. The learners in a particular class may have test performances that are on average well above or well  below the average performance associated with all learners of the corresponding grade in an entire school district or province, using the same or an equivalent test. Moreover, the variation of individual test performances within any particular classroom will generally be substantially less than the overall variation in performances on the same instrument across the school district or  province. Systemic assessment While classroom assessment is generally fine-grained and topic specific, systemic external assessment is generally broadly banded, and attempts to ‘cover the curriculum’. From the  perspective of the education departments, and in some cases other stakeholders such as funders of  programmes, major purposes of systemic assessment are to assess the current performance and variability within a particular cohort of learners, according to some sort of external benchmark of desired proficiency, and to monitor progress, also according to some external standards for changes and performance improvements over time. Overall averages (or percentage scores) and the associated pass rates (learner percentages at or above a specified pass criterion) may be deemed particular elements of interest, but their meaningfulness nonetheless has to be argued and established in a suitable robust exposition. These outcomes should be interpreted in relation to other assessment types, for example classroom-based assessment (see Andrich, 2009).  This is a pre-publication draft. The final version of this paper is published in Pythagoras 33(3), Art.#19. http://dx.doi.org/10.4102/pythagorasv33i3.19  Any citations should refer to that version. For systemic and external assessments, the sheer extent of the testing programmes, and the development time period and financial constraints, may constrain resources and available turn-around time for testing, scoring and data capture. Hence systemic test designers may be obliged to limit the type of items to multiple choice or short-answer responses, and to limit testing time to (say) a single period of a school day, and in consequence limit the maximum number of items that can reasonably be attempted. In the ideal situation any systemic assessment is designed to produce, from a single short dipstick event, performance data which is meaningful to stakeholders, district officials and state educational bodies, about the current educational health of systems. This body of data and its interpretation may result in decisions requiring or offering interventions and perhaps some other monitoring functions. The Department of Basic Education 2009 review claims that ‘ externally set assessments are the only credible method   of ‘ demonstrating either to parents or the state whether learning is happening or not, or to what extent ’  ((Dada et al., 2009, p. 36, citing Chisholm et al., 2000, p. 48, emphasis added). We contest that claim and, with Andrich (2009), maintain the view that the results of external assessment must rather be considered in conjunction with classroom assessment, than alone. In fact one may argue that to invoke only external test results to convince stake-holders whether or not learning is happening at the level of individual or class, and even  perhaps of a grade in a school, amounts to dereliction of duty and dangerous unethical practice. The argument is unethical when it does not sufficiently address the complex issues of causation which lurk within the extensive variation of student performance on the test. Similar critique of inordinate emphasis on systemic tests, offered by Bennett and Gitomer (2009), rests primarily on two counts, firstly that systemic testing has unintended detrimental consequences for the teachers, learners, administrators and policy makers, and secondly that the type of assessment generally offers limited educational value, as the assessment instrument is usually comprised largely of multiple choice or short answer questions (p. 45). A systemic assessment may in its totality give a valid overview of system-wide performance on the test instrument (through its constituent items) for the part of the subject and grade curriculum or domain which actually appears within a finite test. Possibly, by astute design and professional concurrence, the test may satisfy further criteria, so as to be viewed as a valid assessment of the
Search Related
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks