Questioning Systems of Deformative Assessment

I argue that state maintained schools within England be brave, reject GCSE’s and their deformative effect upon learning and adopt the IB MYC as a summative assessment framework which provides a more effective means of providing an education for liberation.

Questions and themes relating to low and high stakes assessment practices and reforms are addressed throughout the chapters of Berry, R., & Adamson, B., Eds., (2011) Assessment Reform in Education, Dordrecht, Netherlands: Springer. Featured authors such as my colleague Eleanore Hargreaves (UCL IOE) and Mary James (University of Cambridge) address, respectively, issues concerning the role of teacher feedback within the assessment and learning process and the manner in which assessment is interpreted and applied with variance throughout the UK (with Scotland a lantern in the modern gloom of high stakes assessment).
The most striking article is that presented in Chapter 11 (pp.155-165) through which David Scott (UCL IOE) addresses the topic, Assessment Reform: High-Stakes testing and Knowing the Contents of Other Minds.

David uses Foucault (1979) as a means to enter into an exposition of the false beliefs and dualisms at the heart of high stake assessment beliefs and practices. David uses the oft impenetrable philosophy of Critical Realism (of Roy Baskar and Margaret Archer) to explore assessment thus enabling overlooked aspects such as causality ‘backwash’.

David recognises the existence of two forms of knowledge; Ka and Kb. ‘Ka represents those knowledge sets, skills, and dispositional states of  a person, collectively known as capacities.’ and ‘Kb represents those knowledge sets, skills and dispositional states which allow this person to do well in tests, and, in particular, high-stakes tests.’. A separation of knowledge enables David to explore how assessment favours, incorrectly, one (Kb) over the other and how the resultant assessment systems feedback to shape deformatively Ka knowledge.

If an education system introduces high-stakes testing, that is, testing in which there are significant rewards attached to success in the test both for the individual and the institution in which she works, then there are two consequences. The first is that Kb becomes the dominant form of knowledge in the curriculum and the second is that Ka over time is transformed so that it becomes more like Kb, that is, is has more of its characteristics. (p.155)

Furthermore, conflating these different forms of knowledge into one is common practice, evidenced by approaches taken globally to high-stakes testing (see PISA) and evident within the foundation of false beliefs of which many assessments are built upon.

Testers commonly conflate Ka and Kb, and in doing so make a number of false assumptions about knowledge and its assessment, with the consequence that these two forms of knowledge become indistinguishable in the minds of policy-makers, educational practitioners, students and other stakeholder. (ibid)

David identifies 8 false beliefs at the heart of high-stakes assessment. However I feel that these beliefs also rest within day-to-day assessment practices and as such impact upon attempts to use assessment to both measure and activate effective learning (for example widely applied AfL practices).

False Beliefs 

  1. A person has a knowledge, skill or dispositional set, which is configured in a particular way (a grammar), and it is this configuration, perceived to be uniform across peoples, which is directly assessed when someone is tested. But the presented and observable answer is only the tip of a knowledge iceberg which itself could only be identified through logical inference and or retroductive analysis. 
  2. This grammar is organised into elements, there are relations between those elements, and each element can be scaled, so a person may have more or less of that element, which can then be investigated/tested. But when the assessment only looks for a pre-determined element is fails to recognise that a number of elements may be called upon and enacted thus the assessment fails to recognise a broader and nuanced Ka knowledge. 
  3. In the use of a knowledge-set, or in the performance of a skill, or in the application of a disposition within an assessment context, no internal transformation takes place.
  4. Testing a person’s knowledge, skills and aptitudes has no wash back effects on either Ka, the original knowledge construct, or Kb, the internally transformed knowledge set ready for testing. But Ka and Kb are both activated and both influence and shape each other in a transformative loop. Completing an assessment of Kb requires Ka which itself is transformed by interacting externally and internally with Kb (a process which can be aligned with thinking around Assessment as Learning). Micro (person level) and Macro (system level) ‘washback effects’ are well documented and observable. How often has teaching changed or the curriculum altered in response to a change in or the results of an assessment regime? Such an effect can be deformative altering an approach which then subverts Ka knowledge development. 
  5. The process of testing works in a unidirectional linear fashion. But washback and an individuals place within a society recognises bidirectionality of all and every interaction. Thus a test of X at a single moment in time can not accurately record ones capacity for X from that point forward. 
  6. Different types of knowledge can be tested using the same algorithmic process. Where do I begin with critiquing this state of affairs? 
  7. The performance on the test represents to a greater or lesser extent what the testee can do or show, rather than there being a qualitative difference between the performance on the test and the construct, skill, or disposition of the testee. Capacity in terms of an original construct can not be assessed when the testee must alter that construct to conform to the requirements of the set test. reframing to fit the test rather than actually being tested on ones true capacity. Thus an assessment must recognise and reflect the individuals constructed knowledge and to do so over time. Real world learning and authentic assessment strategies seem wholly relevant here. 
  8. A test can be constructed which is culture-free or free of those issues which disadvantage some types of learners at the expense of others. But of course all tests are shaped by and informed by a prevalent culture. Assessment systems do not exist within a closed system. The very fact that all summative tests in England are written and are written in English reflects a culture and by doing so disadvantages those why may possess, for example, a well formed mathematical capacity but not one for the complexities of exam paper English. In addition chosen references presented within exams may deviate significantly to the reference frames used by individuals to construct their Ka knowledge. 

False belief 4 is extremely powerful as a causal mechanism shaping our world and can be linked back to both Foucault and Freire’s writings. It also enhances my own views about the purpose of education for Education and the goal of enabling a state of Liberated Learning.

Dualism 

Cognitive psychologists and test constructors implicitly adopt a computational or symbol-processing view of mind. Learning…,and the assessment of this learning…, are understood as inputting coded unambiguous information about the world, which is then sorted, stored, retrieved and managed in the same way that a computer processes data… [this is] an impoverished view of the role interpretation plays in learning. (p.160)

David highlights that the application of the above view to testing, assessment and its inevitable washback upon teaching and curriculum fosters a behaviourist approach to learning. In addition the viewpoint suggests a number of dualisms separating out language from reality, mind from body and the individual from society. The dualism between mind and body is of particular interest. Such a separation locates learning and cognition solely in the mind, with this mind physically separate from body, and by default separate from the social world around that body. Such a view sees learning as a passive process and as such assessment as a passive process of isolating and knowing the contents of anothers’ mind. But

Knowledge is not understood as a passive body of facts about the environment but as an interactive process of reconstructing meanings.

Thus assessments should enable interaction and reconstruction of knowledge rather than passive regurgitation. Again authentic assessment is that which is real world, problem passed and interactive assessment.

So where does this leave us? For me, what David offers is a means of looking again at assessment both in their summative and formative forms. What it also highlights is the deformative power of assessment systems and how they, built largely on incorrect assumptions, can have a  ‘washback effect’ upon the structures of education, curriculum, pedagogy and ultimately on a learners experience of their own Education.

The reality is that students must adhere to systems of high-stakes summative assessments. Students sit exams at (arguably indiscriminate) points in their lives and we as informed educators and school designers can make a choice about what form this summative assessment takes. In England when students, aged 16, sit what is possibly the most significant of the high-stakes assessments, a ‘choice’ of sorts exists. Students can sit the iGCSE, BTEC and GCSE forms of assessment. The vast majority sit the GCSE, an assessment regime which is in perpetual flux of design, redesign, tweeks, changes and of questionable validity. Many schools are slaves to the GCSE, as measures of school effectiveness (Ebacc, Progress 8 et.,al.) are determined by student and school performance within this mode of assessment. Thus the tail wags the dog.

Innovative approaches to teaching, learning and curriculum design are needed and wanted by many within the system but while a ‘choice’ is made for us and a certain type of assessment imposed then the Kb knowledge needed to do well in this form of assessment, due to its weighted significance, will shape the Ka knowledge we are able to activate in the five years preceding this high-stakes performance. But we have a choice. Some high-stakes and summative assessment systems seem to have considered many of David’s observations. The International Baccalaureate in both its Middle Years Curriculum (assessed age 16/17) and the Diploma (assessed 18/19) frameworks offers an alternative assessment model which recognises an individuals Ka development over time and recognises the need to assess applied knowledge within real world contexts; authentic assessment. If such a system of assessment were to be adopted the ends would enable the means to be created empowering schools, teachers and learners to focus on Ka knowledge sets, skills, and dispositional states without the need for Kb.

The IB has been traditionally the preserve of private and international schools with some state schools replacing A Level assessments with the Diploma. I am suggesting that state maintained schools within England be brave, reject GCSE’s and their deformative effect upon learning and adopt the IB MYC as a summative assessment framework which provides a more effective means of providing an education for liberation. The IB is not perfect but what assessment framework is? Until we have greater autonomy, as is present in Scandinavian countries for example, to design and administer our own end of phase assessments, then we must look for a system which is more in line with effective learning and in line with the world we wish to shape today and tomorrow. Drawing my own thinking together with the points raised by David, I feel as a teacher, researcher and schools leader that the IB MYC and Diploma offer the best alternative to the present state of our deformative assessment regime.

Thoughts welcome and criticism gratefully received. rgratton.cgl@gmail.com

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s