The failure to reconcile social learning with competitive testing schemes

Hard to believe that seven years have passed since Alina Von Davier and I with the expert assistance of Sue Borchardt created this brief animated video on  collaborative assessment as part of the Pulling to the Edge series to accelerate innovation in educational measurement. Alina offers some glittering insights in this short film such as “We (educational measurement scientists) measure very very well what we know how to measure.” Of course, many instructors and teachers don’t measure very well at all as our stories point out. When it comes to the 4Cs that experts worldwide deemed critical for success in this century — critical thinking, creativity, communication, and collaboration – assessment lags behind the exploration of those concepts. Alina is at the forefront of efforts in “redesigning educational assessments and not merely improving the assessments we have. [making] assessments that reflect the way people actually teach, learn, and work.” 

My ETS Chief Learning Officer area also produced videos explaining game based assessments (with Andreas Oranje) and cognitive based assessment of, by, and for learning (with Randy Bennett and Paul Deane). More on these topics in later posts this January, but I want to follow up on the riff from yesterday about social learning and why our testing should pay more attention to it. First, I highly recommend watching the video above as Alina does a super job of explaining how one type of assessment in this realm might work. But what effect arises from the crossed wires between learning that benefits collaboration and testing that obstructs it? 

If as Howard Mannella suggested in our day two post that some types of testing can discourage teamwork, then we might suppose that they also discourage social learning. I mentioned Vygotsky as one of my inspirations in Graduate School. Another source of new ideas for me that I ended up using throughout my career was Albert Bandura and his social learning theory. We can learn by observation that encompasses more than “watching another person to engage in an activity. Hearing verbal instructions, such as listening to a podcast, can lead to learning. We can also learn by reading, hearing, or watching the actions of characters in books and films.” 

As Ceceilia Heyes has written, “Humans’ thinking machines are built in childhood through social interaction, and were fashioned by cultural, not genetic, evolution. What makes our minds unique are not cognitive instincts but cognitive gadgets.” But if this is an important way in which we learn from the very start, then why are most tests that we endure in our academic and professional lives concerned only with individual accomplishment? Where I’m going with all this through these 31 days of posts is to look where testing could fulfill its very necessary function of allowing people to make claims about themselves and to assist their learning and development so they can make further claims about themselves.  

We are not going to get there by banning testing. However, I fear that we are all also not going to get there by simply accepting the status quo. One description of assessment of social learning skews too heavily towards self report: “The assessment of social learning is done through involving the learners and asking them to make statement about their perceived level of self-efficacy and expectations and associated behaviours. The learners’ statements of self-efficacy are thus considered as predictive of their levels of performance. For assessment, comparisons are then made of the efficacy expectations that are obtained before and after intervention in order to detect whether there has been increases or decreases in learners’ self-efficacy.” 

But an OECD report uncovered results for such assessment that ranged from mixed to “disappointing”. This conforms with the caution offered by Margaret Gredler in her classic textbook (another return to my Graduate School days) Learning and Instruction: Theory into Practice. Gredler wrote that “the measurement of any unobservable construct, such as motivation, self efficacy, or self regulation, poses problems. Such constructs are not directly observable, and their presence must be inferred from behaviors that have been identified to occur only if individuals possess the particular construct.” We want to make a claim as to what a student knows at a particular time, but the reasons for that desire differ. 

The institution wants a score to conform with various regulations and requirements. The individual wants to know what to learn next. That they learn in part by being around others and through that interaction gaining self efficacy is often unrecognized in their testing. The two realities are at cross purposes when it comes to testing. I will post tomorrow more about some of those realities as I observed them in the sausage factory of testing that was ETS and elsewhere. But let me close by returning to Heyes whose work fascinates me. (This video offers a nice summary of her thought.) She wrote of her fellow humans in her book Cognitive Gadgets: 

“No other animals have so completely transformed their environment, become so dependent on cooperation for survival, and constructed, along the way, the vast edifices of knowledge and skill in which all human lives are embedded: technology, agriculture, science, religion, law, politics, trade, history, art, literature, music, and sports. Why? What is it about the human mind that enables us to live such unusual lives, and why do our minds work that way?” 

If we accept Heyes’ judgment about the spectacular nature of our minds, then why do we accept tests of the properties of those minds, of the abilities and skills held by the people with those minds, that are beyond flawed even dangerous in their improper construction end unwarranted utilization. Not only for ourselves but for everyone else, we should want a personal history of testing that serves each individual. The failure to reconcile social learning with competitive testing schemes, the value of collaborative problems-solving with the exigencies of traditional instruction, is just one dimension of that failure.