NO Tests But For Learning: The Provocation Proceeds

The proceedings of  this blog began back in September with an invocation of the metaphor of exorcism. Some might have found that strange, but the comparison seemed apt to this old altar boy because impressions and ideas, objections and observations, frustrations and fancying associated with the world of testing do afflict me like distracting spirits. Every single day. From 2001 until two years ago today, the nettling of notions that ranged from the validation of score meaning to stochastic learning models to evidence centered design to English language learning and on and on and on came with the job. A really cool job. I was one of the luckiest people to have been named the Chief Learning Officer at ETS especially because my background was not in the business at which that organization is the best in the world — or at least it was the best when I was there — educational measurement. (No, I’m not suggesting that ETS was the best because of my presence; it had been the gold standard of testing and assessment for over half a century before I drove onto its campus in Lawrence Township New Jersey. PS They want you to think it’s in Princeton, but it’s not.)

And because it was my job and because fear of failure motivates me magnificently, immersion in this world became my modus operandi. If there was such a valid concept as learning styles — and trust me there is not as this paper demonstrates — my style would be characterized as reading/writing to obtain mastery of various domains. Early in the morning and late into the night, I attacked books, papers, articles on testing, education, and psychometrics. Then in the curious serendipity enabled by our nervous systems my eyes and ears could not help but latch onto every instance in which something related to testing or education was mentioned.

The Internet fed this obsession and then not long after I got the CLO job the ETS Library became part of my management portfolio. Anyone reading this who like myself was a library rat as a kid will appreciate that I was in the position that Borges once described, “‘I have always imagined that Paradise will be a kind of library.”

Digression Alert

This library that occupied a small building resembling a pie box on the ETS campus not only held historic wonders associated with the very beginning of educational testing but also had the most extraordinary crackerjack staff who could get you literally any book you desired via the glorious interlibrary loan system. In those days, the place was known as the Carl Brigham Library to honor one of the pioneers of educational measurement who is often credited with creating the early structure of what became the SAT. But that naming came to be thought of as a little bit embarrassing because Brigham at one point in his career was both a eugenicist and a racist. Yes, lots of scientists of early 20^th Century America earned both of those merit badges and Brigham later abandoned and disassociated himself from those early positions, but probably not a guy after whom to name your library. That inharmoniousness was noticed by the extraordinary leader of the Library, Karen McQuillen. Our recommendations to change the name — yes, I did hope that at some point the library might get named after somebody really cool like Zenodotus or Golda Meir who started her career as a librarian or Giacomo Casanova who later in life worked as librarian in Bohemia or even Jorge Luis Borges. They mocked our protests and said that Brigham had repudiated those earlier statements, which was true. Then author Daniel Okrent asked to use the library, which was a privilege open to any scholar or writer to research his book “The Guarded Gate: Bigotry, Eugenics And The Law That Kept Two Generations Of Jews, Italians And Other European Immigrants Out Of America.”  In a weird coincidence, he was going to write about Carl Brigham and came to do his research at ETS in… The Carl Brigham library. Very quickly when we tipped those same powers that be of this imminent visit, the library name was changed not to any of my choices but to “The ETS Library”. Subsequently, those same highly creative powers that be at ETS took to treating this treasure like a piece of old exercise equipment banishing it to a corner, making it harder for people to use it, and cutting its budget. Have I mentioned that ETS is a research organization?

Now leaving digression

When the Library was going strong and I was new to ETS I read pretty much everything I could there. But there is a problem with reading pretty much everything you could about something: your mind doesn’t know how to stop. You automatically, reflexively scan the horizon for anything to do with testing like a cheetah looking to lunch on an impala. And then 18 years later, you quit the job and you’re not getting paid to do anything with testing and you’d like to move into your old job as a playwright but your mind keeps reading everything you can about that subject, which in this case was testing.

So to my dear friend Joe C. who noted in a comment on FB that exorcism might not be necessary in my case, I must brook my respectful disagreement. Call it what you will    halsing   (c870),     conjurement   (c1315),     exorcision   (1502),     dispossession   (1600),

Or     undevilling   (a1652), I’ve got to get the sprights of testing out of my life.

But I’m also tormented by something else that went on during those 18 years: I failed at one of the most important aspect of my responsibilities: to facilitate innovation. It was not for lack of trying; there were many product concepts and methods that came out of collaborations that I joined or incited. But they didn’t change the dominant method that ticks off so many people: summative testing. And that’s why I’m giving it one last shot here by trying to provoke the overall consciousness with the outrageous premise that there should be no tests but for learning. Not a single one. I am an absolutist in this regard.

One of the most brilliant people that I met at ETS was Malcolm Bauer and he’s also really nice guy because he wrote the following comment that points out an inconsistency in yesterday’s post:

Interesting position and argument. Always great to read your reflections, T.J. While I agree shifting the amounts of formative and summarize testing might be helpful, you undercut your argument for “no summative” by citing summative NAEP scores to support it when you say “Average reading and mathematics scores for the nation’s 9- and 13-year-olds have remained flat or declined since 2012.” Summative assessment has many purposes including informing policy decisions and knowing if major educational changes are having positive or negative effects at district, state, and national levels. I wonder if your argument is intended to focus on the questionable value of summative assessments when used at the individual level? And to reduce summative assessment to places where there is a strong value proposition (like the NAEP scores you use in your argument) with few negative consequences?

First, Malcolm’s right. NAEP is not a test of individual students. Second, Malcolm is right again in his guess that my argument is “intended to focus on the questionable value of summative assessments when used at the individual level.” I’m advocating that formative assessments be used exclusively in K-12. NAEP should go ahead and continue their group assessments because they are pretty much through our federal government doing the most remarkable things with testing. I’m also not too worried about policy decisions because I don’t see that policy has done as much good in the last forty years. So, if we take a policy break I don’t think anyone will notice. (Not talking about policies on banning books here; that’s a wacky idea for somebody else’s blog) What I’m suggesting is simple: everyone should rise up and tell their elected representatives that they only want tests that are cognitively based assessments of, by, and for learning. One of the most exciting days for me at ETS came out in some hotel on one your Princeton. I was facilitating a meeting of some of the younger leaders of the company and Ida Lawrence and Randy Bennett two of the stars of R&D started to talk about assessments that would aid learning. This grew into CBAL. And tomorrow is all about that story.