Are Problems With Tests Really Problems With Authority?

An unexpected telephone conversation this morning, on the 27^th day of for this blog, exposed me to a loved one who trusts what Joe Rogan and his guests say about the pandemic. In other words, that person believes that those voices speak with authority. To do so requires a corresponding belief that the so-called official sources now hold a diminished or completely devastated authority. When people disagree, authority is a zero-sum game.

What does authority have to do with testing? Educational measurement depends now as it always has upon some authority. Who would trust the claim of the test whether applied to ourselves or to someone else if we did not explicitly or implicitly invest authority in the organization administering the exam? Few could deny that culturally our relationship with authority in education differs from what it was a century ago, even twenty years ago. My interest in the subject of authority and how particularly Americans connect to the concept ranges back years. (I even wrote a comedy about it, Keeping Right, that played on Zoom with a fantastic cast in December 2020.) Some people who know me very well — like my wife, Marjorie — have wondered aloud as to whether my interest isn’t an offshoot of my own problems with authority. In a rare bout of wisdom, I have never disagreed with this conclusion because some people’s authority must be obeyed.

Until fairly recently particularly in Europe as Adrian Wooldridge points out in his book The Aristocracy Of Talent a “hierarchical view of the world was the dominant view in Europe… It also found striking echoes in other premodern societies such as India its cast system of Brahmins and untouchables and Japan with its rigid hierarchy.” Wooldridge continues his description of how authority worked in the premodern world by noting that its inhabitants conceived of their milieu “as a hierarchy of social groups — states, orders or core — that were ordained by God and defined by the relationship to two great verities: their social function and disease those who pray, those who fought and those who worked) and their position in a hierarchy of status that stretch downward from the heavens (the word etat is derived from the Latin for status.” Those were the days when simply claiming that your right to rule came from God was sufficient to get people to do what you said. Of course, I’m pretty sure that swords and lances and battle axes also got involved in those transactions.

Our word in the English-language, authority, derives from the Latin ‘auctoritas’. Jesse Sifuentes notes an important distinction between that kind of power and other types of power in classical Rome: “Authority in ancient Rome was complex, and as one can expect from Rome, full of tradition, myth, and awareness of their own storied history. Perhaps the ultimate authority was imperium, the power to command the Roman army. Potestas was legal power belonging to the various roles of political offices. There was also auctoritas, a kind of intangible social authority tied to reputation and status.” That last meaning relates directly to the kind of authority necessary to impose successfully an examination system upon a population. The intangibility other and its association with reputation and status remains true of the authority associated with pretty much any consequential test that we take from elementary school placement exams to college admission exams to professional certification and licensure: there is an authority behind all of them that we accept. Or we accepted until fairly recently.

Wooldridge in his book of meritocracy argues that Francis Bacon, the 16^th century English philosopher, was wrong in positing that “China was responsible for four of the world’s great inventions — gunpowder, paper, the compass and the printing press.” Wooldridge argues that these inventions were marginal. He maintains with substantial evidence that “China’s most important innovation was the mass examination system, the oldest and longest lived in the world and until the 20^th century, by far the most ambitious.” There can be little quarrel that it is the oldest system as based upon the books of Confucius that appeared in the sixth and fifth century BC and implemented by the Sui dynasty in the seventh century A.D. they controlled entry to the civil service.

Two sources of authority stood behind those exams: the greatly revered Confucian doctrine and the very specific power of figures like the Empress Wu who according to Wooldridge “turns the (Chinese Royal) Court upside down by reserving the top offices for people past the examination.” Wooldridge is quick to point out that this exam system actually looked a lot like the results of our own college admission exam system: “more than 1/3 of tang error examination candidates, and more than half the people who eventually bagged the biggest offices, came from the 10 most prominent families.” Why? Because materials like paper and brushes to take the exam cost money but perhaps more importantly a potential examinee had to have copious free time, often ranging up to two or more years, to study for the exam. In a predominantly agricultural society, free time is a scarce if not almost nonexistent commodity. Is it too much to draw a parallel to the exam systems of today where advantages of wealth and leisure affect test scores?

As other exam systems rose up in Europe hundreds of years later, the same dynamic pertained; a particular power whether Royal or civil held out the prize of admission employment and used certain texts as the foundation of the construct of their exams. Frederick the great subjected his nobles “to a regime of drills and examinations, some of which he administered himself…” As requirements for them holding or retaining certain positions. There was always assertive back-and-forth in these innovations as monarchs changed and nepotism regained its primacy in the awarding of jobs and privileges. Napoleon “pushed French education in an even more meritocratic direction, defined by examinations and selection.” In the British Empire, Thomas Babington Macaulay used the domination of the Indian subcontinent as a giant laboratory for education and exams. Importantly (and imperialistically), he insisted on “distinguishing between ‘ability’ and ‘mere learning’. The job of a well-designed examination, he said, was to ‘test the candidates powers of mind’ rather than just ‘ascertain the extent of his metaphysical reading’”. But with the British military behind him, Macaulay could stand as the authority for what appeared on the exams. Astonishingly to educated people today, not only were the exams never in the language of the Indian people such as Sanskrit but Macaulay himself never bothered to learn a word of Sanskrit during his years as perhaps the preeminent authority in India.

This history of the relationship between authority and testing in part to dispel any belief that authority’s nature need always possess a tinge of rationality or a veneer of humanity. Just because some person or institution issues a warrant for a test doesn’t mean that assertion is grounded in solid educational thinking and psychometric science. Of course, during my time at ETS, those who were establishing or checking the warrant for a test that came out of some sort of authority insisted that it be the rationale or generalization for grounding the claim that the test score will make in the available data provided by the test-taker. They operated under the premise that warrant establishes “the credibility, relevance and strength of the evidence in relation to the target conclusions.”

But surveying the history of educational testing in the United States over the last two decades particularly, attacks upon the authority of various institutions happened not only frequently but effectively in that questioning and denigration of the results of the tests dented their acceptance. Not all tests. My obsessive reading of any article associated with examinations has not revealed a pilot complaining that the FAA denied them the right to get behind the controls of a 737. And while there are libertarians who say that government requires too many certifications for jobs such as plumber, massage therapist, financial advisor, project manager, or dog groomer, the acceptance of that authority by the test-takers and indeed even the general public has remained fairly high.

Yet when we enter the educational domain, authority previously established from state educational departments to the College Board to ACT to GMAC, the story differs significantly. I recall being in several meetings associated with the Common Core Standards and Race to the Top especially during the Obama administration when consortia of state education officials created the standards that would inform both the SmarterBalanced and PARCC tests that would be used in K-12 around the USA. ETS was one of many organizations that consulted with the steering committees for these two consortia. However, the authority for all decisions on the tests rested with state education officials. Curiously, when an alliance of parents protested against these proposed tests – in other words, resisted the authority behind the tests — they did so citing mistakenly that the standards came from the federal government. The uproar of PTA mothers and fathers was such that most city education officials chose pusillanimously not to take responsibility for their own work. The consortia that had the potential to improve education significantly especially in those states with below average learning results for their students largely fell apart as a result of this resistance. In my interactions with their plans and even in some cases with the actual people at those state departments, they seemed amazingly ignorant of the enormity of the change proposed, the resistance that should’ve been expected, and the methodology that does exist to help successfully introduce change in communities. Clueless is a word that comes to mind.

The Feds don’t get a pass on their behavior in this scene. They held the purse strings for the initiative, which is always a sign of authority, and they set up the framework by which decisions would be blessed by the United States Department of Education. The then Secretary of that cabinet level department, Arne Duncan, in a breathtaking misjudgment decided to ‘help’ the effort by telling “a group of superintendents that ‘white suburban moms’ were resisting the implementation of the Common Core. His theory? ‘All of a sudden … their child isn’t as brilliant as they thought they were, and their school isn’t quite as good as they thought they were.’” Insulting your fellow citizens – especially moms — does not appear on any list of tips for facilitating change that I’ve ever seen.

We’ve already covered in a previous post the extraordinary and often inaccurate challenges to the authority of organizations like College Board and ACT; people can keep on saying that those test scores do not predict college success, but that doesn’t erase the pile of studies that indicate their validity and reliability. Critics can carp about advantages for those students whose parents will purchase coaching and other affordances, but the reality is that the differences gained remain minimal.

If we don’t accept authority or if we insist that our own intuitions and research constitute an equal or better authority then changing the way testing happens so that every test administered in an educational domain is a formidable one, one that tells the student how they can improve their mastery of the particular subject, will prove very difficult if not impossible. If Joe Rogan or one of his guests is the authority on Covid-19 than in that zero-sum game the CDC and Doctor Anthony Fauci cannot be the authority. This tendency to see conspiracies in any government intervention has already spilled over into the educational realm. (This article is particularly good the phenomenon.) I hesitate to condemn entirely this suspicion of the authority in education and testing because in some cases those doubts usefully raise questions as to the insularity of expertise, which can also make it difficult for us to affect change. That would be the subject of tomorrow’s post as in these final days of our January Jolts we pivot to what should be different, to solutions for testing before educational measurement becomes history.