Using Assessment to Promote

Equity:

A Call To Action

What started out as a panel discussion focusing on the hot-button issue of equity and accessibility, held at ATP’s Global Virtual Conference in mid-September, turned into a call to action from ATP members to the wider ecosystem of education and technology.

“Broadening access to educational and professional opportunities is a core mission of the assessment industry…along with reliability and validity…equity and fairness, this is at the center of what most of us do, “ stated ATP Chair-Elect John Kleeman,  President of Questionmark, as part of his introduction of an international panel of experts which included leaders from three global regions.

Kleeman went on to lay out the basis for the panel discussion, giving the overview that in the past, tests and exams have been widely accepted as a route for individuals to obtain opportunities. “But current events, coupled with recent trends in thinking, have challenged this worldview,” Kleeman noted.  He pointed to the Black Lives Matter campaign in the U.S. which has caused many people to re-examine if people of all races are being treated fairly within assessment, as well as the #ME TOO Movement, which he said, raises questions as to whether all societies and professions are being fair to women. “And then we’ve had the shocks of COVID with exams cancelled or delayed in some countries, replaced with algorithms elsewhere and requiring expensive technology, which perhaps isn’t fair to all.”

Kleeman turned to the panelists who included Janet Godwin, Interim CEO of ACT, Tim Oates, Group Director of Assessment Research and Development, Cambridge Assessment, Kellye Testy, President and CEO of Law School Admissions Council (LSAC), and Norihisa Wada, Executive Vice President and COO of EduLab,  asking them to respond to a set of recent criticisms of standardized testing starting with the accusation that high stakes tests for young adults correlates highly with family wealth.

Addressing the Wealth Inequity

Godwin opened the discussion by highlighting ACT’s mission which, she noted, has always been to “open doors and help students chase their dreams.”  So the concept and issue of access and equity, she noted, is really part of ACT’s core mission.

Godwin pointed out that if the case is that students from wealthier backgrounds are performing better on these tests, (critics) have to go to the deeper question as to why is this the case? She reiterated that it is already well known that access and equity to rigorous courses in secondary school is essential for student academic success and college readiness, and even before secondary school students need access to extra curriculars and navigational tools that could help them guide their abilities and interests.

So the accusation that the primary result of their college admissions testing is that of wealth is a criticism that is taken seriously at ACT, said Godwin, who pointed to efforts by the Iowa-based testing organization to teach into earlier grades to help inform students, parents and teachers as students progress. “We provide navigation tools for students and counselors to help students make decisions.  We have campaigns in secondary school to help students become excited about college and to learn how to fill out financial aid forms.”

Godwin reiterated that ACT takes extraordinary steps to eliminate bias from their tests, from content development sourcing, to the editing, and to other various panels and reviews, as well as data analysis, to make sure the test scores are not biased.

“However, and to the point,” she added, “if we are measuring differences in academic performance, college readiness and academic coursework we do need to understand what is causing those differences…it’s access to coursework, access to test prep and planning, it’s access to good counseling to help students have goals and progress and that’s where we see wealth make a difference.”

Testy added that  LSAC’s  mission has always been focused on access and equity. “That’s our origination story, and it’s what we’re deeply committed to.”  She went on to note that, “Our world is a deeply unequal one in every way. So whenever you are measuring something, the risk that you will pick up on that inequality is reflected.  And we need to grapple with this issue – not just throw out our statistics about validity and reliability. We have to look at how our assessments are working in the world and how they are being used.”

Oates stepped in with a history lesson from his U.K.- based organization, pointing out the irony that assessments arrived in England in the late 1800s  “precisely to enable equitable access to the next steps in education and into the professions…Previously that access had been determined by who you knew, the wealth that you had and so on, and that really was biased. That really prevented bright people from certain social backgrounds accessing education at all. So, the origins of assessment are important as they were identified very much with opening equity.”

Testy pointed to ways in which LSAC is placing emphasis on access and equity. She reported that the decades-old, non-profit  has partnered with the Kahn Academy to provide free access to preparation and is in the throes of developing a project called Law Hub which, as Testy explained the vision, “will someday unite the entire student journey in law while also providing free preparation and access to the skills…(and tools) that one needs to succeed in law school.”

Wada added that his Japanese-based testing organization has also seen a correlation between family wealth and those students getting into better colleges. But again, he denied that the tests are biased, “instead it is the access to costly preparatory schools causing the issue.”  And just like U.S.- based organizations, he said, EduLab has been combating the issue through broader access to preparatory materials across economic levels.  

Addressing Gender Bias

Wada noted that gender issues are a flash-point in Japan where more men than women are accepted to universities.

Oates likened it to the UK system which has struggled for years with gender issues in a system in which students self-select the areas in which they want to be tested.  “What we see is incredible gender choices…specifically in physics…there are very few girls participating proportionately.”  Once again, Oates noted, it’s not necessarily bias in the exam questions, it’s more likely cultural biases or gender biases or both that make the subjects themselves “attractive or unattractive to different genders.”

Addressing Access Inequity

Kleeman turned the discussion towards what he referred to as “access challenges…a situation exacerbated by COVID in which it has become apparent that not everyone has access to the same (educational) technology.”

Testy noted that “COVID has created an enormous shift in the way that education is delivered.” She reported that LSAC made a very fast change to delivering the LSAT remotely, achieving in a couple of months what likely would have taken years to deliver.  How did they do it?  Most importantly, she said, they consulted accessibility experts in the very design of the exam, and then came up with a plan to provide computers and Internet connections to those who could not access (technology) on their own.

Wada’s EduLab used a combination of remote proctoring and also quickly increased the number of test centers, explaining that since capacity was down by 50%, they had to increase the number.  And for those in truly remote locations, he said they are exploring way to provide  routers and PC software to access the Internet.

Godwin reported that ACT was proceeding a little slower, “because we wanted to make sure we got it right. We wanted to make sure we had the accommodations for students who do not have the Internet, the devices or the quiet places at home to test. We wanted to make sure our accessibility issues were all proven and sound and solid. And we wanted to make sure we had our security protocols in place so that Higher Ed admissions officers could have the confidence in the scores obtained via an at-home experience as opposed to those obtained in a physical test center experience.”  She said ACT is hoping to roll out in December or January, adding also that the non-profit has partnered with thousands of physical locations across the U.S. to provide quiet places to test.

A Call to Action

In wrapping up the panel discussion Kleeman asked how can the industry address the myriad of issues? The solution that bubbled up seemed to be that partnerships within the assessment ecosystem lead most often to success.

As Godwin noted, “We need to use assessment data as a call to action. Use it to dig into the root causes of why there are measurement differences…understand the societal differences, the school district differences, funding issues…I could go on and on about the issues impacting accessibility and equity. Aggregate it, unpack it, and then use it as a call to action. And then partner with others – we are only one part of the ecosystem, we cannot be off on an island. We need to be talking to partners in education, innovators in the educational technology space, use our data to address societal issues that matter so much to us.”

[Editor’s Note: ATP is looking at ways to make the case, going forward, for assessments to be an important way in which society can improve equity. If you have input or ideas, please feel free to contact Panel Moderator John Kleeman: [email protected]]

[The full panel discussion can be accessed at this link: Using Assessment to Promote Equity: How are We Doing?]