What is the College Board Thinking? Pt. 2

Full length AP exams mean poor kids get the shaft. Again. (Again.)

This is part 2 of a two-parter called “What’s the College Board Thinking?" If you missed part 1, you’re really missing out, and you should read it here.

Last week I wrote about how crazy I thought it was that the College Board was going forward with full length AP exams this year. To wit:

“Earlier this year, the College Board stood at a fork in the road and for reasons that I cannot alone find baffling chose the path forward that will harm poor students, particularly those of color, who, due to the limitations of remote school that are obvious to anyone who has spent five minutes on Zoom, and also the general havoc wreaked by the pandemic, have not had the same educational experience as their more affluent peers. So we’re marching them toward a high-stakes exam that through no fault of their own they will be ill-equipped for.”

In this post, I want to discuss why, exactly, the full length AP exams are so unfair and why I believe they will result in an even greater disparity in scores among student groups than in a usual year.

//

The biggest reason why the AP exams are inequitable this year is because they violate a very basic principle of assessment design: you don’t test what you didn’t teach. That is, if you’ve been teaching students how to draw a circle, you don’t give them a test that asks them to draw a square. 

Even if you were to discount that school started later for many students (NYC pushed its September start date back almost two weeks), it is just not possible that any AP teacher was able to cover all of his/her curriculum with any integrity.

This has less to do with the pandemic-related school closings and re-openings that made this year feel like a traffic jam in a cul-de-sac and more to do with how difficult it is to teach online. 

If you’re not a teacher, allow me to explain. (If you’re a teacher, go pour yourself a drink.) Whether you’re online or in the building, in order for a class to be considered successful, many many many things need to happen at once. Students need to be engaged, they need to understand what the teacher is teaching, and they need to demonstrate to the teacher that they understand what the teacher is teaching. The teacher, in turn, has to ensure the students are engaged and constantly monitor if the students understand what is being taught.

Since classrooms, unfortunately, have more than one student, this becomes extremely difficult very quick because at any no point in any lesson are all students engaged, understanding you and demonstrating to you that they understand you. (Even the best teachers on their best days can only hope that most students are engaged and understanding them most of the time.) Thus you are constantly watching, monitoring, pivoting, recalibrating, improvising, editing, etc, for different students in different ways at the same time. An excellent class is one with 30 different parallel universes. Add to that the external circumstances that have a hidden but outsized influence on a class — did the students have the dreaded Miss Trunchbull before you? Did their parents fight last night? Does everyone have healthcare? — and you’ve got a task that makes air traffic control look like emptying the dishwasher. 

This is hard enough in a classroom, and online you may as be trying to do all of this behind a curtain. All of the usual cues you use to gauge where students are — body language, misbehavior, literally looking over their shoulders — are gone. In order to get to a place where you can monitor what students are doing, you have to slow things down. What you might usually cover in 30 minutes takes an hour. 

Now, to be sure, there are certain types of teaching that transfer easily to remote instruction, such as lectures, videos, students reading, students writing, students working through problems. But all of those, in their purest forms, are passive. They don’t require much interaction, so it doesn’t matter whether your students are sitting at a desk three feet in front of you or seven miles away in their beds. 

But if that’s all it took to teach kids — just talking at them or posting work — then this the past year would not have resulted in such massive learning losses for students, particularly low-income students. Au contraire, the past year would have been an unmitigated success and completely upended everything we know about how people learn best.

But even before the pandemic, we knew this was the case. We knew that simply giving kids laptops didn’t work (remember One Laptop Per Child?), that a lot of “ed tech” was Silicon Valley futurism bullshit (turns out even Palo Alto parents didn’t want their kids staring at screens in school so much), and that “personalized learning” was mostly a false promise. So who’s surprised, really?

It just so happens that the most important thing that happens in a class also happens to be the hardest thing to pull off online. I’m talking about discussion. When people say that “learning is social,” this is what they mean. You simply cannot have a successful class if at no point in your lesson your students are not talking to you and/or talking to each other. Discussion is how students make sense of stuff, whether they’re asking and answering questions themselves or listening to others do it. Sure, I’ve seen successful discussions online, but none that are half as boisterous or sophisticated as the better discussions I’ve seen or facilitated in a classroom myself.

And anyone who has been in a Zoom meeting with more than 5 people knows what I’m talking about. Even if, by some miracle, you manage to get most of the kids to turn their goddamned cameras on and participate, class discussions can feel either like messy cable news debates or stilted Quaker meetings. And even if you’re doing the most basic form of discussion (teacher question → student answer → teacher question→ student answer), those are usually slowed down by students multi-tasking, forgetting to unmute themselves, screaming siblings, shouting parents, poor connections, or the Rapture (“Ciara? You there? Hello? Ciara?”) 

In sum, online instruction at its best is a sex doll or a frozen TV dinner. A poor simulacrum of the real thing. 

So what’s all this got to do with the College Board’s decision to administer full length AP tests? Well, as has been well-reported, almost half of all students are learning remotely and most of those students are low income students of color in urban schools, where the pandemic hit hardest.

Therefore, if you know that remote school is inferior to real school...and you know that a disproportionate number of poor students are still mostly only engaged with remote school… then why would you still insist on going ahead with the full length exams?

How is this not going to result in a huge number of low income students who’ve been marooned online scoring significantly lower than their more affluent peers who were able to return to school earlier? How will this not widen the gap in scores that already exists?

I honestly don’t know, but the College Board seems to know this disparity in school experiences is a problem and so came up with a solution: AP Daily. 

AP Daily are short, free videos that offer mini-lessons on almost every single topic in almost every single course and are meant to supplement instruction. Most of these videos feature aggressively enthusiastic teachers in Skullcandy headphones talking over power points and probably play alright in Plano or Naperville, but I think most city kids would tap out after 5 minutes. But even discounting the endearingly dorky nature of these videos, it’s unfair to place the burden of such learning on students, especially given everything we about how isolating and tumultuous this past year has been for students hit disproportionately hard by the pandemic, which, in case you haven’t been paying attention, is low-income students.

And all of which is to say nothing about how fallacious it is to think that you can supplement remote instruction with more remote instruction.

//

A smaller, but not less significant issue has to do with the functionality of the digital exams. This is from the AP Digital Testing Guide, which came out last month:

[Digital exams will] not allow students to return to answered questions or move back-and-forth between unanswered questions

This means that students must start at question 1 and move forward. They cannot start at question 25. They cannot skip to another question. They cannot go back. Once a question’s answered, that’s it. 

Almost literally no one takes AP exams like this. 

If you are thinking, Well, what’s the big deal? then you either haven’t taken a multiple choice exam in a while or you were never very good at them. (It’s okay. Neither was I.) 

The first part of most AP exams is an hour-ish multiple choice section. Each exam has a different number of questions, but you’re looking at around a minute per question. You have to move fast¹.

One of the first strategies you learn when taking a timed test is to get a lay of the land. When the timer starts, you open the exam, scan the passages and questions and quickly develop a plan of attack. Most students do not start at question #1. Maybe questions #45-48 center around a concept you’re familiar with. Maybe questions #11-23 feature a simpler poem. You knock the easy stuff out of the way first. Build your confidence. Earn yourself some breathing room. You do not work linearly. 

Even more granularly, when it comes to a run of questions, you don’t work in a straight line. You move back and forth within that run of questions because, particularly when it comes to reading passages, some of those questions refer to specific lines (“In lines 19-27, the author’s tone can best be described as…”) and some of those questions refer to the entire text (“Which of the following can best be described as the author’s thesis?”). In order to save precious time, a useful strategy is to skip actually reading the passage and answer the line-specific questions first. By the time you answer them, you will have basically read the whole passage and can then answer the other questions. You can earn yourself valuable minutes.

But, okay, let’s say you’re not that savvy of a test taker. Let’s say you like to keep things simple and straightforward and you just want to start at question #1. Even so...you can’t go back to review your answers! Who doesn’t go back to look at previously answered questions? At all? Ever? This also puts students with special needs and multi-language learners who benefit from re-reading at a major disadvantage, and yes, both of those groups of students take AP exams, and yes, they often do very well.

This is such a confounding decision that when I first read the testing guide I thought I’d misinterpreted it. The digital guide’s reason for this is testing security, but why not just do what every single teacher who has ever existed does when they don’t want their students to cheat: make different versions of the same test. Why doesn’t the College Board just make 25 versions of each multiple choice section?

Even if students were able to find another student with the same version of that test, how possible is it that those students would be able to coordinate and execute a joint test taking session? Have you ever seen students try to do group work? They spend half the time arguing about who’s not going to be the leader and half the time reading the directions and then one of them starts decorating the poster paper and two start talking about basketball and the leader ends up doing all the work herself. Then the bell rings and you have to pick up all the fucking markers. 

In that same testing guide, the College Board assures us that “there’s nothing about an AP Exam being digital [. . .] that inherently affects a student’s ability to get a particular score,” but when the functionality of your test forces students to take the exam in an unnatural, unintuitive way, that’s simply not true. 

//

I’ll conclude by saying that I hope I’m wrong. To be clear, I actually think AP classes are pretty great for low-income students, who have been historically excluded from tough college-level classes, and which is why I have found the College Board’s decision so disappointing.

There will be anecdotal reports from students and teachers floating around on social media after the exam, but the real tell will be the results, which we won’t get until this summer.

The College Board provides thorough data on how well students do on each AP test each year. They chop up the data by state, gender and race. Notably missing is socioeconomic status. (Again, why?) Though it doesn’t appear there was much of a difference between the 2020 scores (when the test was shortened) and 2019 (the last full length exam), we shouldn’t conclude that the pandemic had no effect on the scores of sub-groups since there were many significant differences between those two testing administrations.

The true(r) comparison will be between the 2021 and 2019 tests.

So I guess we’ll see.

Best of luck to all of the students taking the AP exams in May.

Photo: Schomburg Center for Research in Black Culture, Photographs and Prints Division, The New York Public Library. "Flatbush art students showing examples of their work" The New York Public Library Digital Collections. 1935.

1. My beef with timed standardized tests will be the subject of a future post.