Skip to main content

The Last AP Post?

I'm not sure if this will be the last AP post I make, but it sure seems that way, unless people (including those of us who are members of the, you know, membership organization called College Board) put enough pressure on them to continue providing data in the detailed format like they've always done.

In case you don't know, College Board used to put very granular data on its website, for anyone to download and examine.  You could look at data by state, by student ethnicity, by specific exam, and by AP score.  For instance, this was a sample from one of the years showing the state of Alabama.  You can see the data, and at the bottom, see other breakouts: Public schools, male/female, 11th/12th graders, etc.  It was a gold mine of data, if you wanted to spend the time looking and digging and calculating.  I've only scratched the surface in the three or four posts I've done on this blog, partly because the data are in spreadsheet format, and it takes a lot of cleanup to get it into useable shape for Tableau. 


If you use Higher Ed Data Stories in your work at a college or a for-profit company or practice, consider supporting the costs of software, time, and web hosting by buying me a coffee at this link.  If you're a high school counselor or you work at a CBO, please don't contribute; it's on me.

So anyway, I had taken a look at AP scores in 2019 and 2020 to see how they changed during COVID.  It looked to me that scores had risen, which was probably not a problem in itself, for a lot of reasons:

  • We were dealing with COVID, and there was major disruption everywhere, including schools, but of course things that happen in a student's home affect school, too
  • The exams were converted to 45-minute, take-at-home formats.  College Board assured us that they could predict a student's actual score with fewer and shorter questions, but I don't recall being shown that data.  You just have to believe.
  • The populations of students who took the test probably changed, too.  Students with lower performance may have dropped out.  Students from lower-income families might not have access to computers at the precise time they needed to test.  It was suggested that if bandwidth at home was a problem, students could just take their exams in a parking lot of a McDonald's using the free wifi. There were some difficulties in the technology. too as students who completed the exam couldn't upload their results.
  • The grading process may have been compromised in unintentional ways: Teachers may have been confused about the new format.  And, during a time when we could all use a little grace and understanding and compassion, those things might have entered into the grading calculations, too.  No one could fault people for giving kids the benefit of the doubt.
And no one could have blamed the College Board for the effects of the pandemic.  Except in one area.  

When College Board announced the new 45-minute exams, they said the colleges would accept them just like they accepted the results from previous years.  Problem is, they didn't exactly ask the colleges, so once they announced it, it was unlikely that any college was going to unilaterally be the bad guy.  It was a genius move from a strategy angle, of course.  A stroke of strategic (if evil) genius, if you will.

College Board, of course, claims it did ask colleges, but it seems like the only ones it really asked were its trustees, who are, of course, trustees because they align philosophically with College Board.  And the trustees helped out by making a video telling students that it was their responsibility to take the exams. Sweet.

Because the alternative would have been to cancel the tests.  And College Board likes its revenue, so that wasn't going to be an option.

But once the worst of COVID was over, I wanted to see if the 2021 scores had returned to some semblance of normal.  I think all colleges have a right to this information, given that we grant academic credit--the coin of the realm in higher education--trusting that the exams measure and assess what they say they do.  Without warning, College Board pulled the data off its website, and did not publish the granular data for inspection for 2021.

I complained about it on Twitter.  Not only do colleges have a right to this, so do students, teachers, and parents, even if most will never look at it.  And College Board is touting AP as a good substitute in a test-optional college admissions world as the SAT has taken a life-threatening blow, so more transparency is critical, I think.  I re-tweeted it later.  The whole time I was reminded of the frequent tweets of Akil Bello, pointing out David Coleman's claim of "transparency" in all they do. 

One day this article by Jeremy Bauer-Wolf appeared, and it got a little attention (not enough to change minds at College Board, but that's another issue into and of itself). And I thought that was that.  It wasn't.

Some right-leaning and even alt-right sites picked it up, and made it seem like the race issue was the big one.  It wasn't, and in fact, I've visualized the data from prior years showing differences by ethnicity.  It pissed me off, but I suppose that's where we are in America these days.  So on the visualization below, you won't see any comparisons of scores by race.  

The interesting thing is that if College Board had published the data, and I had visualized it, maybe 2,500 people would have seen it.  As it is, I'm sure tens, if not hundreds of thousands of people have had the opportunity to think about this: About College Board's responsibility to members (colleges and high schools) and to students, not to mention society writ large.

So, after all that, the data, below, in six views, using the tabs across the top.   The first view Changes 2018--2020 shows changes in volume by state from 2018 to 2019.  If you hover over a state bar, you'll see a pop-up of the changes in that state by score distribution: The 5s increased by 2%, or the 3s were down 8%, for instance.  Again, causality is a dangerous thing.  Don't rush to judgements. 

The second view, Scores and Household Income is for the racists.  Nothing is labeled, so you can't tell which point shows which, but you should see the relationship between income and performance on exams.  Chalk it up to race if you want, but it's not quite that simple.  Income and ethnicity are tied together in America, and if you don't believe me, argue with the data.  The filter will take you from national to state data if you wish.  Actually, I encourage you to do so.

The third view, Income and Scores by State shows the same thing, broken out in more detail, for the skeptics.  Again, no labels.  Just look at the patterns.

The fourth view, Mean Scores by State, shows mean scores for 2018, 2019, and 2020 by state.  You can filter to specific tests using the control at the top.

The fifth view, Changes in Mean Scores, shows the difference between means in 2018 and 2020.  Use the controls to change the state or ethnicity if you'd like.

And finally, Participation by Ethnicity shows three years of exam volume by student race on the bars, and year-over-year percentage changes on the lines.

As always, email me or leave a comment if you see something of interest. Thanks for reading.


Comments

Popular posts from this blog

On Rankings, 1911, and Economic Mobility

If you're alive today, you have lived your whole life with college rankings.  Yes, even you.  You may not have knows you were living in the time of college rankings, but indeed, you have been, unless you were born before 1911 (or maybe earlier.)  If you're interested, you can read this Twitter thread from 2020 where I discuss them and include snippets of those 1911 rankings as well as those from 1957, written by Chesley Manly. You can read for yourself, or you can trust me, that in fact the rankings as we know them have been surprisingly consistent over time, and most people would have only minor quibbles with the ratings from 1911.  Perhaps that's because they have always tended to measure the same thing. But what if we did different rankings?  No, not like the Princeton Review where they make an attempt to measure best party school, or best cafeteria food, or worst social life.  Something more quantifiable and concrete, although still, admittedly, a hard thing to get rig

The Highly Rejective Colleges

If you're not following Akil Bello on Twitter, you should be.  His timeline is filled with great insights about standardized testing, and he takes great effort to point out racism (both subtle and not-so-subtle) in higher education, all while throwing in references to the Knicks and his daughter Enid, making the experience interesting, compelling, and sometimes, fun. Recently, he created the term " highly rejective colleges " as a more apt description for what are otherwise called "highly selective colleges."  As I've said before, a college that admits 15% of applicants really has a rejections office, not an admissions office.  The term appears to have taken off on Twitter, and I hope it will stick. So I took a look at the highly rejectives (really, that's all I'm going to call them from now on) and found some interesting patterns in the data. Take a look:  The 1,132 four-year, private colleges and universities with admissions data in IPEDS are incl

Freshman Migration, 1986 to 2020

(Note: I discovered that in IPEDS, Penn State Main Campus now reports with "The Pennsylvania State University" as one system.  So when you'd look at things over time, Penn State would have data until 2018, and then The Penn....etc would show up in 2020.  I found out Penn State main campus still reports its own data on the website, so I went there, and edited the IPEDS data by hand.  So if you noticed that error, it should be corrected now, but I'm not sure what I'll do in years going forward.) Freshman migration to and from the states is always a favorite visualization of mine, both because I find it a compelling and interesting topic, and because I had a few breakthroughs with calculated variables the first time I tried to do it. If you're a loyal reader, you know what this shows: The number of freshman and their movement between the states.  And if you're a loyal viewer and you use this for your work in your business, please consider supporting the costs