I'm not sure if this will be the last AP post I make, but it sure seems that way, unless people (including those of us who are members of the, you know, membership organization called College Board) put enough pressure on them to continue providing data in the detailed format like they've always done.
In case you don't know, College Board used to put very granular data on its website, for anyone to download and examine. You could look at data by state, by student ethnicity, by specific exam, and by AP score. For instance, this was a sample from one of the years showing the state of Alabama. You can see the data, and at the bottom, see other breakouts: Public schools, male/female, 11th/12th graders, etc. It was a gold mine of data, if you wanted to spend the time looking and digging and calculating. I've only scratched the surface in the three or four posts I've done on this blog, partly because the data are in spreadsheet format, and it takes a lot of cleanup to get it into useable shape for Tableau.
If you use Higher Ed Data Stories in your work at a college or a for-profit company or practice, consider supporting the costs of software, time, and web hosting by buying me a coffee at this link. If you're a high school counselor or you work at a CBO, please don't contribute; it's on me.
So anyway, I had taken a look at AP scores in 2019 and 2020 to see how they changed during COVID. It looked to me that scores had risen, which was probably not a problem in itself, for a lot of reasons:
We were dealing with COVID, and there was major disruption everywhere, including schools, but of course things that happen in a student's home affect school, too
The exams were converted to 45-minute, take-at-home formats. College Board assured us that they could predict a student's actual score with fewer and shorter questions, but I don't recall being shown that data. You just have to believe.
The grading process may have been compromised in unintentional ways: Teachers may have been confused about the new format. And, during a time when we could all use a little grace and understanding and compassion, those things might have entered into the grading calculations, too. No one could fault people for giving kids the benefit of the doubt.
And no one could have blamed the College Board for the effects of the pandemic. Except in one area.
When College Board announced the new 45-minute exams, they said the colleges would accept them just like they accepted the results from previous years. Problem is, they didn't exactly ask the colleges, so once they announced it, it was unlikely that any college was going to unilaterally be the bad guy. It was a genius move from a strategy angle, of course. A stroke of strategic (if evil) genius, if you will.
College Board, of course, claims it did ask colleges, but it seems like the only ones it really asked were its trustees, who are, of course, trustees because they align philosophically with College Board. And the trustees helped out by making a video telling students that it was their responsibility to take the exams. Sweet.
Because the alternative would have been to cancel the tests. And College Board likes its revenue, so that wasn't going to be an option.
But once the worst of COVID was over, I wanted to see if the 2021 scores had returned to some semblance of normal. I think all colleges have a right to this information, given that we grant academic credit--the coin of the realm in higher education--trusting that the exams measure and assess what they say they do. Without warning, College Board pulled the data off its website, and did not publish the granular data for inspection for 2021.
Some right-leaning and even alt-right sites picked it up, and made it seem like the race issue was the big one. It wasn't, and in fact, I've visualized the data from prior years showing differences by ethnicity. It pissed me off, but I suppose that's where we are in America these days. So on the visualization below, you won't see any comparisons of scores by race.
The interesting thing is that if College Board had published the data, and I had visualized it, maybe 2,500 people would have seen it. As it is, I'm sure tens, if not hundreds of thousands of people have had the opportunity to think about this: About College Board's responsibility to members (colleges and high schools) and to students, not to mention society writ large.
So, after all that, the data, below, in six views, using the tabs across the top. The first view Changes 2018--2020 shows changes in volume by state from 2018 to 2019. If you hover over a state bar, you'll see a pop-up of the changes in that state by score distribution: The 5s increased by 2%, or the 3s were down 8%, for instance. Again, causality is a dangerous thing. Don't rush to judgements.
The second view, Scores and Household Income is for the racists. Nothing is labeled, so you can't tell which point shows which, but you should see the relationship between income and performance on exams. Chalk it up to race if you want, but it's not quite that simple. Income and ethnicity are tied together in America, and if you don't believe me, argue with the data. The filter will take you from national to state data if you wish. Actually, I encourage you to do so.
The third view, Income and Scores by State shows the same thing, broken out in more detail, for the skeptics. Again, no labels. Just look at the patterns.
The fourth view, Mean Scores by State, shows mean scores for 2018, 2019, and 2020 by state. You can filter to specific tests using the control at the top.
The fifth view, Changes in Mean Scores, shows the difference between means in 2018 and 2020. Use the controls to change the state or ethnicity if you'd like.
And finally, Participation by Ethnicity shows three years of exam volume by student race on the bars, and year-over-year percentage changes on the lines.
As always, email me or leave a comment if you see something of interest. Thanks for reading.
If you're not following Akil Bello on Twitter, you should be. His timeline is filled with great insights about standardized testing, and he takes great effort to point out racism (both subtle and not-so-subtle) in higher education, all while throwing in references to the Knicks and his daughter Enid, making the experience interesting, compelling, and sometimes, fun. Recently, he created the term " highly rejective colleges " as a more apt description for what are otherwise called "highly selective colleges." As I've said before, a college that admits 15% of applicants really has a rejections office, not an admissions office. The term appears to have taken off on Twitter, and I hope it will stick. So I took a look at the highly rejectives (really, that's all I'm going to call them from now on) and found some interesting patterns in the data. Take a look: The 1,132 four-year, private colleges and universities with admissions data in IPEDS are incl
(Note: I discovered that in IPEDS, Penn State Main Campus now reports with "The Pennsylvania State University" as one system. So when you'd look at things over time, Penn State would have data until 2018, and then The Penn....etc would show up in 2020. I found out Penn State main campus still reports its own data on the website, so I went there, and edited the IPEDS data by hand. So if you noticed that error, it should be corrected now, but I'm not sure what I'll do in years going forward.) Freshman migration to and from the states is always a favorite visualization of mine, both because I find it a compelling and interesting topic, and because I had a few breakthroughs with calculated variables the first time I tried to do it. If you're a loyal reader, you know what this shows: The number of freshman and their movement between the states. And if you're a loyal viewer and you use this for your work in your business, please consider supporting the costs
Each time I publish this visualization I get a lot of traffic on the site, and I can see why. It shows all doctoral recipients (in 2016-2020) broken out by where they received their bachelor's degrees. So, for instance, the top level view shows that UC Berkeley is the alma mater of more doctoral recipients than any other institution, followed by The University of Michigan and Cornell University. That would be interesting, but of course, these are large institutions, and it's natural to think lots of graduates will lead to lots of doctoral degrees. No surprise there. So the visualization allows you to look at the types of colleges you or your students might want: Select a state, select public or private, select by Carnegie type if you wish. You can also look at HBCUs if you'd like. Most important, you can filter by doctoral degree, so if you want to see which institution sends the most students to doctorates in chemistry, you can do so. To head off the questions I get ev
Post a Comment