The Decision-Making Process for Reinstating Standardized Testing in Admissions

In an email to faculty on January 30, 2024, President Rosenbaum wrote that, “Two questions that have arisen in recent campus conversations concern the value of standardized testing, and the appropriate role of extracurricular activities, in admissions decisions. We have asked the Advisory Committee [on Undergraduate Admissions Policy] to accord these questions high priority and, if possible, to offer their recommendations on standardized testing by the end of March and on extracurricular activities by the end of April.”

The Faculty Advisory Committee on Undergraduate Admissions Policy, reported on in the February 6 edition of the Tech, orally presented their recommendation to the Faculty Board at its monthly meeting on April 8. After the presentation, the Faculty Board held a formal vote and decided to take up the recommendation. Caltech’s reinstatement of the standardized testing requirement in admissions was announced on April 11. This decision ended the moratorium that was originally set to expire in 2025.

In a conversation with Provost David Tirrell and Advisory Committee Chair Professor Steve Mayo, the Tech was told that the committee “spent a lot of time thinking about what the implications would be if we decided to end the moratorium a year early” and it was clear based on what they were seeing that it would be advantageous to do so. The factors that the committee considered included the number of admitted students taking standardized tests during the moratorium, the ability to see AP scores, and decisions from peer institutions.

The vast majority of admitted students in the last four enrollment years took and reported their standardized test scores. During the moratorium, AP test scores were also not available. The committee took into account research showing that AP test scores are highly predictive of student performance, much more than SAT/ACT scores. Professor Mayo also stated that “a large fraction of [Caltech] applicants apply to MIT and Stanford.” MIT has reinstated standardized testing requirements and Stanford is test-optional, so “a significant fraction of students will be taking the test anyway.”

When researching for this article, the Tech was made aware of work being done by the Student Success Analysis Working Group. This group was composed of Professors R. Michael Alvarez and Jonathan N. Katz, as well as the Institute’s Chief Institutional Research Officer, Lindsey Malcom-Piqueux and the Dean of Undergraduate Studies, Lesley Nye. Formed in 2022 at the instigation of Professor Kevin Gilmartin, the Working Group has collated and analyzed data related to standardized testing in admissions decisions, among other things.

In research reviewed by the Tech, the Working Group found that although a correlation exists between SAT/ACT scores and undergraduate students’ academic performance, especially in first-year core curriculum classes, it is small in magnitude. They showed that adding standardized test scores to predictive models of grades that account for basic socio-demographic variables does not significantly improve the accuracy of its predictions. Also, their findings demonstrated that the distribution of standardized test scores for admitted students who took a test after the moratorium was enacted was identical to the distribution observed for previous cohorts.

When asking Professor Mayo about these findings and whether the committee took them into consideration, he stated that the Working Group’s findings have been “extremely useful and timely,” but the committee also did its own research which showed stronger results. The main point Professor Mayo emphasized is that there is no debate about whether or not there is a correlation between standardized test scores and student performance. He noted that both the Working Group and the committee found correlations. While the size of the correlation in the Working Group’s research was not high, it was statistically significant. The committee’s study recapitulated this finding. The committee focused primarily on shadow GPA (first, second term first-year STEM GPA) where the strength of the correlation was higher. Their analysis also showed that there was a statistically significant correlation between standardized test scores and later terms. The strength of the correlation was smaller, but the correlation was there and statistically significant. Professor Mayo recognized that there are differing opinions regarding the extent to which admissions should rely on that correlation. However, the committee found that having additional data is helpful, particularly for validating choices of applicants who are from schools where less information is available, so it is harder to calibrate the applicants.

The Tech also asked about the reliability of shadow grades, given that having the first two terms as pass/fail is to allow students to adjust to Caltech without having to worry too much about their performance. Professor Mayo acknowledged this point but stated that the challenge with using grades after the first two terms is that Caltech, like many other institutions, has seen significant grade inflation. As the grades all compress around a high value, the ability to see a strong correlation in that data decreases. A correlation is much more obvious in shadow GPA than in other data sets.

Additionally, the Tech was told that the formulation of the Faculty Advisory Committee on Undergraduate Admissions Policy was not in response to the faculty petition; the decision to form the committee was made prior to its circulation. For context, on February 6, the Tech reported that a group of professorial faculty submitted a petition to President Rosenbaum in the beginning of January. It alleges that there has been a decline in student readiness for and performance in classes. The primary call of the petition was to reinstate the SAT/ACT at some level in admissions decisions.

At the April 8 Faculty Board meeting, the committee and the board discussed what impact reinstatement would have on current students at Caltech, as well as the potential perception that the faculty think that current students are not actually qualified to be here. Professor Mayo stated, “That is completely, absolutely not true. We support the students that are here. This is about getting more information in the future about future admits to make more informed decisions. Most times, more data are better. The decision had nothing to do with criticism or deficiencies in current classes; it was all about thinking about what the rest of the world was doing, the other kinds of things we wanted to see like AP test scores, and providing as much information as possible to make the best decisions about future incoming classes.”

More on this topic: [Athletics in Admissions at Caltech — A Look at Some Numbers]