Correlations Between Grades and Recruit Success

Grades matter in predicting the number of OCIs extended, but after that point, grades are generally no better at predicting success than non-academic factors

The 2021 Toronto 2L Summer Recruit was unique, to say the least. Unlike Ultra Vires’ previous Recruitment Specials, where a full set of 1L grades were available for analysis, the majority of this year’s 2L applicants only had a single semester of 1L Fall grades because the Faculty implemented a mandatory Credit/No Credit system for all 1L Winter grades and full-year course grades. However, this year’s recruit participants had their Fall 2L grades available, which makes this analysis even more interesting. Unlike in 1L, students choose their classes in 2L.

1L Grades and Recruit Success

This analysis included participants who responded to all questions asking about grades in all seven required 1L courses. Participants were excluded if they reported a blank number (i.e., not a positive integer nor zero) in the number of OCI applications sent, non-OCI applications sent, OCIs received, OCIs accepted, in-firms received, in-firms accepted, and offers received.  Of 127 total participants, 91 qualified to be included in the analysis.This did not exclude them from analysis in the demographics and open-ended survey questions.

Before we proceed, it is important to acknowledge the effect of self-selection bias. It is to be expected that students are more likely to participate in the survey and share their results if they performed better academically or were more satisfied with the outcome of the recruit. In the table below showing the distribution of 1L and 2L grades, this self-selection bias is supported by the overrepresentation of Hs at the expense of Ps. Of the total 91 qualified participants 60 secured at least one offer.

Scaled GPA Calculation

Because students took the seven mandatory 1L courses in different orders (with the exception of LRW in the Fall), and only 1L Fall grades were available, most students had a limited number of graded (i.e., non-CR/NCR) courses.

An unweighted numeric GPA was assigned for each reported grade.As given in the Academic Handbook: 5 for HH, 4 for H, 3 for P, 1 for LP, 0 for F. Unlike in past surveys, the vast majority of students (85%) had only three non-CR/NCR grades. Eight students had a full set of 1L grades, likely because they were in a dual degree program. In previous years, we computed students’ total GPAs by adding their individual courses’ GPAs and ignoring the courses’ weights. Because we can no longer simply add the unweighted numeric GPA from each course like in previous years, we introduced the Scaled GPA. The Scaled GPA takes the average GPA of a student’s reported grades, regardless of how many, and scales that GPA to the desired equivalent number of courses.For example: a student with 7 reported grades, all Ps, has a full GPA of 7*3 = 21; this is then scaled to three grades as follows: (21/7)*3 = 9. Note how a student with 3 grades, all Ps, will have an identical Scaled GPA: 3*3 = 9. This accounts for the varying number of available grades for each student while capturing as many 1L grades as available.Scaling is not without limitations. Although for the 85% of students with 3 reported grades, scaling does not have any effect (since it is the same as adding the 3 GPAs). For the remaining 15%, it has the effect of ignoring employers’ reactions to more or less reported grades. For instance, it ignores the possibility that an employer may treat a student’s 7 reported Ps more favorably than a student’s 3 reported Ps. GPA was scaled to three courses because that was the most common reported number of grades (85%).

Once a Scaled GPA was calculated for each student Again, only 14 students required scaling. The total number of students is 91., we grouped students with similarFor the 14 students who required scaling, their Scaled GPA ended-up being a decimal number. So, to group them in their respective groups, Scaled GPAs were rounded to the nearest whole number accordingly (e.g., Scaled GPA 7 included those at or above 6.5 and those below 7.5). Scaled GPAs together, and compared each Scaled GPA group with three main variables of interest: average OCIs received, average in-firms received, and average offers received.

The Findings

There was an overrepresentation of Hs and underrepresentation of Ps. This could be the outcome of self-selection bias or a result of the limited number of grades available per course, given the mandatory CR/NCR policy in Winter 1L grades. 

For the 1L grades, the average participant had a mean and median Scaled GPA of 10.9 and 11, respectively. A Scaled GPA of 11 is equivalent to: P, H, H, or P, P, HH. Like previous years, there is a   strong upward trend in the average number of OCIs received as Scaled GPA increases. The trend is even more evident when the grades are divided into more granular units. However, take these numbers with a grain of salt — especially at the extremes of the distribution, since their smaller sample sizes may not be representative.

The positive correlation was also evident in the average number of in-firm interviews received, although much less pronounced. The mean and median number of in-firm interviews received were 4.29 and 3, respectively.

Finally, with respect to the number of offers received, among all Scaled GPA groups, the number fluctuated around one. The mean and median were 1.36 and 1, respectively.

2L Grades and Recruit Success

The same exclusion criteria used for analyzing 1L Fall grades were used for the 2L Fall grades. From the same pool of students who proceeded in the 1L grades analysis, 12 students were excluded from this analysis, for not reporting their 2L Fall grades.Students who reported full-year courses in-progress and those with CR/NCR courses were included. Only those with no reported status on their courses were excluded (12 students). Also note that courses from other divisions were excluded because they do not follow the Faculty of Law’s grading system (this only affected one grade for one student, and the course was ignored, so their remaining courses were still part of the analysis). This left us with 79 responses.

Additionally, the same technique of Scaled GPA proved useful in analyzing Fall 2L grades. Because students took a different number of courses, we could not simply add the unweighted GPA grades of all courses for each student, as this would simply inflate the GPAs of students with more courses.Additionally, unlike 1L grades, where the weights of each course were ignored and all courses were treated as equivalent in weight, 2L courses varied in weight. Since it was very impractical to weigh every reported 2L grade with its Faculty-listed weight, the technique of scaling overcame this challenge. Full-time students are required to take between 13-16 credits each term. Therefore, to satisfy this requirement, a lower number of courses taken must carry a higher weight per course taken, and vice versa. Hence, scaling 2L grades will, at least in theory, better reflect academic performance of the entire semester than 1L grades where only 3 out of 7 courses were available for 85% of the eligible students.

All students’ GPAs were scaled to four courses because most students were enrolled in four graded courses.4 students took 2 graded courses, 36 students took 3 graded courses, and 39 students took 4 graded courses. A student who took 3 courses with 3 Ps had a Scaled GPA of (3*3)/3*4 = 12, which is identical to a student who took 4 courses with 4 Ps: (3*4)/4*4 = 12. Like with the 1L grades, students’ 2L grades that required scaling (i.e., those who took more or less than 4 graded 2L courses) ended-up with a decimal Scaled GPA and were rounded to the nearest whole number. 

The Findings

Because 2L GPAs were scaled to 4 courses, unlike 3 courses in 1L grades, there are more possible configurations of grades, which make up the 11 groups of distinct Scaled GPA that range from 10-20 (unlike the 8 distinct groups in 1L, ranging from 7-14). Please use additional caution when interpreting the data given the smaller sample sizes in each group, especially in the extremes.

There was an overrepresentation of Hs and HHs at the expense of Ps (see table below outlining the distribution of 2L grades), which was more heightened than in 1L grades. This outcome could be the result of the following two reasons: (i) self-selection bias favoring those who performed better academically; or (ii) students tended to do better in 2L courses because they had discretion to select the courses they were most interested in taking.

For the 2L grades, the average participant had a mean and median Scaled GPA of 14.9 and 14.67, respectively. A Scaled GPA of 15 is equivalent to: P, H, H, H or P, P, H, HH. As with the 1L grades, we see a positive correlation between average OCIs received and Scaled GPA. This correlation, however, was moderate in strength.The Pearson correlation coefficient was calculated for each of the following independent variables: 1L Scaled GPA, 2L Scaled GPA, and 1L & 2L Scaled GPA. Each variable was paired separately with each of the following dependent variables: number of OCIs received, in-firms received, and offers received. Only the following pairs had a moderate association (defined as 0.5 < r < 0.7): 2L Scaled GPA and OCIs received (r = 0.54) and 1L & 2L Scaled GPA and OCIs received (r = 0.59). Also similar to 1L grades, average in-firm interviews received increases slightly with grades. However, this trend is barely visible when we look at the number of offers received, which also fluctuates around 1. 

Combined 1L & 2L Grades and Recruit Success

This is where it all comes together. Presumably, employers will have taken into account both 1L Fall and 2L Fall grades. For each of the 79 students that qualified for both 1L and 2L grades analyses,91 in the 1L grades analysis and 79 in the 2L grades analysis. we added their 1L Scaled GPA to their 2L Scaled GPA. This addition (“1L & 2L Scaled GPA”) is automatically scaled to 7 courses. No further scaling was necessary. 

Because the 1L & 2L Scaled GPA is representative of seven graded courses, there are even more possible configurations of grades than earlier analyses. Specifically, there are 14 different groups of grades, ranging from 20-33. Please use additional caution in interpreting the data given the smaller sample sizes in each group, especially in the extremes.

The Findings

For the combined 1L & 2L Scaled GPA, the average participant had a mean and median Scaled GPA of 25.78 and 26, respectively. A Scaled GPA of 26 is equivalent to: 2 Ps, 5 Hs or 3 Ps, 3 Hs, 1 HH or 4 Ps, 1 H, 2 HHs, etc. As with the 2L grades, we see the same moderate positive correlation between OCIs received and Scaled GPA. The persistence of this trend, despite the presence of 14 different grade categories/intervals, supports the idea that grades are correlated with the number OCI received. In other words, had this grades-to-OCI association been weaker, one would expect to see external factors, such as resume and cover letter factors, disturb the increase in OCIs as grades increase. We see this phenomenon more clearly in the red bars representing average in-firm interviews received. There, a “U” shaped trend is visible starting at the GPA of 24 (which is equivalent to 4 Ps and 3 Hs). A similar trend is also evident in the number of offers received (blue bars), where at GPA of 24 and upwards, the average number of offers in each category fluctuates around 2. 

Although the average number of offers fluctuated around 2 in most grade categories, when comparing the 1L & 2L Scaled GPA of those who ended up with at least one offer  to those without any offers, there was a statistically significant difference: mean Scaled GPA for those with at least one offer was 26.33 while those with no offers was 24.60.p = 0.004422 on a one-tailed, two-sample t test (two-tailed also had p < 0.05). Two years ago, grades between employment groups were not significantly different. It remains to be seen whether this finding is a one-off event or a start of a new trend. 

With the exception of grades being statistically significantly different for those with offers and no offers,  the three preceding trends mirror previous years and suggest a recurring theme: grades matter a great deal in predicting the number of OCIs extended, but after that point, grades are generally no better at predicting success than non-academic factors. Hopefully this offers some reassurance that grades are not everything.

Appendix