Either good writing isn’t helpful for law school or LRW doesn’t assess writing ability well
With the proposed 1L curriculum changes (see “Slashing Small Group” by Avnee Paranjape), there has been much discussion about Legal Research and Writing (LRW). There has been, to put it mildly, a wide range of opinions on LRW’s efficacy—from Professor Chiao: “LRW is the most important 1L course,” to Professor Shaffer: “Let’s be frank, LRW is the absolute worst course we have in first year.”
However, as noted by members of the faculty and student population, there has been a distinct lack of evidence in support of either of these positions. While I cannot claim to have definitive proof that LRW is either the most important or the worst 1L course, I want to share some observations from the Recruit Survey data that Ultra Vires has collected in the past few years, to provide some evidentiary grounding for these positions.
For the past four years, Ultra Vires has collected data on 1L students’ grades as part of our annual Recruit Survey.
One piece of analysis that I performed last year looked at how well a student’s grade in one class predicted their grade in another class. The underlying assumption was that there are generally transferable skills between classes (such as clear writing, good study habits, and legal reasoning) such that students who perform strongly in one class perform strongly across classes.
The data did not support this conclusion. The correlations between classes are very weak, such that knowing someone’s grade in one class gives little information about their grade in another class.
That said, there is a clear divide between the correlations as between LRW and other classes, and as between any other two classes.
As I said last year:
LRW…[has] negative correlations with several other classes … Even when not negative, LRW … [has] correlations with other classes that are very small or essentially zero … Intuitively, it feels like the skills of “doing research” and “writing” should be common to all law classes (and law generally). However, the data suggest that either LRW is accurately measuring student’s writing skills, but having strong writing skills is not predictive of success in other classes or that LRW is not accurately measuring student’s writing skills.
There are a few inferences about LRW that you can draw from this (none of them good):
- LRW DOES accurately assess students’ writing abilities, but writing ability IS NOT predictive of student’s grades in other classes;
Alternatively, writing ability is predictive of student’s grades in other classes, but the style of writing that LRW assesses is different enough from the writing in other classes such that your level of ability at one does not predict your level of ability with the other. - LRW DOES NOT accurately assess students’ writing abilities, but writing ability IS predictive of student’s grades in other classes; or
- LRW DOES NOT accurately assess students’ writing abilities, and writing ability IS NOT predictive of student’s grades in other classes.
I find it difficult to accept that writing ability does not help predict students’ grades. Intuitively, better writing should lead to better grades (and worse writing to poorer grades), because every evaluation involves some form of persuasive writing. That would seem to rule out inferences 1 and 3, leaving us with the conclusion that LRW is not doing a good job of assessing students’ writing ability. This would also seem to align with the many (many) voices during the March 9 1L Curriculum Consultation that raised concerns about LRW. That said, in an article about firm evidence, I do not have any such evidence to suggest inferences 1 and 3 are less likely to be correct than conclusion 2.
However, regardless of which inference you prefer, LRW stands out from the other 1L classes. For four years, how well students did in LRW has had essentially zero predictive value on how well students did in their other classes. Whatever comes out of the proposed curriculum changes, LRW needs a serious look.
Some Numbers If You’re Interested
So as to not scare anyone with numbers, I put them at the end. You can feel free to skip this part. Correlations range from 1 (i.e. you got the same grade in both classes) to -1 (an HH in one class and an LP in the other). A correlation of 0 means that both grades are random. Correlations greater than 0.7 are generally considered strong, correlations of 0.3 to 0.5 are generally considered weak, and correlations less than 0.3 are generally considered very weak (or no correlation). The table below shows the correlation between LRW and every other 1L class since the school adopted the HH/H/P grading scheme.
Table 1 – Grade Correlation between LRW and other Classes
| Class of | Constitutional | Contracts | Criminal | Legal Process | Property | Torts | Average |
| 2021 | 0.11 | -0.015 | 0.17 | -0.0042 | 0.068 | 0.032 | 0.060 |
| 2020 | -0.040 | -0.056 | -0.040 | 0.21 | 0.068 | 0.033 | 0.056 |
| 2019 | 0.14 | 0.14 | 0.22 | 0.090 | 0.14 | 0.063 | 0.13 |
| 2018 | 0.24 | 0.19 | 0.11 | 0.10 | 0.19 | 0.12 | 0.17 |
For reference, the average correlations between any two classes (excluding LRW) were 0.18, 0.15, 0.24, and 0.26 for the Classes of 2021, 2020, 2019, and 2018 respectively. For four years, LRW has been at least 0.1 points less well-correlated than any two classes other classes.
1 Ultra Vires also collected data from earlier years, but grading prior to introducing the HH/H/P scale was not looked at.
2 For more details on all of these issues, see the Recruit Specials; they discuss the data in depth.
3 Though the faculty knows where to find me if they have a suitably anonymized dataset that could use some attention.
4 The difference is statistically significant with p = 0.002.
5 Alternatively, writing ability is predictive of student’s grades in other classes, but the style of writing that LRW assesses is different enough from the writing in other classes such that your level of ability at one does not predict your level of ability with the other.
