by Jeff Sovern
There's been a lot of talk among law professors about the US News plan to measure faculty scholarly impact in part by citations to faculty scholarship (see here for a blog post citing to commentary). While for now US News says it will not incorporate the citation rankings into its general law school rankings, US News may in the future replace the faculty reputation measure to some extent with a measure of how frequently professors are cited. That could create disturbing incentives for faculty hiring and retention, as well as affect what professors write about.
To be more concrete, imagine that a law school is hiring a new professor and has two candidates. One candidate writes about criminal law and the other writes about consumer law. The law school wants to maximize its ranking, and so wants to hire the candidate whose work will be cited more. The universe of people writing scholarly articles about criminal law is much larger than the universe of professors writing about consumer law, and so, all other things being equal, the criminal law professor is likely to rack up more citations and so help with the school's ranking more. How do we know more people write about criminal law? I did a search for "consumer law" on SSRN and got 637 hits. "Criminal law" by contrast elicited 7,867 hits, or more than a dozen times as many. Every law school offers criminal law courses, probably all have a full-time professor teaching in the area, and many have one or more professors writing in the area. But as of 2014, only about a third of law schools had a consumer law course of one sort or another (yes, I need to update that), and many of the courses were taught by adjuncts who have a day job and so are unlikely to find time to write about consumer law. Fewer professors writing in an area means fewer people likely to cite your work. And so that means hiring the consumer law scholar could hurt your ranking as compared to hiring the criminal law professor. It also means that those seeking to become law professors should write in widely-taught areas to make themselves more attractive to law schools.
Now maybe you're thinking that even within broad areas like criminal law, professors are apt to specialize when it comes to writing and so as a practical matter writing about a consumer law topic is no different than writing about a criminal law topic. So let's compare the number of papers on SRRN in a couple of specific areas within criminal law and consumer law. Homicide drew 767 hits while "mens rea" got 479 hits. "Debt collection" elicited 179 hits while "credit reporting" got 135. Those differences are less dramatic, but the number of criminal law hits are still multiples of the consumer law hits. No doubt SSRN is an imperfect measure of the amount of scholarship in a particular area, but it would have to be off by a lot to make those differences insignificant. It makes intuitive sense that a subject that is taught at fewer law schools will be written about less–and so generate fewer citations.
This isn't just bad for consumer law scholars. It's also bad for consumer law, and indeed any subject that isn't taught at all or nearly all law schools. If you think, as I do, that scholarship is valuable, and law schools stop producing consumer law scholarship because they stop hiring consumer law professors, then society will have lost something. Society will have gained instead more articles about widely-taught subjects, but surely the marginal value of yet another article on criminal law is less than the marginal value of an article in an area less traveled.
I believe your concern would be largely resolved if they simply use the average number of citations per tenure-track/ tenured faculty (or whatever faculty category for which writing and publishing is expected). When you have more people in a field, you would naturally expect there to be more publications in that field. The total number of articles in a field over any period of time tells us more about the number of people writing in a field than they do the productivity of faculty (e.g., publications/faculty).