Caron and Black have posted their paper, Ranking Law Schools: Using SSRN to Measure Scholarly Performance. Here’s the abstract:
There are several methods for ranking the scholarly performance of law faculties, including reputation surveys (U.S. News, Leiter); publication counts (Lindgren & Seltzer, Leiter); and citation counts (Eisenberg & Wells, Leiter). Each offers a useful but partial picture of faculty performance. We explore here whether the new “beta” SSRN-based measures (number of downloads and number of posted papers) can offer a different, also useful, albeit also partial, picture. Our modest claim is that SSRN-based measures can address some of the deficiencies in these other measures and thus play a valuable role in the rankings tapestry. For example, SSRN offers real-time data covering most American law schools and many foreign law schools, while citation and publication counts appear sporadically and cover a limited number of U.S. schools. The SSRN measures favor work with audiences across disciplines and across countries, while other measures are more law-centric and U.S.-centric. SSRN is relatively new and thus favors younger scholars and improving schools, while other measures favor more established scholars and schools. At the same time, the SSRN measures have important field and other biases, as well as gaming risks. We assess the correlations among the different measures, both on an aggregate and on a per faculty member basis. We find that all measures are strongly correlated; that total and per faculty measures are highly correlated; and that SSRN measures based on number of papers are highly correlated with measures based on number of downloads. Among major schools, all measures also correlate with school size.
Here’s my thought: I agree that SSRN downloads are a relevant statistic. But it has holes, as do each of the alternatives. So why not a composite statistic for each faculty, which would then be aggregated into faculty quality? Combining the data and adding total publications would yield a useful third statistic.
For example, I do pretty well in SSRN (around 30th all-time). But, you would say, not much of an achievement because I write in corporate (though not exclusively). So therefore my statistics are jacked up because there's all these people who download rather than write articles, or something. But I rank higher in articles posted. Probably because I'm one of those people who writes without thinking about what I'm saying, or just because I'm so old. But I also do pretty well in Westlaw cites (“larry w/2 ribstein” = 1246). But that's because I have treatises, and again because I'm so old - Westlaw gives me an advantage because of the viral nature of citations that accumulate over time.
So when do I start getting some credit for being REALLY FAMOUS? My informal check indicates that not many people rank above me in all the measures listed above. Not to blow my own horn or anything. My aim is modest: I just want a statistic that RANKS ME HIGH. Is that too much to ask?
Comments