21 November 2012

Bibliometrics

'Are more senior academics really more research productive than junior academics? Evidence from Australian law schools' by Vinod Mishra and Russell Smyth in (2012) Scientometrics1-15 examines
the relationship between academic seniority and research productivity through a study of a sample of academics at Australian law schools. To measure research productivity, we use both publications in top law journals, variously defined, and citation metrics. A feature of the study is that we pay particular attention to addressing the endogeneity of academic rank. To do so, we use a novel identification strategy, proposed by Lewbel (Journal of Business and Economic Statistics 30:67–80, 2012), which utilises a heteroscedastic covariance restriction to construct an internal instrumental variable. Our main finding is that once endogeneity of academic rank is addressed, more senior academics at Australian law schools do not publish more articles in top law journals (irrespective of how top law journals are defined) than their less senior colleagues. However, Professors continue to have greater impact than Lecturers when research productivity is measured in terms of total citations and common citation indices, such as the h-index and g-index.
'Academic Inbreeding and Research Productivity in Australian Law Schools' (Monash Department of Economics Discussion Paper 46/12) [PDF] by Mishra and Smyth compares
 the research productivity of inbred and non-inbred faculty employed at Australian law schools. The sample consists of 429 academics, employed at 21 law schools. To measure research productivity we use both articles and pages published in top law journals, defined in six different ways, as well as total citations and two different citation indices. We report results including, and excluding, publications in the home law review. We find evidence that silver-corded faculty outperform other faculty on one of the measures of research productivity, once the endogeneity of academic seniority and grant history is addressed, but this finding is not robust across alternative measures of research productivity. We find that there is no statistically significant difference between the research productivity of inbred and non-inbred faculty. This finding is robust to a range of different ways of measuring research productivity and alternative econometric approaches, including using two-stage least squares to address the endogeneity of academic seniority and grant history.
The authors conclude -
The main finding in this study is that there is no significant difference in the research performance of inbred and non-inbred staff. Hence, there is no support for either of the competing hypotheses presented earlier. While there are conceptual arguments suggesting that academic inbreeding could have a positive or negative relationship with research performance, and the existing empirical findings are not unanimous, intuitively we expected to find that academic inbreeding would have a negative effect on research performance. As Eisenberg and Wells (2000, p. 310) put it, when they told colleagues that they found a negative relationship between academic inbreeding and research performance in US law schools, “no-one seem[ed] surprised by the result”. 
Our findings are surprising in the sense they are negative results and refute popular (mis)conceptions about academic inbreeding. Over the last few years there has been increased recognition that scientific progress in several disciplines has been hampered by researchers‟ tendencies to confine negative results to the file-drawer (see Gumpenberger et al., 2012). An example in the empirical legal studies literature is the debate on capital punishment, in which Donohue and Wolfers (2005) have argued that “reporting bias” has distorted the debate by producing a situation where primarily only studies reporting a deterrent effect get published. We believe that our findings have value in that we use a rigorous methodology to reach results contrary to many recent studies on the relationship between academic inbreeding and research productivity and we do so in a different institutional context. 
Most recent studies which have found a negative relationship between academic inbreeding and research productivity have used data from countries and disciplines in which the proportion of inbred staff is relatively high (see eg. Horta et al., 2010; Inanc & Tuncer 2011). Even in the Eisenberg and Wells (2000) study, the proportion of academic inbreeding was high in the Ivy League schools. By contrast, the degree of academic inbreeding is relatively low in Australian law schools and findings in this setting add to the literature. 
Almost three decades ago, Wyer and Conrad (1984) published one of the first studies on this topic to use a multivariate model and also reached the conclusion that there was no statistical difference in the research productivity of inbred and non-inbred staff. Given that finding was “contrary to most previous results” the authors considered it “appropriate to summarize the methodological strengths of [their] study” (at p. 224). We do likewise. First, our results are robust to a number of alternative ways of measuring research productivity. Second, we focus on publications in top journals, variously defined, and citations to measure impact. This contrasts to most previous studies which have focused on the quantity, and not quality, of research. Third, we control for a number of factors likely to be correlated with research productivity. Fourth, the results are robust to the inclusion, or exclusion, of publication in the home law review. Fifth, we corrected for the endogeneity of seniority and grant history and the findings for inbreeding were robust. 
Thus far, studies of the academic inbreeding-research productivity nexus outside of the US have tended to focus on disciplines and countries in which there is a high prevalence of academic inbreeding. In terms of future research, there need to be more studies for countries and disciplines in which academic inbreeding is not as prevalent to verify the results here. The relationship between academic inbreeding and research productivity is just one aspect of the broader debate around academic inbreeding. There is also much speculation around the organizational effects of inbreeding. Compared with the larger literature on the relationship between academic inbreeding and research productivity, few studies have examined the organizational effects or institutional implications of academic inbreeding (Horta et al. 2010 is a recent exception). This is a topic that could usefully be the subject of future research.