Academic and government institutions in many countries usually offer generous
financial support to PhD students. A wildly invoked reason for such support is
that most candidates would not otherwise have the financial means to pursue
such degree. However, an evaluation of these policies based on their impact on
the publication performance of their beneficiaries has surprisingly remained
unexplored. Enter Hugo Horta, Mattia Cattaneo and Michele Meoli.

As these authors point out an evaluation along these lines is more important
than ever given that the number of PhDs has grown by an astonishing 40%
annually in OECD countries during the decade 1998-2008. All sorts of
speculations can be put forward when trying to advance an answer for this
question. On one hand, one may favor the hypothesis that those who get the
means to focus without distractions on research will certainly produce on
average an output of a higher quality. On the other hand, the pursuit of a PhD
degree even in the absence of financial support may be something that only
those who are truly committed to an academic career are willing to undertake.

The stage is set but the challenge is big. Note that the question presents a
textbook example of what researchers in social sciences refer to as “selection
bias”. Indeed, when comparing those doctoral candidates with financial support
with those without it, the former have normally a head start since it is likely that
those with better skills for the successful completion of a research project are
also on average the ones who have a larger probability of getting financial
support. Hence, a superior research output may entirely be the result of better
pre-PhD qualifications and have little to do with financial support.

By using a representative sample of all PhD holders residing in Portugal on
December 2009, Horta et al (2018) embark on a mission to solve this riddle.
Their first step is to address the potential bias mentioned above by building a
control group whose ex-ante probability of getting a PhD grant is similar to those
in the “treatment” group (that is, those who actually receive the grant). “Similar”
here should be understood as a control group that on average has similar
average values along three dimensions: prestige of the university they graduated
from before starting their doctoral studies, the proportion of subjects who
change their scholarly field, and the average number of pre-PhD degrees
obtained. The second step is now simple, since if one accepts these three
variables as comprehensive enough to address the above selection bias, a
straightforward comparison of average scientific publications and citations in
the two groups should do the trick. As it turns out, the authors find that average
output in the treatment group is significantly large both during and after the
completion of the PhD. For example, the average number of publications and
citations during their studies is 25% and 93% larger, respectively, for those
subjects in the treatment group.

Can we proffer a complacent “Mission Accomplished” given these results? Well,
not really. There are two major concerns that overshadowed an otherwise
flawless implementation. The first one has to do with an important omission in
the choice variables that shape the control group. Note that an emphasis is
usually given in evaluating applicants for PhD funding to their grades in their
bachelor/masters degree. It is thus likely that selection bias remains a problem.
The second concern represents a much more serious caveat. Many PhD
candidates may not get funding just because they do not apply for it in first place.
Their careers are focused on their current jobs and they chase a PhD degree to
enhance their job market options outside the academia. For this kind of student,
research output is less of a priority. This could actually be compatible with the
surprising finding that graduation from a prestigious university is correlated
with negative research performance for the students in the sample. Note that a
prestigious university is usually the choice of career-oriented students. The
authors simply address this by pointing out that “graduates from prestigious
international universities are not necessarily successful and productive”. It is
unfortunate that their bar in scrutinizing their own findings is not place a bit
higher.

Text: Iñaki Rodriguez Longarela, Stockholm Business School

Hugo Horta, Mattia Cattaneo & Michele Meoli (2018) PhD funding as a
determinant of PhD and career research performance, Studies in Higher
Education, 43(3), 542-570.