NPR Science Correspondent Richard Harris spent a week at UW in January 2018, and some of us (including postdoc leadership) were able to spend time with him. His recent book brings together much of the published literature and personal stories about the “reproducibility crisis ” in research, biosciences in particular.
We pressed him to identify possible solutions, and — as this is a multi-faceted issue — there are several. As postdocs, we are the current and next-gen researchers and is truly up to us to collectively shift research culture and practice.
We feature a few of the top issues individual researchers can take on now, and you may consider how they can play into your current or next phase of work:
Seek diverse opinions. When developing a research methodology, study design, statistical analyses, or interpreting results for publication, consult with several different people with diverse expertise, experience and backgrounds. Daily judgment calls are made in research, and as you develop your best approach and continue your own training, it can help to get a wide breadth of input.
Be a good scientific citizen. There is a broader movement toward open science with the goal of accelerating progress, minimizing waste and identifying errors to improve our collective learning and potential impact. Since negative results, null findings, or reproductions of experiments are not published in peer-review journals (though Richard Harris says journals are changing practice on this and we need to catch up), it can help to have back channels to share findings, data sets, and analytic strategy so your field can move forward and we can use less research dollars on ineffective studies. There are badges and ways of annotating your CV that can help demonstrate your citizenship.
Get beyond impact factors. The editors of top-tier publications have banded together to speak against the use of impact factors in hiring or promotion decisions, as they don’t mean what we have come to make them mean within academia. Some Schools and Colleges, Universities, and even the NIH, are evolving their criteria to look for a broader spectrum of metrics to demonstrate real impact of your work. Whether your results allowed a research group across the country to move ahead and make a breakthrough, or your publicly shared findings resulted in a policy change, there are other ways to track the reach of your work. When seeking future positions — inside or outside academia — ask about promotion criteria. What things are genuinely valued? If the department is still just counting papers or dollars, and you do not share those values, you may do well to keep looking.
UW graduate student and Lindau Nobel Laureate Fellow Blythe Adamson wrote a book review of Harris’ work and summarized his tips for young scientists:
- Use valid ingredients.
- Show your work.
- No HARKing (Hypothesizing After the Results of the study are Known).
- Don’t jump to conclusions (and discourage others from doing this with your results).
- Be tough. People may try to discredit you if your hypothesis goes against their life’s work, or for any number of reasons.
- Be confident in your science.
- Recognize the tension between your own achievement and communal scientific advancement.
The more we talk together as a community, the more we learn, avoid reproducing outdated assumptions about what works, and begin shaping our collective futures. We encourage you to start a conversation in your research group or department, or come chat with us about any of these issues if you are looking for a way to get started! Office hours are always open.