Etikettarkiv: peer review

Peer review glossary

Publicerat i english, så kallad humor, the practice of science av

‘Misleading’ — not exactly as I would have written it

‘Somewhat confusing’ — using terminology from adjacent sub-subfield

‘Confusing’ — completely illegible

‘Poorly structured’ — not exactly as I would have written it

‘Conversational’ — in need of adjectives

‘Descriptive’ — using technology that isn’t fashionable anymore

‘Potentially’ — definitely

‘by a native English speaker’ — by the Microsoft Word spell checker

‘due to insufficient enthusiasm’ — because it’s trite

‘gratefully’ — begrudgingly

‘adequate’ — perfunctory

‘constructive’ — fairly polite

Various positions

Publicerat i dear diary, english, the practice of science av

What use is there in keeping a blog if you can’t post your arbitrary idiosyncratic opinions as if you were an authority? Here is a list of opinions about life in the scientific community.

Social media for scientists

People who promote social media for scientists by humblebragging about how they got a glam journal paper because of Twitter should stop. An unknown PhD student from the middle of nowhere must be a lot more likely to get into trouble than get on a paper because of Twitter.

Speaking of that, who thinks that that writing an angry letter to someone’s boss is the appropriate response to disagreeing with someone on Twitter? Please stop with that.

Poster sessions

Poster sessions are a pain. Not only do you suffer the humiliation of not begin cool enough to give a talk, you also get to haul a poster tube to the conference. The trouble is that we can’t do away with poster sessions, because they fulfill the important function of letting a lot of people contribute to the conference so that they have a reason to go there.

Now cue comments of this kind: ”That’s not true! I’ve had some of my best conference conversations at poster sessions. Maybe you just don’t know how to make a poster …” It is true that I don’t know how to make a good poster. Regardless, my ad hoc hypothesis for why people say things like this is that they’re already known and connected enough to have good conversations anywhere at the conference, and that the poster served as a signpost for their colleagues to find them.

How can one make a poster session as good as possible? Try to make lots of space so people won’t have to elbow each other. Try to find a room that won’t be incredibly noisy and full of echos. Try to avoid having some posters hidden behind pillars and in corners.

Also, don’t organize a poster competition unless you also organize a keynote competition.

Theory

There is way way way too little theory in biology education, as far as I can tell. Much like computer programming — a little of which is recognized as a useful skill to have even for empirically minded biologists who are not going to be programming experts — it is very useful to be able to read a paper without skipping the equations, or tell whether a paper is throwing dust when it states that ”[unspecified] Theory predicts …” this or that. But somehow, materials for theory manage to be even more threatening than computer documentation, which is an impressive feat. If you disagree, please post in the comments a reference to an introduction to coalescent theory that is accessible for, say, a biology PhD student who hasn’t taken a probability course in a few years.

Language corrections

That thing when reviewers suggest that a paper be checked by a native English speaker, when they mean that it needs language editing, is rude. Find a way of phrasing it that won’t offend that one native English speaker who invariably is on the paper, but doesn’t have an English enough name and affiliation for you!

Peerage of Science Reviewer Prize 2017

Publicerat i dear diary, english av

I won a prize! Hurrah! I’m obviously very happy.

If you want to hear me answer a couple of questions and see the Peerage of Science crew engaged in some amusing video editing, look at the interview.

How did that happen? After being told, about a year ago to check out the peer review platform Peerage of Science, I decided to keep reviewing manuscripts that showed up and were relevant to my interests. Reading and commenting on unpublished manuscripts is stimulating, and I thought it would help improve my reviewing and, maybe, writing.

Maybe this is a testament to the power of gamification. I admit that I’ve occasionally been checking my profile to see what the score is even without thinking of any reviewer prize.

Peerage of science, first impressions

Publicerat i english, med mera, the practice of science av

After I wrote a post about reviewing papers, Craig Primmer suggested on Twitter that I look into Peerage of Science. Peerage of Science is a portal and community for peer review. It has a lot of good ideas. It decouples reviewing from journal submission, but it is still made for papers aimed to be published in a conventional journal. It collects reviewers and manuscripts from a different fields in one place, allows interested reviewers to select papers they want to review, and provides anonymity (if the authors want it). I once wrote a few sentences about what I thought ”optimal peer review” would be like, for a PLOS early career researchers’ travel grant. (I did not get the grant.) My ideas for better peer review were probably not that bright, or that realistic, but they did share several features with the Peerage of Science model. Naturally, I was interested.

I’ve tried reviewing for Peerage of Science for a couple of months. My first impression is that it seems to work really well. The benefits are quite obvious: I’ve seen some of the papers get more reviews than they would typically get at a journal, and the reviews usually seem no less elaborate. The structured form for reviewing is helpful, and corresponds well with what I think a good review should be like. I think I’ll stick around, look out for the notifications, and jump in when a paper is close to my interests. I really hope enough people will use Peerage of Science for it to be successful.

There are also downsides to this model:

There seems to be an uneven allocation of reviewer effort. Some papers have a lot of reviewers, but some have only one. Of course, only the people at Peerage of Science know the actual distribution of reviews. Maybe one reviewer processes are actually very rare! This is a bit like post-publication review, except that there, you can at least know who else has already commented on a paper. I know some people think that this is a good thing. Papers that attract interest also attract scrutiny, and thus reviewer effort is directed towards where it is most needed. But I think that in the ideal case, every paper would be reviewed thoroughly. This could be helped by an indicator of how many other reviewers have engaged, or at least already posted their essays.

There is also the frustration of coming late to a process where one feels the reviewers have done a poor job. This was my first experience. I joined a review process that was at its last stages, and found a short, rather sloppy review that missed most of what I thought were the important points, and belaboured what I thought was a non-issue. Too late did I realize that I could do nothing about it.

Who reviews the reviewers? The reviewers do. I see the appeal of scoring and weighting reviews. It certainly makes reviewing more of a learning experience, which must be a good thing. But I feel rather confused about what I am supposed to write as reviewer feedback. Evidently, I’m not alone, because people seem to put rather different things in the feedback box.

Since the Peerage of Science team have designed the whole format and platform, I assume that every part of the process is thought through. The feedback forms, the prompts that are shown with each step, the point at which different pieces of information is revealed to you — this is part of a vision of better peer review. But sometimes, that vision doesn’t fully make sense to me. For example, if the authors want to sign their manuscripts, Peerage of Science has the following ominous note for them:

Peerage of Science encourages Authors to remain anonymous during the review process, to ensure unbiased peer review and publishing decisions. Reviewers usually expect this too, and may perceive signed submissions as attempts to influence their evaluation and respond accordingly.

Also, I’d really really really love to be able to turn down the frequency of email notifications. In the last four days, I’ve gotten more than one email a day about review processes I’m involved in, even if I can’t do anything more until after the next deadline.

Reviewing, postscript

Publicerat i english, the practice of science av

Later the same day as the post on reviewing was published, I saw the paper by Kovanis and coworkers on the burden of peer review in biomedical literature. It’s silly of me that it didn’t occur to me to look for data on how many papers researchers review. Their first figure shows data on the number of reviews performed 2015 by Publons users:

kovanis_reviewers_figure

Figure 1B from Kovanis & al (2016) PLOS ONE (cc:by 4.0).

If we take these numbers at face value (but we probably shouldn’t, because Publons users seem likely to be a bised sample of researchers), my 4-6 reviews in a year fall somewhere in the middle: on the one hand, more than half of the researchers review fewer papers, but it’s a lot less than those who review the most.

This paper estimates the supply and demand of reviews in biomedical literature. The conclusion is lot like the above graph: reviewer effort is unevenly distributed. In their discussion, the authors write:

Besides, some researchers may be willing to contribute but are never invited. An automated method to improve the matching between submitted articles and the most appropriate candidate peer reviewers may be valuable to the scientific publication system. Such a system could track the number of reviews performed by each author to avoid overburdening them.

This seems right to me. There may be free riders who refuse to pull their weight. But there are probably a lot more of people like me, who could and would review more if they were asked to. A way for editors to find them (us) more easily would probably be a good thing.

Morning coffee: reviewing

Publicerat i english, morning coffee av

20160417_125609

(It was a long time since I did one of these posts. I’d better get going!)

One fun thing that happened after I received my PhD is that I started getting requests to review papers, four so far. Four papers (plus re-reviews of revised versions) in about a year probably isn’t that much, but it is strictly greater than zero. I’m sure the entertainment value in reviewing wears off quite fast, but so far it’s been fun, and feels good to pay off some of the sizeable review debt I’ve accumulated while publishing papers from my PhD. Maybe I’m just too naïve and haven’t seen the worst parts of the system yet, but I don’t feel that I’ve had any upsetting revelations from seeing the process from the reviewer’s perspective.

Of course, peer review, like any human endeavour, has components of politics, ego and irrationality. Maybe one could do more to quell those tendencies. I note that different journals have quite different instructions to reviewers. Some provide detailed directions, laying out things that the reviewer should and shouldn’t do, while others just tell you how to use their web form. I’m sure editorial practices also differ.

One thing that did surprise me was when an editor changed the text of a review I wrote. It was nothing major, not a case of removing something inappropriate, but rewording a recommendation to make it stronger. I don’t mind, but I feel that the edit changed the tone of the review. I’ve also heard that this particular kind of comment (when a reviewer states that something is required for a paper to be acceptable for publication) rubs some people the wrong way, because that is up to the editor to decide. In this case, the editor must have felt that a more strongly worded review was the best way to get the author to pay attention, or something like that. I wonder how often this happens. That may be a reason to be even more apprehensive about signing reviews (I did not sign).

So far, I’ve never experienced anything else than single-blind review, but I would be curious to review double-blinded. I doubt the process would differ much: I haven’t reviewed any papers from people I know about, and I haven’t spent any time trying to learn more about them, except in some cases checking out previous work that they’ve referenced. I don’t expect that I’d feel any urge to undertake search engine detective work to figure out who the authors were.

Sometimes, there is the tendency among scientists and non-scientists alike to elevate review to something more than a couple of colleagues reading your paper and commenting on it. I’m pretty convinced peer review and editorial comments improve papers. And as such, the fact that a paper has been accepted by an editor after being reviewed is some evidence of quality. But peer review cannot be a guarantee of correctness. I’m sure I’ve missed and misunderstood things. But still, I promise that I’ll do my best, and I will not have the conscience to turn down a request for peer review for a long time. So if you need a reviewer for a paper on domestication, genetic mapping, chickens or related topics, keep me in mind.