A Preprint Experiment: Four Pillars and a Foundation for the Future of Scholarly Publishing

So, we got together, had two working group meetings to discuss the future of scholarly publishing in Ecology, Evolutionary Biology, and the Earth and Ocean Sciences. What were were thinking that entire time?

We’ve just submitted a piece that brings together our broad ideas (some of which have been seen before), but, simultaneous to publication, we’ve also decided to put up a preprint. Why? Simply put, immediate access is one of our four pillars of the future of scholarly publication. Once you feel something is ready for public consumption, put it out there! We’ve been delighted to watch the evolution of PeerJ Preprints, so we’ve placed our piece there.

Byrnes et al. (2013) The four pillars of scholarly publishing: The future and a foundation. PeerJ PrePrints 1:e11 http://dx.doi.org/10.7287/peerj.preprints.11

This immediate access to the piece goes hand in hand with another of our four pillars. Open Review. We want to know what you think. And now. We hope you give us feedback over at the preprint. Or, if you want to give us more detailed annotated comment, we’ve put it in a comment-open Google doc. Highlight something you disagree with. Argue with us. We welcome it! We’d ask that you put your name with the comment. We want a discussion, as discussion will improve this manuscript and help us shape our argument rather than just one-way commenting. This will also allow *you* to get full recognition for your comments, and we will include this in future acknowledgements.

So, enjoy the piece – our commentary is not a straight experiment-analysis-discussion piece, but rather part of a broader ecosystem of scholarly products that we feel are important to get out there. We look forward to hearing what you think of the piece!

Wrapping Up Meeting Two of the Future of Publishing Working Group

We’ve wrapped up the second meeting of the NCEAS Future of Publishing in Ecology, Evolution, & Evironmental Sciences working group, and it was tremendous. The group set out trying to merely examine issues of access in EEB (EEEB? or EEEOEB as we ended up expanding out to ecology, evolutionary biology, earth, ocean, and environmental sciences by the end – let’s just go with EEB for brevity). We soon realized that access alone is not the future, but rather opens up a wide range of other issues – from diversity of scholarly products to immediate access via preprints to the very review and discussion process itself. Indeed, we cemented four general principles that we feel must be addressed in the future of publishing fairly early on – a diversity of products besides just the narrative paper, immediate access to new work before even review, open review and commentary on that new work, and a robust system of reputation and recognition for participation in the publishing process. All of this leads towards a better underlying flow of information to aid people in the discovery of new work and new information – speeding along science at a faster clip.

The group has generated several products, some of which are still in progress, that all advance our goals of better understanding of the future of scholarly communication. Briefly, they are a special issue and new section in the journal Ideas in Ecology and Evolution; a survey of Editors in Charge regarding preprint practices to complement a crowdsourced exploration of what those practices are; a survey of EEEOEB scientists on their current publishing practices, and what they would like to see in the future; and last, an experimental preprint server that will test the community’s reaction to many of the ideas contained in that survey.

The Special Issue

We have launched a new section of Ideas in Ecology and Evolution on the Future of Publication. Chris Lortie is heading it up as section editor, and the section has already seen a number of thoughtful pieces ranging from Stefano Allesina’s piece on new ways of modeling peer review to Jeremy Fox taking on whether blogging can change how Ecologists share ideas. There are 5 more in the pipeline, so it’s a definite net win for this discussion.

The Survey

We finalized and launched our survey of scholarly publishing practices and needs during this second meeting after obtaining IRB approval and revising our survey questions to avoid ambiguity. The survey can be found at http://bit.ly/eebpublishing. In the first 24 hours, we received over 200 responses, and as of writing, are up to at least 550 click throughs. We’ve advertised on ECOLOG, evoldir, and AGU. Please, if you feel your community has not seen this survey yet, forward it around to your colleagues and any relevant mailing list.

We’ll be keeping it open for 2-3 months, at which time we will write up the results for publication. Should be interesting!

The OpenPub Project

Surveys and writing can only take us so far. As we identified the four pieces of the future of peer review we were most interested in seeing experimented with – a diversity of contribution types, immediate publication or preprints, open review and commentary on products, and a system for assessing the quality of participation – we realized the only way we could see if any of those ideas worked was to experiment with them.

And so, we are in the process of creating an experimental preprint server for the EEB community. Two of our working group members have significant experiment in development, and by this meeting have a toy version of the system up and running. There is much work to be done, and we spent much of this meeting nailing down a bare-bones feature set to test the ideas we’re most interested in. These include:

– Paper uploading
– Comments
More Detailed Reviews
– Revision
– Stable Identifiers
A Reputation and Karma System
– Discovery, Tagging, & Search
– User Pages
– Notification of action

Our tentative timeline is to have a beta together by September. We’re in the processes of finding funders and partners interested in the work so that we can fund continued development on our September timeline. Any suggestions are welcome. We have a sign up for an announcement list up now.

The idea behind the experiment is to engage our communities through a large-scale outreach campaign. Interested parties can sign up for the server, agreeing to a minimal level of participation, and agreeing to participate in a survey 6 months after joining. With the survey results, we hope to assess what works, what doesn’t, and what are some new avenues that we should be thinking about for the future of scholarly publishing.

We are currently reaching out to different journals and societies, letting them know about our efforts, and seeing if they have any interest in participating as partners, willing to accept both manuscripts and comments that have been posted on the server. Indeed, one of our goals here is to help existing journals by giving them better quality manuscripts and a ready pool of moderated opinions about the quality of the work.

If the experiment works, we hope that the server can continue on, providing a valuable service that is currently absent from the EEB community.

What are good toplevel categories for an EEB preprint service?

While preprints will be able to have tags assocaited with them (like keywords), we also want some toplevel categories that people can browse generic viagra by. Our initial ‘category’ list is as follows:

population ecology
community ecology
ecosystem ecology
evolutionary ecology
population genetics
conservation biology
animal behaviour

This would be for something like a fine-grained page, perhaps with some of those categories nested within bigger toplevel categories. Picture this like the breakup plos one does. We’d initially populate a list, but then after 2 months of usage, or so, see what tags had come up as new ‘categories’ through sheer commonness of usage.

For initial display on the front page, however, we wanted an even coarser grained list of Toplevel Topics. Here’s the list we ended up with. Picture a front page with 2-3 articles listed from each topic – say, the latest submitted, or hottest, or highest rated in the last week.

Evolutionary Biology
Environmental Sciences
Earth Sciences

This seems sufficient as toplevel topics…but, thoughts?

OpenPub Project Features and Rough Timeline

Frozen Feature Set for The OpenPub Project: Where Science Preprints are On Tap
– Paper uploading
– Comments
– Reviews
– Minimal Reputation
– Minimal Discovery, Tagging, & Search
– Revision
– Stable Identifiers
– User Pages
– UI
– Notification
– Logo – Amber&Mark

Build Timeline (roughly that will change)
Now – Paper uploading and commenting
March – Reviewing
April – Versioning and Identifiers
May – Tagging
June – Discovery/Search/Tagging
July – User Pages & Notifications
August – Reputation
September – DOI

UI Whiteboards

openpub, page 5

openpub, page 4

What do we need on the front page?
Lists of papers
Search box
Account features in a top bar
Login, etc.
New &Interesting/Hot(activity)/Top Rated

Point of the front page is to suck someone in…

Top level categories with the latest 3 in each category…
– can click to resort to hot and highest rated this week

Subheadings on Front Page
Evolutionary Biology
Environmental Sciences
Earth Sciences

Tags – 10 top tags in each toplevel category

Submit images that relate to their work? Cycle fun image randomly? Get random images…and papers on the side

Search…with google?

How will we rate papers in the system?

What should a score be: words or numbers?
Should they relate to the current journal paragidm?
Technicalities v. importance – facet out the score?
– too complicated!
Do we let this emerge?
What kind of ratings would people like – later in the survey?
Discussion board topic on ratings

Jarrett’s Scale
Seriously flawed
Major science concerns
Minor science concerns
Science is good, but poorly written
Science is good, text needs minor cleanup
Ready for publication
Outstanding paper in the field

Cameron’s scale
This paper:
is seriously flawed – 1
has some technical issues – 2
is useful – 3
is a significant contribution – 4
is an outstanding contribution to the field -5

We all agree on Cameron’s scale. The slightly more vague scale actually has some very large advantages.

How does this contribute to Karma?
Reviews do not reduce Karma
But a score of 1 does not add to Karma
Nonlinear function of scores

We need to come up with a scoring formula for papers!