Enabling Efficient Discovery via Linking Information

The discussion focused around the attributes of scholarly projects and the comments and people associated with them that can then be used for discovery tools.

Attributes that describe a single work that can be used to enable discoverability of new works

  • who is reading it
  • properties of people reading it, and of the authors (university, location, field, etc.)
  • tags (user generated and author generated)
  • citation network, and network of citations to reviews/responses
  • reading habits of people participating in a discussion of a paper
  • geocoding
  • associated funding
  • corpus of a paper
  • associated social media and habits of social media

How do we use those attributes to enhance discoverability?

  • Aggregation by tags, sorting by scores or other properties – date, time, etc.
  • Build networks of influence based on group of people, then see what the ‘most influential are reading/reviewing, or build a network of papers based on a circle of people and their reading habits
  • You want to find things that are near each other in a given ‘space’ – can use information for people as well as information about a product and its content
  • Must have ability to stumble on something highly related, even if not reviewed.
  • Concentrating on pieces that bridge multiple fields and seeing where else they go
  • Visualize connections of these tags
  • Overlays of a paper that convey some of this information

Reputation & Credit

What are ways that people can accumulate reputation & credit for participating in the scholarly publishing enterprise? If we start by assuming the open reviewing system of yesterday, what are ways things can be ‘scored’ to give people public reputation?

Products need to accumulate reputation via use and re-use:

  • citations
  • pagerank (2nd and 3rd generation citation) – how are papers that cite a paper themselves then cited to make a paper pagerank
  • altmetrics
  • reader and reviewer opinion
  • Question: can you accumulate negative things?
  • Qualifying all of this by kind of contributions

A lot of this information can be culled from total-impact.org and altmetric.com.

These are all evidence of use & re-use, but give different information. We tossed around the idea of creating an aggregate number, but agreed that fine-grained information HAS to be there. What does an aggregated number really mean?

Reviews need some different metrics to be assessed, so that reviewers can be assessed. The must have metrics associated with re-use, reader scores, etc. But also –

  • utility to the author
  • how do they lean on a paper…keep track of rejecty rejecterson
  • reader score – does it weight the review’s use and utility to an author?
  • ‘Badging’ for activity – track quantity

Chris: also maybe something about last 5-paper read, or reading history? But privacy issues abound.
But – other people who read this, also read these…

What are the elements of peer review 2.0?

The whiteboard from our discussion of what should be part of our ideal peer-review system:

  • continuous
  • community rejection—the “quack” button
  • “reputation” for reviewing quality, meta-reviews
  • a review is a citable object (that therefore can be peer-reviewed)
  • scoring:
    • different levels of depth: review (continuum & content) vs. reader score
    • commenting: no score assigned
  • anchored reviewing (line #/annotation): can review pieces of an article
  • notification after revision, old versions marked as deprecated
  • versioning: what is the user interface to prevent confusion about the timeline?
  • private review possible
  • >comment as real person or pseudonym?
  • comments tied to a single account
  • conflict of interest for scored reviews

For Those Missing the Meeting: The Building Blocks of a New System

For both those who are here at the meeting and were not able to meet it, here are the fundamental areas that we’re going to discuss that we feel need to be pulled apart and put back together to see how we can improve the current system of scholarly communication.

Please, if you have any thoughts about any of these but are not here, drop me a line with relevant questions, notes, thought, etc.

  • What is a product?
  • The Review Process & Opening it Up
  • Credit and Reputation for Participation
  • Linking Information for Efficient Discovery
  • The Roles of Agents/Stakeholders
  • Distribution Mechanisms (publishers, libraries, societies, etc.)
  • Funding
  • Software & User-interface architecture

Preprint language at Science and Nature

Science

http://www.sciencemag.org/site/feature/contribinfo/faq/

“WHAT ABOUT MANUSCRIPTS THAT HAVE BEEN POSTED ONLINE BEFORE SUBMISSION?

We do not consider manuscripts that have been previously published elsewhere. Posting of a paper on the Internet may be considered prior publication that could compromise the originality of the Sciencesubmission, although we do allow posting on not-for-profit preprint servers in many cases. Please contact the editors for advice about specific cases. We provide a free electronic reprint service to authors that allows visitors to the authors’ web site free access to the published version of the Sciencepaper on Science Online immediately after publication.”

Nature

http://www.nature.com/authors/policies/confidentiality.html

“More details about the Nature journals’ pre-publicity policy:

Nature journal authors must not discuss contributions with the media (including other scientific journals) until the publication date; advertising the contents of any contribution to the media may lead to rejection. The only exception is in the week before publication, during which contributions may be discussed with the media if authors and their representatives (institutions, funders) clearly indicate to journalists that their contents must not be publicized until the journal’s press embargo has elapsed. Authors will be informed of embargo dates and timings after acceptance for publication of their articles.

Presentation and discussion of material submitted to a Nature journal at scientific meetings is encouraged, but authors must indicate that their work is subject to press embargo and decline to discuss it with members of the media. Authors are free to publish abstracts in conference proceedings and to distribute preprints of submitted or ‘in press’ papers to professional colleagues, but not to the media.

Occasionally, journalists and editors hear about work at talks given at scientific meetings and mention this work in meeting reports or editorials in their journals. In these cases, a Nature journal will assess the extent to which authors have solicited this interest or cooperated with journalists. If, in the judgement of the editors, the journal’s embargo policy has been broken, the submitted paper may be rejected, even if it is technically ‘in press’.

Contributions being prepared for or submitted to a Nature journal can be posted on recognized preprint servers (such as ArXiv or Nature Precedings), and on collaborative websites such as wikis or the author’s blog. The website and URL must be identified to the editor in the cover letter accompanying submission of the paper, and the content of the paper must not be advertised to the media by virtue of being on the website or preprint server. Material in a contribution submitted to a Nature journal may also have been published as part of a PhD or other academic thesis.

Taxonomic descriptions. Authors of papers that contain taxonomy (that is, the formal nomenclature and description of a newly discovered species) should be aware that it is possible for third parties to exploit the prior publication of nomenclature at any time between online posting of a preprint and the print publication date in a journal, by publishing the name in print and asserting priority according to the rules of the Code of Zoological Nomenclature. Nature Publishing Group takes no responsibility for such assertions of priority in the case of manuscripts it publishes if the content of those manuscripts have previously appeared in the public domain as online preprints or other form of online posting.”