This hasn’t been formalized into an academic context, but iNaturalist provides an interface for microarticles.
Microarticles for Observations
2
Category Archives: From the meeting
Enabling Efficient Discovery via Linking Information
The discussion focused around the attributes of scholarly projects and the comments and people associated with them that can then be used for discovery tools.
Attributes that describe a single work that can be used to enable discoverability of new works
- who is reading it
- properties of people reading it, and of the authors (university, location, field, etc.)
- tags (user generated and author generated)
- citation network, and network of citations to reviews/responses
- reading habits of people participating in a discussion of a paper
- geocoding
- associated funding
- corpus of a paper
- associated social media and habits of social media
How do we use those attributes to enhance discoverability?
- Aggregation by tags, sorting by scores or other properties – date, time, etc.
- Build networks of influence based on group of people, then see what the ‘most influential are reading/reviewing, or build a network of papers based on a circle of people and their reading habits
- You want to find things that are near each other in a given ‘space’ – can use information for people as well as information about a product and its content
- Must have ability to stumble on something highly related, even if not reviewed.
- Concentrating on pieces that bridge multiple fields and seeing where else they go
- Visualize connections of these tags
- Overlays of a paper that convey some of this information
Reputation & Credit
What are ways that people can accumulate reputation & credit for participating in the scholarly publishing enterprise? If we start by assuming the open reviewing system of yesterday, what are ways things can be ‘scored’ to give people public reputation?
Products need to accumulate reputation via use and re-use:
- citations
- pagerank (2nd and 3rd generation citation) – how are papers that cite a paper themselves then cited to make a paper pagerank
- altmetrics
- reader and reviewer opinion
- Question: can you accumulate negative things?
- Qualifying all of this by kind of contributions
A lot of this information can be culled from total-impact.org and altmetric.com.
These are all evidence of use & re-use, but give different information. We tossed around the idea of creating an aggregate number, but agreed that fine-grained information HAS to be there. What does an aggregated number really mean?
Reviews need some different metrics to be assessed, so that reviewers can be assessed. The must have metrics associated with re-use, reader scores, etc. But also –
- utility to the author
- how do they lean on a paper…keep track of rejecty rejecterson
- reader score – does it weight the review’s use and utility to an author?
- ‘Badging’ for activity – track quantity
Chris: also maybe something about last 5-paper read, or reading history? But privacy issues abound.
But – other people who read this, also read these…
What are the elements of peer review 2.0?
The whiteboard from our discussion of what should be part of our ideal peer-review system:
- continuous
- community rejection—the “quack” button
- “reputation” for reviewing quality, meta-reviews
- a review is a citable object (that therefore can be peer-reviewed)
- scoring:
- different levels of depth: review (continuum & content) vs. reader score
- commenting: no score assigned
- anchored reviewing (line #/annotation): can review pieces of an article
- notification after revision, old versions marked as deprecated
- versioning: what is the user interface to prevent confusion about the timeline?
- private review possible
- comments tied to a single account
- conflict of interest for scored reviews
Preprint server directory
OSTI (Office of Scientific and Technical Information) maintains the Eprint Network, which lists available preprint archives by subject. One subject is environmental sciences and ecology and they list many.
http://www.osti.gov/eprints/pathways/environmentalsciencesA.shtml
An idea about a rough do-able now experiment
Area51.stackechange is a site for people to propose nascent mathoverflow-like sites. As a first pass experiment on open review and a reputation economy (while we develop or find a way to develop a more sophisticated model) could we get the community to support an Ecology Preprint stackechange? We could have NCEAS host preprint pdfs from validated authors (i.e., have a ecopreprint.stackechange user account) and an individual ‘question’ would concern people’s comments on a paper.
We’d ask the community to volunteer to post preprints there fore at least, say, 2 months for submission, and given them the option of including their ‘review’ trail, etc. when they submit their paper to a journal.
Users would shape things like tags, etc. as the do on any stackoverflow site.
Granted, ‘review’ would be public, with usernames revealed. But, it’s an experiment.
After running this for ~6 months, we survey participants about their experience.
Thoughts?
Comment from Dave: You’d need some journals and high profile people to buy-in before doing this.
The Pillars of “There”
If we are to go from here to there in scholarly publication, what is there? This was the question of the afternoon, and we ended up agreeing on four broad pillars. The details of some of these pillars is up for debate, but, there was general consensus on the Four Pillars of There, as it were.
1) More types of products than a narrative paper
2) Preprints must become part of our culture
– We talked a lot about fraud/plagarism/quack detection. It could be handled by people, or by crowsourcing quick abuse reporting.
3) Public review
– What this means was unclear. Anonymity, meta-review, etc. are all up for grabs. Experiments needed.
4) Public reputation for activity
– A reputation economy of some sort with a number of different associated metrics for reviewing, meta-reviewing, authoring, etc.