Jarrett’s Scale
Seriously flawed
Major science concerns
Minor science concerns
Science is good, but poorly written
Science is good, text needs minor cleanup
Ready for publication
Outstanding paper in the field

Cameron’s scale
This paper:
is seriously flawed – 1
has some technical issues – 2
is useful – 3
is a significant contribution – 4
is an outstanding contribution to the field -5

We all agree on Cameron’s scale. The slightly more vague scale actually has some very large advantages.

How does this contribute to Karma?
Reviews do not reduce Karma
But a score of 1 does not add to Karma
Nonlinear function of scores

We need to come up with a scoring formula for papers!

Activities Generating Karma

– adding tags
– suggesting tags
– moderating comments
– moderating tags
– commenting
– voting up a comment
– voting down a comment
– having a comment voted up
– having a comment voted down
– posted a paper
– reviewing a paper
– scores from review: review formula – ratings normalized by review score and maybe reviewer
– voting up a review
– voting down a review
– having a review voted up
– having a review voted down
– author upvotes your review
– author upvotes your comment
– moderating users
– using a social media button

Privileges that Karma Unlocks
– Post a paper (0 Karma)
– Post a comment (0 Karma)
– Post a review (0 Karma)
—
– Vote up a comment
– Vote up a review
—
– Vote down a comment
– Vote down a review
—
– Flag a paper
– Flag a tag
– Flag a comment
– Flag a review
– Flag a user
—
– Add a tag to someone else’s paper
—
– Create a tag (starts 3-4 monts in)
—
– moderation priviledges

Add a feedback email link

What do we want to achieve from a survey?

The purpose/motivation is

1) Increase speed of the scholarly communication process

2) Provide a place for preprints and other works

3) Demonstrate the value of conversation as it relates to scholarly works

4) Demonstrate the value of assigning value to review

5) Demonstrate the value of preprints

General Questions

Did you find all of this useful as an author?

Did you find all of this useful as an reading consumer?

Did you find all of this useful as an actively participating consumer?

Do you now feel more engaged and interested with preprints, conversation, and the review process?

Which of these features are useful?  Which are not? Tear it down to each piece?

What other features would you find useful?

Has this changed you as a participant in the scholarly communications process?

How active are you? As a reader? As a participant? As an author?

Issues to consider

When do we survey people? Email their survey after their account is 6 months old.

Prod people to participate if they haven’t held up their end of the bargain? Don’t show karma until they have done the thing

Our goal is to establish a baseline of your opinions on the current state of scholarly communication for our fields so as to highlight potential gaps and improvements.

Please, head on over and take the survey. We want your input. We’ll be using it both to write up a paper, and inform a few future projects from the group.

Thursday:

The Eco-Preprint-Commentary Project
– Ed presents what has been done (morning)
– Open discussion on the project

Afternoon, small groups
-designing a post-use survey for the preprint server
– devs continue work based on morning discussion

Friday
Morning
– plenary bringing survey and devs together, make sure goals are aligned

Afternoon small groups
– devs continue work based on morning discussion
– Strategize roll-out of project and assign roles

Saturday
– final presentation by developers
– Strategize roll-out and Science Online presentation
– Discuss future administration of project
– Discuss funding of future OpenPub efforts

Here’s our whiteboard sketch to keep in the back of your head.