What should a score be: words or numbers?
Should they relate to the current journal paragidm?
Technicalities v. importance – facet out the score?
– too complicated!
Do we let this emerge?
What kind of ratings would people like – later in the survey?
Discussion board topic on ratings

Jarrett’s Scale
Seriously flawed
Major science concerns
Minor science concerns
Science is good, but poorly written
Science is good, text needs minor cleanup
Ready for publication
Outstanding paper in the field

Cameron’s scale
This paper:
is seriously flawed – 1
has some technical issues – 2
is useful – 3
is a significant contribution – 4
is an outstanding contribution to the field -5

We all agree on Cameron’s scale. The slightly more vague scale actually has some very large advantages.

How does this contribute to Karma?
Reviews do not reduce Karma
But a score of 1 does not add to Karma
Nonlinear function of scores

We need to come up with a scoring formula for papers!