Currency Design: Stones, Tumbles & Gems

This is a currency intended to be used by Communities of Practice.

 

This write-up awas originally provided by Arthur Brock as an exercise in providing feedback and helping develop the Omidyar.net community, though surely the information can be valuable for other communities as well.

 

Overview

People tend to think of any currency or points system as the balance of positive and negative values earned over time. But a reputation system may not be about "good" and "bad" but rather a way of extrapolating the kind of future value that somebody will provide, based on the value they've provided in the past.

 

For healthy and honest interactions in a community, it's very important for reputation data not to be merely personal or egoistic, but rather informational to support the community in fulfilling its purpose.

 

Omidyar.net Purpose

The home page of omidyar.net states: "We believe every individual has the power to make a difference. We exist for one single purpose: So that more and more people discover their own power to make good things happen."

Members may join or participate for a variety of reasons, but it seems clear that we want the community's activity to revolve around discovering your power, making a difference, and making good things happen.

In operational terms for an online community, the purpose might translate into:

1. Attracting and connecting like-minded individuals for brainstorming, collaboration, encouragement, sharing resources and building teams (successful)
2. Sharing good ideas, discussing ideas, creating a knowledge base of the most empowering ideas (successful)
3. Creating, supporting and accomplishing real-world results on projects making good things happen (could be stronger)
4. Leveraging the knowledge and wisdom of thousands of community members to find and filter projects appropriate for the Omidyar Network to fund and/or support (could be stronger)

My personal assessment is that Omidyar.net has done quite well with #1 for doing so little marketing of the site, and is well on the way with #2 with numerous active and high-quality discussions. However, further progress may be impeded by the sheer mass and volume of participation (past, present & future).

 

Signal to Noise Filtering: a "non-reputation" rating

A number of online communities have emerged and then fallen under their own weight as they attracted more people and couldn't maintain the same quality that originally attracted people. Walking into an online community can feel like being plopped into the middle of a packed stadium with everybody talking all at once. It is virtually impossible to figure out what's going on, whom to listen to, where the best conversations are happening and how to find the people it would be valuable to connect with.

Luckily, there are some technical tools which help experienced users filter out noise such as keyword search tools, content ratings, hierarchal categorization, meta-tags, visible activity levels, and such. Unfortunately, most of these are currently incompletely or confusingly implemented on Omidyar.net.

• Search - Currently Great. The only additional enhancements would be to provide for more sophisticated searching/filtering on some of the other elements we're about to cover.
• Visible Activity Levels - Currently Good. Marking threads with new comments in bold, and having listings by activity level help to make sense of a chaotic world of active discussions.
• Hierarchal Categorization - Currently Confusing. Groups join other groups willy-nilly making for random and odd order. People name things very nonsensically (such as "Lost in the Food Chain" or "Targeted Currencies") and there doesn't seem to be a process for re-negotiating the naming or the hierarchal relationships to make them more orderly when clearer purpose for a group or thread emerges. (After writing that, I just discovered the ability to remove groups from groups. Cool.)
• Content Ratings - Currently Extremely Confusing. Ratings of content are are collapsed with the reputation system of people. The site should mobilize collective intelligence to have everybody be filtering for quality and relevance as much as possible, but instead you have to spend scarce points from your "bank." A high volume of quality comments doesn't boost the reputation of the thread (and it should). Also, the reputation of a thread or workspace decays rapidly so that there is no way to differentiate past excellence from stuff that has always been garbage. This is the opposite effect of what would be desirable for working toward building a persistent knowledge base of the best ideas and practices. See next section for details.
• Meta-Tags / Folksonomy - Currently Missing. This could be an alternative for people to navigate random naming practices (and search for appropriate terms) and for groups/threads to refine their purpose and focus on a continually evolving basis.

Content Ratings as a Signal/Noise Filter: One approach to helping reduce information overload is to adapt something similar to the Slashdot rating system for O.net content. Let everybody vote on each thread, post, and workspace. On a post in a thread, people could give a 1-5 rating for the quality of the post (and a special tag like funny, insightful, or important) as well as rating the relevance of the post in forwarding the intent of the thread. This acknowledges people posting a valuable tangent, but allows readers who want to filter for relevance to stay on topic. People could set their personal preferences for filter levels when viewing content. Quality content does not need to decay, but rather be distilled into a persistent knowledge base to make it easier for future readers to find high-quality content.

Please note that ratings as a signal to noise filter are not really a "reputation" system. Rating a post does not imply any future expectations one should have about that post. It isn't going to re-write itself. I don't believe there is any reason why you'd want to discourage people from doing the work for you of helping filter signal from noise. Give them all the votes or points they need to do that job. The ability to do so should not be doled out in scarce quantities.

 

Actual Reputation Ratings (for People)

Mapping Non-Reputation onto Reputation: Content ratings/filters as described are not really a reputation since they aren't predictive, but we can derive some predictive information from cumulative ratings for individuals, threads, and groups. There are identifiable patterns, some people may have overall high quality ratings, low relevance ratings, or high activity ratings. Some groups may have consistently high-quality discussions or workspaces. Threads which have a number of highly ranked responses may be ranked higher. Computers are good at doing these kinds of calculations and statistics for us. It's a great way to get extra mileage out of the work that people invest in the signal/noise filtering.

Rating Participation and Performance not Personality: It is possible to set up reputation systems which can give people who are doing the rating greater freedom and latitude to be honest and forthcoming. People in a collaborative community are typically careful about offending each other. It is obviously less offensive to say "You scored 62% on this test," than to say "You suck at math." Clear and descriptive feedback is not only easier to give, but also more useful to the person receiving it. When you ask people to rate people, make the questions very specific for more useful information.

A well-segmented reputation provides the most valuable information about community members. The following is an example of how you might structure feedback systems in a community of practice distinctly to identify different types of strengths - providing original ideas, building community and inciting participation, and translating ideas into successful action.

 

Stones - Originality / Ideas / Eloquence

Balance Calculation - Average Percentile Ranking of most recent 6 months (Average Post Rating, Number of Quality Posts, Number of Tagged Posts, Number of Citations in Initiatives)

Balance Display - 0-10 scale (add 1 and divide by 10 rounded to nearest integer)

All-Time Balance (for a hall of fame or such) removes 6-month limit

• Average of Post Ratings
• Number of Posts (If Relevant=Y & Contribution > 6)
• Tagged for Knowledge Base
• Cited as building blocks in projects/initiatives

Tumbles - Participation / Processing / Community Building

Balance Calculation - Average Percentile Ranking of most recent 6 months of (# Posts rated > 3, # of Ratings, # of Taggings, # of friends)

Balance Display - 0-10 scale (add 1 and divide by 10 rounded to nearest integer)

All-Time Balance removes 6-month limit

 

Gems - Implementation / Practice / Action / Leadership

Gathered from:

• Posting Initiatives
• Completing Tasks/Actions
• Initiatives Producing Impact
• Initiatives Creating Participation

Sample Composite Reputation Display

Jane User-Doe (3s | 6t | 4g)

Reputation for Personal Relationship and Trust: It is also possible to include the personal dimension, but let it be defined by configurable relationships. For example, we could provide the means for people to add users to their "ignore list"to filter out their posts, or to select their top 10 users with street cred - the users whose ratings they trust the most and who they'd be most interested in reading whatever they're rating highly. We could not only let people jump to the things they're rating highly, but we can again aggregate information about these relationships to be reflected in their individual reputation.

 

Project Reputation / Timeliness, Effectiveness & Impact

 

Note: Initiatives is a term we coined for something like emergent projects. A place for people to start to frame out a project intention and actions that would need to be taken that would be associated with it. Ideally, these would include tools for identifying tasks which people could sign up for completing and then report on their completion. For an example of something handling initiatives well, you could take a peek at the way group commons works.

Distinct from the ratings of information or reputations of people, it is possible to create useful information for initiatives and projects.

Rating Initiatives:

• Number of people participating (joined?)
• Number of Active People (completed an action in the past month)
• Number of Supporting People (posted inside initiative or taken on a task in the past month)
• Subjective Initiative Ratings
• Impact 1-5 (How valuable do you see the impact of this project being?)
• Practicality 1-5 (How likely is this approach to succeed?)
• Innovation 1-5 (How innovative an approach is this?)
• Meeting Project Milestones/Goals (%) (specific & measurable)
• Impact Units Achieved (equalized by weighting factors) (displayable by category)
• Funding commitment ??