Digg and Potential User Rating System Problems

On Josh Porter’s excellent Bokardo blog, he recently wrote a post called Digg’s Design Dilemma. The post deals with some of the “gaming” of the system done lately at Digg, a social news site.

First, here is a background provided by Josh:

The most recent controversy happened on September 5th, when someone named jesusphreak posted Digg the Rigged?, an in-depth article exposing some of the curious details of recently-popular stories on digg. Many of the stories, jp pointed out, were dugg by members of the Digg Top 30, or the 30 most popular digg members (popular being measured by number of stories submitted that were promoted to the frontpage). The Top 30 includes Digg founder Kevin Rose.

This was not the first time that someone has pointed out this phenomenon. On April 18 of this year Macgyver at ForeverGeek posted Digg Army, which included screenshots of who dugg two recent articles on the site. Each article had the exact same 16 people digging it in the exact same order. Of the first 19, 18 were the same. Included in that list of people was, again, Kevin Rose. (for an in-depth history see Tony Hung’s excellent: A Brief History of the Digg Controversy)

So, is gaming something I need to worry about with the site I’m developing? I recently posted ideas about a user rating system, and many of the problems that Digg has had could pertain to my system. Of course, there are huge differences between the two sites (not the least of which is that Digg is wildly popular and mine is still in my head).

Josh correctly points out that we should not blame the users of Digg. We should blame the design that allows the users to behave this way. So, how can I make sure I don’t experience the same problems down the road?

Josh lists some of Digg’s “features in question” in his post. Let’s take a look at what I should watch out for.

Rankings list
If you want people to compete, rank them. This is a big part of the reason why there is gaming on Digg. Getting a higher ranking becomes an incentive to game because if you do then others will notice and you’ll get recognition. (that’s important to people, even in social software) In addition, with the recent offer by afformentioned Calacanis to pay people for this type of work, high rankings may also be a source of income.

This one opened my eyes. I wrote the other day about listing top contributors and even allowing them to post a little message (or advertisement) with their name as INCENTIVE to try to move up the charts. However, Digg’s user base is broad. Visitors to my site will be professionals within a specific field. Most users (I assume) will be interested in improving their work, not improving their own popularity.

Porter comments about Jason Calcanis’ offer to pay top Diggers. Could a high standing on my site be a potential source of income for the users? Not in the same way as a move from Digg to Netscape. But, a high standing could result in increased exposure which might lead to job offers, book offers, speaking offers, etc. But is that a bad thing? Probably not, since the benefits are all directly related to the community.

Friends feature
The Digg friends feature is the means by which the top users promote stories so quickly and with such synchronicity…

Right now, I’m not planning a “friends” feature. Of course, I’m not anticipating a user base the size of Digg. For my site, I think the community will be much smaller and much more manageable.

Exposing who diggs what
At the bottom of each digged entry is a list of people who have dugg it, and serves as the evidence that the two articles above used to expose the issue going on at Digg.

In my post, I actually addressed this, saying “And how do you deal with someone getting mad at someone else and giving them minus after minus every time they post? Well, make the rating history for a particular post/comment public. Any sneaky work can be policed by the community. The questionable ratings can be removed.”

Stories at a distance
It is very possible to interact on Digg, digging stories and burying others, without actually reading a story. That’s because Digg only shows summaries of posts. If you want to read a post, you actually have to click on them and go to the external site to do so. Many people will make this extra effort. But many people won’t.

I suppose my users could rate stories without actually reading them, but one thing worth pointing out is that rating them does not help THEM PERSONALLY in away way. But I suppose it could help friends they have within the system. But where this is a professional community, I don’t think too many of the users are going to know each other personally. Tendencies to assign a “+” to a certain users’ posts is bound to happen, but that’s probably going to happen because you like the poster’s work.

Ease of voting
While it takes extra effort to read posts, it takes almost no effort to digg them. This might be backward…digg is essentially making it possible to vote without knowing what you’re voting on. Although the digg feature is amazing, an excellent example of technology that makes our lives easier, it is also in danger of trivializing them.

My current site mockups essentially allow a user to rate a story or comment without reading it. The other day, I was considering an approach where a user was required to give a rating (+, –, or indifferent) with each comment they post. This was a way to really PROMOTE the user rating system. A way to ensure ratings are more relevent might be DISALLOWING rating unless you’re leaving a comment. Then again, this may just be a problem for a larger community. However, it’s something I need to think about.

Josh then talks about how Digg is a much different approach than something like del.icio.us. Del.icio.us is a social bookmarking site where users bookmark (similar to “digging” in some ways) a site, but it’s mostly for their own person use. But because it is a social site, users can view what others are bookmarking. So, you can bookmark sites for the purpose of sharing with others (for example, I could share a list of desktop app replacing web apps by sharing my bookmarks with the “webapp” tag).
Josh says:

The voting on Digg is in contrast to a site like Del.icio.us, where voting (saving a bookmark) is done more independently, often without having any idea whether or not someone else even viewed it, let alone voted on it. Del.icio.us isn’t immune to gaming, however, as there is a popular list, and it’s very easy to simply copy those bookmarks into your own, driving up the numbers just like on Digg.

I’m not sure there would be any benefit to hiding article and user ratings on my site. Perhaps I’m naive, but I don’t see much gaming going on with a professional community. However, it can’t hurt for me to start thinking about these things and employing backup plans if things ever start to get a little shady.

All in all, it was a great article by Josh and it made me realize that with a successful web app can come these very serious issues to worry about. When the integrity of the content can be questioned, it can be devastating for a social media site. Now I know a bit of what to look for.

Comments are closed.