The proliferation of crap content and the rise of content reputation systems


For a number of years I’ve talked about how we are effectively reaching a world infinite content, and the implications of that. That is becoming more real by the day, as in an economy increasingly driven by search and links, people find new ways to generate content that participates in this new information infrastructure.

I wrote last year about Philip Parker, who created programs that have automatically generated 200,000 books by aggregating and structuring content on the web. I haven’t read any of the books, but I’m told that they are – unsurprisingly – pretty poor, though of possible value to some people. However this is probably at the quality end of the spectrum of auto-generated content. For many years blog spammers have been auto-generating blog posts which have plausible language constructions, so they are picked up by search engines, but in fact are nonsense.

Adding to the morass of content are non-native speakers who lack background and context writing articles that are far more coherent than anything generated by computers, but which are still basically crap i.e. a waste of time to read.

Michael Arrington has written a pessimistic post titled The End of Hand Crafted Content, writing about what he calls fast food content, saying:

My advice to readers is just this – get ready for it, because you’ll be reading McDonalds five times a day in the near future. My advice to content creators is more subtle. Figure out an even more disruptive way to win, or die. Or just give up on making money doing what you do. If you write for passion, not dollars, you’ll still have fun. Even if everything you write is immediately ripped off without attribution, and the search engines don’t give you the attention they used to. You may have to continue your hobby in the evening and get a real job, of course. But everyone has to face reality sometimes.

Forget fair and unfair, right and wrong. This is simply happening. The disruptors are getting disrupted, and everyone has to adapt to it or face the consequences. Hand crafted content is dead. Long live fast food content, it’s here to stay.

I am far more sanguine. What is required, and what will inevitably happen, is the rise of effective content reputation systems, that allow you to assess the likely quality of articles before you read them or even find them. This was one of the ideas we explored at our Future of Influence Summit, held August 31 in San Francisco and Sydney, and I’ve written about one aspect of this in an article Who will provide the credibility ratings for the journalists of the planet?

One of the primary issues in a content reputation system is that it must be easily accessed and embedded, especially so that aggregators can use the data in assessing which articles to include. The system needs to be very open to be effective. Segregated systems will be of limited value. This will slow the development of content reputation systems, however I believe they will still proceed apace.

This doesn’t mean that there won’t still be an infinite amount of crap content, and among it a wealth of extremely valuable content. However we will far more easily distinguish between them, making the business models for creating crap content more marginal, and increasing the value and rewards for quality content.

  • What does this mean for my hand crafted blogsite?
    Do I have to be a part of Technorati and Digg to get a reputation? These two sites offer no benefit to me and my niche market, so must I rely on real people within my Twitter and LinkedIn networks to promote and trust my content?

  • I really hope it doesn’t come down to having to join credibility guilds just to prove ones content worth. very elitist – and once again brings up the question of who’s accrediting the “credible”?

  • Though I agree that reputation systems are often the missing component in content evaluation, it is not at all clear how to implement a generally applicable solution.
    In our upcoming O’Reilly/Yahoo! Press Book: Building Web Reputation Systems, Bryce Glass and I point out that all reputation takes place in a context.
    There is no global, all content, context.
    There is a lot of work ahead.

  • Thanks for your comments!
    Wedge: Content’s value should become evident – but there are things you can do if you think it’s been overlooked… Twitter really is the most important single platform for this now.
    Keith: I certainly don’t believe in closed systems for content rating – these will be primarily unbounded, though there might be a role for the ‘guilds’ you mention – interesting concept.
    Randall: I very much look forward to reading your book! Yes there are no easy answers here, but you, I, and many others will be working hard at it in years to come – the degree of value from these justifies whatever it takes to get there.

  • will such systems be enough – early enough – to save traditional newspaper journalism?