The proliferation of crap content and the rise of content reputation systems

For a number of years I’ve talked about how we are effectively reaching a world infinite content, and the implications of that. That is becoming more real by the day, as in an economy increasingly driven by search and links, people find new ways to generate content that participates in this new information infrastructure.

I wrote last year about Philip Parker, who created programs that have automatically generated 200,000 books by aggregating and structuring content on the web. I haven’t read any of the books, but I’m told that they are – unsurprisingly – pretty poor, though of possible value to some people. However this is probably at the quality end of the spectrum of auto-generated content. For many years blog spammers have been auto-generating blog posts which have plausible language constructions, so they are picked up by search engines, but in fact are nonsense.

Adding to the morass of content are non-native speakers who lack background and context writing articles that are far more coherent than anything generated by computers, but which are still basically crap i.e. a waste of time to read.

Michael Arrington has written a pessimistic post titled The End of Hand Crafted Content, writing about what he calls fast food content, saying:

My advice to readers is just this – get ready for it, because you’ll be reading McDonalds five times a day in the near future. My advice to content creators is more subtle. Figure out an even more disruptive way to win, or die. Or just give up on making money doing what you do. If you write for passion, not dollars, you’ll still have fun. Even if everything you write is immediately ripped off without attribution, and the search engines don’t give you the attention they used to. You may have to continue your hobby in the evening and get a real job, of course. But everyone has to face reality sometimes.

Forget fair and unfair, right and wrong. This is simply happening. The disruptors are getting disrupted, and everyone has to adapt to it or face the consequences. Hand crafted content is dead. Long live fast food content, it’s here to stay.

I am far more sanguine. What is required, and what will inevitably happen, is the rise of effective content reputation systems, that allow you to assess the likely quality of articles before you read them or even find them. This was one of the ideas we explored at our Future of Influence Summit, held August 31 in San Francisco and Sydney, and I’ve written about one aspect of this in an article Who will provide the credibility ratings for the journalists of the planet?

One of the primary issues in a content reputation system is that it must be easily accessed and embedded, especially so that aggregators can use the data in assessing which articles to include. The system needs to be very open to be effective. Segregated systems will be of limited value. This will slow the development of content reputation systems, however I believe they will still proceed apace.

This doesn’t mean that there won’t still be an infinite amount of crap content, and among it a wealth of extremely valuable content. However we will far more easily distinguish between them, making the business models for creating crap content more marginal, and increasing the value and rewards for quality content.