How Algorithms and Human Journalists Will Need to Work Together

By

Ever since the Associated Press automated the production and publication of quarterly earnings reports in 2014, algorithms that automatically generate news stories from structured, machine-readable data have been shaking up the news industry. The promises of this technology—often referred to as automated (or robot) journalism—are enticing: Once developed, such algorithms could create an unlimited number of news stories on a specific topic at little cost. And they could do it faster, cheaper, with fewer errors and in more languages than any human journalist ever could.

This technology provides an opportunity to make money creating content for very small audiences—even, perhaps, customized news feeds for an audience of just one person. And when it works well, readers perceive the quality of automated news as on par with news written by human journalists.

As a researcher and creator of automated journalism, I’ve found that computerized news reporting can offer key strengths. I’ve also identified important weaknesses that highlight the importance of humans in journalism.

Identifying automation’s abilities

In January 2016, I published the “Guide to Automated Journalism,” which reviewed the state of the technology at the time. It also raised key questions for future research, and discussed potential implications for journalists, news consumers, media outlets and society at large. I found that, despite its potential, automated journalism is still in an early phase.

Right now, automated journalism systems are serving specialized audiences, large and small, with very particular information, producing recaps of lower-league sports events, financial news, crime reports and earthquake alerts. The technology is constrained to these types of tasks because there are limits to what sorts of information it can take in and process into text that humans can easily read and understand.

It works best when handling structured data that is accurate like stock prices. In addition, algorithms can only describe what happened – not why, making it best for routine stories based solely on facts that have little room for uncertainty and interpretation, such as when and where an earthquake happened. And because the major benefit of computerized reporting is that it can do repetitive work quickly and easily, it is best used to cover repetitive topics that require producing a large number of similar stories, such as sporting event reports.

Covering elections

Another useful area for automated news reporting is election coverage—specifically regarding results of the numerous polls that come out almost daily during major campaigns. In late 2016, I teamed up with fellow researchers and the German company AX Semantics to develop automated news based on forecasts for that year’s U.S. presidential election.

The forecasting data were provided by the PollyVote research project, which also hosted the platform for publishing the resulting texts. We established a completely automated process, from collecting and aggregating the raw forecasting data, to exchanging the data with AX Semantics and generating the texts, to publishing those texts.

Over the course of the election season, we published nearly 22,000 automated news articles in English and German. Because they came from a fully automated process, the final texts often had errors, such as typos or missing words. We also had to spend much more time than we had expected troubleshooting problems. Most of the issues came from errors in the source data, rather than the algorithm – highlighting another key challenge of automated journalism.

Finding the limits

The process of developing our own text-generating algorithms taught us firsthand about the potential and limits of automated journalism. It’s crucial to make sure the data is as accurate as possible. And it is easy to automate the process of creating text from a single set of facts, such as the results of a single poll. But adding insights, like comparing that poll to others in the past, is much harder.

Perhaps the most important lesson we learned was how quickly we reached the limits of automation. When developing the rules governing how the algorithm would turn data into text, we had to make decisions that might seem easy for people to make – such as whether a candidate’s lead should be described as “large” or “small,” and what signals could suggest a candidate had momentum in the polls.

Those sorts of subjective decisions are very hard to formulate into predefined rules that should apply to any situation that has occurred historically – much less to any situation that might occur in future data. One reason is that context matters: A four-point lead for Clinton in the run-up to the election, for example, was normal, whereas a four-point lead for Trump would have been big news. The ability to understand that difference and interpret the numbers accordingly is crucial for readers. It remains a barrier that algorithms will have a hard time overcoming.

But human journalists will have a hard time outcompeting automation when covering routine and repetitive fact-based stories that merely require a conversion of raw data into standard writing, such as sports recaps or company earnings reports. Algorithms will be faster at identifying anomalies in the data and generating at least first drafts of many stories.

The ConversationAll is not lost for the people, though. Journalists have plenty of opportunities to take on tasks algorithms cannot perform, like putting those numbers in proper context – as well as providing in-depth analyses, behind-the-scenes reporting and interviews with key people. The two types of coverage will likely become closely integrated, with computers using their strengths and the humans focusing on ours.

Andreas Graefe, Endowed Sky Research Professor, Macromedia University of Applied Sciences This article was originally published on The Conversation.

Keynote slides: The Future of Work and Education

By

I had the great pleasure today of doing the opening keynote at CEE2017 Enterprising Minds Conference in Melbourne, organised by the Centre for Educational Enterprise, run by Melbourne Girls Grammar School.

My session pulled out to a very big picture view, starting with the key drivers of Acceleration, Society and Structure, delving into the disruption of Work and the resulting human Capabilities we need, and finally on to the fundamental shifts in Learning, Education and the resulting Leadership that is required.

The slides to my keynote are below. As always, my slides were designed to support my presentation and not to stand alone, but may be somewhat useful to those who weren’t present for my keynote. Many of the slides were in fact videos, in this deck only shown as images.

Read more

Everyone needs to understand the potential impact of AI and automation

By

On Tuesday ABC ran a prime-time special program The AI Race, supplemented by other content including analysis and an interactive tool on the impact of AI on jobs.

The program was excellent, looking at people working in a variety of jobs from truck driver to lawyer and how AI might impact them.

I appeared on the program as part of an ‘expert’ panel discussing the implications of automation on the future of work with a group of young people. The segments where I appeared are in the video below.


Read more

List of the top influencers in the future of work

By

Influencer lists should be taken with considerable caution, however they can be useful general indicators of those who are shaping conversations in a particular domain.

Atos Netherlands recently shared an analysis taken from Right Relevance of the top influencers in the future of work, shown below.


Read more

Should Google and Facebook Be Considered Media Companies?

By

Facebook and Google are making conscious efforts to distinguish themselves as technology companies rather than media companies. This distinction is important to define considering their vast reach and power to shape the future news landscape.

Examining their roles becomes even more noteworthy as major news organizations around the world seek to curb the power of this duopoly. This is in part to preserve the integrity of traditional journalism and to loosen the platforms’ grip on the global ad market.

In the US, major news publishers are seeking collective bargaining rights against the Facebook-Google duopoly. The effort is spearheaded by the News Media Alliance, an organization whose members include The New York Times, the Washington Post, and The Wall Street Journal, as well as numerous regional newspapers in the US.

Similar actions are being taken in Britain. The News Media Association, representing 1,100 British newspapers, wrote a letter to Parliament calling for the regulation of the two companies for their dominance over the ad market and the massive role they play in distributing information.

Why it’s important

In 2016 Google serviced over 9 billion searches and Facebook’s user base continues a steady upward climb, reaching up to 2 billion worldwide users.

The ubiquity of these platforms and their ability to spread information to users makes them highly influential sources for news. Even more, their global reach makes them attractive platforms for publishers to use as primary hosts for originally produced content.

In a recent paper published by peer-reviewed journal First Monday, authors Philip M. Napoli and Robyn Caplan counter Google and Facebook’s resistance to the “media company” label.

In their analysis, the characterization of these companies is integral to their role in disseminating information and shaping public opinion. It also could influence the regulatory and policy rules that govern them.

What is a ‘media company’

In simple terms, a media company can be defined as an entity that deals in the mass distribution of content. According to Napoli and Caplan, there are three main criteria essential to the function of a “media company,” although they are not mutually exclusive.

  1. The production of content; original material created by users or, in this case, news outlets.
  2. Distribution of content by moving it from producers to consumers.
  3. Exhibition, or the process of providing content directly to audiences.

The move toward digital news media, however, has disrupted the traditional relationship between publisher and consumer. In fact, the ease in which users or outlets can generate content and then distribute it via web-based platforms has significantly changed the media landscape.

Curation vs. creation of content

One of the central arguments tech companies use to differentiate themselves from media companies is that they do not generate content. Instead, they provide a platform for users to disseminate their own or third-party content.

For example, Eric Schmidt, the chairman of Google’s parent company Alphabet, explained, “we don’t do our own content, we get you to someone else’s content faster.” Schmidt acknowledges Google’s role in the distribution and exhibition of content, both of which are key functions of a media company as discussed above.

At a 2016 Q & A session at Rome’s Luiss University, Facebook founder Mark Zuckerberg elaborated on this view regarding his social network platform:

“We are a tech company, not a media company. When you think about a media company, you know, people are producing content, people are editing content, and that’s not us. We’re a technology company. We build tools.”

Some of these tools include Facebook Live and Facebook Instant Articles which allows users, including news publishers, to directly host content on the platform.

In refuting this argument, Napoli and Caplan claim that the lack of creation/ownership of content does not exempt a company from its role in distributing and exhibiting media content. As an example, they point to the fact that cable and satellite companies are often under the same regulatory authority as the content producers they serve. This is true even if they are not involved in the content creation process.

Human vs. algorithmic editing

A less explicit argument from Facebook and Google is that their lack of human editorial processes separates them from traditional media companies that rely on human decision-making to choose content.

However, the “gatekeeping” duty usually assigned to human editors is still being carried out, albeit in a different matter. With more media companies trying to incorporate artificial intelligence into editorial and writing processes, the argument that relying on algorithms over human editors indicates the practices of a tech rather than media company begins to hold less weight.

Can a media company be run by computer scientists?

Another argument these companies use is that the makeup of their employees distinguishes them from media companies. For example, Eric Schmidt claims that Google is a technology company “because it is run by three computer scientists”.

In debunking this argument, Napoli and Caplan comparatively look at the symbiotic evolution of media and technology over time. They state,  “Technological advancements—and the associated technical expertise—have been fundamental to the media sector since at least the advent of the printing press.”

Therefore the view that the tech backgrounds of a company’s staff separates it from the media industry does not hold up based on precedent. Historically, media companies have a track record of embracing new technologies to minimize costs, increase distribution, and aid in content creation.

Duopoly pushing traditional media out of ad market

Finally, competition for ad revenue between online platforms and traditional media outlets implies that they operate within the same business sector.

According to a report by Axios, Google and Facebook are dominating the global market with about 50% of all global ad-spending going to the two companies.

In comparison with traditional media like print and radio, Google’s ad revenue alone matches that of all print media outlets globally and Facebook’s ad revenue out-earns all global ad revenue generated by radio.

Preserving high-quality journalism

Important players in the news industry are catching on to the potential negative effects of underestimating Facebook and Google’s dominance in the digital media ecosystem.

In 2016, both Facebook and Google came under harsh scrutiny for their alleged role in the spread of “fake news” during the US presidential election.  As the New York Times’ Jim Rutenberg puts it, “The maneuvering is about more than the fight for digital territory. It’s about the endurance of quality journalism.”

Do Facebook and Google primarily deal in the curation of content? Yes. But with the increasingly important role these companies have in spreading news media stories, perhaps it is time for them to take responsibility for their role in affecting the quality of journalism in the future of digital news.

Developing the political policies that will drive a prosperous future for jobs

By

I recently was interviewed on ABC News about the future of jobs, in a panel dicussion with policy advisor Terry Barnes.

You can see a video of the interview below.


Read more

Why Indian Publishers Should Be Wary Despite Growing Newspaper Circulation

By

For years, the Indian newspaper industry has been the envy of the rest of the world. It has grown consistently since India’s economic reforms in 1990, and advertising revenues have occasionally hit double-digit growth rates, while the industry declined steadily in much of the western world.

According to a recent report released by India’s Audit Bureau of Circulations (ABC), a non-profit organization founded in 1948, newspaper circulation of its 967 member publications grew 60% between 2006 and 2016, reflecting an annual growth rate of 4.9%.

Total daily circulation for these newspapers audited by the ABC reached 62.8 million, meaning on average each newspaper boasted a daily circulation of about 65,000.

Commentators justifiably took pride in the steady growth of print newspapers, and news outlets like CNN reported on how these figures demonstrated the country’s “thriving” print media industry.

However, this positive coverage glosses over a bout of newsroom buyouts, layoffs, and shutdowns in India, notably by the English-language press and including one of the only four listed newspaper companies in the country. Few, if any, have gone beyond the headline numbers and therein lies the proverbial glass half empty.

The bigger picture

For a comprehensive picture of print media in the country, here are some other numbers to consider:
&nbsp

    • Between 2006 and 2016, average daily print circulation of the 967 publications audited by the ABC grew by 23.7 million. In the same period, India’s population grew 160 million, meaning incremental penetration of newspapers was less than 15% despite youth literacy sharply rising to 90%.

&nbsp

    • The combined annual revenues of India’s print industry declined marginally from $13.6 billion in 2013 to $13.4 billion in 2016, according to the German statistical research firm Statista.

&nbsp

  • GroupM estimates print advertising in India will grow by 4.5% in 2017, just above the previous year’s 4%, but barely above the inflation rate of nearly 4%.

&nbsp
The first bullet point indicates that a majority of youth (50% of India’s population is below the age of 25) may not buy into print, and may instead embrace digital news sources. The second suggests a flattening of print revenues, or even a decline in real terms, despite rising circulation.

Lastly, the third suggests near-flat advertising revenues. If these interpretations are right, India’s print industry may have hit a peak and must prepare for the slide, perhaps by embracing digital with a little more enthusiasm.

Blinded by revenue

Historically, newspapers thrived in a monopolistic position–one in which readers and advertisers had few choices. However, the Internet has broken down both walls, as we have known since at least the turn of the century. Yet the ripple effects are just beginning to show themselves in the Indian market.

In fact, the shift to digital could conceivably play out at a faster pace than it did in other countries, notably the US, for several reasons. After all, technology and newsonomics know no boundaries.

By 2020, India’s Internet users are projected to rise 60% to 730 million and smartphone users are expected to more than double to 702 million, according to a joint study by Internet technology firm Akamai and Indian software industry group Nasscom. Such rapid adoption will likely accelerate already changing reading habits.

So far, Indian print publishers, with only a few exceptions, have made only a feeble push into digital. Perhaps this seemed to make business sense because they were making money hand over fist from print, but no longer.

Most newspapers run a token website with an editorial staff that works in isolation from its print counterparts. Consequently, there is little collaboration, experimentation, or innovation in newsrooms or in marketing departments. Most publishers earn less than 10% of their revenue from online, compared to about 25% for American publishers.

Digital startups with a head start

On the other hand, a range of pure digital news media companies has emerged to tap into the fast-growing online readership overlooked by print publishers. These digital startups are also targeting the fast-growing digital advertising market, which is expected to double its share of the advertising pie by claiming 24% in 2020.
&nbsp
Since digital news media does not face restrictions on foreign ownership, it has allowed many global companies to throw their hats into the ring. They include New York-headquartered IBT Media and the venture-funded Scroll.in, based in Boston.
&nbsp
UC Web, the Chinese media company owned by Alibaba, is also in the process of setting up massive news media operations in multiple Indian languages is In addition, the British Broadcasting Corporation is in an advanced stage of adding more languages to its mix of news websites in India. Billionaire Mukesh Ambani-owned Network 18, through its ETV subsidiary, is similarly planning to start news websites in a dozen Indian languages.
&nbsp
Some of the pure digital startups deliver quality news stories with several multimedia elements. One, The Wire, is a nonprofit initiative. However, some are simple aggregators or rewriting shops and already have stolen a march over many print publishers’ digital offerings.
&nbsp
In fact, some rank among the most trafficked news websites in India, according to the web traffic-monitoring firm SimilarWeb. They include IBTimes India and Oneindia, which is venture-funded and publishes in multiple Indian languages.
&nbsp
India’s print publishers still have a lot going for them. They have corporate advertising support (40% vs. low teens in many other countries, according to The Economist), and generous advertising support from the federal and state governments. But they risk losing print audiences as well as digital ones, if they don’t act soon.
&nbsp

First step is building quantum computers, next step is scaling them

By

It was a great pleasure to get a tour of the University of New South Wales’ Centre for Quantum Computation today from Prof Andrea Morello.

The Centre was last year awarded A$70 million in funding from private and public sectors with the aim of achieving ‘10 qubits in 5 years’.
Read more

Overcoming Fake News and Echo Chambers in the Age of Social Media

By

Online news sources have grown significantly in recent years. With this increase, more people are relying on social media sites to discover and share news stories.

According to the Reuters Institute Digital News Report of 2016, 51% of a global sample said they use social media as a source of news every week, and about 1 in 10 use social media as a main source of news.

Along with this emerging trend, access to free online platforms as well as tools that simplify the creation of websites have contributed to the spread of unverified stories. Together these factors influence the way news is discovered, consumed, and published.

This article aims to define two important media trends that have sprung up as a result: the proliferation of “fake news” and “echo chambers.” It will also provide insight into the actions of leading media companies to adapt to this changing framework and develop a new model for the future of journalism.

Defining ‘fake news’

Throughout 2016 and 2017, “fake news” was used to label misinformation campaigns that used social media and automated bots to intentionally spread false information. It has since evolved to become a toxic media brand used to describe inaccurate news.

To the alarm of many news professionals, politicians have also started using the term to undermine the credibility of unfavorable media outlets. For the sake of this article, the term “fake news” will be used to address any news stories that are simply false or purposely misleading.

According to the Reuters Institute’s 2017 trends and predictions report, by the end of the 2016 US Presidential election, there was actually higher engagement with fake news stories on Facebook than with accurate journalism (see below).

Due to the ease of sharing on social networks, untrue or exaggerated articles can spread quickly. Also, as the accepted definition of “fake news” remains ambiguous, these stories may be difficult to identify.

For these reasons, it has become increasingly important for news publishers to ensure their credibility with audiences in order to maintain a high level of integrity for their readers and to avoid being labeled as fake.

Fact-checking and data-based journalism to combat fake news

According to a 2017 report from Harvard’s Shorenstein Center on Media, Politics and Policy, the issue of fake news can addressed by three primary methods: prioritizing fact-checking, promoting bipartisan discussion, and fostering a collaborative research environment. We have already seen shifts in influential media companies to meet these goals.

At its 2017 F8 conference, Facebook announced plans to outsource fact checking to third parties like Politifact and Snopes in order to provide analysis of claims being made by news outlets online. This year, the Google Digital News Initiative (DNI) also awarded funding to projects such as the British prototype: Fact-checking Automation and Claims Tracking System (FACTS). This platform seeks to be the first to fact-check claims automatically using statistical analysis.

The same report claims, “we should seek stronger future collaborations between researchers and the media.” In addition to reducing the cost of data-based journalism, some publishers are pursuing “open-data” based platforms that focus on shared use of data, research, and user collaboration.

In 2017, Wikipedia founder Jimmy Wales announced the launch of WikiTribune. “Articles are authored, fact-checked, and verified by professional journalists and community members working side by side as equals,” states the website. The project aims to produce “evidence-based” journalism that is founded on data, research, and user collaboration.

News aggregation and user-interaction to break echo chambers

When it comes to self-tailored media, or “echo chambers,” the main causes are twofold. The first is that many people share and interact with news stories they agree with or of which they approve. In addition, social media users are free to select which media outlets to follow, and which ones to block.

The result is a potential bubble of information that may discourage engagement with challenging viewpoints.

Even more, if the view or information reinforced by these media choices are inaccurate, it may lead to a personalized information trap of unreliable sources. This has created concern among readers who don’t want to miss challenging viewpoints.

To address this issue, some consumers are turning to news aggregators. In fact, according to the Reuters Digital News Report, 57% of respondents said they prefer news aggregators in order to access a variety of sources.

News aggregators like Google News and Yahoo Japan are already quite popular with people whom prefer to receive news from multiple sources.

In China a new app called Bingdu combines news aggregation, user-driven advertisements, and Facebook-style recommendation algorithms to attract around 10 million active users.

Another Google DNI funded initiative comes from Europa Press Comunicación via a news platform that aims to “facilitate the use of open-data both as a source of news and as a fact validation instrument.”

Solutions require collaboration and innovation 

The problems of fake news and echo chambers will not be solved overnight. We have already seen a shift in news production by leading media companies to ensure credibility in this changing media ecosystem. Innovative emerging publishers should continue to foster a strong relationship between publishers, readers, and researchers. This is an essential first step in building a more positive media landscape for the future.

 

Image sources: Reuters Institute’s 2017 trends and predictions reportReuters Digital News Report,

Phase transition: I am launching a new group of companies

By

I am delighted to announce a massive transition in my work and life: launching a new group of companies with other amazing founders that expands and accelerates my vision.

It is now over 20 years since I left employment and started building the Advanced Human Technologies group of companies, which spanned professional services, publishing, events, apps and software development. We had many landmarks and successes along the way.

The reason to bring these ventures into a new group is simply to enact the same vision I have long had on a larger stage and scale. Read more