By Paula Klein
“AI is perhaps the most profound technology to affect our era,” according to MIT Professor and Director of the Initiative on the Digital Economy, Erik Brynjolfsson. Now, there is data to prove it—lots of data.
The Artificial Intelligence Index Report, one of the most comprehensive reports on the state of AI, was released December 11 to “track, collate, distill, and visualize data relating to artificial intelligence.”
The 2019 edition is the third report produced by the AI Index, an independent initiative of Stanford University’s Human-Centered Artificial Intelligence Institute (HAI). Its mission is to provide “unbiased, rigorously vetted data for policymakers, researchers, executives, journalists, and the general public” about the complex field of AI. MIT’s Brynjolfsson is a member of the AI Index Steering Committee, a group of experts from academia and industry working with more than 35 sponsoring partners and data contributors such as McKinsey & Company, Google, PwC, OpenAI, Genpact and AI21Labs.
Until now “we’ve been sorely lacking good data about basic questions like ‘How is the technology advancing’ and ‘What is the economic impact of AI?’ ” Brynjolfsson said. The new index, which tracks three times as many data sets as last year’s report, goes a long way toward providing answers.
Included are updates on all aspects of AI including technical performance, the economic impact, education and social issues, and R&D investments. Nearly all indicators point to huge growth and advancements. Globally, for instance, investment in AI startups continues a steady ascent from a $1.3 billion raised in 2010, to $37.4 billion as of early November 2019. Funding has increased at an average annual growth rate of more than 48%.
Outpacing Moore’s Law
Technology-wise, AI computational power is accelerating faster than traditional processor development. “Prior to 2012, AI results closely tracked Moore’s Law with compute doubling every two years,” the report said. “Post-2012, compute has been doubling every 3.4 months.”
“Some of the technical advances are eye-popping,” Brynjolfsson said. Machine learning models that took three hours to be trained as recently as 2017, are now trained in a couple of minutes to achieve the same level of accuracy. “This reflects both more and better computer power, as well as improvements in the underlying algorithms.”
Although “there’s a lot to be excited about,” we’re still quite far from human level understanding of machine learning and natural language right now, Brynjolfsson added.
Other key insights include the following:
- Research and Development
- From 1998 and 2018 the volume of peer-reviewed AI papers has grown by more than 300%, accounting for 3% of peer-reviewed journal publications and 9% of published conference papers.
- China now publishes as many AI journal and conference papers annually as Europe; it passed the U.S. in 2006.
- Singapore, Switzerland, Australia, Israel, Netherlands, and Luxembourg have relatively high numbers of deep learning papers published per capita.
- North America accounts for over 60% of global AI patent citation activity from 2014 to 2018.
- Technical Performance
- Since 2017, the time required to train a large image classification system on cloud infrastructure has fallen from about three hours to about 88 seconds. During the same period, the cost to train such a system has fallen similarly.
- Progress on some broad sets of natural-language processing tasks has been remarkably rapid; performance is lower on some NLP tasks requiring reasoning or physical dexterity. Drinking coffee and polishing furniture are still among the most difficult tasks for machines to master.
- The Economy
- Singapore, Brazil, Australia, Canada, and India experienced the fastest growth in AI hiring from 2015 to 2019.
- Fully 58% of large companies report adopting AI in at least one function or business unit in 2019, up from 47% in 2018.
- The share AI-related jobs in the U.S. increased from 0.26% of total jobs posted in 2010 to 1.32% in 2019, with the highest share in Machine Learning (0.51% of total jobs).
“The flow of people into the field is rapid, though from a small base,” Brynjolfsson noted. “There are four times as many people working on machine learning and AI in the U.S. now as in 2010, though it’s still only a bit over 1% of the workforce.”
- Education
- At the graduate level, AI has rapidly become the most popular specialization among computer science PhD students in North America. In 2018, over 21% of graduating Computer Science PhDs specialize in Artificial Intelligence/Machine Learning.
- Industry is the largest consumer of AI talent. In 2018, over 60% of AI PhD graduates went to industry, up from 20% in 2004.
- In the U.S., AI faculty leaving academia for industry continues to accelerate, with over 40 departures in 2018, up from 15 in 2012 and none in 2004.
Challenges Remain
Of course, there are still many unresolved issues and challenges to be addressed. Societal considerations such as fairness, interpretability, and explainability were the most frequently mentioned ethical challenges cited across 59 documents.
Additionally, in over 3,600 relevant news articles last year the dominant topics of concern were framework and guidelines on the ethical use of AI, data privacy, the use of face recognition, algorithm bias, and the role of big tech. There is also a significant increase in AI-related legislation recorded in congressional records, committee reports, and legislative transcripts around the world.
Moreover, actions to mitigate risk are not keeping pace. Only 19% of large companies surveyed say their organizations are taking steps to reduce risks associated with explainability of their algorithms, and just 13% are alleviating risks to equity and fairness, such as algorithmic bias and discrimination.
Gender disparities are still very evident, too. Diversifying AI faculty along gender lines has not shown great progress, with women comprising less than 20% of the new faculty hires in 2018. Similarly, the share of female AI PhD recipients has remained virtually constant at 20% since 2010 in the U.S. By contrast, many Western European countries, especially the Netherlands and Iran, show relatively high presence of women in AI research.
To better navigate the nearly 300-page report, a Global AI Vibrancy tool is available to compare countries across 34 axes. It gives the choice to measure by overall numbers as well as per capita trends to recognize hot spots in places such as Israel, which produces more per capita deep learning research than any other country, or advanced AI leaders like Finland and Singapore.
For the IDE, the report represents continuation of its efforts to explore the global impact of AI. Earlier this year, IDE announced a collaboration with the Partnership on AI (PAI). PAI’s multi-stakeholder coalition numbers over 95 organizations, representing some of the world’s most influential technology companies, think tanks, policy and research centers, and human rights organizations, active on five continents and in 14 countries.