Comment: Is Big Data Taking Us Closer to the Deeper Questions in AI?

Even though there’s a lot of hype about AI and a lot of money being invested in AI, I feel like the field is headed in the wrong direction. There’s been a local maximum where there’s a lot of low-hanging fruit right now in a particular direction, which is mainly deep learning and big data. People are very excited about the big data and what it’s giving them right now, but I’m not sure it’s taking us closer to the deeper questions in artificial intelligence, like how we understand language or how we reason about the world.

Marcus, G. (2016). Is Big Data Taking Us Closer to the Deeper Questions in Artificial Intelligence?

Gary Marcus, a developmental psychologist who trained with Steven Pinker, discusses his reservations about deep and reinforcement learning as the hype around these models keeps growing (this presentation is from 2016). Gary explains the limitations of current neural networks, while also acknowledging the kinds of problems that they’re well-positioned to solve.

He argues that the progress in those domains are interesting but aren’t the most important problems that AI could potentially help us with. For example, it’s great that deep learning helps us get better recommendations from Amazon but it can’t help us to cure cancer, which we’d all agree is a more important problem. We can acknowledge the value in building narrow AI systems (for example, natural language processing systems that help us with machine translation but which have no understanding of language) while also recognising that those same systems may never help us move closer to what we might value more dearly.

This is a really nice presentation that provides some background around deep and reinforcement learning, the kinds of problems those methods are useful for, and why they’re not really a plausible route towards strong, general artificial intelligence. If you’re interested in a more academic critique of deep learning, see Deep Learning: A Critical Appraisal, by Marcus.

A potentially interesting side note: Marcus’ critical position on deep and reinforcement learning is not uncontroversial and not all AI researchers agree with him. See here for an example of how emotionally charged the topic is.