Stay Curious

SIGN UP FOR OUR WEEKLY NEWSLETTER AND UNLOCK ONE MORE ARTICLE FOR FREE.

Sign Up

VIEW OUR Privacy Policy


Discover Magazine Logo

WANT MORE? KEEP READING FOR AS LOW AS $1.99!

Subscribe

ALREADY A SUBSCRIBER?

FIND MY SUBSCRIPTION
Advertisement

How Leaky Datasets Undermine AI Math Reasoning Claims

Questions over tests of AI math abilities suggest we may never know how capable intelligent machines computers can become.

(Credit: Phonlamai Photo/Shutterstock) Phonlamai Photo/Shutterstock

Newsletter

Sign up for our email newsletter for the latest science news

Sign Up

Back in 2019, a group of computer scientists performed a now-famous experiment with far-reaching consequences for artificial intelligence research. At the time, machine vision algorithms were becoming capable of recognizing a wide range of objects with some recording spectacular results in the standard tests used to assess their abilities.

But there was a problem with the method behind all these tests. Almost all the algorithms were trained on a database of labelled images, known as ImageNet. The database contained millions of images which had been carefully described in human-written text to help the machines learn. This effort was crucial for the development of machine vision and ImageNet became a kind of industry standard.

In this way, the computer scientists used a subset of the images to train algorithms to identify a strawberry, a table, a human face and so on, using labelled images in the dataset. They then used a ...

Stay Curious

JoinOur List

Sign up for our weekly science updates

View our Privacy Policy

SubscribeTo The Magazine

Save up to 40% off the cover price when you subscribe to Discover magazine.

Subscribe
Advertisement

0 Free Articles