For Kirk Borne, the information revolution began 11 years ago while he was working at NASA’s National Space Science Data Center in Greenbelt, Maryland. At a conference, another astronomer asked him if the center could archive a terabyte of data that had been collected from the MACHO sky survey, a project designed to study mysterious cosmic bodies that emit very little light or other radiation. Nowadays, plenty of desktop computers can store a terabyte on a hard drive. But when Borne ran the request up the flagpole, his boss almost choked. “That’s impossible!” he told Borne. “Don’t you realize that the entire data set NASA has collected over the past 45 years is one terabyte?”
“That’s when the lightbulb went off,” says Borne, who is now an associate professor of computational and data sciences at George Mason University. “That single experiment had produced as much data as the previous 15,000 experiments. I realized then that we needed to do something not only to make all that data available to scientists but also to enable scientific discovery from all that information.”
The tools of astronomy have changed drastically over just the past generation, and our picture of the universe has changed with them. Gone are the days of photographic plates that recorded the sky snapshot by painstaking snapshot. Today more than a dozen observatories on Earth and in space let researchers eyeball vast swaths of the universe in multiple wavelengths, from radio waves to gamma rays. And with the advent of digital detectors, computers have replaced darkrooms. These new capabilities provide a much more meaningful way to understand our place in the cosmos, but they have also unleashed a baffling torrent of data. Amazing discoveries might be in sight, yet hidden within all the information.
Since 2000, the $85 million Sloan Digital Sky Survey at the Apache Point Observatory in New Mexico has imaged more than one-third of the night sky, capturing information on more than 930,000 galaxies and 120,000 quasars. Computational analysis of Sloan’s prodigious data set has uncovered evidence of some of the earliest known astronomical objects, determined that most large galaxies harbor supermassive black holes, and even mapped out the three-dimensional structure of the local universe. “Before Sloan, individual researchers or small groups dominated astronomy,” says Robert Brunner, an astronomy professor at the University of Illinois at Urbana-Champaign. “You’d go to a telescope, get your data, and analyze it. Then Sloan came along, and suddenly there was this huge data set designed for one thing, but people were using it for all kinds of other interesting things. So you have this sea change in astronomy that allows people who aren’t affiliated with a project to ask entirely new questions.”
A new generation of sky surveys promises to catalog literally billions and billions of astronomical objects. Trouble is, there are not enough graduate students in the known universe to classify all of them. When the Large Synoptic Survey Telescope (LSST) in Cerro Pachón, Chile, aims its 3.2- billion-pixel digital camera (the world’s largest) at the night sky in 2019, it will capture an area 49 times as large as the moon in each 15-second exposure, 2,000 times a night. Those snapshots will be stitched together over a decade to eventually form a motion picture of half the visible sky. The LSST, producing 30 terabytes of data nightly, will become the centerpiece of what some experts have dubbed the age of petascale astronomy—that’s 1015 bits (what Borne jokingly calls “a tonabytes”).
