Computational Challenges in Cosmology
In the past 15 years cosmology has undergone a renaissance,
transforming from a data-starved science to a data-driven one. The COBE
satellite and subsequent observations of the cosmic microwave background have
begun to provide a detailed picture of the early Universe. Telescopes have
found galaxies at distances corresponding to the Universe at one-tenth of its
present age, while large-scale redshift surveys have begun to map out the
structure of the nearby Universe.
At the same time, according to an NSF award abstract, "the
size of these datasets threatens to leave cosmology data-swamped. Realizing our
scientific goals depends on meeting the qualitatively new computational
challenges set by quantitatively new data."
This multi-disciplinary research project, led by the
University of California at Berkeley and funded under the National Science
Foundation's KDI (Knowledge and Distributed Intelligence) program, has involved
a focused collaboration between astrophysicists, statisticians, and computer
scientists to develop computational tools, techniques, and technologies to cope
with the new challenges posed by these massive datasets.
According to Professor Marc Davis of the
Astronomy Department at Berkeley, a major part of the KDI project has dealt
with handling data from Phase 2 of the DEEP Survey (Deep Extragalactic
Evolutionary Probe). DEEP is a multi-year program that uses the twin 10-meter
Keck Telescopes in Hawaii, the world's largest, and the Hubble Space Telescope
to conduct a large-scale survey of distant, faint, field galaxies. The goals
include investigating the formation and evolution of galaxies, the origin of
large structure, the nature of dark matter, and the geometry of the Universe.
The DEEP2 Redshift Survey utilizes a new
DEIMOS spectrographfeaturing very high resolution and efficiencyto
obtain spectra of about 65,000 faint galaxies. It is a collaboration between
astronomers at Berkeley, the University of California at Santa Cruz, and the
University of Hawaii.
"The instrument was completed last year, and we started
taking data last summer," Davis says. "It's going extremely well. The data are
magnificent. We think we can set powerful constraints on the dark energy of the
Universe, among other thingsa whole host of cosmological issues will be
addressed by our data."
The Berkeley astrophysicist notes that "KDI support was used
to help us cope with this huge data onslaught, because the machine, this new
spectrograph, generates data at approximately 1 gigabyte per hour, and the
processing of the data is pretty complicated. We had to build an automated
pipeline to cope with it, because it's just overwhelming."
Davis says he believes the research team has effectively
met the computational challenges posed by DEEP2. "When we started the project
many years ago, we really were afraid we wouldn't be able to deal with the data
at allthat it would just pile up on the floor and we'd be just swamped in
it, it'd be hopeless. But it turned out we did meet that challenge."
Full datasets from the DEEP2 Redshift Survey will be made
available to the rest of the research community by depositing them with a
data-archiving service, Davis says, while summary datasets will be publicly
available on the Internet.
The KDI grant also was utilized in connection with work
on AMANDA, a project at the South Pole that has involved drilling deep holes in
the icecap and installing in them strings of photomultiplier tubes. Most of the
photomultiplier tubes are pointed downward. The aim is to detect light signals
emitted when extremely high-energy neutrinos, which have come from deep space
and passed through the Earth, interact with the ice.
According to Davis, "funds from the KDI grant were used to
support AMANDA researchsome postdoctoral fellows who were working on data
processing and understand how to cope with large datasets."
Other principal investigators involved with the NSF grant
were George F. Smoot, Lawrence Berkeley National Laboratory; Philip B. Stark,
Department of Statistics, UC Berkeley; Andrew H. Jaffe, formerly at Berkeley's
Center for Particle Astrophysics and now at Imperial College, London; and
Joseph I. Silk, formerly with the Astronomy Department, UC Berkeley, and now at
Oxford University, U.K.
Back to Top of Page