Almadena Y. Chtchelkanova
CCF Division of Computer and Communication Foundations
CSE Directorate for Computer & Information Science & Engineering
Start Date:
September 15, 2001
Expires:
November 30, 2008 (Estimated)
Awarded Amount to Date:
$4369999
Investigator(s):
Dennis McLaughlin dennism@mit.edu (Principal Investigator)
Alan Willsky (Co-Principal Investigator) Paola Malanotte-Rizzoli (Co-Principal Investigator) Dara Entekhabi (Co-Principal Investigator) Kerry Emanuel (Co-Principal Investigator)
Sponsor:
Massachusetts Institute of Technology
77 MASSACHUSETTS AVE
Cambridge, MA 02139 617/253-1000
NSF Program(s):
ITR MEDIUM (GROUP) GRANTS
Field Application(s):
0000099 Other Applications NEC
Program Reference Code(s):
HPCC, 9216, 4444, 1687, 1661, 1652
Program Element Code(s):
1687
ABSTRACT
ITR/AP: An Ensemble Approach to Data Assimilation in the Earth Sciences
New data sources are beginning to have a dramatic impact on our ability to understand the earth as an integrated system. Our prospects for dealing with the environmental issues of the 21st century -- climate change, population pressures on natural resources, and major modifications in global element cycles -- depend largely on this new information. However, our ability to process and interpret environmental data is not keeping pace with the dramatic increase in available information, especially information from airborne and orbital remote sensing platforms. If we are to realize the potential benefits of new sensing technologies we will need to develop intelligent environmental data assimilation procedures that are able to efficiently extract useful information about the earth from a diverse set of data sources.
Environmental data assimilation can be posed as a problem of estimating a large number of unobservable or highly uncertain variables (e.g. sea surface heights, atmospheric pressures, hydrologic fluxes, etc.) from a large number of related but noisy measurements (e.g. microwave radiances or backscatter detected by a satellite sensor). The estimation procedure relies on mathematical models that relate unknowns to measurements. Environmental estimation problems are challenging because the systems of interest: 1) are spatially distributed and highly variable over a wide range of space and time scales, 2) are difficult to describe with precision, 3) are often nonlinear, even chaotic, and 4) are often characterized by non-unique relationships between unknowns and measurements.
This project is concerned with very large problems (many measurements and many unknowns) which are not amenable to traditional data assimilation techniques but are of crucial interest to researchers in the earth sciences. An interdisciplinary team will develop a better understanding of the issues of dimensionality reduction and uncertainty propagation that are crucial to large-scale data assimilation. So-called ensemble methods provide a particularly informative way to identify these key features. A new generation of "intelligent" data assimilation methods will be developed that build on the understanding gained from the reduced problem. The applicability of these methods will be investigated on problems of broad interest in the earth sciences, including problems that 1) deal with coupled systems, 2) cut across traditional disciplines, and 3) work with remote sensing data sets.
This ITR project brings together acknowledged experts on environmental data assimilation. It is a group ITR project, rather than several individual projects, which cuts across earth science disciplines. The research will be coordinated with: 1) a seminar series, 2) joint supervision of Ph.D. students and post-doctoral researchers, 3) a Ph.D. mentoring program, 4) a selection of cross-cutting sample problems, and 5) co-authored publications.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Buehner, M., P.Malanotte-Rizzoli,A. J Busalacchi, and T. Inui. "Estimation of the tropical Atlantic circulation from altimetry data using a reduced-rank stationary Kalman filter," In "Interhemispheric water exchanges in the Atlantic Ocean", Elsevier Oceanographic Series, v.68, 2003, p. 49.
Buehner, M; Malanotte-Rizzoli, P. "Reduced-rank Kalman filters applied to an idealized model of the wind-driven ocean circulation," JOURNAL OF GEOPHYSICAL RESEARCH-OCEANS, v.108, 2003.
Buehner, M; Malanotte-Rizzoli, P; Busalacchi, A; Inui, T. "Estimation of the tropical Atlantic circulation from altimetry data using a reduced-rank stationary Kalman filter," INTERHEMISPHERIC WATER EXCHANGE IN THE ATLANTIC OCEAN, v.68, 2003, p. 49-92.
Caparrini, F; Castelli, F; Entekhabi, D. "Mapping of land-atmosphere heat fluxes and surface parameters with remote sensing data," BOUNDARY-LAYER METEOROLOGY, v.107, 2003, p. 605-633.
Caparrini, F; Castelli, F; Entekhabi, D. "Estimation of surface turbulent fluxes through assimilation of radiometric surface temperature sequences," JOURNAL OF HYDROMETEOROLOGY, v.5, 2004, p. 145-159.
Caparrini, F; Castelli, F; Entekhabi, D. "Variational estimation of soil and vegetation turbulent transfer and heat flux parameters from sequences of multisensor imagery," WATER RESOURCES RESEARCH, v.40, 2004.
Choi, MJ; Chandrasekaran, V; Malioutov, DM; Johnson, JK; Willsky, AS. "Multiscale stochastic modeling for tractable inference and data assimilation," COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING, v.197, 2008, p. 3492-3515.
Choi, MJ; Chandrasekaran, V; Willsky, AS. "Maximum entropy relaxation for multiscale graphical model selection," 2008 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING, VOLS 1-12, 2008, p. 1889-1892.
Choi, MJ; Willsky, AS. "Multiscale Gaussian graphical models and algorithms for large-scale inference," 2007 IEEE/SP 14TH WORKSHOP ON STATISTICAL SIGNAL PROCESSING, VOLS 1 AND 2, 2007, p. 229-233.
Crow, WT; Chan, S; Entekhabi, D; Hsu, A; Jackson, TJ; Njoku, E; O'Neill, P; Shi, J. "An observing system simulation experiment for hydros radiometer-only soil moisture and freeze-thaw products," IGARSS 2005: IEEE International Geoscience and Remote Sensing Symposium, Vols 1-8, Proceedings, 2005, p. 2737-2740.
Crow, WT; Chan, STK; Entekhabi, D; Houser, PR; Hsu, AY; Jackson, TJ; Njoku, EG; O'Neill, PE; Shi, JC; Zhan, XW. "An observing system simulation experiment for Hydros radiometer-only soil moisture products," IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, v.43, 2005, p. 1289-1303.
Dunne, S; Entekhabi, D. "An ensemble-based reanalysis approach to land data assimilation," WATER RESOURCES RESEARCH, v.41, 2005.
Dunne, S; Entekhabi, D. "Land surface state and flux estimation using the ensemble Kalman smoother during the Southern Great Plains 1997 field experiment," WATER RESOURCES RESEARCH, v.42, 2006.
Dunne, SC; Entekhabi, D; Njoku, EG. "Impact of multiresolution active and passive microwave measurements on soil moisture estimation using the ensemble Kalman smoother," IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, v.45, 2007, p. 1016-1028.
Dutkiewicz, S; Follows, MJ; Heimbach, P; Marshall, J. "Controls on ocean productivity and air-sea carbon flux: An adjoint model sensitivity study," GEOPHYSICAL RESEARCH LETTERS, v.33, 2006.
Emanuel, K; Sundararajan, R; Williams, J. "Hurricanes and global warming - Results from downscaling IPCC AR4 simulations," BULLETIN OF THE AMERICAN METEOROLOGICAL SOCIETY, v.89, 2008, p. 347-+.
Entekhabi, D; Moghaddam, M. "Mapping recharge from space: roadmap to meeting the grand challenge," HYDROGEOLOGY JOURNAL, v.15, 2007, p. 105-116.
Entekhabi, D; Njoku, EG; Houser, P; Spencer, M; Doiron, T; Kim, YJ; Smith, J; Girard, R; Belair, S; Crow, W; Jackson, TJ; Kerr, YH; Kimball, JS; Koster, R; McDonald, KC; O'Neill, PE; Pultz, T; Running, SW; Shi, JC; Wood, E; van Zyl, J. "The hydrosphere state (Hydros) satellite mission: An earth system pathfinder for global mapping of soil moisture and land freeze/thaw," IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, v.42, 2004, p. 2184-2195.
Ferreira, D; Marshall, J; Heimbach, P. "Estimating eddy stresses by fitting dynamics to observations using a residual-mean ocean circulation model and its adjoint," JOURNAL OF PHYSICAL OCEANOGRAPHY, v.35, 2005, p. 1891-1910.
Forget, G; Wunsch, C. "Estimated global hydrographic variability," JOURNAL OF PHYSICAL OCEANOGRAPHY, v.37, 2007, p. 1997-2008.
Fox, EB; Fisher, JW; Willsky, AS. "Detection and localization of material releases with sparse sensor configurations," IEEE TRANSACTIONS ON SIGNAL PROCESSING, v.55, 2007, p. 1886-1898.
Gebbie, G; Heimbach, P; Wunsch, C. "Strategies for nested and eddy-permitting state estimation," JOURNAL OF GEOPHYSICAL RESEARCH-OCEANS, v.111, 2006.
Gorenburg, IP; McLaughlin, D; Entekhabi, D. "Scale-recursive assimilation of precipitation data," ADVANCES IN WATER RESOURCES, v.24, 2001, p. 941-953.
Hansen, JA. "Accounting for model error in ensemble-based state estimation and forecasting," MONTHLY WEATHER REVIEW, v.130, 2002, p. 2373-2391.
Hansen, JA; Emanuel, KA. "Forecast 4D-Var: Exploiting model output statistics," QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, v.129, 2003, p. 1255-1267.
Hansen, JA; Penland, C. "On stochastic parameter estimation using data assimilation," PHYSICA D-NONLINEAR PHENOMENA, v.230, 2007, p. 88-98.
Heimbach, P., C. Hill, and R. Giering. "Automatic generation of Efficient Adjoint Code for a Parallel Navier-Stokes Solver," Lecture notes in Computer Science (LNCS), J.J. Dongara, P.M.A. Sloot, and C.J.K. Tan (Eds.), Springer-Verlag, v.2330, 2002, p. 1019.
Heimbach, P; Hill, C; Giering, R. "Automatic generation of efficient adjoint code for a parallel Navier-Stokes solver," COMPUTATIONAL SCIENCE-ICCS 2002, PT II, PROCEEDINGS, v.2330, 2002, p. 1019-1028.
Heimbach, P; Hill, C; Giering, R. "An efficient exact adjoint of the parallel MIT General Circulation Model, generated via automatic differentiation," FUTURE GENERATION COMPUTER SYSTEMS, v.21, 2005, p. 1356-1371.
Jafarpour, B; McLaughlin, DB. "History matching with an ensemble Kalman filter and discrete cosine parameterization," COMPUTATIONAL GEOSCIENCES, v.12, 2008, p. 227-244.
Johnson, JK; Willsky, AS. "A recursive model-reduction method for approximate inference in Gaussian Markov random fields," IEEE TRANSACTIONS ON IMAGE PROCESSING, v.17, 2008, p. 70-83.
Khade, VM; Hansen, JA. "State dependent predictability: Impact of uncertainty dynamics, uncertainty structure and model inadequacies," NONLINEAR PROCESSES IN GEOPHYSICS, v.11, 2004, p. 351-362.
Kim, J; Cetin, M; Willsky, AS. "Nonparametric shape priors for active contour-based image segmentation," SIGNAL PROCESSING, v.87, 2007, p. 3021-3044.
Kim, JM; Fisher, JW; Yezzi, A; Cetin, M; Willsky, AS. "A nonparametric statistical method for image segmentation using information theory and curve evolution," IEEE TRANSACTIONS ON IMAGE PROCESSING, v.14, 2005, p. 1486-1502.
Krajewski, WF; Anderson, MC; Eichinger, WE; Entekhabi, D; Hornbuckle, BK; Houser, PR; Katul, GG; Kustas, WP; Norman, JM; Peters-Lidard, C; Wood, EF. "A remote sensing observatory for hydrologic sciences: A genesis for scaling to continental hydrology," WATER RESOURCES RESEARCH, v.42, 2006.
Lawson, WG; Hansen, JA. "Implications of stochastic and deterministic filters as ensemble-based data assimilation methods in varying regimes of error growth," MONTHLY WEATHER REVIEW, v.132, 2004, p. 1966-1981.
Lawson, WG; Hansen, JA. "Alignment error models and ensemble-based data assimilation," MONTHLY WEATHER REVIEW, v.133, 2005, p. 1687-1709.
Lea, DJ; Haine, TWN; Allen, MR; Hansen, JA. "Sensitivity analysis of the climate of a chaotic ocean circulation model," QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY, v.128, 2002, p. 2587-2605.
Losch, M; Heimbach, P. "Adjoint sensitivity of an ocean general circulation model to bottom topography," JOURNAL OF PHYSICAL OCEANOGRAPHY, v.37, 2007, p. 377-393.
Losch, M; Wunsch, C. "Bottom topography as a control variable in an ocean model," JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY, v.20, 2003, p. 1685-1696.
Luo, ZZ; Stephens, GL; Emanuel, KA; Vane, DG; Tourville, ND; Haynes, JM. "On the use of CloudSat and MODIS data for estimating hurricane intensity," IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, v.5, 2008, p. 13-16.
Mahadevan, A; Lu, J; Meacham, SP; Malanotte-Rizzoli, P. "The predictability of large-scale wind-driven flows (vol 9, pg 173, 2002)," NONLINEAR PROCESSES IN GEOPHYSICS, v.9, 2002, p. 173-173.
Malioutov, DM; Johnson, JK; Willsky, AS. "Walk-sums and belief propagation in Gaussian graphical models," JOURNAL OF MACHINE LEARNING RESEARCH, v.7, 2006, p. 2031-2064.
Margulis, SA; Entekhabi, D. "Variational assimilation of radiometric surface temperature and reference-level micrometeorology into a model of the atmospheric boundary layer and land surface," MONTHLY WEATHER REVIEW, v.131, 2003, p. 1272-1288.
Margulis, SA; Entekhabi, D. "Boundary-layer entrainment estimation through assimilation of radiosonde and micrometeorological data into a mixed-layer model," BOUNDARY-LAYER METEOROLOGY, v.110, 2004, p. 405-433.
Margulis, SA; Entekhabi, D; McLaughlin, D. "Spatiotemporal disaggregation of remotely sensed precipitation for ensemble hydrologic modeling and data assimilation," JOURNAL OF HYDROMETEOROLOGY, v.7, 2006, p. 511-533.
Margulis, SA; McLaughlin, D; Entekhabi, D; Dunne, S. "Land data assimilation and estimation of soil moisture using measurements from the Southern Great Plains 1997 Field Experiment," WATER RESOURCES RESEARCH, v.38, 2002.
McLaughlin, D. "An integrated approach to hydrologic data assimilation: interpolation, smoothing, and filtering," ADVANCES IN WATER RESOURCES, v.25, 2002, p. 1275-1286.
McLaughlin, D.. "A probabilistic perspective on nonlinear model inversion and data assimilation," In: Subsurface Hydrology: Data Integration for Properties and Processes Geophysical Monograph Series 171, American Geophysical Union, v.171, 2007.
McLaughlin, D; Zhou, YH; Entekhabi, D; Chatdarong, V. "Computational issues for large-scale land surface data assimilation problems," JOURNAL OF HYDROMETEOROLOGY, v.7, 2006, p. 494-510.
Moghaddam, M; Entekhabi, D; Liu, M; Teneketzis, D. "Soil moisture smart sensor web concept using data assimilation and optimal control," 2007 IEEE AEROSPACE CONFERENCE, VOLS 1-9, 2007, p. 2294-2299.
Naumann, U; Heimbach, P. "Coupling tangent-linear and adjoint models," COMPUTATIONAL SCIENCE AND ITS APPLICATIONS - ICCSA 2003, PT 2, PROCEEDINGS, v.2668, 2003, p. 105-114.
Ponte, RM; Quinn, KJ; Wunsch, C; Heimbach, P. "A comparison of model and GRACE estimates of the large-scale seasonal cycle in ocean bottom pressure," GEOPHYSICAL RESEARCH LETTERS, v.34, 2007.
Ponte, RM; Wunsch, C; Stammer, D. "Spatial mapping of time-variable errors in Jason-1 and TOPEX/Poseidon sea surface height measurements," JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY, v.24, 2007, p. 1078-1085.
Ravela, S; Emanuel, K; McLaughlin, D. "Data assimilation by field alignment," PHYSICA D-NONLINEAR PHENOMENA, v.230, 2007, p. 127-145.
Ravela, S; McLaughlin, D. "Fast ensemble smoothing," OCEAN DYNAMICS, v.57, 2007, p. 123-134.
Reichle, RH; McLaughlin, DB; Entekhabi, D. "Hydrologic data assimilation with the ensemble Kalman filter," MONTHLY WEATHER REVIEW, v.130, 2002, p. 103-114.
Sini, F; Boni, G; Caparrini, F; Entekhabi, D. "Estimation of large-scale evaporation fields based on assimilation of remotely sensed land temperature," WATER RESOURCES RESEARCH, v.44, 2008.
Smith, LA; Hansen, JA. "Extending the limits of ensemble forecast verification with the minimum spanning tree," MONTHLY WEATHER REVIEW, v.132, 2004, p. 1522-1528.
Stammer, D., C. Wunsch, I. Fukumori, and J. Marshall. "State estimation in modern oceanographic research
," EOS
, v.83, 2002, p. 289.
Stammer, D; Kohl, A; Wunsch, C. "Impact of accurate geoid fields on estimates of the ocean circulation," JOURNAL OF ATMOSPHERIC AND OCEANIC TECHNOLOGY, v.24, 2007, p. 1464-1478.
Stammer, D; Ueyoshi, K; Kohl, A; Large, WG; Josey, SA; Wunsch, C. "Estimating air-sea fluxes of heat, freshwater, and momentum through global ocean data assimilation," JOURNAL OF GEOPHYSICAL RESEARCH-OCEANS, v.109, 2004.
Stammer, D; Wunsch, C; Giering, R; Eckert, C; Heimbach, P; Marotzke, J; Adcroft, A; Hill, CN; Marshall, J. "Global ocean circulation during 1992-1997, estimated from ocean observations and a general circulation model," JOURNAL OF GEOPHYSICAL RESEARCH-OCEANS, v.107, 2002.
Stammer, D; Wunsch, C; Giering, R; Eckert, C; Heimbach, P; Marotzke, J; Adcroft, A; Hill, CN; Marshall, J. "Volume, heat, and freshwater transports of the global ocean circulation 1993-2000, estimated from a general circulation model constrained by World Ocean Circulation Experiment (WOCE) data," JOURNAL OF GEOPHYSICAL RESEARCH-OCEANS, v.108, 2003.
Sun, W; Cetin, M; Thacker, WC; Chin, TM; Willsky, AS. "Variational approaches on discontinuity localization and field estimation in sea surface temperature and soil moisture," IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, v.44, 2006, p. 336-350.
Troch, PA; Paniconi, C; McLaughlin, D. "Catchment-scale hydrological modeling and data assimilation," ADVANCES IN WATER RESOURCES, v.26, 2003, p. 131-135.
Valstar, JR; McLaughlin, DB; Stroet, CBMT; van Geer, FC. "A representer-based inverse method for groundwater flow and transport applications," WATER RESOURCES RESEARCH, v.40, 2004.
Van Horne, MP; Vivoni, ER; Entekhabi, D; Hoffman, RN; Grassotti, C. "Evaluating the effects of image filtering in short-term radar rainfall forecasting for hydrological applications," METEOROLOGICAL APPLICATIONS, v.13, 2006, p. 289-303.
Vivoni, ER; Entekhabi, D; Bras, RL; Ivanov, VY; Van Horne, MP; Grassotti, C; Hoffman, RN. "Extending the predictability of hydrometeorological flood events using radar rainfall nowcasting," JOURNAL OF HYDROMETEOROLOGY, v.7, 2006, p. 660-677.
Willsky, AS. "Multiresolution Markov models for signal and image processing," PROCEEDINGS OF THE IEEE, v.90, 2002, p. 1396-1458.
Willsky, AS; Natti, S. "Excerpts from "Some solutions, some problems, and some questions"," IEEE CONTROL SYSTEMS MAGAZINE, v.27, 2007, p. 13-16.
Wunsch, C., and D. Stammer. "Global Ocean Data Assimilation and Geoid Measurements," in G. Beutler, R. Rummel, M.R. Drinkwater, and R. von Steiger, Earth Gravity Field from Space - from Sensors to Earth Sciences, Space Sciences Series of ISSI, v.1, 2003.
Wunsch, C; Heimbach, P. "Estimated decadal changes in the North Atlantic meridional overturning circulation and heat flux 1993-2004," JOURNAL OF PHYSICAL OCEANOGRAPHY, v.36, 2006, p. 2012-2024.
Wunsch, C; Heimbach, P. "Practical global oceanic state estimation," PHYSICA D-NONLINEAR PHENOMENA, v.230, 2007, p. 197-208.
Wunsch, C; Stammer, D. "Global ocean data assimilation and geoid measurements," SPACE SCIENCE REVIEWS, v.108, 2003, p. 147-162.
Zang, X; Malanotte-Rizzoli, P. "A comparison of assimilation results from the ensemble Kalman filter and a reduced-rank extended Kalman filter," NONLINEAR PROCESSES IN GEOPHYSICS, v.10, 2003, p. 477-491.
Zhou, YH; McLaughlin, D; Entekhabi, D. "Assessing the performance of the ensemble Kalman filter for land surface data assimilation," MONTHLY WEATHER REVIEW, v.134, 2006, p. 2128-2142.
Zhou, YH; McLaughlin, D; Entekhabi, D; Ng, GHC. "An ensemble multiscale filter for large Nonlinear data assimilation problems," MONTHLY WEATHER REVIEW, v.136, 2008, p. 678-698.
