Context and objectives
The forecasting of ecosystem response to climate extremes is crucial for understanding and mitigating the impacts of climate change. However, current vegetation models used for scenario simulation and sensitivity analysis are limited by their parametric simplicity and the lack of accurate data to constrain them. This hinders the reliable prediction of ecosystem responses to future climate extremes. To address this challenge, an innovation project called DeepV is proposed, aiming to develop data-driven, spatially explicit ecosystem response models based on remote sensing data and conditional Generative Adversarial Networks (cGANs). These cGANs have shown success in generating realistic data, such as true color and multi-spectral satellite images, using deep learning techniques. The objective of the DeepV project is to expand and explore the use of cGANs for generating fake yet realistic satellite image time series of ecosystem response.
Project outcome
The project follows a structured approach combining dataset development and advanced machine learning:
-
Development of GeoClimaData, a harmonised dataset integrating multi-year Sentinel 2 imagery with meteorological and environmental data into spatio-temporal data cubes.
-
Implementation of conditional generative models (cGANs) for predicting satellite imagery conditioned on environmental variables, including uni-temporal and multi-temporal approaches.
-
Integration of biophysical constraints through model conditioning and loss functions, improving ecological realism (e.g. NDVI-based constraints).
-
Extension towards spatio-temporal modelling and handling of real-world EO data, including irregular temporal sampling.
-
Adaptation of state-of-the-art transformer-based models (e.g. Contextformer) to overcome limitations of cGANs for time series modelling.
This framework enables the combination of data-driven modelling with physically meaningful constraints, bridging machine learning and Earth system science.
The project resulted in several key scientific advances:
-
Development of GeoClimaData, a novel EO–climate dataset enabling flexible and realistic modelling of ecosystem dynamics.
-
Demonstration of generative modelling of satellite imagery using cGANs, including evaluation of different architectures and loss functions.
-
Evidence that temporal context improves model performance, particularly for vegetation-related indicators such as NDVI.
-
Demonstration that biophysical constraints are essential to ensure ecological realism in generative models.
-
Identification of limitations of cGANs for complex time series modelling, leading to the adoption of transformer-based methods adapted to EO data.
Importantly, the project also shows that conditional generative modelling of EO data forms a general methodological framework, applicable beyond the original climate–ecosystem focus.
| Project leader(s): | KULeuven - Department of Earth and Environmental Sciences (EES) | |
| Location: | ||
| Related presentations: | ||
| Website: | http://www.earthmapps.io/projects/DeepV.html | |