DeepFeatures: Learning Latent Representations from Spectral Indices for Ecosystem Monitoring

Monitoring and understanding Earth system dynamics and their response to climate change and human activity requires innovative approaches to analyse complex and multivariate remote sensing data. However, the current trend is towards large models that require a lot of memory and computational power to be trained. The DeepFeatures project addresses this challenge by developing an embedding approach to create Feature Data Cubes, which capture the underlying spatio-temporal ecosystem dynamics as a low dimensional representation in latent space. These reduced representations enable the use of simpler, resource-efficient downstream models, which are easier to train and require minimal computational resources.Specifically, the project builds on the rationale that each spectral index (SI), which is calculated from spectral bands and represents certain surface properties such as vegetation greenness, reflects a specific aspect of ecosystem behaviour. Despite the development of over two hundred spectral indices, current studies often narrow their focus to individual SIs, overlooking the broader context of land surface processes represented by not considered SIs. The DeepFeatures project addresses this challenge by adopting a spatio-temporal multivariate approach. The SIs are derived from Sentinel-2 observations to generate a SI Data Cube. A deep learning embedding algorithm is applied to reduce the SI dimension and extract a latent space to create the Feature Data Cubes.To demonstrate the potential of the Feature Data Cubes, the project focuses on inference across a range of scientific applications, including modelling gross primary production, analysing tree mortality and greening trends, biodiversity monitoring for conservation, comparing phenological features using satellite and crowd-sourced data, and studying the ecological impacts of open-pit lignite mining.DeepFeatures emphasises the deployment of transparent and reproducible workflows, from generating Sentinel-2 derived Training Data Cubes to creating Feature Data Cubes. It aims to have an accessible, extensible, and modifiable framework for diverse applications, fostering broad community engagement and enabling open exploration of Earth system dynamics.This presentation will showcase the methodology, scientific cases, and transformative potential of the DeepFeatures framework, highlighting its contributions to Earth observation and climate research.The project DeepFeatures is funded by ESA’s AI4Science activity. Website: https://rsc4earth.de/project/deepfeatures/