A repository holding code and example notebooks for change detection methods and post-classificaiton processing for the Dynamic World Land Cover product. Dynamic World is a joint iniative between the World Resources Institute, Natioanl Geographic Society, Google, and Impact Observatory. The Dynamic World Land Cover product is a 10-meter resolution, Sentinel-2 based land cover dataset that runs globally. Dynamic World classifies Sentinel-2 scenes and can be reduced to annual classifications.
Due to noise and classification errors, some changes in classifications from year to year may not represent actual change on the ground. Therefore many land cover products apply change detection or post-classification filters to improve the consistency of the land cover product through time or improve the classification accuracy on the training data. The Resource Watch team within the World Resources Institute has developed this repository to demonstrate five change detection and post-classification approaches that can be used in Dynamic World.
All Python modules are contained in the wri_change_detection folder which will soon be installable via pip as well. Jupyter notebooks demonstrating methods for change detection and post-classification processing are contained in the demo_notebooks folder.
You can view the package on PIP at https://pypi.org/project/rw-dynamicworld-cd/0.0.1/
The demo notebooks contain 5 notebooks:
- MapBiomas_Spatial_Temporal_Filters.ipynb: demonstrating the application of gap filling, spatial filters, temporal filters, frequency filters, and incidence filters following code from the MapBiomas land cover product.
- NeighborhoodPrediction_LC_ChangeDetection_Example.ipynb: allows the user to train a model to predict whether change that occurs from yeari to yeari+1 stays consistent in yeari+2 using properties of the neighboring pixels as predictor variables.
- SeasonalProbabilitiesPrediction_LC_ChangeDetection_Example.ipynb: allows the user to train a model to predict whether change that occurs from yeari to yeari+1 stays consistent in yeari+2 using seasonal class probabilities in yeari and the difference in seasonal class probabilities from yeari to yeari+1.
- Probability_Filters.ipynb: allows users to apply probability filters as general rules in post-processing, such as "classify all pixels with built-area probability greater than 0.3 as built-area" or "replace all pixels with forest probability less than 0.2 with the neighboring class". This method was inspired by Malinowski et al. 2020.
- HiddenMarkovModel_LC_Postprocessing_Example.ipynb: allows users to apply a Hidden Markov based post-classification algorithm, based off of Abercrombie and Friedl 2016, which is used in the MODIS Collection 5.1 Land Cover Type product (MCD12Q1).
The approach for each of the five methods is defined below.
MapBiomas is an initiative of the Greenhouse Gas Emissions Estimation System (SEEG) from the Climate Observatory's and is produced by a collaborative network of co-creators made up of NGOs, universities and technology companies organized by biomes and cross-cutting themes. The MapBiomas land cover products are 30-meter resolution, Landsat based land cover products that covers Brazil and other regions in South America. You can learn more about the MapBiomas project at their home page. You can read more of the methodology in the Algorithm Theoretical Basis Document (ATBD) Page on their website, including the main ATBD and appendices for each each biome and cross-cutting themes.
The ATBD goes on to describe five post-classification filters that are applied. For each filter, the Resource Watch team converted original code from MapBiomas from Javascript to Python when code was available, and otherwise coded the filters as close to the description. You can view the associated demo notebook MapBiomas_Spatial_Temporal_Filters.ipynb and Python modules in post_classification_filters.py to view how this method can be applied to the Dynamic World land cover product.
From Section 3.5 of the ATBD, MapBiomas defines post-classification filters, "[due] to the pixel-based classification method and the long temporal series, a chain of post-classification filters was applied. The first post-classification action involves the application of temporal filters. Then, a spatial filter was applied followed by a gap fill filter. The application of these filters remove classification noise. These post-classification procedures were implemented in the Google Earth Engine platform"
- 3.5.1. Gap fill
- The Gap fill filter was used to fill possible no-data values. In a long time series of severely cloud-affected regions, it is expected that no-data values may populate some of the resultant median composite pixels. In this filter, no-data values (“gaps”) are theoretically not allowed and are replaced by the temporally nearest valid classification. In this procedure, if no “future” valid position is available, then the no-data value is replaced by its previous valid class. Up to three prior years can be used to fill in persistent no-data positions. Therefore, gaps should only exist if a given pixel has been permanently classified as no-data throughout the entire temporal domain.
- 3.5.2. Spatial filter
- Spatial filter was applied to avoid unwanted modifications to the edges of the pixel groups (blobs), a spatial filter was built based on the “connectedPixelCount” function. Native to the GEE platform, this function locates connected components (neighbours) that share the same pixel value. Thus, only pixels that do not share connections to a predefined number of identical neighbours are considered isolated. In this filter, at least five connected pixels are needed to reach the minimum connection value. Consequently, the minimum mapping unit is directly affected by the spatial filter applied, and it was defined as 5 pixels (~0.5 ha).
- 3.5.3. Temporal filter
- The temporal filter uses sequential classifications in a three-to-five-years unidirectional moving window to identify temporally non-permitted transitions. Based on generic rules (GR), the temporal filter inspects the central position of three to five consecutive years, and if the extremities of the consecutive years are identical but the centre position is not, then the central pixels are reclassified to match its temporal neighbour class. For the three years based temporal filter, a single central position shall exist, for the four and five years filters, two and there central positions are respectively considered.
- Another generic temporal rule is applied to extremity of consecutive years. In this case, a three consecutive years window is used and if the classifications of the first and last years are different from its neighbours, this values are replaced by the classification of its matching neighbours.
- 3.5.4. Frequency filter
- This filter takes into consideration the occurrence frequency throughout the entire time series. Thus, all class occurrence with less than given percentage of temporal persistence (eg. 3 years or fewer out of 33) are filtered out. This mechanism contributes to reducing the temporal oscillation associated to a given class, decreasing the number of false positives and preserving consolidated trajectories. Each biome and cross-cutting themes may have constituted customized applications of frequency filters, see more details in their respective appendices.
- 3.5.5. Incident filter
- An incident filter were applied to remove pixels that changed too many times in the 34 years of time span. All pixels that changed more than eight times and is connected to less than 6 pixels was replaced by the MODE value of that given pixel position in the stack of years. This avoids changes in the border of the classes and helps to stabilize originally noise pixel trajectories. Each biome and cross-cutting themes may have constituted customized applications of incident filters, see more details in its respective appendices.
The goal of this method is to predict whether change that occurs in one year is "permanent" or "consistent", which can be defined by the user, using properties of the neighboring pixels as predictor variables in the model. One definition of consistent change is if change occurred from yeari to yeari+1, then did not change from yeari+1 to yeari+2. This approach builds a binary classification model and gathers training data from the land cover classification product. The model uses properties of the neighboring pixels as predictor variables for example how many of the surrounding pixels transitioned, how many of the surrounding pixels are of each class, etc. Neighborhoods are defined using kernels, and there are many options for the kernel including shape and size. The output of the model is a probability ranging from 0 to 1 that the transition is consistent.
You can view the associated demo notebook NeighborhoodPrediction_LC_ChangeDetection_Example.ipynb. The demo notebook uses Python modules for preprocessing in preprocessing.py and for training a classifier in gee_classifier.py.
This approach is similar to the one above, except now the predictor variables are seasonal probabilities for each land cover class for yeari and the difference in seasonal probabilities from yeari to yeari+1. The goal of this method is to predict whether change that occurs in one year is "permanent" or "consistent", which can be defined by the user, using seasonal probabilities of the current year and the following year. One definition of consistent change is if change occurred from yeari to yeari+1, then did not change from yeari+1 to yeari+2. This approach builds a binary classification model and gathers training data from the land cover classification product. The predictor variables can include the seasonal probabilities for yeari and the difference in seasonal probabilities from yeari to yeari+1, but could also include the difference in seasonal probabilities from yeari-1 to yeari. The user controls the definition of the seasons, including the number of seasons and date ranges of the seasons. The output of the model is a probability ranging from 0 to 1 that the transition is consistent.
You can view the associated demo notebook SeasonalProbabilitiesPrediction_LC_ChangeDetection_Example.ipynb. The demo notebook uses Python modules for preprocessing in preprocessing.py and for training a classifier in gee_classifier.py.
In classifying land cover, pixels are assigned probabilities for each land cover class that the pixel belongs in that land cover class. Oftentimes the land cover class with the highest probability is chosen as the final classification. However some land cover products choose to apply rules to these classification probabilities in order to increase the final accuracy, such as the 10m Sentinel-2 Based European Land Cover map created by Malinowski et al. 2020.
In this approach, there are two ways the probabilities can be filtered.
- Greater Than Filter: This defines a minimum probability for that land cover class to be chosen, regardless if there are other classes with higher probabiliites. If the class probability is higher than the defined threshold, then the pixel is assigned that class. For example, all pixels with the "built-area" class probability greather than 0.3 will be reclassed as the "built-area" class.
- Less Than Filter: This defines a maximum probability for the land cover class to be replaced by the mode of the neighboring pixels. If the central pixel's class probability is lower than the defined threshold and the central pixel was assigned that class, it will be replaced by the mode of the neighboring pixels. For example, all pixels that were assigned "snow-cover" with a "snow-cover" class probability less than 0.5 will be reclassed as the mode of the neighboring pixels.
You can view the associated demo notebook Probability_Filters.ipynb. The demo notebook uses Python modules for preprocessing in preprocessing.py and post_classification_filters.py to apply the post-classification probability filters.