Automated Detection of Hazards
Detecting and Monitoring of Illegal Sand Mining Activities using Advanced Computing Methods
Summary
Sand is 2nd most valuable resource we have today. The exponential increase in the demand for this resource has resulted in illegal sand mining activities. Detection and monitoring of illegal sand mining networks is a time-consuming and laborious process. These illegal activities have catastrophic effects on the environment, often results in natural disasters and the destruction of local ecosystems. We developed Sandsat tool to autonomously detect and monitor illegal sand mining activities. In this hackathon, we apply this tool to a use-case which is a portion of Poyang Lake (one of the worst affected areas due to illegal sand mining activities) to identify these activities.
How We Addressed This Challenge
Why Sand Mining?
Although sand mining is probably not the first issue that comes to mind when talking about natural or environmental hazards, yet its consequences are far reaching. More over: sand is the most extracted resource after water.
The human population is growing fast and more people are moving to cities than ever before. Building and expanding these cities cannot be done without sand that is used to make concrete and glass, the primary materials of nearly all buildings. Yet not all sand can be put to use, making sand a finite resource. To use sand in construction, angular sand is needed since the small irregularities in the sand grains keep them together when making concrete. This sand is found in rivers, beaches, and islands, usually places with high ecological value, but often also places where humans live.
Extensive sand mining was found to cause collapsing bridges, river banks, destroy fishing grounds, impact wildlife and lower river levels. In the long-term sand mining causes coastal erosion that makes countries more vulnerable for flooding.
What Is Sand Sat Toolbox?
The Sand Sat Toolbox is a processing pipeline combining remote sensing with machine learning to detect sand mining from shoreline dynamics over time. It uses data from Landsat- and Sentinel missions and is operational via Google Earth Engine. Sand mining activities are filtered out from natural trends and flagged over space and time. Shoreline dynamics can even be predicted to evaluate the effects of current sand mining intensities on coastal erosion.
Sand Sat Toolbox combines the detection and monitoring service with a mobile app for citizens and activists to report suspicious activities or illegal sand mining. The crowd-sourced data is then collected and used as input for the toolbox to detect sand mining activities.
How Does It Work?
The Sand Sat Toolbox takes images from Landsat 5,7 , and 8, as well as Sentinel-2 data. It applies pre-processing to enhance image resolution and to get rid of cloudy pixels, after which every image is classified in the classes 'sand', 'water', 'whitewater' , and 'other'. Then the shoreline is mapped on every sand-water interface, creating poly-lines on every image. These shorelines are then intersected with a transect that is stable over the years, to see the development of the coastline over time. The data from the transects are passed to a time-series decomposition machine learning model called SARIMAX to separate natural trends and residues. When shoreline decline exceeds natural levels, a transect is flagged over space and time as a location with suspicious activity.
What's Up Next?
Sand Sat Toolbox is not validated yet, so the first step from now is validating results from various sand mining locations around the world, as well as locations without sand mining.
We want to build and launch our citizen-science app that will give us crowd-sourced information about sand mining from anywhere in the world. Together with the app, we want to invest in good relationships with the citizens and activists.
With more data, more sand mining locations can be uncovered, and with higher spatial and temporal resolutions, activities can be better flagged over space and time and perhaps even at near real-time.
We want to extend current capabilities with a ship tracking algorithm that would be capable of signaling (illegal) sand mining as it happens, and possibly identify the owners of the sand mining vessels.
We want to extend the tool as well with remotely sensed bathymetry data, which could ultimately be used in contract compliance monitoring. This is useful because sand mining contracts are often exploited to mine beyond the legal limits. When a sand mining company makes a contract with local authorities, we can monitor whether the contract is being adhered to by both parties, using the remotely sensed bathymetry data.
How We Developed This Project
Sand is one of the most valuable resources we have available today. It forms the very basis of the modern civilization. We have seen an exponential increase in the demand for this valuable yet exhaustible resource. These alarming rates are a result of ever increasing building construction projects and other infrastructural development. Consequently, the process of acquiring it has become expensive and laborious for builders. Ergo, numerous networks have cropped across the world who conduct illegal sand mining activities. Even though, we have vast expanses of sand in the Sahara and other deserts, this type of sand is not suitable for construction.Therefore the sand found near rivers and lakes is more desirable. These regions are regarded as a critical resource in view of its recreational, environmental and economic importance.
At present, the detection and monitoring of illegal sand mining is highly tedious and in-efficient. The open data on illegal sand mining today comes from in-situ measurements, however this is limited to only a few spots in the world. The primary source of information on such activities comes from inhabitants & sand activists who would like to raise awareness. However, in most cases their claims are disregarded due to lack of evidence. Furthermore, this large manual effort restricts the ability of governments and environmental agencies to identify and curb these illegal activities.
During the course of the challenge we aim to find a solution for the aforementioned problem with the use of satellite remote-sensing data and autonomous computer methods. We propose the Sahara Mobile application (user end platform) & SandSat toolbox (toolbox to detect & monitor illegal sand mining activities.) as a solution for the aforementioned problem. We apply this approach to detection of illegal sand mining activities at a specific region of Poyang Lake.
An overview of the methodology can be found in the figure below:
Figure 1: The figure shows an end-to-end pipeline we use to identify illegal sand mining.
The first step of the pipeline, is the development of a mobile application in order to provide inhabitants & activists a safe platform to raise concerns regarding illegal sand mining activities. This application will facilitate the users to request an in-depth analysis of a potential illegal sand mining area. In the next step the available data for the "flagged" area is downloaded and processed for further analysis. However, for the purposes of a working prototype, we have shortlisted worst areas affected by sand mining activities. In the following step, we take this information and feed it into the SandSat toolbox. This toolbox classifies the data into different classes to establish a boundary between sand & water - shoreline. The variation of this shoreline (change of the shoreline over a period of time) is then processed to identify potential illegal sand mining activities in that area. Finally, in Step-4 the results of the analysis are sent to all stakeholders (such as government agencies, media outlets, environmental agencies) including the original requester, while concealing the identity of the user upon request.
For the purposes of this challenge we are using the optical remote sensing observations from Landsat (Landsat 5, 7, 8 satellites) & Copernicus constellations (Sentinel-2 satellite). In the Sandsat toolbox we utilise the open source remote sensing data of coastal region available from Earth observation satellites. This data is a low-cost alternative to the in-situ measurements of the coastal region, that is limited to only a few sites around the world. We have modified a pre-existing tool Coastsat for the extraction of shoreline from the remote sensing images as per our requirements. Coastsat is an open source project developed by Kilian Vos. Interested readers are encouraged to explore the original project at https://github.com/kvos/CoastSat/
The processed satellite images are classified into the following classes: sand, water, white-water & other features with the use of Neural network classifier called Multilayer Perceptron. Now, in order to identify potential sand mining activities we need to analyse the rate of change in shoreline over the period of time. As per Step-4, this is done with the use of time series decomposition method. The results of the shoreline detection algorithm is provided as an input for the time series analysis of the rate of change of shoreline. The data from various transects is analysed using a Time Series Decomposition Method. As the name suggests, this method decomposes the signal (the observations of the shoreline location with respect to the reference shoreline) into three components, namely; Trend, Seasonality and Residual (noise). The results provide a structured way of understanding the data and hence a means to better detect anomalies in the coastline dynamics as well as to perform forecasts. In this analysis, a so-called additive model is used which indicates that the three components of the signal are added as per the following equation:
O(t) = T(t) + S(t) + R(t)
where O is the observation, T is the trend, S is the seasonality and R is the residual. Of course, the real-world data may not necessarily follow this model and locally might be a multiplicative function. However, after a parametric study, it was concluded that the additive model provides a better forecast. Furthermore, in Step-5 we utilise the past data to predict the coastal dynamics and the rate of change of shoreline with the use of Seasonal Auto-Regressive Integrated Moving Averages with eXogenous regressors (SARIMAX) model.
With this tool we were able to map the autonomously map the shoreline for a use case scenario and also identify potential sand mining spots for poyang lake which can be seen in the figures below:
Figure 2: The shoreline is mapped autonomously with Sandsat for a particular region of Poyang lake
Figure 3: The highlighted areas indicate potential areas for illegal sand mining activities.
Furthermore, with the Sarimax model we were able to forecast the trend for in the future. This trend could be extremely useful to make informed decisions and curb sand mining activities. In the figure below you can see the forecast:
How We Used Space Agency Data in This Project
In the SandSat Toolbox we create time-series of shorelines of lakes, coasts, and river banks to understand the development of the position of sand-water interface over time. From this time-series we analyze natural trends of shoreline dynamics against sharp trend breaks that are unlikely caused by natural phenomena.
To create the time-series data, NASA's Landsat program has been paramount to our success. We used data from Landsat 5 TM, Landsat 7 ETM+, and Landsat 8 OLI, as well as ESA's Sentinel-2 MSI. The Landsat series allowed us to go far back in time, since it covers a time-span of 36 years, with its earliest images dating back to 1984. Without Landsat 5 TM, our analyses could not be as robust as they are now. In addition, the data from other Landsat missions and the Sentinel-2, used in our project helped to have a regular retrieval of cloud-free imagery.
To analyze this data, each image was pre-processed using cloud masking, pan-sharpening, and downsampling. This resulted in imagery without cloudy pixels and enhanced spatial resolution, thanks to the panchromatic band on board of Landsat 7 and 8 that has a higher spatial resolution than its multi-spectral bands and can be used to enhance image resolution. After pre-processing, each image was classified according to a land cover classification scheme separating sand, water, white water, and other land features. With these classified images, shoreline mapping could be applied by drawing a sub-pixel resolution contour line around the water interface using a Marching Squares algorithm.
The result of this pipeline is a mapped shoreline on each input image. These shorelines were then included in a multi-year transect and analyzed in a time-series signal decomposition algorithm called SARIMAX.
Project Demo
We used a portion of the Poyang Lake as a use case scenario to autonomously detect illegal sand mining activities. With initial analysis, we were able to identify "potential" sites for when the sand mining took place. The results along with a description of the methodology can be found here:
https://docs.google.com/presentation/d/10q3hpdYHNlhiEPsPhsqmoCeQn5Lkc-sQODPzRi6QUok/edit?usp=sharing
Project Code
Data & Resources
- Landsat 5 TM
- Landsat 7 ETM+
- Landsat 8 OLI
- Sentinel-2 MSI
- Coastsat tool (https://github.com/kvos/CoastSat)