Lighteners

Lighteners| Data Discovery for Earth Science

Awards & Nominations

Lighteners has received the following awards and nominations. Way to go!

Global Nominee

Data Discovery for Earth Science

Websites like the NASA Earth Observatory showcase the many uses of satellite data to highlight interesting natural events. International partner instruments on NASA satellites such as Japan’s ASTER instrument and Canada’s MOPITT instrument, both onboard the Terra satellite, are also included as part of the Observatory. This challenge will ask you to devise a tool or technique to guide users to relevant datasets to study specific events.

Summary

According to research, 19.8 % of the working hours – the equivalent of one day per working week – is wasted searching for information to do our job effectively.So, launching from the idea that time is a double-edged sword, We developed a solution to save scientists' valuable time in searching for their wanted articles in tons of links. Our solution is a graphical user interface (GUI) tool that enables the scientist to enter his queries of interest or search with an image. Related data will then appear containing links to related articles, common keywords, and a summary of each link. He can also specify a date to retrieve the data after that date.

How We Addressed This Challenge

We developed a Graphical User Interface (GUI) tool with a variety of searching ways: queries, images, and dates:

Searching through an image can help scientists to know more about a new strange phenomenon taken by a satellite as an example. As our GUI gives the related data to it, this enables scientists to predict the reality of this phenomenon and know more information about it.
Searching through a query can help scientists to search using a word or a sentence and retrieve the related data.
Related data of the images or the queries contain similar articles' hyperlinks, common keywords in each link to enable scientists to take an overview without the need to open it, and a summary of each link.
Scientists can also Filter by date and visualise the number of related articles in each year. This allow them to study the phenomenon of interest in detail by relating the year parameters to the number of articles written, as the number increases by changing the parameters like in our Corona era.
Users can also specify the number of retrieved data and save the retrieved images for future use.

How We Developed This Project

Inspiration:

Being interested in artificial intelligence technology and Natural Language Processing (NLP) in particular, we wanted to enhance our knowledge by solving this interesting problem. As engineers, we know the value of time, so we became more interested in this project to be able to minimize scientist's lost time.

Approach:

Our approach begins by clarifying the project's requirements and its scopes, setting our schedule, determining our risks, collecting needed resources, revising the technologies that we will need, and dividing the problem into several tasks.

Languages and tool:

We developed this code in python, applying NLP for text preprocessing, using BERT, and sci_BERT to train and fit our model as a deep learning tool. For collecting the data, we used web scraping technology to extract the content of the URLs.

Problems :

Main problem we faced is the data. There was no data specifically for this competition.

Achievements:

We were able to collect our data using web scraping in a very effective way that gets the most out of the given URLs like images, date, and content.
We mixed between two dominant tools in deep learning (BERT and sci_BERT) to get more accurate results.
We visualized the output results to help researchers take a look at the relevant datasets as a whole.
We summarized each topic in the output datasets by visualizing the main keywords and its frequency.

How We Used Space Agency Data in This Project

We collected the data for our challenge by collecting the URLs of some of NASA's web pages from the given resources. Then, by applying web scraping on them, we were able to retrieve the content, the date, and the images of each URL.