The Ontic 42| Hey! What Are You Looking At?
Awards & Nominations
The Ontic 42 has received the following awards and nominations. Way to go!
Hey! What Are You Looking At?
Cosmic Engine: An interactive visualization tool to query HEASARC data
Summary
We have created "Cosmic Engine", a user-friendly web application to automate the querying of HEASARC data from the STScI NAVO Directory. Our UI is designed as an interactive dome that enables the user to search for objects from all the HEASARC catalogues in NAVO within a specific angular radius centered at a set of inputted RA and Declination coordinates in the sky (or an inputted name for a celestial object).The user can select one catalogue from a number of available results and view the entire resulting set of objects under the search radius. Each of these objects will have specific data available for them, and the user has the option to display or download this data.
How We Addressed This Challenge
We, The Ontic 42, have created "Cosmic Engine", an interactive, user-friendly web application that endeavors to automate the querying of HEASARC data from the STScI NAVO Directory.
Accessing or querying data for a specific service, be it SSA (Simple Spectral Access), SIAP (Simple Image Access Protocol) or Simple Cone Search, is either a long, repetitive & arduous procedure that involves understanding the highly specific formatting of URLs and XML files, or it requires proficiency with computational tools like Python packages (PyVO) that have high friction owing to their inherently user-unfriendly nature. Interfaces such as Xamin exist, but acquiring a firm grasp over these tools is limited only to professionals or those who have had to learn and repeat these procedures out of necessity. Access to the data, thus, is limited to such a minority.
To increase the accessibility of the information that such platforms store, we create an alternative tool that outsources all the manual labor to our computational system and makes the process of data acquisition much more intuitive. Cosmic Engine provides an interactive dome that displays the night sky with the functionality to pan and zoom over its specific patches. The user can also toggle backgrounds with celestial grids, constellations, or celestial boundaries as per their preference. Its utility over conventional night sky viewing software such as Stellarium lies in its ability to directly query scientific data from the aforementioned directory and its ease of access owing to its web-app nature of deployment.
To fetch the data centered around a specific set of Right Ascension and Declination co-ordinates, the user need only input these values and the angular radius around which they would like to perform their search. Alternatively, the user can acquire the co-ordinates of a specific object in the sky by its name in the Sesame Name Resolver. Additionally, the user can select the type of service (SSA, SIAP, Cone Search) and a specific catalogue from a number of available catalogues displayed depending on the service type. For a specific catalogue, the user will be shown a list of celestial objects it stores that are observed within the search radius. Each of these objects will have specific data associated with it depending on the type of service selected by the user (including access to images and spectral data). Finally, the user can, depending on their purposes, choose to either display this data or download it.
Cosmic Engine is a web application. The interactive dome itself is created using the three.js, a JavaScript 3D library, and the querying of the data from Virtual Observatories is carried out using standard Python libraries. In the future, we hope to relax the specifications to HEASARC data and, instead, generalize to archives of other astronomical phenomena. With each subsequent release of Cosmic Engine, we hope to increase functionality of our tool, to optimize the rendering of the celestial dome and fetching the data, and to make our code more robust in general. In addition, we intend to add the ability for the user to analyze the acquired data using our interface itself (starting with spectral data). The pipeline for such real-time analyses will be made such that they correspond to Use Cases of this data in science.
We hope that tools such as these can serve as long overdue alternatives to the inefficient and laborious conventional procedures that one has to follow simply to acquire the data. Besides increasing the efficacy of the process of data acquisition for professional astronomers, we hope that the visual nature of this tool can be beneficial for many of the citizen science initiatives that have emerged around the world. Ease of access to data such as these can help increase the number of individuals and institutions performing analyses on them, thereby potentially increasing the rate at which astronomical research is conducted and aiding the prospect of human discovery.
How We Developed This Project
Our team, a mixture of astrophiles and tech-savvy developers, all share the inherent wonder that comes with looking up and gazing at the night sky. Observational astronomy enables us, as a species, to look up and peer into the depths of the universe. With the amount of data that NASA missions gather, and the increasingly open culture of astronomy when it comes to publicly releasing this information and open-sourcing the tools that enable its analyses, we believe that ease of access is an imperative part of both the research process and the outreach of this data.
To contend with querying data for HEASARC missions, we utilized the NAVO Directory and approached the problem in such a way that would minimize the manual labor for the user for whom this tool has been developed. Cosmic Engine is currently a web application.
Our code is split into a frontend and a backend.
- For the front-end, we used three.js, a JavaScript 3D library, to render the celestial dome that the user can interact with. The user can pan and zoom around different patches of the dome to change their field of view of the night sky. Here, the user can select the "VO Service" (Virtual Observatory query) button to select from an option of three types of services that correspond to options in the "Compatibility Type" filters in the NAVO Directory. They can then select the catalogue they would like to explore simply by clicking on one of the displayed items from a list of available catalogues (they can also search for a catalogue by title). After selecting a catalogue, the users are prompted to input either the name of an object or the celestial coordinates around which they would like to perform a search. If the user chooses to input a name, Astropy functions in the back-end use the Sesame Name Resolver to search for corresponding RA and Declination values. The user also inputs the search radius in degrees. The catalogue ID, celestial co-ordinates, and search radius is then fed into functions in the back-end to obtain our final search results- a list of celestial objects. Each of these objects will have specific metadata depending on the catalogue and the type of service chosen. For SIAP, the user has the option to view the image in our interface itself if it is returned in a JPG format.
- For the back-end, we created a dedicated Python module to handle the access of data from the NAVO directory as well as perform some conversion and search tasks implicitly. For data access, the search parameters taken from the front-end are used to request an XML file from the NAVO directory (several patterns for the URLs of the XML files were manually identified, others were identified using rules provided on the HEASARC website) by using web-scrapers like BeautifulSoup. This fetched XML file is then parsed and formatted in a manner that is consistent across all the different catalogues. This functionality is currently available for the Simple Image Access Protocol, Simple Cone Search, and Simple Spectral Access. The back-end was coded in such a way that would be easily generalizable to other capability types (as such, future releases could include this generalization). For each type of service (SIAP/SCS/SSA), to enable the functionality to search for catalogs (and their information such as names, descriptions, and so on), these search results were web-scraped from the NAVO Directory website and stored on the Cosmic Engine server previously. This stored data will be periodically updated to include more recent HEASARC data as they are added onto the NAVO Directory.
The front-end and the back-end work in tandem to provide an intuitive interface that the user can access through a browser either on a computer or a mobile device.
The main challenge for our team was to strike the right balance between providing ease of access or user-friendliness and how scientifically informative we can make the tool (more specifically, the level of complexity of the data that we can enable the users to acquire). We decided to include the functionality of accessing the following three types of services in the virtual observatory- SIAP, SSA and Cone Search. For the purposes of this release, we limited our search only to those catalogues that have "nasa.heasarc" in their ID. Our UI allows the user to, thus, access all the available HEASARC missions on the NAVO directory without having to be proficient in Python packages or other sophisticated interfaces designed to navigate the archive.
Ultimately, we determined that the real-world utility of this visualization tool gives us incentive to continue developing it under future releases with additional functionalities such as the following.
- Generalize to include datasets from beyond HEASARC missions. To have better visualization power, map points in the sky corresponding to a selected catalogue's set of data points.
- Acquisition of time series data (for example, for Gravitational Wave events from platforms like GW Open Science Center or APIs like GraceDB when operational, or using Xamin to obtain images with certain time stamps so the user can access data from the past). Make data acquisition more compatible with conventional methods by downloading data using Perl scripts (as described here).
- Addition of functionality to analyze data (for example, analysis of Spectral Data using tools like PyXspec or analysis of FITS files returned by the services) by interacting with the Cosmic Engine UI itself. To make the tool more handy for scientists, add direct pipelines in the front-end to analyze data as in the Use Cases provided here such that the same functionality is provided within the interactive UI without performing any manual coding.
- Incorporate dynamic rendering of higher-resolution pictures limited to the field of view of the celestial dome when zoomed in, and add a dialog box to input parameters and query images directly from the SkyView VO. Rotate to inputted co-ordinates and add a "hyperdrive" effect in the front-end before displaying the returned image. Automate the acquisition of SkyView VO images for all types of queries when a particular RA, Declination and search radius is provided and "zoom in" to these images regardless of the type of query.
- Using Astropy functions, include the option to vary the night sky depending on the latitude and longitude of the observer (add a "relative" night sky) by rotating the celestial dome.
How We Used Space Agency Data in This Project
We utilized the STScI NAVO Directory to access all the HEASARC archive data on the platform by filtering by (IVO)IDs to display only those that include "nasa.heasarc". We then accessed data for specific service/compatibility types on the directory through our web-scraping functions (this is analogous to filtering by compatibility type on the NAVO Directory). Within each catalogue, we fetch data only around specific co-ordinates that the user enters. In theory, the user could input a large enough search radius to obtain all data points within the catalogue with the available access type, but this would be computationally expensive as our API dynamically scrapes the NAVO directory to obtain XML files in real time.
In particular, we followed the conventions and the tutorials offered on the HEASARC website to query VO-enabled services to generate URLs that point to the XML files we've parsed. This parsed data is displayed on the front-end.
Project Demo
Google Slides: https://bit.do/ontics42
YouTube Video (Overview): https://youtu.be/ZG6BgPM5Isw
Project Code
Data & Resources
Data:
We utilized the STScI NAVO Directory to access all the HEASARC archive data on the platform by filtering the IVOIDs to display only those that include "nasa.heasarc". Accessing a catalogue for a specific service type in our interface is equivalent to selecting a capability type on the directory.
Resources:
Programmatic VO interfaces to HEASARC catalog holdings and services through Python