NTUT Smart Medical Cloud - Space Apps Challenge

Team Updates

Hello all! We'd like to share our project with the global community so we made an English version of its details. Feel free play around with our program!

Project

I will be there before the next snooze.

Summary

Fatigue monitoring and sleeping schedule management

How We Addressed This Challenge

This project consists of:

- a scheduling tool (a database application -- astronaut-scheduling-master.zip),

- a human-face analysis application (a Python 3 computer vision application -- FatiqueDetect.py) , and

- a program to work with the Arduino-MAX30100 oximeter module (MAX30100_Minimal.ino).

The scheduling tool database application serves to manipulate tasks data from an SQL database, while the oximeter module program polls sensor data via I2C from an Arduino Nano device.

The Python 3 human-face analysis application uses the shape_predictor_68_face_landmarks.dat model, which perceives the human face as a combination of multiple oval shapes and applies facial landmark detection to localize important regions of the face. By calculating the default (eyes open and mouth closed) aspect ratios of such facial regions (the eyes and the mouth) with the vertical and horizontal landmark distances of their respective oval shapes, the program can then observe changes in such ratios to detect blinking and yawning.

How We Developed This Project

According to NASA's Life Sciences Data Archive, an astronaut's sleep schedule management should include two features:

1. A scheduling tool

Space missions could vary in terms of difficulty and duration. Some might be rudimentary or routine, while others might pose a severe degree of difficulty. Thus, it is crucial to take into account all aspects of the astronaut's conditions. The mission schedule should be planned according to the following parameters:

- SpO2 level, pulse frequency and heart rate

- gender and age

- health conditions

2. Real-time detection

Every individual has their own physical conditions and lifestyles. In space, every astronaut could respond to the environment and its effects differently. Hence a one-size-fits-all measuring system in terms of fatigue might not be applicable in space missions.

Real-time fatigue detection should be incorporated into the scheduling system so that once an astronaut is found unfit for duty due to exhaustion or other physical conditions, appropriate measures could be taken immediately to avoid accidents or errors.

Therefore, we conclude that our project should include the following programs:

- a scheduling tool (a database application -- astronaut-scheduling-master.zip),

- a human-face analysis application (a Python 3 computer vision application -- FatiqueDetect.py) , and

- a program to work with the Arduino-MAX30100 oximeter module (MAX30100_Minimal.ino).

The scheduling tool database application serves to manipulate tasks data from an SQL database, while the oximeter module program polls sensor data via I2C from an Arduino Nano device.

The Python 3 human-face analysis application uses the shape_predictor_68_face_landmarks.dat model, which perceives the human face as a combination of multiple oval shapes and applies facial landmark detection to localize important regions of the face. By calculating the default (eyes open and mouth closed) aspect ratios of such facial regions (the eyes and the mouth) with the vertical and horizontal landmark distances of their respective oval shapes, the program can then observe changes in such ratios to detect blinking and yawning.

How We Used Space Agency Data in This Project

NASA's Life Sciences Data Archive

Project Demo

Fatigue detection by observing eye blinks and yawns:

https://www.youtube.com/watch?v=1gGkXZ4Altk&t=72s

Presentation deck:

https://drive.google.com/file/d/1pd4o_e8rQli2Gq6aAPl9do_hL3BcJO__/view?usp=sharing

Project Code

https://drive.google.com/drive/folders/1uwPtX-XRXEmaE2UCoC3jfQwH_bBAjam7?usp=sharing

Data & Resources