Recognizing and understanding emotions is vital for mental well-being. In our interview with Min-Xiang Zhang, a clinical psychologist specializing in depression and stress management, we learned that many people lack emotional awareness, and he emphasized the harmful effects of stress on managing emotions. To address this, we developed an app that measures stress levels. Unlike other stress measurement apps and gadgets, our goal is to track stress and offer a cost-effective, user-friendly solution that detects stress levels and provides comprehensive stress management tools.
The application incorporates the following two main features:
Stress measurement: Utilizes Finger Photoplethysmography (FPG) to assess Heart Rate Variability (HRV), converting this data into a quantifiable stress index.
Emotional relief tips: Provides various stress-relief strategies, including breathing exercises, meditation & soothing music, and planning for work & rest.
In addition to these core features, the app is designed with a healing mini-game to enhance user engagement. Daily check-ins are encouraged to foster consistency. Furthermore, the app is integrated with SERENE Beverage by featuring a QR code on the seal that allows users to collect points.
This app is dedicated to helping users recognize their emotions and then gradually follow the app’s recommendations to alleviate negative feelings. It fully embodies SERENE’s mission to deliver a comprehensive solution for managing negative emotions.
▲ The app's homepage.
Heart Rate Variability (HRV) measures the variation in time between heartbeats and reflects how well the body's autonomic nervous system balances between the sympathetic (fight or flight) and parasympathetic (rest and digest) responses. Low HRV is associated with stress, anxiety, and poor health, while high HRV indicates better stress resilience and recovery.
HRV can be analyzed using time-domain and frequency-domain methods:
Time-domain calculations: focus on the direct measurement of time intervals between heartbeats.
Frequency-domain calculations: break down the HRV signal into different frequency bands to analyze the balance between sympathetic and parasympathetic activity.
When stressed, HRV decreases as the sympathetic system takes over, but during relaxation, HRV increases with parasympathetic activation. Chronic stress keeps HRV consistently low, signaling prolonged strain. Monitoring HRV can help manage stress, and activities like deep breathing, meditation, and mindfulness can raise HRV by activating the parasympathetic system.
Wearable devices can now provide real-time HRV monitoring, offering insights into both time and frequency domains. This makes HRV a valuable tool for managing stress, improving emotional and physical well-being, and optimizing recovery, particularly for athletes and those using biofeedback therapies.
However, wearable devices are not widely used and require additional costs. To enable everyone to measure stress anytime and anywhere, we need to find alternative methods for stress measurement.
To help everyone become aware of their negative emotions in daily life, we decided to develop an app, “PokeFish”, that allows users to measure their stress anytime, anywhere, without any additional cost.
No need for wearable devices, stress can be measured simply by placing a finger on the phone's flashlight and camera simultaneously. By shining a flashlight through your finger and observing the subtle color changes caused by blood flow, the app can detect your current stress level.
But, how does the app work?
Photoplethysmography (PPG) is an optical method used to measure volumetric changes in blood. Originating from pulse oximetry, PPG can be obtained through transmission or reflection, typically using the fingertip or forehead. Due to the transparency of human fingernails and the abundance of capillaries and blood at the fingertip, detecting the heart rate cycle is relatively easy.
When a fixed wavelength of light is directed at the fingertip, the contraction and expansion of blood vessels with each heartbeat influences the transmission (e.g., light passing through the fingertip in transmission PPG) or reflection of light (e.g., light near the wrist in reflection PPG). As blood pressure rises with each heartbeat, this change is reflected in the amount of red and infrared light absorbed or reflected by the fingertip, which is captured by a sensor, such as a smartphone camera. Smartphone lenses are highly sensitive to light—although they cannot fully capture the wide spectrum of colors and light in the real world, they are much more sensitive than the human eye and can even detect and process near-infrared light. The camera photoreceptor supports wavelengths approximately from 300 to 1200 nm, while the visible light has a wavelength approximately from 380 to 780 nm.
▲ Visible light spectrum.
After the camera and flashlight are turned on, the first 10 seconds of the captured images are discarded to eliminate inaccurate measurement results and warm up the camera, which further ensures measuring accuracy.
The total measurement time is 60000 milliseconds, which is 60 seconds. Although 70000 milliseconds is set here, 10000 milliseconds (10 seconds) are provided for eliminating the use of preliminary data. That is, the 10 seconds after the camera is turned on is for warming up (not included in the measurement), and then the measurement is taken for 60 seconds, a total of 70 seconds.
private final int measurementLength = 70000;
private final int clipLength = 10000;
To convert raw images into data, it is essential to understand the principle of PPG and how it can be applied to heart rate measurement.
When light passes through skin tissue and reflects to a light-sensitive sensor, it undergoes some attenuation. The absorption of light by tissues like muscle, bone, and veins remains relatively constant (as long as the measurement site does not shift). However, blood behaves differently. Because blood flows through the arteries, light absorption varies over time — the number of red blood cells at a given location changes, rather than remaining constant.
The DC (direct current) component of the signal is largely absorbed by skin tissue. In contrast, the AC (alternating current) component reflects changes in blood volume caused by the pulsatile pressure of the heart cycle. This means that when light is converted into an electrical signal, the arteries' varying light absorption creates an AC signal, while other tissues maintain a constant DC signal. By isolating the AC signal, we can track changes in blood flow.
The phone's LED white light is turned on to capture these changes, and images of the finger under the light are continuously recorded through the camera lens. Over a set period, the red component of each frame is extracted at regular intervals. Repeating this process over time allows us to track changes in the red component, which forms the PPG signal.
▲ Changes in fingertip blood flow cause corresponding variations in the PPG signal.
In order to extract the red component in a raw image, according to the official documentation on the Android Studio Developers website [1], the red component can be extracted by using bitwise operators: specifically, use the bitwise operating code as follows:
int R = (color >> 16) & 0xff;
In this case (in which we use a textView in Android development to represent the recorded view from the smartphone camera), a pixel's color value is an integer composed of four 8-bit bytes representing the alpha (A), red (R), green (G), and blue (B) components, respectively. The order of these components from the most significant byte to the least significant byte is ARGB. The >> operator is a bitwise right shift operator, which shifts an integer value right by the specified number of bits. Here, the operation pixels[pixelIndex] >> 16 shifts the pixel value 16 bits to the right, moving the red component to the least significant 8 bits. Then, the & 0xff operation is a bitwise AND operation, which sets the higher-order bytes to zero and retains only the low 8 bits, which is the value of the red component.
So we need to obtain the pixelated length and width of the camera view, traverse all pixels in that view, extract each pixel's red components, add it to an integer variable, and then send it into a specified object “store” in another java class for further processing.
int pixelCount = textureView.getWidth() * textureView.getHeight();
int measurement = 0;
int[] pixels = new int[pixelCount];
for (int pixelIndex = 0; pixelIndex < pixelCount; pixelIndex++)
{
measurement += (pixels[pixelIndex] >> 16) & 0xff;
}
// Add the value of measurement into the store object
store.add(measurement);
Here (in another java class), we defined a function named “add”, which serves the purpose of storing different timestamps of measurements (i.e. the amount of red components) into an array list, and checks if the new measurement value is less then the current minimum of larger then the current maximum, to find the maximum and minimum of the array list “measurement”. Finally, the array list “measurement” is exactly the PPG signal we want.
private final CopyOnWriteArrayList> measurements = new CopyOnWriteArrayList<>();
private int minimum = 2147483647;
private int maximum = -2147483648;
void add(int measurement)
{
Measurement measurementWithDate = new Measurement<>(new Date(), measurement);
measurements.add(measurementWithDate);
if (measurement < minimum) minimum = measurement;
if (measurement > maximum) maximum = measurement;
}
We can compare the PPG signal with the electrocardiogram (ECG) signal obtained using electrocardiography.
▲ Variations in the PPG waveform. [2]
By analyzing the waveforms of PPG, it is possible to identify the peak values of vascular contraction, respectively. From this, we can calculate the time it takes for blood to be pumped from the heart to the measurement site, known as the pulse transit time (PTT). PTT is closely related to blood pressure: higher blood pressure results in a faster pulse wave velocity, while lower blood pressure results in a slower velocity. In other words, by measuring the PPG signal, we can convert it to the measurement used by an electrocardiogram, which is the pulse-pulse interval (PPI, or pulse peak interval), and obtain the heart rate variability index (R-R interval, RRI) with a certain degree of accuracy: the time interval between R-Wave Peaks (that is, R-R interval, RRI) is approximately as long as both the systolic pulse peak interval and the diastolic pulse interval in the PPG wave graph. Numerous papers and experiments have verified the feasibility of obtaining RRI and heart rate variability (HRV) indicators from the PPI signal. [3]
Now in the Android programming, first we need to define a time interval between each measurement point; that is, how long it is from one data point to another, here we set it as 45 milliseconds, meaning that the time interval between two sampled data points is 45 milliseconds.
// The time interval for each measurement is 45 milliseconds
final int peakDetectionWindowSize = 13;Then, we need to find the peak (or valley) values during a specific length of data points, here we set it as 13; which means we check 13 points of “measurements” each time, including the middle point itself with the 6 data points before and after it, and find the largest/smallest value(s) amongst the array of these 13 values. The code representation is as follows:
final int peakDetectionWindowSize = 13;In other words, we define a single PPG wave as one that takes a time length of 45 milliseconds x (13 - 1) points = 540 milliseconds. How long this time is does not matter, since the only factors we care about are the HRV indices derived from the time intervals between two wave peaks (or valleys) rather than the time length for a single PPG signal wave to happen.
This value has no deterministic factor and is mostly decided via the experiments and experiences during the demo and debug process of the application. Setting a too-large window size may result in detecting too many “fake” peaks, while setting a too-small window size may result in detecting too few peaks.
After calculating the PPG data and converting it into HRV data, we need to convert the HRV data into a stress index. Although research shows no global standard for stress indices, we found a study that provides standardized criteria for stress indexes.
▲ Comparison of HRV parameters between various levels of stress. [4]
This study involved 150 participants in an experiment examining the relationship between HRV values and stress indices. After measuring HRV and applying various stress scales, three distinct stress levels were identified based on HRV parameters, which we will follow. To ensure the study’s accuracy, we interviewed Associate Professor Hsin-Chin Chen from the Department of Psychology at National Chung Cheng University. Following the interview, we confirmed the study's reliability and used only time-domain calculation parameters based on Professor Chen's recommendation.
Why did we decide not to use the frequency-domain method, such as LF and HF, besides using the time-domain parameters such as SDNN, RMSSD, NN50, and pNN50 The sensors in different smartphone models vary, each smartphone may receive different frequencies in the light spectrum; also, most smartphone lenses cannot capture frequencies far beyond the visible light spectrum very well. Such capturing can only be achieved via special hardware gadgets such as health monitoring watches.
▲ Left: The CIE chromaticity diagram. (Courtesy of the General Electric Co., Lighting Division.)
Right: Illustrative color gamut of color monitors (triangle) and color printing devices (shaded region).[5]
Due to this phenomenon, frequency-domain-based stress indexes, such as LF or HF, become unreliable and cannot be easily measured by solely using the camera. That results in inconsistencies when applying the frequency-domain method. For this reason, we chose to rely on the time-domain method to measure stress levels instead.
These are the time-domain method parameters we use for measuring stress levels:
SDNN: Standard deviation of all NN (RR) intervals.
RMSSD: The square root of the mean of the sum of the squares of differences between adjacent NN intervals.
NN50: Number of pairs of adjacent NN intervals differing by more than 50 ms in the entire recording; three variants are possible counting all such NN interval pairs or only pairs in which the first or the second interval is longer.
pNN50: NN50 count divided by the total number of all NN intervals.
After obtaining the HRV parameters, we still needed to convert them into stress indexes. Based on the standards outlined in the study, we quantified stress levels by assigning 'Well' as 0, 'Normal' as 1, 'High' as 2, and 'Severe' as 3. These values were then used as stress indexes (stress level scores), to conduct further normalization.
Finally, we add all scores to create a total score, which is then naturalized into 4 different categories of stress levels: well, normal, high, and severe:
// Judge the pressure condition based on the total score
if(totalScore >= 0 && totalScore <= 2)
result[4] = 0;
else if(totalScore >= 3 && totalScore <= 5)
result[4] = 1;
else if(totalScore >= 6 && totalScore <= 8)
result[4] = 2;
else if(totalScore >= 9)
result[4] = 3;
// Invalid result
else
result[4] = -1;
A GUI (Graphic User Interface) popup window is displayed to the user to notify their current stress level. Furthermore, they can see their detailed values of HRV parameters and export all data to any other applications they wish via the share button.
▲ The GUI popup window for demonstrating the stress testing result.
To verify whether our stress measurement system meets our requirements, we released a beta version of the app to team members, friends, and family for daily stress measurement and asked for their feedback. All participants provided highly positive reviews and expressed excitement for the official release.
In addition to ensuring the system meets our requirements, we also need to verify the accuracy of the HRV measurements. Therefore, we plan to use specialized HRV measurement devices to validate the accuracy of the data.
Theoretically, all stress indexes (SDNN, RMSSD, … etc.) are proven to be accurate if the statistics they derived from, R-R intervals and heart rate, are proven to be accurate. For the sake of proving our software's accuracy, the software developer decided to use himself as the sole experimenting target. The following images are heart pulse rates measured by using two different specialized measuring gadgets on both hands’ different parts right after waking up (approximately noon), after being without food and water for nearly 9 hours.
The first instrument we used is a Japanese blood pressure and heart pulse measuring device - OMRON Intellisense HEM-7071. We deployed the device on both upper arms. The measurement result indicated that the BPM (heart beats per minute) is around 63~64 times per minute.
▲ Left: Heart rate measurement on the left upper arm using OMRON Intellisense HEM-7071. The measured heart rate is 63 beats/minute.
Right: Heart rate measurement on the right upper arm using OMRON Intellisense HEM-7071. The measured heart rate is 64 beats/minute.
The second device we used is a FORA TN'G SpO2 Fingertip Pulse Oximeter. The measurement result on the right forefinger indicated that the BPM (heart beats per minute) is 62 times per minute. All measurement results are approximately the same as those in our new Android application.
▲ Left: FORA TN'G SpO2 Fingertip Pulse Oximeter.
Right: Heart rate measurement on the right forefinger using FORA TN'G SpO2 Fingertip Pulse Oximeter.
▲ HRV indices were measured right after using a sphygmomanometer and oximeter, proving the software has enough measuring accuracy.
While under significant stress, indicated by a high or severe level of stress in the test result, the app will immediately offer stress-relief recommendations, including breathing practices, meditation, and purchasing our SERENE Beverage. Furthermore, the app provides daily reminders at 14:00, based on the user’s smartphone time. These notifications will not be disabled even if the app is completely closed (unless the user force restarts the smartphone), and will loyally send a notification and start playing the app's background music in the smartphone background to inform the user it's time for their stress measurement.
▲ The demonstration of daily notifications.
After users become aware of their stress levels, what should they do if their stress is too high? To provide comprehensive stress management, we not only measure stress but also researched various stress-relief methods and considered the recommendations provided by Associate Professor Hsin-Chin Chen from the Department of Psychology at National Chung Cheng University during our interview.
Here are three emotional relief tips currently offered within the app:
After reviewing the literature, we found that breathing exercises can effectively improve negative emotions. Based on our interview, we adopted the abdominal breathing method recommended by the professor.
Abdominal breathing is a relaxation technique focusing on taking slow, deep breaths. It involves inhaling slowly through the nose and exhaling through the mouth, using the diaphragm and abdominal muscles. This technique increases oxygen levels in the blood, lowers blood pressure and heart rate, and reduces muscle tension. Abdominal breathing can help relieve stress, pain, and anxiety. It is also referred to as deep breathing or diaphragmatic breathing. [7]
We recommend a video from REACH Rehab + Chiropractic Performance Center - “Diaphragmatic Breathing for Relaxation & Mobility”. The speaker explained how abdominal breathing is beneficial to our health, and taught the audience how to perform abdominal breathing step-by-step. [8]
Conducting meditation [9] and listening to soothing music can rapidly hasten the process of stabilizing heart rate variability, thus indirectly reducing anxiety. [10] We decided to implement the functionalities of a music player (with our meticulously crafted music) and a meditation tutorial video recommendation for app users.
▲ The music page interface.
We recommend a video from TED - “Happiness is all in your mind: Gen Kelsang Nyema at TEDxGreenville 2014”. The speaker perfectly explained how one obtains happiness, and taught the audience how to perform meditation step-by-step. [11]
(https://www.youtube.com/watch?v=xnLoToJVQH4&t=0s)
According to the Creative Commons Attribution license (reuse allowed) (CC BY License) announced in the original videos, all copyrights of the abdominal breathing video and the meditation video go to the original creators. We only give the users a recommendation. The publication of these videos in the app is granted permission by the CC BY License from the original creators.
Do you know that keeping a daily routine can relieve stress? According to the World Health Organization (WHO), having a daily schedule can help us use our time efficiently and feel more in control. Also, our interview revealed that maintaining consistent wake-up and bedtimes, rather than just following a sunrise-to-sunset routine, is more effective in reducing stress. We developed a work and rest planning system within the app to support users in establishing a routine and managing stress.
This system allows users to set their wake-up and bedtime, with the app sending notifications when it’s time to sleep or wake up. To support maintaining a routine, the app also tracks and displays the user's sleep times and durations throughout the week, making it easy for users to review and monitor their sleep patterns.
▲ The interface for work and rest planning.
However, one or two days of training is not sufficient to effectively relieve negative emotions. To achieve meaningful results, we recommend that users practice breathing exercises or meditation at least once a day and maintain a regular daily routine to maximize the benefits of reducing negative emotions.
In addition to stress measurement and simple suggestions for relieving negative emotions, limiting the app to these two features would make it somewhat dull. To enhance user engagement, we made several adjustments to make the experience more enjoyable and integrated the app with SERENE Beverages.
Here are the additional changes we implemented:
To boost user engagement, we designed the app as a soothing fishing mini-game. Users can "fish" by measuring their stress daily and collecting different species in a catalog. We created 50 unique fish illustrations for users to gather. The fun of collecting these illustrations is intended to motivate users to measure their stress daily, helping them form a habit of regular stress tracking.
▲ The interface when a fish is caught.
In addition to the soothing fishing mini-game, we also designed a check-in system. Users can check in by measuring their stress daily, and after several consecutive check-ins, they can redeem SERENE Beverages as a reward. Also, in the future, we intend to work this out with the owner of a tea chain and provide some financial incentive for them to participate.
▲ The interface of daily check-in.
We have also integrated the app with our product, SERENE Beverages. Users will find a QR code on the seal when purchasing and consuming SERENE Beverages. Scanning the QR code allows them to download the app and collect points, which can be redeemed for SERENE Beverages, similar to the check-in system.
Also, by integrating the app with SERENE Beverages, users can track whether the drink effectively alleviates their negative emotions, providing insight into its impact on stress relief.
The app is provided in 4 different languages - English, Traditional Chinese, Japanese, and French - based on the app user's smartphone language preference settings.
We plan to launch the app, but there are still many features that need to be reorganized before the release. It is known that there is no global standard for stress indices, so in the future, we aim to integrate the entire system with artificial intelligence (AI).
We selected the Perceived Stress Scale (PSS), one of the most widely used stress measurement tools worldwide, developed by Sheldon Cohen in 1983. The PSS is favored for its simplicity, cultural adaptability, and suitability for both clinical and research environments. It has been translated into numerous languages and validated across diverse populations. When measuring stress, users will also complete the PSS. The results, combined with HRV data, will be analyzed and processed by artificial intelligence. The AI-analyzed data will then be used to create a comprehensive stress index customized for different countries, genders, and age groups. This way, while measuring stress, AI will provide more accurate stress measurements, making it easier to relieve negative emotions afterwards.
In addition, we hope to add more stress-relief features in the future, making the app even more comprehensive in helping users manage their stress.
Supported Platforms
| Android |
|---|
| ✅ 5.0 (Lolipop 🍭, Android SDK API level 21) and above |
| ✅ optimized at 12.0 (Snow Cone 🍧, Android SDK API level 34) |
There are two different ways to install this application.
The following DISCLAIMER is for clarification to judges; for normal users, please skip and directly refer to the installation methods.
DISCLAIMER
Upon reviewing the software requirements, our team noticed that it is mentioned we cannot upload binary executables such as .exe or .bin files in GitLab. Nevertheless, since there is no explicit, trustworthy and reliable definition of whether an APK file is a binary executable or not, we provided two different installation methods: the A method is for APK file direct installation, which provides normal users with a much easier app installing process; while the B approach is fairer and prevents controversy, but much more complicated.
The APK file is locked in the compressed folder do_not_touch.zip and is protected by strong passwords; only the main developer holds this password, so no one can access the file unless the provision of the APK file is granted by iGEM officials.
We understand the importance of adhering to the competition's guidelines and wish to ensure that our approach does not inadvertently affect our submission or evaluation; therefore, for software judging, please ignore the file do_not_touch.zip and installation method A. Thank you very much and sorry for any inconvenience caused.
Note: .apk is a file format of APK files. APK files are saved in the ZIP format and are typically downloaded directly to Android devices for app installation. However, there exists continuous debates on the Internet concerning whether an APK file is a ZIP file, or a binary executable, or even perhaps having both properties.
Preparatory Work On Your Smartphone
There is a ZIP folder named "install tutorials", which contains screenshots of demonstrations on a Windows 10 virtual machine and Android Operating System. You can refer to those images if getting confused with the following instructions.
Installation Method A
You can also choose other compressed file format options as long as you can extract the folder; since the focal point is to extract the folder, obtain the APK file, and then copy and paste that file on your smartphone to finish installation.
Installation Method B
BEWARE: this method is much more complicated and requires extra caution, please be prepared with a Windows pc that has enough disk space.
```shell
sdkmanager "platforms;android-33"
sdkmanager "platform-tools"
sdkmanager "build-tools;33.0.2"
sdkmanager emulator
sdkmanager "system-images;android-33;google_apis;x86_64"
sdkmanager --licenses
```
You can select an arbitrary area and right-click in the terminal to copy or paste the selected area. Also, the command may take some time to complete running, so please be patient.
```shell
sdk.dir = C:\\android\\sdk
```
```shell
cd Downloads\ccu-taiwan-main
gradlew installDebug
```
The command may take some time to complete running, so please be patient.
In conclusion, the "Pokefish" app offers a unique, cost-effective approach to stress measurement and emotional management by combining cutting-edge technology with user-friendly features. By utilizing Finger Photoplethysmography (FPG) to assess stress levels, users can monitor their emotional states anytime without the need for additional devices. The app's comprehensive set of stress-relief strategies, such as breathing exercises and meditation, empowers users to manage negative emotions effectively. Furthermore, engaging features like the soothing fishing mini-game and daily check-ins with SERENE Beverage integration enhance user engagement and promote daily stress tracking.
With its focus on emotional awareness and holistic stress management, "Pokefish" provides users with the tools to better understand their emotions and make positive lifestyle changes. As we move forward, future integrations with AI and expanded features will continue to improve the accuracy and effectiveness of stress management, helping users around the world lead healthier, more balanced lives.