wrangler Face scanning technology in personalised nutrition and health

In your Face: A look at emerging Face Scanning technology and its role in nutrition and health

Mar 15, 2024 5:11pm

 

Did you know your ‘selfie’ camera can now be used to measure vitals such as blood pressure and pulse rate? No?  Face scanning technology has been integrated in a number of industries such as for pay,ets and beauty, however it is slowly making its way into health and nutrition. Keep reading as we look into face scanning and what it could mean for personalised nutrition

 

 

What is a Face Scan “Healfie”

Face scanning technology, a form of computer vision, uses your smart phone camera along with artificial intelligence (AI) and computer learning to provide insights and in the landscape of health, can check vitals like heart rate. The tech that makes this possible, was first developed and made accessible by Apple to unlock the iPhone in 2017 and subsequently can be used for payments and security using face recognition. Now, it has the potential to help doctors, nutritionists and other healthcare professionals to monitor patients' health digitally.

 

Is Face scanning the same as Face recognition?

Face recognition is a form of face scanning with the end result being to match the face for security purposes such as signing into an App or paying for a service; this same technology however is being leveraged to find further patterns such as information on your health. 

 

How does Face Scanning Technology work?

Using high resolution imaging available with RGB cameras, inbuilt in most smart phones and laptops, a 3D picture of your face is captured (250,000 points of data!) and analysed as digital information. This digital information is compared to other data;  It can look at how your face is shaped and textured, or go a step further and use light and optics to pick up on pulse and blood flow. Combined with AI and deep learning algorithms, these data points can be used to tell a variety of health information. 

The light and optics element is done via Photoplethysmography (rPPG), a form of optimal imaging, also known as pulse oximeters; a common PPG method is Transdermal Optical Imaging. PPG uses light and optics to measure the amount of and detect changes in, light absorbed or reflected by the skin, and as oxygenated blood, deoxygenated blood, and skin tissue all have different absorption characteristics, can provide analysis of various health vitals. PPG is also what is used by wearables to pick up on heart rate and heart rate variability. 

 

What health data can be seen with 3D Face Scanning? 

 

 

Face scanning software can detect:

  • Emotions and pain
  • Syndromes and genetic disorders
  • Malnutrition
  • Early dehydration
  • Changes that may mean endocrine or neurological disorders 
  • Skin conditions and characteristics such as:
    • Spots, texture, dark circles, redness, pores, wrinkles, oiliness, moisture, eye bags, acne, skin firmness, skin radiance, upper eyelid droopiness, and lower eyelid droopiness

 

 

Health Vitals:

  • Heart rate (Blood volume pulse)
  • Heart rate variability (Blood volume pulse)
  • Pulse rate and variability
  • Respiration and breathing rate
  • Arrhythmias 
  • Oxygen levels and saturation 
  • Blood pressure
  • Blood flow
  • Arterial stiffness
  • Stress level
  • Anxiety & Depression
  • Glucose / HbA1c
  • Body Shape Index
  • BMI
  • Waist-Height Ratio

Using the above, subsequently predict risk for conditions such as: cardiovascular disease, Stroke, Hypertension, Heart attack, Type 2 Diabetes

 

 

Combined with AI and deep learning algorithms, these data points can be used to tell a variety of health information

 

Current innovators of Face Scanning technologies 

Despite it being early days, there are some apps already offering a face scan or “healfie” to check health vitals, the same AI and technology is now also available via a mirror (NuraLogix) to provide daily real time health checks. 

Artificial intelligence companies such as AHI and NuroLogix developed FaceScan and Anura (respectively) all of which use PPG. These apps, along with Vastmindz can be integrated into other platforms (Anura can also be used as a standalone app in itself).

In addition to this companies are leveraging this technology to get ahead; FACESCAN was recently launched by ICICI Lombard in India, a health insurance company, AXORA, a tech marketplace for heavy industry, is using the technology amongst mining companies to improve staff health and Mana Dr, a private online Doctor service in Singapore, is using “MaNaDr AI Face Scan”, promising Early Detection and Prevention, Personalized Health Insights and Streamlined Care Coordination. Similarly Dr Renee “Together with Renee” is an app available to download that promises to support loved ones and support medical care 

In addition, technologies to analyse the health of the skin such as wrinkles have been developed, for example “FitSkin” which has already become mainstream and being used by big brand cosmetics such as Neutrogena (a Johnson & Johsnon company) and No7, as well as being used to validate skin care products such as a green tea cream (Liao et al., 2022). Mudho has also added this technology along with the rest of their portfolio for personalising health, lifestyle and nutrition.

On the other hand, using facial morphology (the differences in landscape of the face) scientists have developed Face2Gene, an app that uses AI, deep learning and phenotype data to identify over 200 genetic syndromes, which is already available and being used by Doctors.

 

 

 

 

Face scanning is an emerging technology that is advancing beyond skin care into health. Whilst still a small segment within the digital health sphere, it is expected to grow owing to its low cost and high smartphone penetration making accessibility easy

 

 

The Current Market of Face Scanning

Face scanning for health is yet to reach a category in itself, however the global wearable  technology (providing similar information) market size was worth around USD 55.5 billion in 2022 and is predicted to grow to around USD 142.4 billion by 2030 (Zion); fitness monitoring, telehealth and digital health along with facial recognition have grown in recent years and set to continue. Read the full report on the current market, here. 

 

What is the opportunity of Face Scanning in Personalised Nutrition?

Nutrition can touch many areas of health and wellness from weight management, skin health, stress and sleep, exercise, disease prevention and so forth. The many measurements that can be provided with face scanning offers an opportunity for cheap and accessible personalisation, monitoring and feedback. 

  • Wellness 
  • Weight management
  • Stress and sleep
  • Sport nutrition
  • Hydration
  • Health and prevention
  • Malnutrition 
  • Skin health

Face scanning can provide readily available information, daily, digitally and with ease, on clients’ stress, blood pressure, emotions, hydration etc. and can therefore provide nutritionists new ways of monitoring clients and providing feedback and advice. Monitoring of stress, already a growing area within fitness wearables (think HRV and WHOOP) and hydration can also give nutritionists and sports coaches a way of being able to support clients and athletes further than they may have been able to before, especially as face scanning is inexpensive, if not free. 

Whilst there is potential to estimate BMI using a face scan, and various studies have looked to validate the option of using a smart phone to predict BMI, it seems to be less of a focus when developed into a solution, with only Anura boasting BMI as a possible measurement (although the App does ask for your weight and height when you sign up…). This does make sense considering BMI is already an easily accessible measurement and face scanning can instead offer more valuable insights that wouldn’t have otherwise been accessed digitally and non-invasively. 

Personalised nutrition on the whole is a growing area and face scanning may be able to expand this further due to its accessibility. We recently discussed the outlook of digital twins and how it can revolutionise the nutrition and health field by providing doctors and nutritionists a more accurate picture of the patients’ needs and responses; face scanning can add to this catalogue of data. With self-tracking being a key component in behaviour change, face scanning could support healthful behaviour change as a stand alone or in conjunction with other wearables and devices. 

 

 

How accurate is 3D Face Scanning?

Accuracy is very promising for multiple measurements; Face2Gene boasts 91% accuracy for genetic syndrome identification which was replicated in another study whilst AHI reports their face scan can measure vitals such as blood pressure, HR and respiration with 96% accuracy and 98% reliability. 

 

Accuracy of Vitals

Overall, AHI’s reported accuracies are mirrored elsewhere with vitals such as blood pressure, heart rate and respiration accurately measured using PPG with around 96% accuracy. Various studies, including a meta analysis looking at HR, HRV, RR and SpO2 show promising results with often over  90% accuracy  mirrored in similar accuracies for blood pressure and atrial fibrillation when compared to gold standard; in fact AHI has recently launched a medically approved mobile phone assessment of atrial fibrillation (Listcorp). Hydration, on the other hand, is less researched but sits at 76% accuracy.

 

Accuracy for Glucose Monitoring using PPG

Measurements for blood glucose however, only managed to reach 66% accuracy but it has been reported PPG can be used to detect prevalent diabetes, so may be an area to watch out for considering the rise of glucose monitoring. 

 

Accuracy of Skin Conditions

According to research, most common skin conditions can be predicted up to 97% accuracy, and skin characteristics with 69% average agreement was found in a validating study, but this was not replicated in a later independent review.

Facial Scanning for Personalised Nutrition 

A recently published study, found 73.1% accuracy in predicting malnutrition when using an already established nutrition assessment of malnutrition (NRS-2002) for AI to compare to, offering the potential to be able to monitor at risk patients for malnutrition and hopefully reduce progression of ill health; further supported by another study looking at malnutrition in the older adults.

 

BMI - Body Mass Index

There has been some attempt to predict BMI with a facial images in multiple formats and in comparison to waist to hip ratio; one study reported 90% accuracy, however it is difficult to ascertain if this result is replicated elsewhere and the results are not specific to smart phone face scanning and more general facial images.

It is prudent to mention that BMI seems rarely accounted for or discussed in the research for the other health vitals, so perhaps not considered important or a confounding variable for PPG, however when it comes to certain conditions, such as Cushing’s syndrome, when BMI was considered, accuracy went from 91.7% to 61.1% suggesting potential implications in ccertain situations.

 

Overall it is noted that many of the results are in clinical settings, with calibrated devices, and often the research is extrapolated and not necessarily tested in situ; Anura (NuraLogix) specifically says the app is for information only and the phone application  “performance” has not been established. Nevertheless, the science is promising. 

 

 What impacts the accuracy of Face scanning?

Whilst research in face scanning technology is promising in many areas of health, face scanning for health vitals such as blood pressure and stress is sensitive and relies on good ambient lighting along with a still hand. Moreover, whilst reliable findings can be found, much of the research is done using limited skin tones and in healthy participants. A study published in 2023 does show when AI is trained with different skin tones, the results are still as accurate, however, as earlier discussed another study shows how obesity can impact accuracy yet is not often reported nor considered. As we discussed in our white paper on  the ethics of AI in nutrition, AI data sets will only provide accurate information based on the data you give it and therefore needs to be created using different skin colours, ages and health status to ensure the technology is valid across multiple populations and not just healthy white people.

 

Benefits of Face Scanning

  • Face scanning is easy to use 
  • Requires minimal training and therefore a low learning curve
  • It is inexpensive, if not free and could therefore make personalized nutrition and recommendations much more accessible
  • Accessible - smartphones are highly adopted already and no other equipment like a watch needed
  • Can provide real time feedback  
  • Can be another tool to add to a nutritionists toolbox to improve and personalise advice 

 

Major challenges and gaps of Face Scanning

  • More scientific support and research required, especially against gold standard protocols
  • Needs behaviour change element 
  • Sensitive to light and movement 
  • Lack of regulation and regulatory risk - Clearview recently fined $9.4 million 
  • Face recognition concerns over application across all skin colours - for example Rite Aid recently banned from using facial recognition due to inaccuracies in black, latino and asian people
  • Concerns for privacy and involuntary opt in to surveillance - like concerns over TSA (Transportation Security Administration) roll out in airports 



In Conclusion

Face scanning is an emerging technology that is advancing beyond skin care into health. Whilst still a small segment within the digital health sphere, it is expected to grow owing to its low cost and high smartphone penetration making accessibility easy. Face scanning may contribute a simple and effective daily health management tool. 

This is a short version of our Executive data briefs created by our experts. At Qina we track the market in terms of scientific advances, new innovations and their potential impact on diet and health. We leverage the latest in technology to track developments and share our insights via reports, dashboards as well as expert created content.

To learn more about our services sign up for a free account

 

References

Selfie Face Scan to detect Health vitals? Really? Medium

Liao R, Parker T, Bellerose K, Vollmer D, Han X. A Green Tea Containing Skincare System Improves Skin Health and Beauty in Adults: An Exploratory Controlled Clinical Study. Cosmetics. 2022; 9(5):96. https://doi.org/10.3390/cosmetics9050096 

Qiang J, Wu D, Du H, Zhu H, Chen S, Pan H. Review on Facial-Recognition-Based Applications in Disease Diagnosis. Bioengineering (Basel). 2022 Jun 23;9(7):273. doi: 10.3390/bioengineering9070273. PMID: 35877324; PMCID: PMC9311612.

Bautista, Melissa Joanne et al. “Clinical applications of contactless photoplethysmography for monitoring in adults: A systematic review and meta-analysis.” Journal of clinical and translational science vol. 7,1 e129. 15 May. 2023, doi:10.1017/cts.2023.547

Luo H, Yang D, Barszczyk A, et al. Smartphone-Based Blood Pressure Measurement Using Transdermal Optical Imaging Technology. Circ Cardiovasc Imaging. 2019;12(8):e008857. doi:10.1161/CIRCIMAGING.119.008857

Gill S, Bunting KV, Sartini C, et al. Smartphone detection of atrial fibrillation using photoplethysmography: a systematic review and meta-analysis. Heart. 2022;108(20):1600-1607. Published 2022 Sep 26. doi:10.1136/heartjnl-2021-320417

Saha, P., Zulfiker, S.M.I., Hashem, T., Islam, K.A. (2023). Dehydration Scan: An Artificial Intelligence Assisted Smartphone-Based System for Early Detection of Dehydration. In: Tsanas, A., Triantafyllidis, A. (eds) Pervasive Computing Technologies for Healthcare. PH 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 488. Springer, Cham. https://doi.org/10.1007/978-3-031-34586-9_20

Avram R, Olgin JE, Kuhar P, et al. A digital biomarker of diabetes from smartphone-based vascular signals. Nat Med. 2020;26(10):1576-1582. doi:10.1038/s41591-020-1010-5

Muhaba KA, Dese K, Aga TM, Zewdu FT, Simegn GL. Automatic skin disease diagnosis using deep learning from clinical image and patient information. Skin Health Dis. 2021;2(1):e81. Published 2021 Nov 25. doi:10.1002/ski2.81

Madison K. Cook, Margaret A. Kaszycki, Irma Richardson, Sarah L. Taylor & Steven R. Feldman (2022) Initial validation of a new device for facial skin analysis, Journal of Dermatological Treatment, 33:8, 3150-3153, DOI: 10.1080/09546634.2022.2127305

Zaar O, Larson A, Polesie S, et al. Evaluation of the Diagnostic Accuracy of an Online Artificial Intelligence Application for Skin Disease Diagnosis. Acta Derm Venereol. 2020;100(16):adv00260. Published 2020 Sep 16. doi:10.2340/00015555-3624

Wang J, He C, Long Z. Establishing a machine learning model for predicting nutritional risk through facial feature recognition. Front Nutr. 2023;10:1219193. Published 2023 Sep 15. doi:10.3389/fnut.2023.1219193

Tay W, Quek R, Kaur B, Lim J, Henry CJ. Use of Facial Morphology to Determine Nutritional Status in Older Adults: Opportunities and Challenges. JMIR Public Health Surveill. 2022;8(7):e33478. Published 2022 Jul 18. doi:10.2196/33478 

Wang Y, Jin Z, Huang J, Lu H, Wang W. Facial Landmark based BMI Analysis for Pervasive Health Informatics. Annu Int Conf IEEE Eng Med Biol Soc. 2023;2023:1-5. doi:10.1109/EMBC40787.2023.10340239

Yao Y, Song L, Ye J. Motion-to-BMI: Using Motion Sensors to Predict the Body Mass Index of Smartphone Users. Sensors (Basel). 2020;20(4):1134. Published 2020 Feb 19. doi:10.3390/s20041134

Fook, C. Y., Teen, L. W., Chin, L. C., Ali, H., Vijean, V., & Nasir, A. S. A. Investigation on Body Mass Index Prediction from Face Images.

G. Bolukbaş, E. Başaran and M. E. Kamaşak, "BMI Prediction From Face Images," 2019 27th Signal Processing and Communications Applications Conference (SIU), Sivas, Turkey, 2019, pp. 1-4, doi: 10.1109/SIU.2019.8806503. 

Mayer C, Windhager S, Schaefer K, Mitteroecker P. BMI and WHR Are Reflected in Female Facial Shape and Texture: A Geometric Morphometric Image Analysis [published correction appears in PLoS One. 2017 Feb 9;12 (2):e0172205]. PLoS One. 2017;12(1):e0169336. Published 2017 Jan 4. doi:10.1371/journal.pone.0169336

Chanda A, Chatterjee S. Predicting Obesity Using Facial Pictures during COVID-19 Pandemic. Biomed Res Int. 2021;2021:6696357. Published 2021 Mar 10. doi:10.1155/2021/6696357 

Dhanamjayulu C, Nizhal U N, Praveen Kumar Reddy Maddikunta, Thippa Reddy Gadekallu, Celestine Iwendi, Chuliang Wei, Qin Xin. IET. Identification of malnutrition and prediction of BMI from facial images using real-time image processing and machine learning. 2021 https://doi.org/10.1049/ipr2.12222

Qiang J, Wu D, Du H, Zhu H, Chen S, Pan H. Review on Facial-Recognition-Based Applications in Disease Diagnosis. Bioengineering (Basel). 2022;9(7):273. Published 2022 Jun 23. doi:10.3390/bioengineering9070273  

Bautista MJ, Kowal M, Cave DGW, Downey C, Jayne DG. Clinical applications of contactless photoplethysmography for monitoring in adults: A systematic review and meta-analysis. J Clin Transl Sci. 2023;7(1):e129. Published 2023 May 15. doi:10.1017/cts.2023.547

Talukdar D, De Deus LF, Sehgal N. The Evaluation of Remote Monitoring Technology Across Participants With Different Skin Tones. Cureus. 2023;15(9):e45075. Published 2023 Sep 12. doi:10.7759/cureus.45075

https://www.zionmarketresearch.com/report/wearable-technology-market

Gurovich Y, Hanani Y, Bar O, et al. Identifying facial phenotypes of genetic disorders using deep learning. Nat Med. 2019;25(1):60-64. doi:10.1038/s41591-018-0279-0

https://www.listcorp.com/asx/ahi/advanced-health-intelligence-ltd/news/ahi-to-launch-medically-approved-mobile-phone-based-af-asses-2965440.html 

Xing W, Shi Y, Wu C, Wang Y, Wang X. Predicting blood pressure from face videos using face diagnosis theory and deep neural networks technique. Comput Biol Med. 2023;164:107112. doi:10.1016/j.compbiomed.2023.107112

Harford M, Catherall J, Gerry S, Young JD, Watkinson P. Availability and performance of image-based, non-contact methods of monitoring heart rate, blood pressure, respiratory rate, and oxygen saturation: a systematic review. Physiol Meas. 2019;40(6):06TR01. Published 2019 Jul 3. doi:10.1088/1361-6579/ab1f1d