PREDICTING ENERGY EXPENDITURE FROM PHYSICAL ACTIVITY VIDEOS USING OPTICAL FLOWS AND DEEP LEARNING

Abstract

This thesis presents a novel approach for predicting energy expenditure of physical activity from videos using optical flows and deep learning. Conventional approaches mainly rely on wearable sensors, which, despite being widely used, are constrained by practicality and accuracy concerns. This proposal introduces a new strategy that utilizes a three-dimensional Convolutional Neural Network (3D-CNN) to evaluate video data and accurately estimate energy costs in metabolic equivalents (METs). Our model utilizes optical flow extraction to analyze video, capturing complex motion patterns and their changes over time. The results are good indicating potential for this method to be deployed in various healthcare applications, such as automatic health monitoring and physical activity surveillance. This research contributes towards accurate automatic estimation of energy expenditure of physical activity simply from recorded videos and thus creates opportunities for non-invasive health monitoring systems.