Estimating Energy Cost of Physical Activities from Video Using 3D-CNN Networks

Abstract

This research proposes a machine learning model that can estimate the energy cost of physical activities from video input. Currently, wearable sensors are commonly used for this purpose, but they have limitations in terms of practicality and accuracy. A deep learning model using three dimensional convolutional neural network (3D-CNN) architecture was used to process the video data and predict the energy cost in terms of metabolic equivalents (METs). The proposed model was evaluated on a dataset of physical activity videos and achieved an average accuracy of 71% on energy category prediction task and an root mean squared error (RMSE) of 1.14 on energy cost prediction task. The findings suggest that this approach has the potential for practical applications in physical activity surveillance, health interventions, and at-home activity monitoring.