If you have ever had words with your smartwatch activity monitor when you discover it has ignored your 30-minute near death workout then you will welcome GymCam.

Smart watches and similar wearable activity monitors are often frustrating because their simple sensors miss valid exercise activities. It is well known that many sensors simply give up if you are running on a treadmill, for example. Recording exercise is important because it provides motivation and data on improving performance. Many people don't like exercise and when a fitness monitor starts to miss activities there is a tendency to think "so what does it matter if I don't do it".

Rather than use wearable sensors, GymCam, developed by researchers at the Carnegie Mellon Human-Computer Interaction Institute and Institute for Software Research, uses an off-the-shelf video camera, a Logitech C922 camera with a resolution of1920 × 1080, to detect and count the exercises that are being performed. It can track multiple people at a time and keep count of their exercises. The idea is that a single GymCam set up in a gym will monitor everyone. An ambitious idea, but even more so when you take into account the fact that the aim was to monitor people when they are only partially visible to the camera.

Rather than an end-to-end approach that just plugs a neural network into the camera, computer vision algorithms were used to create a hand-selected feature vector. Optical flow trajectories were computed and converted into a 27-dimensional feature vector which was fed to a neural network. The system doesn't attempt to track users or estimate their poses. Extracting skeletons for each user would allow for working out body positions and hence what exercise they were doing, but it is computationally expensive and doesn't work when the user is partially obscured. As an exercise is repeated, GymCam makes use of the periodic variation in the image to characterize an exercise. The fact that the first step is to compute an optical flow is also an advantage when it comes to privacy, as this is hard to reverse to get a video image.

Of course, a great deal of the work was in establishing suitable training videos, which involved recording people in the gym and then manually classifying activities and counting repetitions.

You can see the result in action in the following video:

It seems to work well:

"To develop and evaluate our machine-learning algorithms, we collected data in CMU’s varsity gym for five days. We segmented all concurrently occurring exercises from other activities in the video with an accuracy of 84.6%; recognized the type of exercise (acc.=92.6%) and counted the number of repetitions (± 1.7 counts). GymCam advances the field of real-time exercise tracking by filling some crucial gaps, such as tracking whole body motion, handling occlusion, and enabling single-point sensing for a multitude of users."

There are some problems that have to be solved before a commercial version is deployed. The first is that the system doesn't recognize users. The suggestion is that users are tracked using another device - their smart phone or a QR code. The camera also has to be told where in a scene exercise is going to happen so it needs some customization when it is installed. However, the fact that it uses only a single video camera and doesn't require powerful hardware makes it a practical proposition even for home use.

More Information

GymCam: Detecting, recognizing, and tracking simultaneous exercises in unconstrained scenes (pdf) Rushil Khurana, Karan Ahuja, Zac Yu, Jennifer Mankoff, Chris Harrison and Mayank Goel. Carnegie Mellon's Human-Computer Interaction Institute

Related Articles

Fitness for Geeks

Breaking Fitness Records Without Even Moving - Spoofing The Fitbit

Monitoring a fitness regime

GPS Always Overestimates Distances

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info