Tonemeter accuracy check

 A good trainer pushes your game to the next level. But sometimes the workout is mental. 

Meet Matt Moran, owner of Crossfit Metropolis in NYC. The guy not only knows workouts and diet, but he's math strong too. Phi Beta Kappa math degree from UNC and experience as an actuary.

This background came in handy in giving Tonemeter a workout. Matt drilled in deep on the methodology behind Tonemeter, but ultimately wanted to know how it performs under stress. 

Matt knew the instructions suggest even lighting and flexing slightly for photos, but sometimes people don't follow the manual. He asked an interesting question: What happens when people don't follow the directions?

I told him I didn't know what would come out, but to give it a shot. Here's what happened.

The setup

Matt took four pictures under different conditions - changed lighting and flexed muscles.  

The first step in optical measurement is cropping the photo. This preserves anonymity and builds more accurate results. Matt was a good sport and allowed me to show the cropped versions.

The next step was feeding the images through the system. We gather hundreds of human judgements. Next, we run them into an algorithm to compile the input into a reliable bodyfat percentage. What numbers would you expect? 

Raw results

Here are the straight numbers. Averaged 12.1% (since he weighs 170, this is 21 lbs of fat). There's a range of 1.6% (13.0% - 11.4%). For the statistical geeks out there, 0.8% standard deviation.

One subject, different conditions

What does it mean?

Data is great, but knowledge is better.

Are we in range?

The first question: does 12.1% make sense? Lots of gym rats claim to be under 10% bodyfat without getting tested. Since DEXA (measures fat with X-rays) is widely viewed as the gold standard, a few DEXA-verified photos may help.

Matt's pictures are between these two. Visual inspection confirms we're in the correct range.

Is the variation acceptable?

There's some variation in our readings (0.8% standard deviation). How much variation in DEXA?

It was difficult to test Matt with DEXA (New York requires a doctor's prescription because of the X-rays, costs about $150/test). But two scientific studies help provide an answer.

One study measured the differences between different types of DEXA machines. I plugged in Matt's measurements. There's a range of 1.9%, with only three readings. There's more variation, just between machines

One subject, different DEXA machines

There's a range of 1.9%, with only three readings. There's more variation, just between different kinds of machines.

But what about repeatability on the same machine? Researchers placed bags of fat on the body to measure DEXA accuracy. DEXA was quite accurate on average, but there's a little variation on individual readings (Bod Pod was less accurate). The single machine variation doesn't significantly increase errors.

Does flexing matter?

Matt "dropped" 1% between the flexed picture and the others. Definitely a good idea to just flex a little each time you use the app.

Is the app accurate?

Tonemeter beat DEXA in this one test. We'll need more tests to say it's definitely more correct. But I'm confident in one thing: these two methods are at least in the same range of accuracy.

---

The important thing is get accurately tested regularly on either Tonemeter or DEXA. Bodyfat % is key to your overall health. This applies double if you're working out or dieting.

Throw out your scale.