Data Fluency Series: Variation and Validity

If you are using a tool to measure people data, there are two things it has to do:

• It has to reveal actual variation in the real world.
• That variation has to validly relate to something else.

When it comes to performance reviews or surveys, most of the questions that are asked create data with absolutely no variation – everyone will give all fours and fives. If you ask a question like “I trust my coworkers.” You’ll get no variation in responses – everyone will answer “Agree” or “Strongly Agree.” Without variation, that question doesn’t validly measure anything (except maybe someone’s ability to use a computer).  Most employee surveys don't validly predict anything. Share on X

The question must be worded so precisely that it creates range. Asking someone to rate the phrase “I trust my coworkers” won’t do this, but asking someone to rate the phrase “I know my teammates have my back” does. You can only learn this by asking question after question after question, and sorting out the ones that actually create variation in the responses.

Next, you have to learn if the variation you’ve created relates to something valid in the real world. Do the people who answer, “Strongly Agree” to the question “I know my teammates have my back” end up being more productive in the real world? (As it turns out, they do!) Employee deserve valid and reliable #HR tools. Share on X

This is called Criterion Related Validity. If a question can measure an increase in employee engagement, it should also be able to predict an increase somewhere else – something like productivity, or retention. Not only should it reliably and validly measure one thing, but we need that to predict something else to make the tool effective.

A good HR tool will yield valuable data that matters in the real world.

Check back next week for more information on how to become data fluent, and the impact of bad data on our businesses.