Bad Metrics

It's one of the two times each year where people are dusting off their strategic plans, hosting their parties and strategy workshops, and making sure the KPIs and metrics on their scorecards are the ones they want to be watching in the next period.

But most people really aren't that religious about measurement systems, or tightly aligning specific actions with the needle they are most likely to move. The goal of "becoming data-driven" usually isn't accompanied by the discipline and perseverance to make it happen, even though the payoffs are huge.

And none of us are immune to bad metrics. Sometimes, a metric is just too emotionally enticing to give up.

I use one bad metric myself, and no matter how bad I know it is, I keep using it to evaluate (one dimension of) my personal value. PSA: It is never good to tie your worth as a human to a metric (any metric). Gen Z may have more luck than us Gen Xers on this one.

My bad metric, the one I can't emotionally detach from, is number of citations on Google Scholar. I proudly achieved my goal of "more citations in 2020 than 2019" on December 20th, triggered an intense flow of dopamine.

Here's why the "citations" metric is so terribly bad:

• Number of citations is a lagging indicator.  The lag can often be 3-5 years.
• By the time the needle moves, it's hard to figure out exactly what happened to make it move.
• There are very few actions I can personally take to make that needle move.
• Any actions I do take will be indirect. I can make people more aware of my papers, but can't force anyone to cite one... so the actions I can take will influence reach, but not citations.
• There's an interesting social dimension to past number of citations. The more citations you have on a paper, the more likely you are to attract additional citations; similarly, the more citations you have, the better SEO you get on sites like Google Scholar. It's a "fit get fitter" scenario.
• I can't monitor this metric on a weekly or monthly basis. If it dips, I won't be able to respond by taking an action to restore growth.
• I haven't even thought about using this metric as a signal for when I should take action. Because of all the problems I listed above.
• I didn't do anything to achieve my 2020 goal. I just helplessly watched that number creep up, kept my fingers crossed, and (now) celebrated on December 19th when I (just barely) went over the wire before New Year's.
• The calendar is arbitrary anyway. What if I achieved the goal on January 5? Would I feel unaccomplished? Probably yes (this is pathetic)!
• Ultimately, I am not in control of citations. I should have picked an intermediary metric that I am in control of... but that's really difficult, and I'm not in academia any more so I really don't even need to pay attention to this (another giant problem! Why am I still even paying attention? Attention is expensive!)

My Holiday wish for you is: Select your metrics mindfully! Pick ones that are (ideally):

• Not limited to lagging indicators with extraordinarily long lags
• Monitorable on (at least) a weekly or monthly basis
• Designed so that if you aren't achieving your target level, you can immediately figure out where the problem is happening, and even know how to dig down deeper to figure out why that problem is happening
• Triggers for action: every metric that's not where you want it to be should be link to a thing you can do -- that's in your control -- that you know has a pretty good chance of making that needle move the direction you want it to

When your metrics aren't revealing, actionable, or in your control, you've just set yourself up for a special kind of paralysis the entire year.

_{Reprinted from Q&I (December 19, 2020) with permission.}