FROM THE HBR.ORG INSIGHT CENTER “FROM DATA TO ACTION” | 2© 2014 Harvard Business Publishing. All rights reserved.
WHAT TO ASK YOUR “NUMBERS PEOPLE” BY TOM DAVENPORT
If you’re a manager working with the analysts in your organization to make more data-driven business decisions, asking good questions should be one of your top priorities. Many managers fear that asking questions will make them appear unintelligent about quantitative matters. However, if you ask the right kinds of questions, you can both appear knowledgeable and advance the likelihood of a good decision outcome. In my new book (co-authored with Jinho Kim) Keeping Up with the Quants, and in a related article in this month’s HBR, we list a lot of possible questions for various stages of analysis. But in this short article, I thought it might be useful to mention not only a couple of the most important questions you can ask about data, but also what some of the ensuing dialogue might involve.
1. Questions about assumptions You ask: What are the assumptions behind the model you built? You think in response to the answer: If your numbers person says there are no particular assumptions, you should worry—because every model has assumptions behind it. It may be only that you’re assuming that the sample represents a population, or that the data gathered at a previous time are still representative of the current time. Follow-up: Is there any reason to believe that those assumptions are no longer valid? You think in response: You are really looking only for a thoughtful response here. The only way to know for sure about whether assumptions still hold is to do a different analysis on newly gathered data— which could be very expensive. Perhaps a particular relationship holds only when the values of a variable are moving in a particular direction (e.g., “This mortgage risk model holds true only when housing prices are going up”—nah, that could never change!).