When hiring an expert, you want someone who knows their stuff. Not just someone who has “been there,” but someone who understands the domain deeply in unique ways and can contribute special value to your team. But how do you know if someone is really an expert or just someone who has simply hung around long enough to claim expertise.

To me, one of the most important qualities of an expert is that an expert can anticipate where things will obviously fail, and where it is not obvious that things will fail. An expert is someone who has seen failure, learned from that failure, and can see where things might fail in ways that others don’t.

Conversely, another important quality of a true expert is that she can anticipate where things will obviously work, and where it is not obvious that things will work. A true expert can weigh equally the possibility of failure and observe that there’s still value in pursuing the goal. They know the odds, and play them.

In law, an expert knows the caselaw sufficiently well to know and articulate the “hard questions.” In technology, an expert knows where you will need to make tradeoffs and where you might be able to gain the unique edge. In chess, the expert knows that sacrificing a piece may be the key to victory.

Which brings me to the surprising mistake we often make when thinking about hiring experts. We often assume that experts are those with experience. But that’s necessary, not sufficient. Hiring true expertise requires asking about failure, and where the candidate has experienced failure, and what they learned from that failure that others haven’t. And it requires asking where candidates made bets that others weren’t willing to make, and why.

When asking for past performance for experts, we need to learn that expertise isn’t the same as experience. Asking about their past successful work is fine. But what we really to learn is to ask about their worst failures and their best bets.