This post was original part of Kellblog, written by Dave Kellogg, former CEO of MarkLogic, a database company specializing in unstructured information.  His posts cover next-generation database management, search, and content management technologies along with commentary on Silicon Valley, venture capital, and the business of software.

As you read his post, consider what you have been learning about how science shapes the way we know things.  This post is about the fact that some products that are developed do not succeed.  Seemingly, this has nothing to do with “science.”  But consider the assumptions he is making about the scientific process:

• What kinds of assumptions does he make about variables?
• Why is it important to account for different variables?
• He asks two “key questions” at the end.  Why are those important questions to ask?  What other questions arise as you reflect on the answers to his questions.

 

If We Can’t Have Repeatable Success, Can We At Least Have Repeatable Failure?

I’ve always found business to have a fair amount of accidental, built-in hubris, largely resulting from the strategy formulation process.  I remember one time at Business Objects we had a strategy offsite where, in our infinite wisdom and with a fair bit of groupthink, we came up with the idea for a BI workflow solution, which we dubbed Sundance for the name of the lovely venue at which it was conceived.

I remember coming home from the offsite and having a conversation akin to the following:

Me:  What if Sundance doesn’t work?

Exec:  What do you mean, “doesn’t work?”

Me:  Well, for example, what if nobody wants to buy it?

Exec:  What do you mean, what if nobody wants want to buy it!?  It’s strategic.  We can put incentives in the salesforce compensation plans and bundle it.  Don’t worry, we can sell it.

Me:  I didn’t say what if no one wants to sell it.  I said what if no one wants to buy it.

Exec:  But, it’s strategic.  We decided it at the offsite.  You’re talking crazy Dave.  Come have another beer.

As a marketer by background, I tend to view most everything as an experiment.  That is the nature of marketing.  You never know what’s going to work.  You can try different things.  You can measure them.  You can see what works and what does not.  You can even try to build explanations for why certain things work and certain things don’t.  But you are trained to approach business with humility and with an experimental spirit.

Exec:  Look.  Sundance is not an experiment.  We can’t tell Wall Street it’s an experiement.  We need to tell them it’s the future of the company.

Me:  But what if it isn’t?  What if we’re wrong?  Heck, it’s not a bad idea, but we dreamed it up in two hours on a white board.

The problem is that things fail all the time in business.   Products fail.  Startups fail.  Business models fail.  Heck, Sundance failed.  And the bigger problem is that when we dismiss the possibility of failure in our planning, we dismiss the possibility of learning along with it.

Yes, business — and particularly so in startup-up land — is the quest for finding a repeatable, scaleable model.  (Why?  So you can “just add water” and create an arbitrarily large business — and valuation to go with it.)  But quite often in the hurry for repeatable success, managers fail to design things scientifically so they actually have some degree of repeatability and can thus learn from either success or failure.

Example:  you take a new product and put it in the hands of 8 different salespeople (all of whom are “world-class” as defined by the VP of sales) with 8 different backgrounds in 8 different cities selling to numerous types of different target customers with a variety of different sales pitches.  Consider these scenarios:

• Everybody sells.  Buy more stock quick.  Anyone can sell this stuff to anybody saying pretty much anything.  (Hint:  this does not happen very often.)

• Some sell and some don’t.  This is tricky.  Did the folks who sold sell because of their background, their territory, the target customer, their approach, or their salespitch?  Well, we don’t know.  We can look for patterns but we haven’t designed the experiment to make things easy.  The quick assumption is that the folks who sold did everything right and the folks who didn’t did everything wrong, but if you think about it, you can’t assume that’s the case.  Did we have a great salesperson in Chicago pitching the wrong message?  The guy in DC who sold a lot carries a rabbit’s foot — should we dispatch rabbit’s feet instantly to the whole salesforce?  After firing the VP of sales, we learn that “world-class” actually meant “I liked him/her” and can find virtually no additional common traits among the salesforce.  Hum.

• Nobody sells.  This is hard, too.  Was everybody doing everything wrong?  Unlikely.  Yet, no one came together with the right combination to sell.  But are we sure the lady in Chicago is a bad salesperson?  Can we be sure that the pitch they’re using in DC doesn’t work?  Can we be certain that there is “no market” for the product as the VP of sales is insisting?  What can we learn from such a random experiment?  The answer is nothing.

Thus, my statement:  if we can’t have repeatable success, then can we at least have repeatable failure?

If instead of hiring 8 salespeople, we hired only 3,  put them all in NYC, and called only on the same handful of titles within investment banks using the same sales presentation and demo, could we have learned more?   Yes.

• If it works, it’s great news, because you know exactly what to go scale.

• If it doesn’t work, it’s still good news (perhaps for your successor) because you can, with a pretty high degree of certainty, conclude that mix of levers tried will not work.  Or, in the spirit of Thomas Edison, you’ve learned one more way not to make a light bulb.

I’ve picked two extreme cases and there certainly is middle ground in between, but the two key questions are:

• Are we assuming only successful outcomes in our planning?
• Are we designing things as an experiment from which we can learn no matter the outcome?