Mo Data, Mo Problems

If you’ve spent any time in high tech the last few years, you’ve probably heard the term “big data” more than you care to recall.  It’s become a constant refrain, and the subject of plenty of breathless cheerleading, much like “the cloud”, “social media”, and countless other trends that preceded it.  This is not to say that big data is not important, but context and meaning are essential.  Big data has many roles to play, but it’s not an end in itself, as Shira Ovide explains so concisely in her recent Wall Street Journal piece

“Data for data’s sake” is the first major weakness of the big data obsession cited by Ovide, and it’s probably the most salient.  This a classic case of valuing inputs over outputs – the idea that if we only collect enough data, good things will happen.  This sort of magical thinking is somewhat reminiscent of past crazes for purely A.I./algorithmic approaches to data science, but at least in those cases there was some concept of outputs and programmatic attempts at sense-making. 

Of course, big data also isn’t going anywhere, and many worthy analytical endeavors demand that we address it.  However, it is essential to distinguish between warehousing, searching and indexing, and actual analysis.  Focusing solely on storage and performance creates a sort of computational uncertainty principle, where the more we know, the less we understand.

As Ovide also notes, there is also a critical gap in analytical talent, which big data has done more to expose than mitigate.   Computing power can go a long way towards making big data manageable and facilitating insight – if paired with a sufficient dose of human ingenuity.  Simply put, humans and computers need each other.  "Pattern recognition” is frequently cited as a benefit of a big data approach, but computers can't learn to spot patterns they've never seen.  As a result, the value of the analyst in defining the correct patterns and heuristics becomes all the more important. 

Appropriately enough, the most valuable and elusive elements lurking within big datasets are often human: fast-moving targets such as terrorists, cyber criminals, rogue traders, and disease carriers who tend to slip through the cracks when algorithms are deployed as-is and left unattended.  The old playground retort that it “takes one to know one” actually applies quite well to these types of situations.

Human capital is a key part of the equation, but it’s not enough to acquire the right talent – you need to address the inevitable organizational challenges that come with retooling for a big data future.  Ovide notes that many companies are installing “Chief Analytics Officers”, and while I want to reserve judgment, the cynic in me suspects this reflects the bias of large organizations to centralize power and create new titles as a first line of defense against unfamiliar problems.  A chief analytics officer could be the catalyst to instill readiness and analytical rigor throughout the organization, but whether this reinforces or dilutes the perception that big data is everyone’s concern is a fair question.

More than anything else, I would analogize the challenges of big data to the differences between conventional warfare and counter-insurgency.  In conventional warfare, the targets are distinct and obvious.  In counter-insurgency, the enemy is hiding among the population.  Much as you can occupy an entire country without knowing what’s really going on outside the wire, you can warehouse and perhaps even index massive data stores without producing actionable insights.  Effective big data approaches, like effective counterinsurgency, require the right balance of resources, sheer power, ingenuity, and strong and constant focus on outcomes.  In the long run, the willingness to pursue a population-centric strategy may well prove to be the difference.

1776 The Ultimate Story of Entrepreneurship

David McCullough’s 1776 is, to my mind, the ultimate story of entrepreneurship.  Starting a company is challenging enough - now imagine starting a country!  Although many orders more complex, America’s founding has much to teach entrepreneurs of all varieties.  And given this heritage, it should also come as no surprise that the United States remains the best place in the world to start something new. 

One of the most valuable things 1776 imparts is an appreciation for the incredibly hard fight endured by the Continental army.  If your most recent lesson on the American Revolution came from a high school textbook, you might dimly recall a few triumphant battles and Valley Forge.  1776 paints a vivid picture of the sheer misery and constant trials of the war – trials few could have anticipated.  The Continental Army’s perseverance is even more impressive when you realize that the Treaty of Paris wasn’t signed until 1783.  For the modern reader, it’s a nuanced lesson: on one hand, you need to be realistic about the challenge ahead, but at the same time, you have no way of really knowing.

The parallels between startups and the Continental army are fascinating.  Some quick observations:

  • Chaos: Compared to the British army, the Continental army seemed completely chaotic. There were no well-defined roles and no visible hierarchy among these ragtag, shoeless countrymen who had taken up arms.  Of course, some of this chaos was real and some was perceived.  The relevant point when starting anything is not how to eliminate chaos, but rather which elements of chaos should be tackled in what order.  Do you address real organizational challenges, or just shuffle everyone’s title? This distinction escaped the British, who underestimated the strength and ability of the “rebels” simply because they looked like a mess. 
  • Meritocracy.  Nathaniel Greene and Henry Knox are two of the better examples.  Greene, a Rhode Island Quaker who had never been in battle before, became Washington's most trusted general due to his exceptional competence and dedication.  Knox was an obese 25-year-old who rose to the rank of Colonel.  He thought up the mission to secure artillery from Ticonderoga, without which the Continental army would have had no such capability. 
  • Talent: Despite Washington’s minor experience in the French and Indian Wars, his principal strength was not military strategy (in fact, his advisors staved off disaster more than once by convincing him not to do something).  His real superpower was his ability to quickly determine who was talented at what. 
  • Food: Food was critical to the Continental army.  Certainly there were times where they were on the move and hardly ate for days on end.  While food was always scarce, the fact that the Army was actually able to feed people with some consistency was critical. The modern startup is obviously not directly comparable, but we’ve seen time and again how providing food pays for itself many times over in terms of focus, productivity and commitment.

But more than simple observations and parallels, there are some real takeaways and strategies for anyone who aspires to start something extraordinary:   

Be Ruthless.

I was shocked by how many times during the course of battle the British would halt their movement to rest or make porridge or something completely non-essential.  There were countless occasions where the side with the advantage could have ended the war, had they only pressed on.  Their reasons should sound a cautionary note even now - stop because it is getting dark?  Stop because that was the plan (despite the ground truth)?  Worst of all: stop because we can finish the job more comfortably tomorrow. 

After routing the Americans and forcing them across a bridge, British General Cornwallis decided to rest.  The Americans retreated brilliantly and swiftly into the night. This was not the Continental Army's first such retreat, so it’s hard to imagine how Cornwallis did not realize the significant risk they posed. Why didn't he send out patrols? Most likely, he thought he would win tomorrow regardless, and preferred not to win under uncomfortable circumstances.  After the fact, he said that he would have kept going, whatever the risks, no matter the orders, if he had only known he would have caught Washington.  The lesson:  Be ruthless as a default setting, not just because victory is seemingly at hand.

Don't Get Overconfident.

Nearly every major mistake by either side in the 1776 campaign was a result of overconfidence.  Minor victories would lead commanders to discard their hard-won knowledge, resulting in terrible decisions.  The tendency to let encouraging signs override our better judgment is actually a fundamental human cognitive bias.  If you’re interested in learning how to recognize and defeat all manner of non-rational thinking, make it a point to read Overcoming Bias

Don't Waste Time Politicking.

General Charles Lee felt slighted that the less experienced George Washington was given command of the Continental army, and constantly sought to undermine him.  When Washington ordered Lee to bring his forces to New Jersey, Lee dawdled, and was captured by the British while seeking some female companionship in a tavern.  Lee was marched to New York in his nightgown, and soon defected.  Much more devastating, however, was a series of letters to Lee from Washington's close advisor and friend Joseph Reed, detailing Reed’s disappointment with Washington.  Why couldn’t Reed have an honest, face to face conversation with his brother in arms to sort through the issues?  In any vital endeavor, there is too much at stake to have closed communications or privately nurse resentments.

It ain't over 'til it's over.

Time after time, each side thought a specific battle was going to be decisive.  In retrospect, it is amazing how incredibly wrong they were, and how often.  So how do you respond? There is a fine line between being jaded and being realistic. Starting something invariably requires commitment in the face of uncertainty.  For this reason, I’d argue that it’s better to be optimistic (even if slightly naïve) than completely cynical, but again, the key is to be aware of our biases.


Business Schools and Employability

According to a recent Wall Street Journal article, business schools are placing increased emphasis on the employability of their students prior to admission.  I won’t speculate to what extent this is motivated by the need to protect their job placement statistics in a grim economy, but it’s worth considering the true consequences of this trend.  As the article notes, business schools have always considered the goals of the applicant – but to what extent are they curating these goals on the front end?  Even if we assume good intentions, the effect is to reinforce the status quo, making business school populations even more risk-averse and less entrepreneurial.

Ironically, this seems to be at least partly motivated by the banking collapse: “when the financial crisis upended the banking sector and sure-thing jobs on Wall Street disappeared, schools began formally tying input (applicants) to output (graduates).”  Why “ironically”?  Regardless of how much blame you want to assign to federal housing and lending policy as opposed to private sector recklessness, the financial crisis wasn’t brought on by entrepreneurial, non-linear thinking. Legions of conventionally smart people who had done everything right, rigorously following twenty year plans including name-brand firms and business schools, managed to get the biggest bets horribly wrong.  This is not meant to be flippant – current market conditions and job statistics are stubborn things that must be acknowledged.  However, if the lesson of the financial crisis is that we should double down on conventional wisdom, regardless of whether anything of value is created, then we’ve indeed learned nothing from the past five years.

As someone who frequently uses the frame of inputs vs. outputs, I took immediate notice of the wording above.  It would be encouraging to see an extremely input-focused sector more concerned with outputs, but I suspect they have confused the two in this case, merely trading one set of inputs for another (the addition of an MBA).  You can also think of this as commoditizing human capital, and this calls the entire purpose of an MBA into question.  Is business school meant to, help develop leaders, or serve as a finishing process on a prestigious kind of assembly line? 

The article goes on to state that “making employability too weighty a factor in admissions can backfire. “ According to Graham Richmond, a former admissions officer at University of Pennsylvania's Wharton School, “Looking at applicants through a narrow vocational lens may deter schools from accepting riskier candidates, such as entrepreneurs or career-switchers, in favor of more sure things, such as aspiring management consultants.”  The fact that aspiring management consultants are considered “sure things” is evidence of how much MBA culture values process over invention.  Candidates and schools understandably want assurances, especially in the wake of 2008.  The world is a chaotic place, even more so since the financial crisis (though I contend that it has always been so, and that the banking industry simply managed to insulate itself unusually well for as long as it could).  Obviously, you have to adapt to the current reality.  Yet I can’t help but wonder if by focusing on doing obvious, “safe” things, to the exclusion of risk-taking and creativity, the MBA community isn’t just constructing an elaborate playpen in which nothing new ever happens.

Calling All Computer Scientists in Southern Europe

One of the most startling yet largely under-reported facets of the European financial crisis is the rate of youth unemployment, especially in Southern Europe.  If you are a young person in Greece (58%), Spain (56%), Portugal (39%), Italy (37%), or France (27%) you are likely looking elsewhere already. There are certainly nearby places with a shortage of qualified workers (such as Germany), and when any job is scarce, it may seem a strange time to be seeking your ideal job.  

Yet, for those of you who studied engineering (especially computer science) that is exactly what I am suggesting.  Palantir is hiring aggressively in Palo Alto, New York, Washington, Los Angeles, London, Australia, New Zealand, Singapore, and beyond.  If you are not only technical, but also passionate about using technology to address problems that matter most in the word, Palantir (and I personally) would love to hear from you. Why Palantir?

Meritocracy: Silicon Valley has the highest concentration of great computer scientists of anywhere in the world.  If you are a gifted young computer scientist, you belong with a Silicon Valley company if not in the Valley itself.  Of all the great things about Silicon Valley, meritocracy may be the greatest differentiator.  There are no long apprenticeship or trainee programs at Palantir (though we are always learning).  Everyone is equipped to begin working on real problems within weeks. Good ideas don’t have to pass through a massive hierarchy - the best idea wins, regardless of whose idea it is.  

Save The World: Palantir is focused on solving the most important problems for the world’s most important institutions, and we are always exploring new uses for our platforms. Some of our major areas of application financial oversight, disease control, narco-trafficking, protection of children, cyber security, protection of civil liberties, and most recently,  disaster response. In the face of global austerity, we are helping governments to get the most out of limited resources, and working with financial regulators to prevent the next financial crisis before it happens.

These are uncertain and volatile times, especially for Europe, yet there has also never been a better time to be part of something extraordinary. 

Apply Here (the path to a better tomorrow): 


The Soft Underbelly of Technology Services

I spend a lot of time thinking about delivery models for technology, especially in an age of shrinking budgets and growing complexity.  So I was struck to read that Avanade, a joint custom software venture between Accenture and Microsoft, had been sued by a customer for major cost overruns.  The key part:

The lawsuit said a software project estimated to cost $17 million and take 11 months instead mushroomed to $37 million over three years, and ScanSource said it still doesn’t have a Dynamics software up and running. Accenture has estimated it will cost $29 million more to complete the ERP project, according to ScanSource’s lawsuit.

What can be learned from this? There are quite a few things.  The cynics among us might point out that an overrun of $20 million and 2+ years is considered a bargain in some areas of government.  That is of course an outrage, but the important takeaway goes beyond the numbers, to the fundamental nature of the delivery model.  Let’s assume for this conversation that actors all good faith and very competent here.  I think that despite that, the model leads to these sorts of outcomes.

Not surprisingly, Avanade turns out to be in the business of renting labor.  Services is the exact wrong model – a catastrophically incorrect model, the more you think about it. These sorts of incidents are really a lagging indicator of the weakness in the model, but it’s taking a whole lot of innocent (and some not-so-innocent) bystanders with it.  More on them in a few.

There are many shortcomings to services model, but most fundamentally it’s the wrong incentive structure.  When you’re renting labor and other nebulous inputs, it’s almost a truism to point out that the longer it takes, the more the company prospers, and the bigger the project, the more room for abuse.  A contractor doing a bathroom remodel might employ a similar cost structure, but could never get away with overruns on a tenth the scale of those alleged in the Avanade lawsuit.  Of course, even if you have reliable cost safeguards in place, custom software development is inefficient, as I’ve often railed about in these pages.  It takes an army of consultants to deliver, and another army of consultants to maintain.  

It’s not all the services company’s fault, though – not even primarily.  In a sense everyone is complicit, from the services company, to the customer who doesn’t demand something better or structure payment to be a premium but based on success, to the tech giants who aren’t working to productize services.  Of course, if product companies dared to do so, the services companies of the world would throw a fit, and professional courtesy runs deeper than you might think in a theoretically competitive marketplace.  

When the world changes, you don’t always get a say in the matter, and evolution has a funny way of sneaking up on those who get too comfortable.  The first indications may just be bubbling to the surface, but two things are clear: services companies are under tremendous pressure, and product companies need to productize services. 

The first point makes sense from a valuation standpoint.  Mature tech companies such as Oracle and Microsoft have market caps of ~5-6x annual revenue, while the multiple is often less than 2x for services firms, even the upper tier.  Yet it’s still not obvious to all that services companies are living in the past (partly because many services companies are so good at convincing people they’re really technology companies).  Mostly, though, it’s because services companies still generate a lot of money.  It’s a dying model that’s still making people rich, so it’s easy to ignore the warning signs even if you see them.  And for an exponential trend, by the time you are 1% there, it is almost done.  You could almost analogize it to the SUV craze: consumers couldn’t get enough gas-guzzling SUVs, and American auto makers happily served them up for several years.  Suddenly (but not all that surprisingly), $3-4/gallon gasoline was a fact of life and those same automakers were all teetering on bankruptcy for giving the customers exactly what they wanted.

In terms of multiplying complexity and data problems, we’re entering an era of $10/gallon gas.  Even if you’re in the product business, if you’re not increasing your productivity per person, you are dying – in some cases more quickly and dramatically than the services dinosaurs.  And for this reason, product companies can’t just deliver products any more – they need to productize services on a continuous basis.  In short, they need to deliver outcomes.  Mere capabilities only work against well understood problems.  They won’t be sufficient for the types of challenges that grow appreciably bigger in the time it takes to read this blog post.  

If that sounds smug, it needs to be acknowledged that building a business based on outcome delivery, as opposed to a static product, is still extraordinarily hard.  Not only are the prevailing incentive and cost structures far behind, but technically speaking it’s a very rugged frontier.  This is perhaps best illustrated by software, where performance at scale, processing, security, stability, and interoperability are often much bigger challenges than achieving the desired functionality.  On the other hand, though, successful technology has always productized services of some kind, dating back as far as the cotton gin or even the wheel.  The entropy of the present and future data landscape adds an enormous degree of difficulty, but along with Moore’s Law, the single biggest lever of the knowledge economy is the ability to repackage experience and lessons learned into a better, more responsive product.  It may take years or even decades, and it’s entirely possible that the first mover will end up being a sacrificial lamb.  Sooner or later, though, the company that gets productization right will eat the legacy companies’ lunch.

Inputs vs Outputs

A defining difference between Silicon Valley and the Old World is that Silicon Valley is intensely focused on outputs as opposed to inputs.  While the shift to an outcome-based  economy remains a work in progress, the high-tech world tends to focus on tangible results, not ingredients.  It’s not just about a different way of thinking about business – it’s a matter of different societies and what they value.

One of the original inputs is ancestry.  No one in Silicon Valley will ask you who your parents were or what they did, whereas people absolutely will in the Old World and East Coast.  At some point in American history, having ancestors who came over on the Mayflower became an indicator of New England aristocracy – funny when you consider that the Pilgrims themselves were people of no social standing, building something from scratch.

Input bias is easy to observe in classically process-oriented companies (and societies).  Fixation on research and development is a prime example: the value of the final product is judged by the input (“it cost us $500million to develop this”) more so than the results. Spending in general is frequently touted as an absolute good or evil ipso facto, but it’s actually one of the least relevant data points on its own.  When we talk about confusing cost with value, we’re really talking about confusing inputs with outputs.

Wall Street is extremely focused on inputs, even though their efforts are ostensibly measured by outputs, and fairly straightforward ones at that.  On Wall Street, input doesn’t just refer to assets under management – it’s about name-brand firm experience, having an MBA from the right school, who designed your suit, even your business cards.  Ironically, Goldman Sachs, the biggest name on Wall Street, transformed itself from a struggling, undistinguished firm to the world’s top investment bank under the leadership of Sidney Weinberg, a junior high school dropout. Weinberg was originally hired at Goldman as a janitor’s assistant making three dollars a week – an anonymous and menial job, certainly, but a job at the firm judged solely on output.  

Where you went to school is an obvious input, but outputs matter for the endurance and success of the school itself, especially young schools.  How did Stanford, founded in 1891, achieve equal footing with the Ivies?  Money certainly helped, but intermingling with Silicon Valley and entrepreneurial culture played a much greater role than simply having wealthy donors.  From legendary engineering dean Frederick Terman, who mentored (and invested in) Hewlett and Packard, to the founding of Yahoo! and Google by Stanford grad students, to Peter Thiel’s recent course on entrepreneurship, Stanford and Silicon Valley have enjoyed a unique symbiosis.  In terms of clear outputs, a recent study found that companies founded by Stanford alumni create $2.7 trillion in annual revenue.   Beyond pure productivity, Stanford arguably introduced the concept of great entrepreneurs as a tangible output of a university, mentioned in the same sentence as Nobel laureates and world leaders.  The willingness of many of these great entrepreneurs to reinvest not only their money but also their wisdom and mentorship into the university is one of the great virtuous cycles in education.

Perhaps the ultimate input is age, and when a society values something simply for being old, it speaks volumes – especially when that something is itself.  The output that matters is enduring impact and relevance.  For the Old World, the danger is that reverence for the merely old is so deeply ingrained that by the time a society realizes it’s stagnating, it is exponentially harder to reverse the tide – witness the number of once-great empires of Europe struggling to stay afloat.  The United States is an obvious counterpoint (not that we can take that for granted), and I’ve often reflected that Silicon Valley values are really American values writ large, but there are new revolutions happening all the time, even in very old societies.  China and India were home to ancient and storied cultures, though neither was a world power as recently as the mid-20th century.  Today, in a post-imperial, post-Soviet world, they are major players, buoyed by high-tech explosions that would have been unimaginable fifty years ago.  Yet I would argue that such transformation only became possible when China and India collectively decided that only outputs, not the systems that produce them, are truly sacred.  



Focus on the First Derivative

In a fast growing company, everyone has less experience than they need for their roles, by definition.  This will continue to be true as the company scales, one's role changing in a fundamental way every 3-6 months, especially when it continues to defy expectations for months and years.  Ultimately, that’s all irrelevant.  In Silicon Valley, we like to talk about visionary leaders making momentous decisions amid great uncertainty, but what really matters is the first derivative of understanding: how are you and your team learning from the experience as it unfolds?  There are many considerations nested in this question – here are some of the most important:

How quickly are you learning? When you are operating within a tornado, speed counts for a great deal.  It’s often been said that even the right decision is wrong when taken too late, and this begins with learning. If the second and third order effects of your original challenge are already on an irreversible course by the time you’ve grasped the nature of the challenge, it’s no longer the same challenge.

Are people taking the same things away from failures? In an ideal world, everyone would not only draw the same conclusions from the experience, but they would also be the correct ones.  More often, the process is a lot messier, but that’s just reality – you learn together through give and take, not some mystical collective unconscious.  The key is that you are unified about your next move.  

Are you making meaningful abstractions, or just reacting to your immediate circumstances? Even when execution is everything, there is such a thing as being too tactical, and morale plummets when people can’t make abstractions (or they aren’t taken seriously).  It’s a delicate line, because your abstractions have to be actionable and part of a continuous cycle of learning and responding.

When dissent occurs, is it productive? Just because you eventually arrive at the same takeaways doesn’t mean there is no room for disagreement.  The question is whether it’s healthy and constructive, or pointed and personal.  The “team of rivals” concept has gained many adherents in recent years, but it’s important to remember that it’s above all a team.  Ideally, iron sharpens iron.

Three Two strikes and you’re out.  In certain areas, such as distribution, you don’t get many chances to course-correct when one approach fails, so extracting the right lessons from the first failure is paramount.  This is not to say that you should impose needless anxiety on these kinds of decisions, but be aware of what the stakes are.

Can you reform your model? Models can be extremely useful and necessary to consolidate your understanding of a complex world and plan accordingly.  However, they can also be an especially insidious kind of blindfold.  Adjusting your model, or abandoning it when necessary, can be incredibly difficult, because it requires you to first recognize and confront your inherent biases, and resist the tendency to rationalize away the model’s shortcomings.

In a hyper-growth environment, you will never have enough information, experience, or foresight.  The first derivative will be the only thing that matters.  We became the ultimate learning animals through many unforgiving eons of natural selection.  This new evolutionary challenge of warp-speed learning and adaptation may feel significantly more abstract, but once again, it all comes down to survival.


Calculus vs. Statistics

If you have more than a passing interest in the future – be it yours, your venture’s, or humanity’s writ large - Peter Thiel’s CS183 lecture #13, “You Are Not A Lottery Ticket” is a feast for thought.  Thiel interrogates the underpinnings and consequences of determinate and indeterminate worldviews in numerous contexts, including as they apply to startups. 

For the aspiring tech entrepreneur, one of the most useful frameworks Thiel invokes is that of calculus (determinate) vs. statistics (indeterminate).  In calculus, you make precise determinations, often concerning discrete futures.  You can figure out exactly how long it will take to drain even the most irregularly shaped swimming pool.  And this enables you to do things of vital importance.  As Thiel notes, when you send a rocket to the moon, you need to know where it is at all times – you can’t just figure it out as you go.  In statistics, on the other hand, there are no certainties.  It’s about bell curves, random walks, and drawing an often uncomfortable line of best fit between limited data points.  Thiel furthermore notes a powerful societal shift towards the belief that statistical thinking ways of thinking will (and should) drive the future.   

The example of landing a rocket on the moon is probably no accident. The 1950s and 1960s (coincidentally the first golden age of Silicon Valley) were a time of widespread American optimism.  The moon landing was a fundamentally optimistic venture that captured the American imagination (and quite literally would not have happened without calculus).  It only makes sense, then, that statistics would be the dominant modality of the cynical world we now inhabit.  If you look at the natural disasters, economic collapses, terrorist attacks, and disease outbreaks of the 21st century, some might seem more or less predictable by conventional wisdom, but the popular perception is that humanity was caught napping, apart from a few obscure Cassandras.  Especially in light of the truism that we’re usually planning for the crisis that just happened, it’s easy to see the appeal of the indeterminate/statistical model.  Statistics couldn’t have predicted exactly which bad things would happen, only that some bad things would happen.  

It’s enough to make you throw up your hands, yet this is exactly what Thiel is not arguing for.  This should come as no surprise. Thiel is a renowned contrarian, and many of his greatest interests reflect a healthy disregard for statistical/indeterminate thinking, life extension being a prime example.  The conclusion of the lecture begins with an acknowledgment that as we embrace the statistical worldview, society is sliding into pessimism, and without indulging in too much pop psychology, it’s easy to see how such thinking becomes self-fulfilling.  The lecture ends with an appeal to “definite optimism”, and posits that computer science offers the best hope.  CS is not only a great way to solve problems, but as Thiel observes, its fundamental determinism may have something to teach startup culture, which is widely presumed to be governed by indeterminacy.

Of course, software itself is greatly misunderstood, and this is one of the primary challenges computer scientists face as entrepreneurs. People who don’t understand software assume that its value is statistical by nature, and fundamentally unknowable (in contrast to hardware, for example).  If you’re a math phobic, single-variable calculus and E = mc2 are just two things you don’t understand, and the differences and relative complexities are immaterial.  To make matters worse, people who truly understand software are relatively rare, especially among those with purchasing authority, and this unknowable fallacy leads to a sort of permanent agnosticism in principle as applied to software.  Within the statistical frame, it’s assumed that two competing software packages lie in the same general area of the bell curve, and therefore the differences are negligible or at least unknowable.  You know that the value of software follows power laws and the differences between good and great are logarithmic, not linear, but the statistical frame ignores all of this.

One consequence is extreme risk aversion: if you believe that the relative merit of one kind of software isn’t calculable, you stick with what you already have, and this has plagued many otherwise forward-thinking institutions.  There is also the simple matter of what’s tangible.  To the layman, hardware seems straightforward, whereas software doesn’t (even if hardware may owe much of its performance to superior software).  As a result, hardware is often seen as a reasonable expenditure, whereas software isn’t.  No one blinks at a $50 million aircraft, even if that aircraft is agreed to be 1980s technology, whereas $50 million for software is not only unthinkable to many, but being newer and better may very well work against you, due to the unknowable fallacy.  

For the aspiring software entrepreneur, there are a few takeaways.  It’s a fact of life that software is misunderstood and undervalued.  However, that doesn’t mean quality doesn’t matter.  In fact, it matters more than ever.  The challenge is that when you are up against a heavy incumbent, it’s not enough to be 10% better – you have to be 10X better, because ultimately your success is dependent on enough people feeling strongly enough about your product to risk rocking the boat.  Earlier we discussed that the idea of any complex product being great enough to sell itself is a myth, and again, concluding that being great is unimportant is absolutely the wrong lesson.  Put another way, if you want to bring people around to viewing software through a calculus frame, you have to make their daily existence demonstrably better.  But wasn’t this always the goal?

This brings up a final point about determinacy: some things are worth doing regardless.  In the last CS 183 class, “Stagnation or Singularity?”, Thiel is joined by several guests, including Dr. Aubrey de Grey, gerontology expert and Chief Science Officer at the SENS Foundation.  De Grey makes the point that while we may have a fair idea what technologies will be developed, the timeline for development is much more tenuous and subject to various externalities.  However, he concludes (paraphrased), “In a sense, none of this matters. The uncertainty of the timeline should not affect prioritization. We should be doing the same things regardless.”

Once again, it all comes down to doing important things, and when this is the stated goal, the inherent pessimism of the statistical approach becomes apparent.  This applies to your own life as well as it does when building a company.  If you wanted to take the statistical view to its logical extreme and hedge against all possible uncertainties, you’d become a jack of all trades/master of none, and consciously choose not to go long on any particular superpower or world-changing problem. If the goal is to live an inoffensive, comfortable life, this might makes sense.  If you want to do anything of lasting value, this is crazy.  In some ways, it’s easier to grasp this concept when designing new technology or building a company – although it’s easy to suffer from too many features or too many business models, most entrepreneurs accept that trying to be all things to all people is a recipe for failure (as software development illustrates so neatly).  Technology needs a problem to solve.  You, on the other hand, are not a problem to be solved – yet what to do with your time and gifts is perhaps the most worthwhile problem of all.


Execution is hard, and distribution is one of the hardest (and not surprisingly, least understood) aspects of execution.  Peter Thiel gives the subject of distribution an extremely thorough treatment in CS183 Lecture 9, “If You Build It, Will They Come?”, including mathematics, psychology, and market-specific models.  Rather than trying to summarize the extensive substance of the lecture, I’d like to focus on how you might think of the distribution challenge as an engineer, in the context of the Your Future series.

Thiel begins by addressing the most basic question – what is distribution? Surprisingly, many people can’t give you a coherent answer, and if they can, there’s a very good chance they underestimate its importance.   If we agree to define distribution as how you get a product out to customers, it becomes a bit more concrete why there’s so much misunderstanding around the topic.  It’s especially difficult when you’re creating software or other technologies that require meaningful user engagement.  If you think of distribution as just getting a product into users’ hands, you’ll likely fail – either because you assume that a product will get used just by virtue of being available, or that the product will remain in users’ hands once it’s reached them.

If you look at two of Thiel’s biggest success stories to date, PayPal and Facebook, you’ll find two companies that nailed distribution, and in very different ways.  It’s worth noting that online payments and social networking sites were both extremely noisy spaces when PayPal and Facebook joined the fray, and neither company had first mover advantage (though as Thiel discusses elsewhere, this may not be such an advantage after all).  Also significant is the fact that online payment processing and social networking sites are both fairly easy to prototype and hack away at.  Of course both PayPal and Facebook hired outstanding engineers and eventually encountered (and overcame) serious technical hurdles – security/fraud in the case of PayPal and scale in the case of Facebook – but I’d argue that those problems only emerged because they got distribution right first.  

As Thiel calls out early in the lecture, engineering bias works against you when it comes to distribution.  As engineers, we are conditioned to think that great products will just reach consumers by virtue of being great (and there’s a dangerous tendency to assume that your idea of “great” is representative).  The concept of a product being “so good it sells itself” is universally appealing - and universally incorrect.  It just doesn’t happen.  It is possible to create an environment where the best idea wins within the confines of your own company, and I urge you to retain this form of idealism, but any market is a fundamentally irrational place, and you need to make peace with that fact.  

Another major difficulty is that so many young engineers in Silicon Valley have been spoon-fed a massive user base, either because they joined a company that already had one, or they piggybacked on one.  Of course, this is a valid distribution channel – the path of least resistance is by no means the wrong approach.  The problem is that it skews the way you think about design and innovation.  Most engineers in non-entrepreneurial roles haven’t had to think about distribution at all.  And that’s fine, as long as you realize that you started on 3rd base and didn’t hit a home run—not for the sake of your ego, but for the sake of your next venture.  You have to approach the might distribution challenge with the humility it deserves, so suffer at her hands.

Whether distribution can really be “engineered” is a topic for another day, but it worth thinking about what makes engineering different from sales, and for the aspiring founder, this is one of the biggest takeaways from the lecture.  I’m not so much concerned with the merits of different distribution approaches as with recognizing the how the skill of distribution (to include sales) lines up with your and your team’s strengths and weaknesses.  It’s no secret that I’m a huge fan of engineering-driven companies, but it’s not enough to focus on your strengths – you also have to even out the competencies you lack, and chances are sales/distribution is among these.  

Why is this? As Thiel notes, sales is a fundamentally irrational enterprise, and engineers are concerned with rationality and truth-telling.  However, their general discomfort with and lack of aptitude for sales isn’t just about purity of spirit, but also about knowing what to look for.  In many cases, it’s not clear what quantifiable skills are actually involved in “sales.” (hint: this ain’t it: Crazy Ernie).  If you convince yourself that these skills aren’t important, or don’t have a place in the kind of utopian company you want to create, you not only ignore one of the central aspects of distribution, but also create a huge talent gap, because you need at least a few folks with these skills.  Think of it as a special sort of project management skill—the ability to get a distribution project across the finish line.  A crude model, but useful in framing the challenge for us engineers.

There are many risks inherent in the worthy goal of starting a company: team risk, innovation risk, technical execution risk, and business execution/distribution risk.  In addition to the first three, distribution is something you need to be thinking about in the foundational stage, not something to be revisited at an undefined point in the future.  Importantly and subtly, distribution risk affects innovation and technical risk in turn - and every form of risk is ultimately a team risk.  Feedback from the field/your customers becomes the fuel (to your creative mind’s spark) for iterating and conquering – you will be on an empty tank without distribution.  If you’re trying to start something, it’s almost more important to ask who on your team is credible in each of these areas than how you’ll specifically get there.  



Think Again About Sticking Around for a Masters Degree

Mark Twain was said to have remarked “I have never let my schooling interfere with my education”, and whether the quote itself is apocryphal, the sentiment should be applauded.  True education is a beautiful thing. A master’s degree, on the other hand, is not only a waste of time (with a few exceptions I’ll get to), but often epitomizes that proverbial interference.  

As I spend a lot of time talking to college students I encounter many who are signing up for the increasingly popular one-year master’s degrees.  I understand the appeal from the student’s perspective and I understand the appeal from a parent’s perspective.  In fact, I pursued one myself.  I had just finished undergrad, I didn’t have a compelling opportunity, and more importantly I had somewhere to get to (Silicon Valley).  For many of today’s brightest engineers, I don’t think a master’s degree makes any sense, and that was exactly the advice I gave my brother who just graduated.  In order to appreciate why a master’s might not be such a wise idea after all, it’s worth considering what makes education meaningful to begin with.  

To begin with, education creates opportunity.  This has probably been drilled into your head from an early age, and for good reason.   If your parents were the first generation in your family to attend college (or you are) this needs no explanation, and it’s a tragedy that education is taken for granted by anyone, let alone so many.

There is also much to learn – much more than most people would ever guess.  When I went off to college, I certainly thought I knew more than I did.  Being disabused of this notion may seem like the first step in one’s education, and it often is, but it’s a really a lesson worth relearning at any stage.  Then there’s the process of learning how to learn, and this is one of the primary reasons you go to school, independent of your field of study.  There are countless dimensions: learning to make abstractions and conceptualize, to interrogate a problem, to work inductively and deductively, to separate first principles from careless assumptions.  You need to experience breadth, both to strengthen your foundation, and to find subjects worthy of exploring in depth.  

Education also provides a unique platform to gain impactful life experience in a low-risk environment.  School is a place to build formative relationships, explore different paths, be in charge of your own time and activities, even start something if you are so inclined.  Perhaps most importantly, it’s a time to learn about your strengths and weaknesses with high upside and low downside.  Of course, college is costly, and time itself is far from trivial, but it’s much easier to avoid loss aversion and do something truly experimental when you’re not deep into your career.   The phrase “do something” is the key here – “finding yourself” has become sort of a cliché of indolence, but it’s while moving forward that you truly find yourself, and college can be the perfect place to do this.

Finally, education validates that your best years really are ahead of you.  High school is certainly a valuable experience, and in the best case can lay the groundwork for the level of exploration that college makes possible.  At the same time, it’s a small pond both socially and in terms of what you’re asked to do.  However triumphant or painful it is, it’s not a place to remain.  College may feel like a time for reinvention, but it’s really a time for original discovery.  

 To understand why master’s degrees are superfluous and even counterproductive for engineering students, I like to use the framework of getting somewhere versus getting something.  You can also think of this as a means to an end as opposed to an end in itself.  Education provides many things of intrinsic value, but much as nine months, give or take, is enough to prepare you for the outside world, so is one degree.

The exceptions tend to fall into the category of getting somewhere: for example to Silicon Valley or to the United States.  A master’s degree can also be a good way to test the waters of academia.  You can take classes with doctoral students and get a feel for the academic life without having to commit to a dissertation or taking on a teaching schedule.  For some friends, a master’s has been an informative gateway to a promising academic career, whereas others consider it worth the price of admission to have been persuaded that doctoral studies aren’t for them after only a year.  I am not coming down on one side or the other, only advocating for informed choices.

On the other hand, if you’re trying to obtain something – experience, distinction, deeper cultivation of your superpower – it’s usually better to just get that thing in its pure form.  If you want entrepreneurial experience, go out and get some – don’t learn about it in an MBA course.  If you want to be a better software engineer, don’t sign on for an extra year of TAing - work on challenging, real-world problems in a production environment, with peers who force you to raise your game.  

I’ve said before that learning for its own sake is not necessarily valuable, and this is especially true of master’s degrees in the workplace, especially degrees in one pure subject (as opposed to MBAs and other first professional degrees).  There is a lot of misleading and unsubstantiated chatter that a master’s degree makes you a more valuable employee ipso facto, and this is just not the case, as many people find out the hard way.

It’s also worth acknowledging that there is often a socio-cultural bias towards more education, especially among generations who experienced firsthand the power of education to achieve an objectively better life.  This is not a perspective to be discounted, but at the same time you need to recognize when the preference for more higher education is no longer contextualized and ignores the question of somewhere versus something. 

Finally, and most importantly, don’t do a master’s because college is fun and it will never get better than this.  If you care about your future as much as I imagine you do, that simply won’t be true (regardless of whatever sentimental projections people offer you).  The fifth year isn’t like the fourth.  Everyone is gone, and you realize that what made college meaningful was the people who went through the experience with you, not the buildings and the campus.  Moving on is not always easy, especially when you strip away the structure and predictability of school, but it’s simply time to forge new experiences.  If there’s one thing I’ve learned since leaving school, it’s that they can all be the best years of your life when you get out there and Do Important Things.