Technology is a wonderfully levered thing, but there is a dangerous temptation to believe that you can change the world without engaging with its institutions, in all their sprawling entropy (setting aside Silicon Valley's frequent delusions about what qualifies as changing the world). When considering how and if to engage with national and global institutions, I've found that people gravitate toward one of two philosophies:
The first holds that the world's institutions are fundamentally evil, and subconsciously people ascribe a certain malicious competence, e.g. the repressive government or ruthless corporate polluter. If this is the norm, your only choices are revolution/anarchy, or much likelier, doing nothing. I disagree, but people are free to believe this.
The second viewpoint holds that the world's institutions are not fundamentally evil, but need help to become great - help that must be renewed as institutions change and people cycle through. Trying to be fair-minded, people often attribute institutional problems to incompetence as opposed to malice (though in my experience this is usually inaccurate or exaggerated).
What I've learned from my struggles in the second camp is that there are no easy answers - but the goal is better, not perfect. These are the convictions that have helped me to contend with all the complexity that follows.
My first principle is to get off the sidelines. Complaining is a dead end. The question needs to be “who will help them?” The answer may be you.
If so, the second principle is to engage with the world as it is: messy and gray.
Overnight change is a myth. The goal is to make today better than yesterday and tomorrow better than today.
In order to bring about change, you need to cultivate some intellectual humility. But early on, it helps to not have excess amounts. Happily, most people lack humility in youth (not a judgment of millennials; it's just human nature). And this allows you to attempt epic things. Along the way, with any luck, some wisdom seeps in. As a student, I thought everything was dumb. Surely I knew a better way. As I got out into the world, I slowly realized I'd been asking the wrong question. I dwelled on the first order: why is this so broken? Eventually, I realized there was a more meaningful question: what must be true for this to make sense? From there, you keep pulling the string until you find the underlying condition that requires fixing.
Politics is a dirty word for idealists and engineers alike (doubly true if you happen to be both!). But you can't address the most pressing problems without also grappling with the common good, which necessitates political considerations. The reality is that politics and engineering each have roles to play:
Politics is about accepting the tradeoffs. We live on an efficient frontier (or so we hope), forcing us to examine the tradeoffs between X and Y. In the political sphere, preference for X or Y, roughly speaking, defines left and right.
Engineering is about innovations that push out the efficient frontier – we can have more of X and Y, and the political viewpoint merely serves up false tradeoffs that rip us apart.
One problem with politics is that in a democracy, both sides are always right, or at least deadlocked. X is important. Y is important, too. Structurally the only good resolution is to invest in more of both. At its best, engineering can render a painful tradeoff false - think power vs. efficiency, human vs. computational acumen, or the former truism that you can only pick two among better/faster/cheaper.
Beware thinking that engineering is a cure-all for political ills. In reality, engineers cannot recuse themselves from the painful organizational aspects of a problem they're trying to solve. Software is fairly unique as something you can open source for the public interest, but it's exceedingly rare that you get to be Johnny Appleseed, sprinkling free technology on grateful soil. The hard work happens at the intersection of products, problems, and people.
It's also worth unpacking some assumptions about the nature of institutions more generally. We tend to think of them as static, almost by definition. This is understandable, especially in the political realm. If you're graduating college this year, you have probably never known a time when bipartisanship and compromise were considered positives, not accusations to be hurled by primary opponents. In reality, though, institutions require people, and the people themselves are only as static as their motivations permit them to be.
On the flipside, one of the most important questions you can ask is whether you are working for a specific person/administration, or for the historic, enduring ideals of that institution. This is a core principle of the US military, and regardless of one's feelings about war, it's a tradition worth emulating. In a democratic society, the failure mode is less likely a cult of personality, and more likely an institution that becomes insidiously focused on consolidating power and resources. Sometimes, institutions bloat past the point of no return, and must be broken up or declawed. Often, though, an infusion of new blood can restore focus to the real mission.
While it's clear that governments should advance causes beyond their own prosperity, I would argue this is not only possible but essential in the commercial realm. We don't remember Steve Jobs for making shareholders a boatload of money; we remember him for giving us new ways to communicate, learn, and experience art. There are many forms of value to be created, and striving for the perfect should not preclude exploring different forms of good. But forms aside, an institution's ability to do more than just enrich itself is also a great indicator of its prospects for enduring - and its worthiness of your involvement.
On a related note, people often discount the ability to work through an institution, not just for it. I don't mean capitalizing on existing channels or infrastructure, but advancing a larger goal. You can think in terms of ceilings vs. floors. Tesla must meet certain safety and efficiency standards to qualify for tax rebates and remain street legal, but these requirements are the floor, not the ceiling. They're catalysts for more aggressive innovations, even those that don't yet have a market.
There is also the truism that if your work is black or white, it probably isn't that important - or worse, you are lying to yourself. The most pervasive problems are subtle, complex, and require deep engagement with both established institutions and emerging forces. It's easy to point to noble aims consumed by unintended consequences and be scared of getting involved. The Arab Spring fueled the rise of theocrats, not democracy. Even seemingly non-controversial developments can force hard choices. Let's say you develop a promising cancer treatment in a startup or research lab. Do you try to take it to market yourself, or through a huge pharma company? Do you optimize on keeping it affordable, or raise prices in order to enable you to expand your reach? Easy answers remain elusive, but to me, these examples are all arguments for more thoughtful engagement, not abdication. You can get off the sideline without getting on the soapbox.
Lastly, when pondering which institutions to join or serve, instead of asking “Why Institution X”, I would first ask, “Why me?” What gifts do I have to offer the world, and what kind of platform would allow me to answer the first question in the most meaningful way possible?
Shyam, I have read all of your Substack posts, and commented on most. This is my last article. It's ironic as my motivation to go down this rabbit hole is because I want to make a difference. I became aware of, and fascinated by Palantir a year ago. I believe the tools you offer provide an objective vehicle of change that can be used by large organizations. They can, for the first time in history, systemically iterate towards providing more for less, ultimately to the point of being valuable. $1 In, more than $1 Out. My expertise is the ability to read organizations, finding and unlocking their behavioral constraints. These constraints are based in emotion. They are irrational. Simply put, it is fear. The fear is rooted in different aspects. Analyzing and alleviating these fear blockers is just as important to maintain the momentum of the journey as are understanding and fixing the rational bottlenecks. It is more art than science. I would like to help Palantir solve this half of the puzzle. My service to the Crown is therefore Pledged.