Software dies in darkness
In an era of yapping, code proliferates, shortcuts abound, and innovation takes a back seat. I want to play a different game.
This is my personal blog, by the way. This is where I go rogue. At bem’s blog, I’m Sandy; in here, I get to be Danny. I haven’t written on these pages for months, my focus being entirely on bem, continuing innovation, and evangelizing people on what we do. I’ve also been busy nesting for our baby girl, who’s coming to this world in February. Am I going to be a fun dad? Will the current world, unfair to women, be fair to my daughter? Is it my job to shape the world to make it so? Will I have the willpower to do so?
I digress, as usual. I came here to talk about software.
About 15 years ago, I emailed Hector García, a well-known blogger and software engineer living in Japan, expressing my frustrations about college, assuming he wouldn’t respond. I was in my first year of Computer Science, and I was disillusioned. In hindsight, it was a long email, but the TLDR is that I was disappointed in the lack of enthusiasm for computer science around me, the failure of 100-people lectures without any practical application, and my chief complaint: people are in this career for the wrong reasons.
To my surprise, he responded. His advice: punch through, finish your degree, eat shit, fuck, get scholarships, get out of Spain, travel, meet people, and more and more interesting things will happen to you. The important thing was to feel like your brain was being smashed, suffer, learn, and, after graduating, feel like you could do anything. I followed his advice. I got out of Spain as soon as I could, traveled, incurred debt to move to the US, and shot way above my horizon line. In the most masochistic way, I always prioritized the long-term investment of learning, rejecting every offer that came my way in the direction of a smooth toll highway and choosing the winding mountain path every single time.
Well, it’s been a while, and again, as a true masochist, it has never been easy. I founded the largest company I’ve worked for, and I’ve always started companies or worked for startups; the sooner, the better. I’ve joined companies with no name or strategy, just a glimpse of a vision, and I’ve ignored all the red flags when building partnerships, all in the spirit of learning. I’ve put my personal life on hold multiple times to grow professionally — my only regret. I’ve had terrible and not-so-terrible bosses, eaten mountains of shit, and have been told no continuously throughout my career.
Every time I’ve been told no, I’ve learned more, gotten more resilient, become more aware of my surroundings, learned more to value talent, learned about the fickle nature of adulation and loyalty, and cared less and less about what anyone has to say. Throughout my career, I hit a point of over-self-awareness, of impostor syndrome, that transformed into the ultimate realization: very few people actually know what they’re doing, and the game people are playing around me is to pretend they do. With the advent of “building in public”, this has become pervasive throughout the software industry. I’m not playing that game. I’ll sure get a lot of things wrong, but I won’t talk about what I don’t know about.
Well, you don’t know what software is, and it sure isn’t code, buddy
That’s a long introduction. I said I would talk about software, and I wouldn’t want you to feel like I’m goofing you around, so let’s dive into it.
Software isn’t code. In fact, I’ll do you one better. Every great bit of progress in software has been in the direction of depending less and less on code. Software is, after all, packaged computing abstractions reflecting business logic. That business logic is human in essence. Software is a way for computers to do what a human can do, just much, much faster and more accurately.
Code is just a communication abstraction. It serves two goals: for humans to communicate with each other about desired functionality, and for humans to communicate to a computer what it should do based on certain parameters. The first programming language, Lovelace and Babbage’s Analytical engine, was along the purest forms of communication we’ve built. IBM’s 12/80 punched card was a more mechanically compatible version of human-to-computer communication.
The creation of new abstractions has marked the last few decades. Abstractions are the reason we can move faster when building software. Abstractions are packaged pieces of computing and business logic. Whether there’s human readable code behind those abstractions or not is irrelevant. Good abstractions should be trusted and relied upon as black boxes; otherwise, they’re what we call “leaky” abstractions. Hardware and mechanical systems also have abstractions and good mechanical systems have modular components that communicate with each other as functions. As a frontend engineer, I can tell you most frontend frameworks are plagued with leaky abstractions that are poorly designed. I’m a big fan of abstractions, but you can end up in dependency hell — most systems today have too many dependencies, and their “stack” is too complicated.
Code screams in horror, with 2 conclusions
When I was building Silo, I started to notice something. It was by far the most complex system I had built, and as our system’s complexity grew, our team took longer and longer to build new features. I convinced myself that was philosophically impossible. After all, we were a great team of engineers and had progressively built abstractions to sustain our growth and further product development.
That wasn’t the problem. The problem is that humans aren’t perfect, and code inherently sucks. Big codebases become towers of babel, where teams start talking past each other in different languages to get anything done. Ensuring good communication between teams requires a herculean effort at architecture design, good software practices, and a resilient network of abstractions. I came to 2 conclusions that I continue to preach wherever I can:
We have to simplify tech stacks. Every few years, as a community, we should revisit our assumptions when building systems and prune intermediate layers. Not doing so is why companies have to literally stop their growth from fixing “tech debt.” Stripe produced a report 6 years ago called “The Developer Coefficient” that values global tech debt at $3 trillion. I’d be surprised if it weren’t double by today.
We have to write less code, and build reusable abstractions to do so. This is probably the most important conclusion I want to come across in this terrible contraption of a blog post you’re reading. Software has nothing to do with code and everything to do with the packaging of business logic. Our goal as a community of builders should be to write less and less code, instead building a universal collection of modular components that speak to each other, communicate as pure functions, and abstract raw computing power.
What is the outcome of this for users and businesses? Systems that fail continuously, are hard to use, and force thousands of people to learn and re-learn obscure ways of achieving very simple tasks.
Is AI being used the right way?
No. In fact, most AI companies are making things worse.
Simplifying tech stacks. Instead of changing the workflow paradigm, most companies today are building patches on top of flawed processes and stacks. The biggest culprit here is RPA. RPA is fundamentally the most flawed paradigm we’ve designed in the last 10 years. To top it off, AI companies are now creating a version of RPA+ they call “agents” that can browse the web and read PDFs. Instead of changing the access pattern to the underlying business logic and the abstractions, we’re creating parodies of humans, anthropomorphized processes that try to emulate humans doing their work, when the goal should be to allow the AI to run these processes as true abstractions that aren’t bound by the “human-esque” way of thinking.
Writing less code? Nope. Codegen is the hottest thing in the market and the usual talking heads in social media are selling us a future where no technical ability is required to code. “Code proliferation” is the name of the game. Codegen companies compete with each other in social media posts about the number of lines of code they’ve generated for their users that week. 1 million. 2 million. The big problem? That code must now be maintained, tested, and validated against requirements. I’ve yet to see true efficiency gains coming from codegen, aside from a marginal improvement in coding workflow. (This is especially painful to me, because I joined Kite as the first engineer years ago, where we built the first AI codegen tool ever).
Just because we’ve built models that write great code, doesn’t mean we use them for that purpose.
OK great then, so what do you propose?
Along the lines of the 2 conclusions above, I’m only excited about platforms that move in the following direction:
Using AI to simplify tech stacks, not bloat them. Specifically, platforms that demystify software stacks that have gotten too fat, like ETL/ELT/Datalakes and Business process/BPM/Mining.
Using AI to create abstractions. Companies and platforms whose goal is to allow their customers to delete code so their engineers can focus on designing systems and architectures, not on writing prompts to generate code no one will ever read or know how to debug. I’m especially interested in abstractions that help engineers build better software for their users, with the outcome being software that has great texture and UX, and speaks the users’ language.
I truly believe teams that build in this direction are poised to become the new generation of great computing companies to shape the next generation of software, business, and human-computer interaction.
This defines the what and why of bem. Our current wave of AI technology is one of the greatest technological gifts we’ve received, a once-in-100-year opportunity. I plan to squeeze every ounce of value out of it.
This defines my professional frustrations, passion, and learnings from the last 15 years. I plan to continue taking the mountain path, like the software masochist I am, and hopefully, in my next post, I can be just as crass but come bearing more learnings.