Posted on
“Looking at the waves scudding outwards and getting lost on the horizon, he could not help but recall the words of his mentor, the Danish physicist Niels Bohr, who had once told him that a part of eternity lies in reach of those capable of staring, unblinking, at the sea’s deranging expanses.”
― Benjamín Labatut, When We Cease to Understand the World.
When my parents came up to Seattle to visit me this past summer, we paid a visit to the Boeing Factory in Everett, just a short drive away from the Seattle-Tacoma (SeaTac) airport. In the industrial heart of Everett, Washington, just miles from the city of Seattle, the sprawling expanse of the Boeing Factory stands as a testament to human engineering. The Boeing Factory in Everett, Washington, is the largest building in the world by volume, spanning an incredible 472 million cubic feet. Opened in 1967, the mammoth structure holds the distinction of housing the production of some of the most iconic commercial aircraft ever built, including the 747, 767, 777, and the 787 Dreamliner. The sheer scale of the factory is awe-inspiring, with doors tall enough to accommodate the world's largest wide-body airliners and a roof that spans over 100 acres. Inside, the factory is a beehive of activity, with assembly lines, cranes, and scaffolding occupying a large portion of the floor space. Despite its industrial design, the factory has a sense of order and precision, with Boeing's teams working meticulously to build and test the thousands of components that go into each airplane.
The sheer enormity of the structure could strike anyone with a sense of profound awe. Beyond the colossal factory doors, a bustling symphony of creation unfolds. Massive airplanes, each at varying stages of assembly, stretch out across the enormous factory floor. Amidst the hum of activity, there's a meticulous dance between workers and machines, a testament to the art of manufacturing. The sight of these gigantic planes, soon to slice through the clouds and conquer the skies, stirs a deep sense of wonder. Witnessing their creation from raw materials to airborne marvels left a mark on me when I was in elementary school, a realization of the boundless potential of human productivity at scale: functionalism.
The Concorde is a similar mechanism of progress within aviation but it represents something that transcends functionalism. Built by the British and French, the Concorde was more than just a sleek, fast airplane that could fly across the Atlantic in just a few hours; it was a symbol of a new era. At a time when the world was still healing from war, this airplane stood for progress and the thrill of what was to come.
The Concorde was different from other big advancements of its time, like the practical and efficient Boeing factory. While Boeing focused on making things that were useful and worked well, the Concorde was about dreaming big. It wasn't just about getting from one place to another quickly; it was about pushing the limits of what we thought was possible. Flying on the Concorde was like stepping into a science fiction story – it made people feel like they were part of a bright, exciting future.
Functionalism and Idealism
Concorde showed that we humans could achieve incredible things, like making a machine that could cross the sky faster than sound. It persists as a reminder of the power of dreams and the excitement of stepping into the unknown. The clash between functional progress and dreaming big is an optimization problem with no obvious solution. Since the turn of the millennium, the world has witnessed remarkable strides in reducing global hunger, child mortality, and extreme poverty, despite significant challenges like the pandemic. This progress, reminiscent of the principles of global capitalism, stands as a testament to human advancement unparalleled in history. Yet, despite these achievements, there remains a pervasive discomfort, a sense of unease that fuels populist and authoritarian movements on both the left and right. This dichotomy in human nature, between openness and tribalism, is at the core of our collective unease with progress. It's a tension between our nature as cooperative traders, willing to share and innovate for mutual benefit, and our instinct as tribalists, inclined to divide and view the world as a zero-sum game.This duality in human nature can be analogized to the concepts of functionalism versus idealism, and to the mathematical notions of differential equations and gradients in the context of human progress. On one hand, our 'trader' aspect, driven by cooperation, language, and an advanced brain, represents functionalism - the practical, tangible advancements we make by sharing knowledge and skills, much like the global improvements in health and poverty. On the other hand, our 'tribalist' instinct echoes idealism - the pursuit of an identity or a cause, often at the expense of pragmatic benefits. This is akin to the idealism in projects like the Concorde, where the thrill and symbolism sometimes overshadow practicality. Concorde notably failed due to its extreme environmental inefficiency and sound concerns, quite literally shattering windows near JFK airport as it took off. In mathematical terms, these aspects of our nature are like the variables and gradients in differential equations, representing the varying rates and directions of human progress. Our challenge, then, lies in balancing these aspects - leveraging our inherent drive for cooperation and innovation while managing our instrinsic creative tendencies, to ensure that the trajectory of human progress continues upward, both practically and ideally.
I’d argue that our economy and rate of global progress can be modeled as a system of differential equations. Various environments and markets have initial conditions which cannot be tampered with- the same way a toddler prefers vanilla to chocolate ice cream. Gradients are the drivers of change across our ecosystems. The usage curves and FLOPs (think raw production and output) consumed across technology companies enables growth in software as we know it. Gradients - rates of change - are thus essential in predicting the future state of any system.
Solving these equations can generally be approached through analytic (including closed-form) and numerical (approximate) solutions. Analytic solutions, which include the precise closed-form solutions expressible in terms of elementary functions, provide a complete and exact understanding of the system. However, such solutions are not always available for complex differential equations. In those cases, we rely on approximate solutions, which, though not perfect, offer a practical insight into the system's behavior. These methods serve as a navigational tool through the intricate landscapes of mathematical models. Regardless of the approach—analytic or approximate—the pursuit of understanding these equations reflects the ever-accelerating gradients of human progress in the mathematical and scientific domains. The recent release of the Apple Vision Pro has begun to blend reality and digital worlds in ways we couldn't have fathomed just a decade ago. The first neuralink chip has been successfully implanted in a human and BCI-enabled applications are starting to show results. OpenAI's chatGPT shook our world with the initial semblances of a universal assistant that captures human knowledge with complex neural reasoning. Big data processing hardware and compute resources are expanding at an unprecedented scale (see NVIDIA). Human existence is fundamentally predicated on mortality and a lack of time. If you were to read one book a week for an entire year, that would equate to 50 books read for that year. Assuming a typical human lifespan, that translates to between 3,500 and 4,000 books read across a lifetime. Most libraries carry 100,000s of volumes of books alone. In earlier times, we used to absorb knowledge from Encyclopedias, Dictionaries, and Atlases. Now, large language models have brought libraries to our fingerprints. Ask a query to ChatGPT and you tap into a body of knowledge that you could never physically traverse nor intellectually capture in your lifetime. Mixture-of-Experts (MoE) models have turned entire disciplines into segments within a multi-billion parameter model that essentially reflects an entire worldview and field of understanding.
Empires come and go but a drive for technological progress and stability has persisted due to the need for the means to be creative: this is satisfied via convenience, productivity, and functionality. For young people like myself, I cannot imagine what the world will look like in 2 years, 5 years, 10 years, and decades from now. It's also hard to predict what will stay the same. What does appear imminent is the simultaneous convergence of advances across hardware (physical devices and neural interfaces) and software (foundation models and search algorithms). It's the best time to build ever: with AI assistants, you can fire up JS templates with the click of a button and learn anything in your field of interest nearly instantly. In short, the euclidean distance to knowledge has been the lowest ever in human history. The whitespace between digital reality and our humanity is closing and it's up to us to invent ways for technology to better our lives. There could soon be a day where you genuinely forget what's real. The question essentially becomes: do we build a Concorde or do we build a factory? While functionalism demands brute force effort, idealism requires strong initial conditions: large amounts of capital, committed long-term partners, and most crucially: a leap into the unknown. In software, we could rephrase with the following modern example: do we build a b2b SaaS product to serve data at a fractionally incremental better cost or do we unleash foundation models into the wild in form factors never seen before. Regardless of the economic tailwinds, it’s not possible to really decipher the exact economic value produced but from general intuition, it’s less risky to go the route of incremental improvement compared to idealistic discovery. I’d like to distill the points alongside two famous heuristics: optimal stopping and exploration versus exploitation.
The optimal stopping problem, or the "secretary problem" in statistics, teaches us about the balance between exploring options and making a timely decision. In this problem, you aim to select the best candidate by initially evaluating a set of options without commitment, and then choosing the first candidate who surpasses all previous ones. This concept is an adjacent metaphor for the balance between functionalism and idealism, especially in a rapidly changing world. Functionally, it emphasizes the importance of making practical decisions at the right time, rather than endlessly seeking the perfect choice, an approach more aligned with idealism. As the future unfolds quickly, marked by technological and societal shifts, the lesson is clear: exploration is valuable, but there comes a crucial point where one must act decisively to seize opportunities and navigate the evolving landscape.
Similarly in RL, an agent learns by exploring its environment and exploiting the knowledge it gains to make better decisions. Exploration involves trying new actions to discover their effects, which is crucial for learning about unknown aspects of the environment. Exploitation, on the other hand, involves using the knowledge already gained to make the best possible decisions. The challenge is balancing these two: too much exploration can lead to inefficiency, while too much exploitation can prevent the agent from finding potentially better strategies. At least with the optimal stopping problem, we find that there is a scalable optimal solution- 1/e or ~37%.
I’d argue that we’re approaching a time where it’s time to be creative and chase idealism. The economic gradients of various industries are such that there are new problems to solve that most of us cannot yet see. Rather than the argument of envisioning Uber just as the iphone was released (a vertical build-up), it’s that problem spaces and domains are overlapping (merging horizontal curves). Search and language modeling are beginning to become interlinked as we see with Perplexity, Glean, and Hebbia. Physical GPU provisioning and software infrastructure are also beginning to become aligned as we see with Modal, Foundry, Lambda, Together, and others. Computer vision and modern visual computing are beginning to become integrated with AR/VR form factors.
With the entropy and randomness that’s configured with humanity, you’ll never be able to find precise analytic estimates for the future. Yet, those who can approximate them with clever “tricks” and think ahead to see where the various curves of growth meet can have unbounded success. Just as a physicist discovers new theorems, we as engineers and builders must also think “outside the box” and dream big. Whether it be applying symmetry or scaling an existing hypothesis, I suggest that idealism and a creative aspiration for the unknown stay at our foundations. Perhaps, this is how one avoids overfitting in the general sense. My suggestion: life's short so work with great people to chase global mountains, not local hills. And just start honestly- that's usually the hardest step.