Made Possible by NVIDIA
If you’ve marveled at incredible visual effects in a movie, struggled with your addiction to Instagram, and wished your car could chauffeur you, you’re benefiting from NVIDIA. Much of the technology we rely on and the miracles we dream of rely on computing. How funny your next Instagram reel will be and our ability to measure the effects of climate change are both impacted by fundamental tools: GPUs. And the best of them are made by NVIDIA.
Nearly every Substack post I’ve written could’ve mentioned NVIDIA. Researching this post has truly been fascinating.
In computing today, there are primarily two main types of chips: Central Processing Units (CPUs) and Graphics Processing Units (GPUs). CPUs conduct billions of operations a second, and mainly do so in sequential order, completing one operation before starting the next. Despite needing to complete one task at a time, they’re incredibly efficient and powerful for most things you do on your computer. However, there are certain problems that benefit from parallel computing, specifically those where many operations occur simultaneously. The ability to render intensive graphics for film or gaming by controlling thousands of pixels at once is the kind of scenario in which GPUs come in handy.
A CPU is like a restaurant with a few delivery drivers. They can handle deliveries simultaneously, but they can also be redirected to run some errands. In comparison, a GPU is more like DoorDash: a large system of delivery drivers. They’re all doing one task at a much larger scale. It’s both specialized and really efficient for that singular purpose. So you might be wondering, why don’t we just use GPUs for everything? Well, GPUs are excellent for tasks that don’t depend on each other. Your hungry neighbor’s order isn’t going to add to your wait time—they’re handled in parallel. However, the DoorDash drivers would be horrible if they also had to get your groceries and run errands for you. Tasks that have dependencies, and need to happen sequentially, are much better suited to CPUs. Hence, in most instances, CPUs and GPUs work in hybrid.
In the early 1990s, the founders of NVIDIA, Jensen Huang (also the CEO), Chris Malachowsky, and Curtis Priem, saw the potential of 3D graphics for gaming and film. At the time, making 3D objects render on your 2D screen was an incredibly power-intensive and expensive process, not possible on a PC. You needed specialized graphics computers like SGI workstations. Unfortunately, these cost upwards of $8000 back in 1995 (about $16,500 dollars today). Jensen and his team realized that specialized chips could democratize advanced graphics.
Thanks to graphics cards, the growing population of PC users would be able to afford new experiences on their desktops, and developers would be able to amplify their creativity. By the end of the decade, NVIDIA had become renowned for its industry leading graphics cards. A few years later, we’d all begin to call those cards GPUs, which, by the way, are blisteringly fast today. A few years ago, when the US banned the export of advanced chips to China to stall their AI development, NVIDIA created less powerful versions of their most powerful chips that were legal to export to China. This crippled chip was still the most powerful chip on the market.
So, NVIDIA, a company I dismissed for a while, mainly because I was tired of my roommate yapping about his new GPU, has actually been intertwined with many exciting technological developments in the last two decades. And most of them have absolutely nothing to do with gaming or film.
Thinking back to our analogy of a delivery network like DoorDash, we used to think that such an approach was only essential for handling complex graphics. But it turned out certain math operations in scientific research could also be run in parallel. Sensing an opportunity that many would dismiss for years, NVIDIA built CUDA—a comprehensive platform that included programming language, software development kit, and tools for developers to easily program GPUs for ‘alternative’ purposes. CUDA, the platform, was an approach akin to what Apple does: you have your hardware shipped to millions of customers globally, but it only runs on a proprietary operating system (iOS or macOS), with several tools for developers to easily make their apps and software available to those within their “walled-garden”. NVIDIA’s platform approach and the scientific community’s wide adoption of GPUs linked the company to two pivotal moments in AI:
AlexNet and the rise of deep learning
Teaching computers to identify objects in the real world was the north star challenge in the field of computer vision. In 2012, two PhDs and their advisor essentially solved this challenge with their poorly named algorithm: AlexNet. AlexNet used a deep learning approach where computers mimic the learning patterns of our brain. Their algorithm (AlexNet) was by far the best approach to the problem. NVIDIA GPUs were used to train AlexNet, solidifying the value of GPUs overall and validating the CUDA platform.
Deep learning became the engine to make sense of “Big Data”. It would later be used in the algorithms that made Facebook and Google giants of the tech industry, and generated billions of dollars of revenue for NVIDIA. The advantage of a GPU over CPU becomes apparent when you think of the scale as described by Bryan Catanzaro, the VP for Deep Learning Research at NVIDIA: “Training deep-learning models is very computationally intensive… To train one speech-recognition model takes more math operations than the number of grains of sand on the Earth.”
ChatGPT
OpenAI launched ChatGPT in 2022. It was the fastest product to achieve 100 million users, but perhaps better remembered as the inflection point that made AI finally intelligible to the average person and made large language models (LLMs) seem like our way to finally creating the kind of AI we’d only seen in science fiction. How did OpenAI train this model? They were using NVIDIA GPU-powered systems. Close ties with researchers and a network of developers once again made sure NVIDIA was going to serve as the chassis to an exciting time in AI.
Unhindered by the dominance of their GPUs, NVIDIA built a platform that allowed innovators across the world to tap into a computing revolution.
As we await our AI dreams—whether it’s making self-driving cars a reality, building advanced robots, curing cancer, or creating a digital twin of Earth—our ambitions are constrained by time, money, and computing power. Computing can cost millions in electricity bills. To power your projects efficiently and (relatively) affordably, you buy NVIDIA chips. When someone’s selling you a dramatic future powered by AI, now you know who is making it possible.
Sources
Several Acquired podcast episodes that were the bulk of my research. It will take seven hours to get through them but if you want to learn as much as possible about NVIDIA, I couldn't recommend a better source:
10 years later, deep learning ‘revolution’ rages on, say AI pioneers Hinton, LeCun and Li
About Shubhstack
Almost two years ago, I began Shubhstack: my tiny corner on this platform of talented writers. It all began with the realization that as much as I liked the outcome of my writing, I never wrote when it wasn’t required of me. The lack of routine beyond college made me question my creativity and productivity. So I began writing to a small audience. In my first post, “Why”, I said, “Through my writing, I want you and I to learn the nuances of new developments and then discuss how they could affect us, our financial systems, and in turn the way we think about the world.” I wanted this to be a place where I could structure the things I learned into informative and thought-provoking content.
If you’re not subscribed already, but you like what you’re reading, please hit subscribe. Feel free to reach out if you have any feedback for me as I continue to learn more from this experiment!