Imagine a modern world without the smartphone. That’s a tough one.
In the mid 2010s, many of us wondered, “What could potentially be next that could have as big of an impact of the iPhone?” Fast forward to the early 2020s, and we all had our answer clear as day: AI LLMs.
Now try to imagine our world without AI. That’s even harder to picture.
Yet, as much as AI has taken over so much of our world by the mid 2020s, it seems to most that it’s primarily impacting software. Hardware, on the other hand, seems to be unscathed. But is this really accurate?
The reality is that AI isn’t just changing software, it’s actually disrupting the global market for physical hardware, as well.
And it all really comes down to RAM.
Data centers and servers can’t function without RAM and neither can laptops or tablets.
The effects are really showing up everywhere.
For instance, though streaming seems to live primarily in the cloud, RAM is actually a big part of streaming hardware, as well.
Consider two streaming encoders, one has 512 MB of RAM, while the other has 4 GB of RAM. On paper, they both stream video, but in the real world, that difference can determine whether or not your stream survives network disruptions.
But I’m getting ahead of myself, let’s first start with a quick refresher of what RAM actually is and why AI suddenly wants so much of it.
Why is AI Consuming So Much RAM?
RAM in Computers vs Streaming Encoders
How RAM Impacts Real World Stream Ability
Final Thoughts
Before we get too far, let’s simply define RAM. It’s the temporary working memory for a computer.
Though that seems straightforward, let’s unpack a quick analogy to help make it even more clear.
Storage is like a filing cabinet, which has a limited amount of room where a limited number of files can be stored. RAM isn’t storage.
Instead, RAM is more like desk space. It essentially allows more information to be processed at once.
So where storage keeps information for the long term, RAM keeps information immediately available.
RAM isn’t an ethereal, unlimited resource. This term covers both the individual memory chips themselves, or a physical package of these memory chips mounted onto silicon with tiny capacitors and transistors, all tied together with microscopic wiring.
Because of its makeup, it takes complex manufacturing in massive fabrication plants with long lead times to meet the increased production demand, since there are just a handful of companies with this expertise.
So overall, you can’t instantly create more RAM when there’s a demand. If there's more demand than supply, things can get tricky.
Ah, simpler times.
Most computers used RAM in a consistent way. Email, web browsing, software applications for work or school, and then video playback as online video content gradually increased.
So for decades, RAM demand largely rose and fell with personal computer sales, making memory pricing relatively predictable. The typical model had 3 tiers of RAM, a starter version, keeping the device low, a standard version for most users, and then a power user amount for enthusiasts.Overall, a predictable amount of humans made up education and the workforce and demand for RAM followed predictable PC sales.
Computers are typically pretty focused and efficient. If a computer is a refrigerator, when milk is needed, it’s easy to grab the 2%. When it’s ketchup and mustard time, just swing open the door and grab just that and leave the rest of your food in the fridge.
AI is different, because if a context window is a refrigerator and only yogurt is needed, it takes everything out of the fridge in order to get you your Chobani. This is a lot more work, and depending on how much food you have in there, there’s a chance that it hallucinates and hands you kombucha when you asked for a Diet Coke.
With this continual refreshing of entire context windows, even when simple commands are entered, AI processes huge amounts of information, and memory is consumed quickly. Or, put differently, large amounts of RAM are used quickly.
Massive data centers have thousands of servers with AI training clusters, constantly consuming enormous collections of public and licensed data to condense all of its knowledge into the smallest and most practical summary. Each time they want to make a better model, they need to dramatically increase the amount of RAM at hand.
On the other hand, there is RAM pressure on end user devices that use local (downloadable) AI models. Since it is getting expensive to pay ChatGPT and Claude to process bigger and bigger problems that customers want, users are resorting to open models that can be downloaded onto your computer. The trouble is, you need LOTS of RAM to make these run. One can’t even buy any Apple Mac Studio computers with 256GB or 512GB – they are all sold out.
So AI’s use of RAM isn’t just the unseen infrastructure required to create and serve those models – it is also demanded by people looking to leverage AI for personal or professional reasons.
AI companies are buying at scale. Data centers are expanding. There’s a greater and greater demand for RAM.
This means more expensive laptops. More expensive workstations. More expensive servers. And more expensive specialized hardware, like streaming encoders. Put differently, as AI continues to grow, the demand on physical RAM will only increase, and this has downstream implications not only for non AI software, but non AI hardware, as well.
Let’s take a quick look at a few consumer laptops on the market today:
Macbook Pro
Dell XPS
Lenovo ThinkPad
What stands out isn’t merely the amount of RAM. It’s the premium manufacturers charge to get it. And in Lenovo’s case, who knew that buying a laptop could cost the same as the downpayment for a starter home?
At the end of the day, one thing is clear, manufacturers are increasingly charging premiums for memory upgrades.
Unlike software streaming encoders, hardware encoders don’t need to spend their memory on internet activity, email, productivity software, or other background applications. Their resources are focused on one job: reliable video delivery.
However, not all encoders allocate resources the same way.
The comparison between two hardware encoders teased in the introduction was actually a real world comparison, as seen below.
Blackmagic Streaming Encoder 4K →
So looking at these encoders at first glance, they both stream 4K60, and one is cheaper. However, that’s not the full picture. BoxCast's Spark has local storage available via SD Card, but more importantly, it has ~8x the amount of RAM.
More RAM costs money. More storage capability costs more money. Additional hardware resources, like Spark’s large touchscreen interface, increase manufacturing costs.
And as we’re about to see, this makes a huge deal when it comes to live streaming at a professional level.
Live streaming requires a lot of active memory in order to accomplish its core task. During a live stream, RAM is the high speed workspace that impacts stream stability and latency.
First, it holds raw video frames temporarily before they’re compressed. Then it aligns audio packets with corresponding video frames in real time. It additionally stores operational data for encoding algorithms like HEVC or AV1. It holds past and future video frames simultaneously so the encoder can calculate motion vectors and compress the data efficiently. When bandwidth fluctuates, it caches packets in memory to maintain a smooth stream. The list goes on, and on.
As you can imagine, this list isn’t exhaustive as not every encoder uses RAM the same way.
What’s clear though is that if you run out of RAM during a live stream because your encoder doesn’t have enough, your stream quality will almost certainly suffer.
Since encoders like the Blackmagic Streaming Encoder 4K are light on RAM, they can end up showing a “Cache Full” error message. This means that as the encoder is trying to stream at a set resolution and frame rate, it can’t keep up, and the device runs out of room to temporarily hold data.
This leads to packets that are lost and eventually can’t be recovered. From a less technical standpoint, this means a cruddy viewer experience who thought they were getting a picture perfect 4K live stream of the important event they’ve been looking forward to for weeks.
RAM and storage serve different purposes, but together they create more opportunities for a streaming system to recover from temporary network disruptions.
More RAM, however, changes the equation.
More temporary storage means more space to recover, and in effect, greater resilience against network challenges. Even further, additional RAM allows custom streaming protocols like BoxCast Flow to enact advanced functions when needed during a broadcast.
For instance, more RAM allows BoxCast Flow to offer configurable latency between 30 and 120 seconds, depending on what the broadcaster chooses, which can allow for larger recovery windows when packets are lost, helping restore missing data and maintain stream quality even when network conditions deteriorate.
The advantage isn’t merely, “4 GB is bigger than 512 MB.” It’s that Spark and its custom protocol BoxCast Flow are able to use the larger table space of more RAM to help ensure a higher quality broadcast, even when the network starts acting up.
AI has fundamentally changed the value of memory.
What was once an inexpensive component has become one of the most important resources in modern computing. From massive AI data centers to the laptop sitting on your desk, RAM is increasingly determining both performance and cost.
The same principle applies to live streaming.
It's easy to compare encoders by price, resolution support, or feature lists. But memory plays a critical role behind the scenes. The amount of RAM available, how it's paired with storage, and how a streaming protocol uses those resources can all affect whether a stream survives real world network disruptions.
As AI continues driving demand for memory, organizations will likely face tougher decisions about how much RAM they're willing to pay for. In live streaming, though, the goal isn't simply buying the most RAM. It's choosing hardware and software designed to use memory intelligently.
That's ultimately why we built Spark and BoxCast Flow the way we did. Not to win a specification sheet comparison, but to help organizations deliver more reliable broadcasts when conditions aren't perfect.
Learn more about Spark and how BoxCast Flow helps organizations deliver more reliable live streams.