Key takeaways
- AI PCs require significant memory and storage for local processing.
- AI features like Microsoft's Copilot require newer computers with specialized NPUs.
- 16GB of RAM is the minimum for AI PCs, with 32GB recommended for those who care about AI features.
The age of the “AI PC” is upon us, and you'll soon have the option to run some AI features natively without an Internet connection, even on relatively modest systems. However, what you may not realize is that even with AI accelerators, these computers will need heaps of memory and storage just to offer native AI features.
AI PCs have local processing power.
Microsoft's Copilot AI feature is one of the main reasons why a new generation of computers with specialized NPUs (neural processing units) is hitting the market. This is one of the reasons why the new Surface Pro devices have the same ARM Snapdragon processors normally found in phones. Currently, the minimum requirement for a Copilot AI PC is an NPU with 40 tops (trillions of operations per second) and that's what these new chips pack in.
There are many reasons for wanting native AI features. It's fast, you're not competing for resources with other users, and you don't have to pay more than the hardware costs. Giving your laptop the smarts of something like ChatGPT and its contemporaries is certainly exciting!
Even “smaller” LL.M.'s are plentiful.
The thing is, these AI “models” are pretty big. The smallest and most efficient LLMs (large language models) may require 1-3GB of memory, while not being that smart. As the model acquires parameters and becomes more capable, it needs more capacity. The super smart LLMs that you access through the web can be hundreds of gigabytes of RAM when it comes to their model size. With hundreds of billions, or in some cases over a trillion parameters, these AI models are not meant to be run on tiny laptops!
Then we have LLMs small enough to run on a smartphone, like the phi-3-mini which runs at 2.4GB. This is in line with what is actually running on these AI laptops, although perhaps not quite as anemic!
Models have to live in RAM.
AI chip hologram with surrounding circuits
While a few GBs of space in an AI model may not seem like a big deal, you have to understand that the entire model must be in RAM for the technology to work efficiently. Those NPUs are specialized parallel processors, which crunch the numbers on these virtual neural networks. with billions of “parameters” at once.
If the model isn't in RAM, the NPU can't be fed fast enough to get you results fast enough. This will affect the speed at which LLM responds, or performs tasks such as real-time translation, or image generation. So if you want AI features to be available at the push of a button or just by talking to your computer, the model basically needs to reserve as much RAM as it needs.
To be sure, these new PCs have super-fast SSDs that can quickly swap data in and out of RAM, but aside from the extra wear and tear on your SSD, it's still so Not fast enough to run everything smoothly, and don't forget you still want to run all your other apps at the same time! It won't be a great experience if your computer has to dump your browser and other apps from RAM to disk every time you want to do something with AI.
16GB is the absolute minimum.
While 8GB of RAM is still fine for a modern office desktop or laptop used for general productivity and media consumption, Windows computers with AI features won't get by with so little space. This is why Microsoft has made 16GB of RAM the absolute minimum for Copilot PCs.
While this is a healthy amount for a typical non-AIPC, I have a feeling that for a PC running a native AI assistant in the background all the time, it should be more like 8GBPC plus AI. It will, unless you disable the AI ​​part. Equality
Some AI PCs will not have upgradeable RAM.
If your new AI Windows PC uses an integrated system chip or has permanently soldered RAM, as is the case with many thin-and-light laptops, you'll need as much RAM as the system has. Ram will come and get stuck in it. With that means you need to get the RAM you need right up front, or if it turns out you misplaced your money you'll need a whole new computer.
Some time ago, I looked into whether 32 GB of RAM wasn't the amount it was meant to be, and my conclusion was that this amount of memory is still too much for the average user. However, with future AI model sizes in mind, I think 32GB should be what anyone who cares about these AI features should be aiming for.
If you care about AI, get more memory.
Whether you care about native AI features is, of course, what matters here. If you don't, it's great that these Copilot PCs are expanding the RAM to at least 16GB. Just disable or ignore the AI ​​part of the equation and you'll be just fine. In fact, unless you have a compelling alternative reason, skip these AI PCs altogether and stick with a more mainstream option.
If, on the other hand, you're excited about native AI features and see many ways to use these tools in the future, don't meet the minimum requirement until you have more to offer later. Don't have the option to add RAM.