Intel has been demoing its upcoming Meteor Lake CPU at the Computex show (via PC World) and the big noise with the new chip is AI acceleration. More specifically, Meteor Lake comes with a new VPU or “Versatile Processing Unit” which apparently is powerful enough to run Stable Diffusion locally rather than in the cloud.
The VPU is essentially a dedicated inferencing accelerator included in Meteor Lake as a separate tile and designed for AI tasks like computer vision and deep learning.
AI is obviously experiencing explosive growth right now, but the likes of ChatGPT, Midjourney, and DALL-E 2 have thus far run in the cloud courtesy of huge training models. It’s hard to see any relevance of a tiny client-based hardware AI accelerator to that, right?
In fact, Intel does see a role for some level of local AI acceleration on PCs. Intel says that the huge costs of rolling out AI apps like ChatGPT to millions upon millions can be reduced by putting some of that processing load on client PCs.
Intel also reckons that local AI acceleration allows for massive distributed scaling, better privacy for users by keeping data local and lower latency because the AI processing is actually happening on your device.
Notionally, that makes sense. But it’s not clear how it will translate into real world applications. Certainly for now there’s now way to offload any part of ChatGPT’s processing to local devices and we’ve heard of no plans from OpenAI to do anything remotely like that.
However, Intel did show off a version of the Stable Diffusion image generator running entirely locally on a Meteor Lake powered laptop that specifically wasn’t connected to the internet. Traditionally you’d want to use a powerful GPU to run Stable Diffusion locally, with the image generator needing a whole lot of silicon to render an image quickly.
A prompt for an “astronaut riding a horse on the moon” was inputted and sure enough, the system spat out an image of just that simply using the chip inside a Meteor Lake laptop.
Exactly how that was being done and what fraction of a larger Stable Diffusion model was running locally isn’t clear, and the demo really raises more questions than it answers, though the increased load on the VPU in Task Manager could be seen.
It’s intriguing to note that the VPU is branded “Intel Movidius VPU” in task manager. Movidius is a chip maker Intel acquired in 2016 that specialises in computer vision and thus far Intel’s AI chip does seem to be very much skewed in that direction.
That Stable Diffusion demo aside, Intel says that the VPU can be used for real-time background replacement and blurring in video calls and noise reduction, tasks that currently run on CPU and GPU cores. Using a dedicated AI engine will do that work much more efficiently. There seems to be some implication from Intel that Meteor Lake’s VPU could accelerate some aspects of Microsoft’s new Co-Pilot feature, too.
One further question that arises from all this is how Intel’s VPU compares to the new AI engine in AMD’s Phoenix APU, which is now appearing in laptops and indeed handheld gaming devices like the Asus RoG Ally, and whether the two AI engines will be software compatible. It’s all very new, and as I say, we have questions.
Intel also confirmed that Meteor Lake will be laptop-only at first but it wasn’t ready to talk about specific CPU models or precise launch dates. We have been expecting Meteor Lake to launch later in 2023, but at this stage it’ll be very late in the year if it does meet that deadline.