Content area
GPUs are at the center of AI's rise to prominence and have seen massive improvements in the past years. Though less in the spotlight, CPUs remain an important player in AI pipelines; data processing tasks that run on the CPU are a necessary part of every inference call. As AI workloads are shifting towards incorporating more modalities and growing in complexity, heavy vision data processing tasks are increasingly common. This work explores various vision AI pipelines and their CPU overheads, specifically CPU time and utilization, under diverse configurations, image resolutions, request rates and serving systems. We find that the CPU is increasingly oversubscribed for vision-language models (VLMs) and multi-model pipelines, higher resolution images and request rates. Additionally, evaluations on newer hardware point to a widening gap between GPU inference and CPU data processing efficiencies, supporting the conjecture that the CPU may become increasingly congested in the future, impacting service level objectives.