Partnership Advantages

We are centering our models to be optimized and compressed with the OpenVINO toolkit. This gives our private LLM deployments several significant advantages:

• Seamless optimization of popular training frameworks
• Accelerated inference
• On-premise deployment
• Scalability - CPUs are much more widely available
• Lower running costs - CPU hardware architectures are 4x more efficient in terms of budget over GPU architectures
• Faster project start and ability to experiment

Our Interplay platform takes advantage of high-performing Intel processors by leveraging Intel Extension for PyTorch for Interplay Drive-Thru automation with full speech recognition, trainable menu knowledge, and a fine-tuned configured LLM (Llama-2-7b-chat). This cutting-edge configuration operates locally on an edge instance of Intel Xeon Sapphire Rapid ensuring real-time response rates, accuracy and customer responsiveness.

Interplay Benefits

Interplay, our low-code AI Application Platform, leverages OpenVINO in several capabilities:

• Interplay Drive-Thru: Full speech recognition, trainable menu knowledge, and LLM configuration for an automated and responsive Drive-Thru experience.
• License Plate Recognition: Utilizes Intel OpenVINO optimized PyTorch and YOLOv8 for video processing and detection, setting a new standard in security.
• Edge Deployments: The organization plans to deploy its services through edge capabilities, ensuring enhanced performance and collaboration opportunities for customers.
• Affordability: In our financial analysis, removing the requirement for a GPU in the edge server can scale the budget and overall running cost efficiency by 4x.

As Iterate continues to innovate and integrate cutting-edge technologies, our commitment to providing intelligent solutions for diverse applications remains unwavering. The convergence of OpenVINO toolkit support and powerful Intel hardware positions Iterate at the forefront of AI and edge computing, delivering unparalleled performance and efficiency to its users.

Deployment Profile

Deployment Profile
Target Application	LLM Manager - Speech Recognition LLMs
Scaled Deployments	4000+ instances on Intel i7 and i9
Target Customer	QSR, Banks, Convenience, Drive-thrus
Target CPU	Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
Base Requirement	Minimum Quad Core

CPU Usage Improvements

Total Request Count (5 requests in parallel)	CPU Usage (%)
	Current (before OpenVINO)	New (after OpenVINO)
10	84.1	75.0
20	134.2	81.0
30	232.0	89.0
40	294.0	96.0
50	253.0	91.0
60	369.3	103.0