Benchmarking Mistral 7B Inference performance on GPUs

Overview This case study examines the inference performance of the Mistral 7B model, a large language model with 7.3 billion parameters, to assess its viability for production-ready Generative AI (GenAI) solutions. As many organisations struggle to move from pilot projects to full-scale deployments due to the high costs associated with GPU-based inference, the study benchmarks […]

Enhancing LLM inference performance on Intel CPUs

Overview Earlier research from OpenAI, particularly the Kaplan scaling laws, indicated that increasing the size of model parameters generally leads to improved accuracy and reasoning capabilities of LLMs. As model parameters grow larger, the requirements for computational parallelization and high-bandwidth memory for production-ready inference also increase substantially. This trend poses a significant challenge to traditional […]

Benchmarking the Indus Language Model on Intel® AI Hardware

Benchmarking the Indus Language Model on Intel® AI Hardware

Overview Bud Ecosystem, Intel, and Tech Mahindra collaborated on benchmarking Project Indus, an innovative open-source language model designed specifically for Hindi and its dialects. Focusing on applications within the Indian linguistic landscape, Project Indus aims to enhance natural language generation and processing capabilities. The benchmarking study emphasises key performance metrics such as Time to First […]

Driving Enterprise RAG Innovation with Intel® Xeon® Processors

The chatbot solution has demonstrated superior performance compared to leading cloud-based ...

Case studies

Benchmarking the Indus Language Model on Intel® AI Hardware

Benchmarking Mistral 7B Inference performance on GPUs

Enhancing LLM inference performance on Intel CPUs

Benchmarking the Indus Language Model on Intel® AI Hardware

Driving Enterprise RAG Innovation with Intel® Xeon® Processors

Company

Product

Resources