Case Study: How Seekr is Building Trustworthy LLMs in Intel® Developer Cloud
Intel Developer Cloud enables AI companies like Seekr to build, train, and deploy innovative solutions on a leading-edge, cost-effective AI acceleration platform.
The full version of this blog was first authored and published by Intel.com on April 9th, 2024.
The Brand Safety Challenge in Advertising
Digital content drives marketing strategies and advertising revenue, but ensuring brand safety and suitability is challenging.
Building a trustworthy brand requires a thoughtful content strategy. Content production and media alignment must be brand-safe and suitable for target audiences. Yet, the tremendous volume of digital content and wide variety of channels makes it nearly impossible for companies to ensure that what they distribute consistently upholds brand standards.
Seekr is helping solve this problem for the advertising industry with SeekrAlign, a solution that leverages proprietary scoring and analysis technology to help brands find the most suitable online content and grow their reach responsibly.
Built, trained, and deployed on an 8-node Intel® Gaudi® 2 AI accelerator cluster in the Intel® Developer Cloud, Seekr’s first-to-market AI solution quickly garnered an impressive cross-industry customer list of leading brands. From Moderna and SimpliSafe to Babbel, companies are realizing the value of using AI solutions to solve enterprise challenges.
Evaluating Content and Context with AI
Misinformation, hallucinations, and inappropriate content can quickly erode brand trust. Even inadvertently associating with platforms that host at-risk content can endanger a brand’s reputation. For this reason, many companies closely screen the content they distribute. This practice is time-consuming, expensive, unreliable, and highly subjective.
A primary use case is podcasts. The popularity of podcasts has grown exponentially in recent years, and for many companies, podcasts are an essential part of their content strategy and advertising revenue. Unfortunately, the podcast landscape has been especially difficult for brands to navigate.
SeekrAlign solves this challenge by allowing companies to quickly and accurately screen hundreds of thousands of podcasts.
“SeekrAlign simplifies finding safe and suitable podcasts by marrying world-class engineering with innovative and transparent ratings, allowing brands to navigate with nuance so they can confidently grow their audio campaign with increased clarity and speed.”
–Dan Granger, CEO of podcast advertising agency Oxford Road¹
Align’s user-friendly dashboard rates podcast content on a scale between 0-100. This patented rating system, called the Seekr Civility Score™, gives marketers and advertisers an objective standard to judge content safety and suitability.
“This challenge, at this vast scale, could only have been solved by combining superior computing power with novel signal detection. The Civility Score synthesizes dozens of considerations into a single metric that opens the door to more brand-suitable opportunities and larger audiences.”
–Rob Clark, President & CTO at Seekr²
The Align platform has already scored 20 million minutes of podcasts, a number that is projected to reach up to 40 million by the end of 2024. In addition, the platform has real-time scores for more than 8,000 podcast episodes, totaling more than 10 million minutes of audio.
“We’ve shown that when AI is applied responsibly and transparently, it can solve real business challenges as this technology continues to develop and advance. I look forward to building on this success through continued innovation, including the development of advanced LLMs that can help solve some of the most vexing social issues and drive business growth with incredible speed and accuracy.”
–Rob Clark, President & CTO at Seekr³
Delivering High Performance with Reduced Cloud Compute Costs
Seekr’s sophisticated AI workload requirements demand immense compute capacity. Historically, this compute power would require a significant infrastructure investment and considerable cloud costs. But with the Intel Developer Cloud, Seekr can access leading-edge Intel hardware and software for a fraction of the price.
“This strategic collaboration with Intel allows Seekr to build foundation models at the best price and performance using a super-computer of 1,000s of the latest Intel Gaudi chip, all backed by high bandwidth interconnectivity. Seekr’s trustworthy AI products combined with the ‘AI-first’ Intel Developer Cloud reduces errors and bias, so organizations of all sizes can access reliable LLMs and foundation models to unlock productivity and fuel innovation, running on trusted hardware.”
–Rob Clark, President & CTO at Seekr
Flexible AI Solutions for Trustworthy Outcomes
SeekrFlow, an end-to-end LLM development toolset focused on building principle aligned LLMs using scalable and composable pipelines, is the foundation of this innovative solution. Designed for maximum efficiency and flexibility, SeekrFlow allows developers to train and build LLMs using scalable and composable workflows. The entire LLM pipeline is managed with an API and can be adapted with custom components. Computational graph specifications such as resources and dependencies are flexible, allowing for countless configuration options.
SeekrFlow’s workflow is compiled into a versioned, repeatable, and reproducible executable. It also integrates robust alignment algorithms to ensure the LLM’s behavior complies with prescribed standards, intentions, rules, or values. These LLMs also benefit from the company’s robust inference pipeline, enabling the platform to transcribe, diarize, and score audio at scale. Once optimized, these custom LLMs are piped into the user-friendly Seekr Align platform where they are used to provide fast and accurate evaluations of digital content.
Seekr Deployment in Intel Developer Cloud
Seekr had been using a regional colocation provider to host a server fleet of GPU and CPU-enabled systems. The self-managed compute infrastructure is used for LLM and multimodal model development and supports Seekr Align.
With customer growth and the increasing size of LLM deployment, Seekr sought a cloud services provider to help scale its businesses through superior price and performance. After successfully fine-tuning and inferencing benchmarking LLMs on Intel Gaudi 2, Seekr decided to leverage Intel Developer Cloud as its preferred computing infrastructure for LLM development and production deployment support.
Figures 1-3 show the performance gained through software optimizations and by using Intel Developer Cloud’s environment.
Figure 1: Seekr LLM fine-tuning using the Hugging Face
Optimum Habana and Habana PyTorch APIs. Source: Seekr.
Intel does not control or audit third-party data. You should consult
other sources to evaluate accuracy.
Figure 2: Seekr LLM inference using Hugging Face
Optimum Habana and Habana PyTorch APIs. Source: Seekr.
Intel does not control or audit third-party data. You should consult
other sources to evaluate accuracy.
Figure 3: Seekr LLM inference using Hugging Face
Optimum Habana and Habana PyTorch APIs. Source: Seekr.
Intel does not control or audit third-party data. You should consult
other sources to evaluate accuracy.
Seekr plans a phased deployment to migrate the company’s solution platform into Intel Developer Cloud. It is using various AI-optimized compute instances in our cloud to optimize the deployment cost/performance of application workloads, including:
- Intel Gaudi 2 accelerators: The 7B LLM model that powers the SeekrAlign solution is deployed on a cluster of Intel Gaudi 2 systems to perform sensitive content categorization and risk assessments.
- Intel® Data Center GPU Max Series 1550: Seekr uses Intel® Extensions for PyTorch* for transcription and diarization processing on a cluster of Intel GPUpowered systems.
- Intel® Xeon® Scalable processors: The company uses a cluster of 4th gen Intel® Xeon® CPU-enabled systems for application monitoring, storage management, and vector and inverted-index database services. These support Seekr’s Retrieval Augmented Generation services and SeekrFlow on building principle aligned LLMs using scalable and composable pipelines.
Plans include utilizing 5th gen Intel® Xeon® Scalable processors later this year.
“The Seekr-Intel collaboration unlocks the capabilities of running AI on hybrid compute. Intel GPUs and Intel Gaudi 2 chips are leveraged to train and serve at scale, large- and medium-size transformers. We saw improved performance for model training and inference compared to other chips in the market. Specifically on LLM inference, the large memory capacity of the Intel HPUs and GPUs has allowed us to adjust workload-specific parameters, such as batch sizes. On the software side, the various Intel extensions enabled us to move our ML stack to Intel hardware seamlessly.”
–Stefanos Poulis, PhD., Seekr Chief of AI Research and Development
As part of the following Intel Developer Cloud integration phase, Seekr plans to leverage large-scale Intel Gaudi 2 clusters with Intel® Kubernetes Service to train its LLM foundation model and add CPU and GPU compute capacity to grow its trustworthy AI service offering.
More Information
Seekr – www.seekr.com
Intel® Developer Cloud – cloud.intel.com
Intel® Liftoff – developer.intel.com/liftoff
Intel Gaudi 2 AI accelerator – habana.ai
Intel Enterprise Software Solutions
About Intel® Liftoff for Startups Program
Intel Liftoff is a free virtual program that helps early-stage AI and machine learning startups innovate and scale. Launched in 2022 by Intel, this self-paced program empowers startups to remove code barriers, unleash performance, and transform their ideas into scalable, industry-defining AI companies.