New Microservices from

NVIDIA Strengthen Sovereign AI

 

NVIDIA microservices empower sovereign AI with regional language models

 

 

 

Nations worldwide are increasingly adopting sovereign AI policies. These policies focus on building AI systems using local infrastructure, data, and expertise. The goal is to ensure AI respects local laws, values, and cultural norms. To support this, NVIDIA has launched four new NVIDIA NIM microservices.

Enabling Locally Customised AI Models

NVIDIA’s microservices streamline the development and deployment of generative AI applications. They enhance understanding of regional languages and cultural nuances. As a result, users experience more accurate, relevant, and engaging AI responses tailored to their local context.

This move aligns with the rapid growth of the Asia-Pacific generative AI market. According to ABI Research, the market revenue is expected to soar from $5 billion this year to $48 billion by 2030.

Regional Language Models for Japan and Taiwan

Among the new NVIDIA offerings are two regional language models: Llama-3-Swallow-70B, trained on Japanese data, and Llama-3-Taiwan-70B, optimised for Mandarin. These models understand local laws, regulations, and cultural details more deeply than generic models.

Additionally, NVIDIA introduced the RakutenAI 7B model family to strengthen Japanese language support. These models come as two separate microservices for Chat and Instruct functions. They built on Mistral-7B and were trained on both English and Japanese datasets.

Rakuten’s models have shown remarkable results among open Japanese large language models. They topped the LM Evaluation Harness benchmark from January to March 2024, earning the highest average score.

Importance of Regional Training for Better AI Output

Training large language models (LLMs) on regional languages improves output quality. These models reflect linguistic and cultural subtleties, enabling more nuanced and precise communication.

For instance, NVIDIA’s regional models outperform base models like Llama 3 in understanding Japanese and Mandarin. They excel at tasks such as legal compliance, question answering, translation, and summarisation.

Global Push for Autonomous AI Infrastructure

Countries such as Singapore, the United Arab Emirates, South Korea, Sweden, France, Italy, and India are investing heavily in sovereign AI infrastructure. This global momentum highlights the growing importance of autonomous AI development.

Rio Yokota, a professor at Tokyo Institute of Technology, explains, “LLMs are intellectual tools that interact with human culture and creativity. Not only do models reflect our data and culture, but they also influence the data we generate. Thus, developing sovereign AI models that respect cultural norms is crucial.”

He adds, “Offering Llama-3-Swallow as an NVIDIA NIM microservice allows developers to easily deploy Japanese applications across industries.”

Benefits of NVIDIA’s NIM Microservices

NVIDIA’s microservices enable businesses, governments, and universities to host native large language models in-house. This improves data control and security while fostering innovation.

Developers can create advanced copilots, chatbots, and AI assistants tailored to local needs. Available via NVIDIA AI Enterprise, these microservices leverage the open-source NVIDIA TensorRT-LLM library. This setup enhances inference performance and speeds up deployment.

Performance gains are clear with the Llama 3 70B microservices, the base for the new Llama-3-Swallow-70B and Llama-3-Taiwan-70B models. They deliver up to five times higher throughput, which reduces operational costs and minimises latency, ultimately enhancing user experiences.