CMC develops a large Vietnamese legal language model

CMC OpenAI Company (C-OpenAI) announced the large Vietnamese legal language model CMC-AI-Legal-32B, and introduced the VLegal-Bench evaluation standard set developed by the unit. These are two fundamental components for developing legal virtual assistants to serve Vietnamese users.

CMC-AI-Legal-32B was built and refined in depth about Vietnamese law. When evaluated on VLegal-Bench, the model leads in overall performance, ranking top 1 in 6/22 tasks, such as problems requiring multi-layer legal reasoning and reasoning.

The research team said that, due to contextual training and standards for citing Vietnamese law, CMC-AI-Legal-32B has an advantage over major foreign language models such as GPT-4o, Claude or Gemini when processing and inferring this specialized content.

To train the model, the team built a set of specialized standards for specific legal assessment. This is a task that is difficult for international standards to replace, due to differences in language and legal systems. VLegal-Bench includes 10,450 data samples with standard answers, divided into 22 tasks, designed according to 5 increasing levels of inference. The standard set is designed according to Vietnam’s specific characteristics. Each data sample is tied to a central-level legal document source to ensure verifiability.

Mr. Nguyen Tien Dong, Technical Director of CMC OpenAI, said that building a set of assessment standards is a “difficult problem” because of both technical and legal requirements. According to him, the biggest challenge is ensuring legal correctness, re-verifiability and compatibility with major language model evaluation standards in the world.

“We pursue the development of a large Vietnamese language model and specialized AI for each field,” said C-OpenAI General Director Dang Van Tu. At the same time, the company will publish the source code, data, evaluation process and call on domestic and foreign experts to participate in standardization to make the toolkit better and better.

AI is currently one of 11 technology groups in the Strategic Technology List signed by the Prime Minister in June, with product groups such as large Vietnamese language models, virtual assistants and specialized AI.

At the AI ​​Forum in the Digital Age held in late August, Minister of Science and Technology Nguyen Manh Hung said that AI is gradually becoming a type of national infrastructure, similar to electricity, telecommunications or the Internet. According to him, developing specialized AI applications not only helps solve national problems but also creates conditions for Vietnamese businesses to perfect and enhance technological capacity.

Many other Vietnamese businesses also pursue the development of a large Vietnamese language model, but at a more general level, instead of following a specialized direction. Zalo AI’s large Vietnamese language model currently has 13 billion parameters and is deployed in many practical applications. In September, VNPT proposed that the Government assign the task of developing a large Vietnamese language model, as the foundation for AI applications built by Vietnamese people.

In another direction, ViGen builds an open source Vietnamese data set, to promote AI applications in Vietnam. The project is jointly run by the National Innovation Center (NIC) and many organizations, including AI for Vietnam and Meta.