VinUni launches V-Bench - A toolkit to evaluate the capacity of AI models for Vietnamese people

(Dan Tri) - AI Research Center - VinUni University introduces V-Bench - a non-profit toolkit developed to serve the community and promote responsible AI development in Vietnam.

Based on an in-depth data set of more than 40,000 questions and tasks, evaluated by a team of domestic and foreign experts, V-Bench can comprehensively evaluate the Vietnamese capacity of major language models, thereby providing an objective, independent reference system, as a basis for applying AI in practice.

V-Bench - a measure of Vietnamese AI capacity, developed by the AI ​​Research Center - VinUni University (Photo: VinUni).

Initiated by the AI ​​Research Center - VinUni University, with the completely non-profit collaboration of a scientific council of 18 leading Vietnamese experts in the AI ​​industry at home and abroad, V-Bench is one of the most comprehensive tools to measure the Vietnamese language processing ability of large language models (LLM - Large Language Model) today.

Not only based on academic knowledge, V-Bench's data set of more than 40,000 questions and tasks is built in both breadth and depth, covering many fields and fully reflecting elements of culture, language, sovereignty, information security and local characteristics.

V-Bench is designed in 2 large categories with 5 specialized evaluation groups, reflecting AI capabilities in the Vietnamese context (Photo: VinUni).

V-Bench divides the toolkit into 2 large categories with 5 specialized evaluation groups:

Implicit culture - evaluates the ability to understand the layers of cultural meaning hidden deep in the Vietnamese language such as customs, beliefs, and history.

Regional diversity - test your understanding of North - Central - South dialects, ways of addressing and typical style of each region.

Digital security and sovereignty - ensuring compliance with national information security principles, combating false information and protecting Vietnam's digital sovereignty.

Practical applications - evaluating effectiveness in education, public health and specific application fields in Vietnam.

Agentic AI capacity - evaluates the ability to plan, use tools, RAG and act autonomously in the Vietnamese environment.

V-Bench's vision in building an AI reference system for Vietnam (Photo: VinUni).

Thanks to that, V-Bench not only measures the ability to answer academic questions, but also evaluates the understanding of culture, regional language and the ability of AI to act in the Vietnamese context, thereby producing results that are objective, comprehensive, close to reality, with reference and high directional value for the community.

Evaluation results by V-Bench are guaranteed by a panel of experts with strict methodology and a large-scale, scientific, and elaborate data set.

Built according to common and common techniques in international LLM research, V-Bench also has high compatibility in terms and processes with the international reference standard ecosystem, helping global LLM developers to participate in assessments without the barriers of technical conventions.

V-Bench helps create a comprehensive perspective when comparing capabilities between models (Photo: VinUni).

Prof.Dr. Duong Nguyen Vu - Vice Principal, Scientific Director of VinUni University's AI Research Center said: "With a comprehensive data set and completely objective, non-profit contributions from the community of prestigious Vietnamese experts globally, V-Bench wishes to contribute to redefining AI standards in Vietnam and building an AI reference system for the country.

V-Bench's evaluation results will provide a panoramic perspective, helping organizations and businesses explore the depth of capacity of each AI model, thereby choosing the optimal model for each usage context and actual needs."

Provided for free at https://vbench.ai/, V-Bench has currently evaluated and published the reference results of 15 typical LLM models today. In the coming time, V-Bench will continue to be developed to add new evaluation criteria based on images and sounds such as: Nom script, regional signs, charts, cultural heritage, recognizing and understanding voices of three regions, understanding cultural context through short videos, understanding long Vietnamese documents of over 100,000 language units such as laws, contracts, textbooks and accurate citations from legal documents, official gazettes...

As one of the typical AI research projects of the AI ​​Research Center - VinUni University, V-Bench is a testament to VinUni's desire to break through and elevate the position of Vietnamese intelligence on the global academic map, while affirming the pioneering mission, spirit of dedication to the community and serving science of Vingroup.