HyperCLOVA X Technical Report Reaffirms its Superiority Over Global Models, Open Source AI, and Translators
HyperCLOVA X Technical Report Reaffirms its Superiority Over Global Models, Open Source AI, and Translators
HyperCLOVA X Technical Report Reaffirms its Superiority Over Global Models, Open Source AI, and Translators
- HyperCLOVA X outperforms global open-source models, supported by pre-training with high-quality datasets
- HyperCLOVA X utilizes Korean and English data for inference in a third language and excels in machine translation with its verified multilingual proficiency
- Touted as a “best practice of sovereign AI”, it employs safety measures such as ‘Red-teaming’ in its development efforts
April 4, 2024
On April 3, NAVER CLOUD released the technical report* for HyperCLOVA X, its hyperscale AI technology. The report provides an in-depth look at the AI model, covering aspects from its learning strategies to its performance metrics. Big tech firms like OpenAI and Google also utilize technical reports to detail the features of their AI models.
HyperCLOVA X outperforms global open-source models, supported by pre-training with high-quality datasets
According to the technical report, HyperCLOVA X outperformed other global open-source models in overall performance tests. Securing the top position in a comparison of 14 models across domains such as the Korean language, general knowledge, math, and coding, HyperCLOVA X showcased its edge as a sovereign AI, with capabilities extending beyond mastering specific languages to solving a wide range of problems across general knowledge and programming.
When compared to closed-source models developed around the world, HyperCLOVA X still demonstrated impressive performance, standing out in Korean language proficiency, where it topped the list among four models evaluated and achieved second place in English compared to the same model group.
The report further details the training process behind HyperCLOVA X’s capabilities, with the majority of its data used in pretraining consisting of a mix of Korean, English, and coding. Efforts to ensure a dataset of the highest quality involved removing overly brief, repetitive, or low-quality content, and any data including personal information. Furthermore, the application of alignment learning techniques fine-tuned the model’s capacity to grasp the nuances of user queries and intentions more accurately.
HyperCLOVA X utilizes Korean and English data for inference in a third language and excels in machine translation with its verified multilingual proficiency
The report also highlighted the ‘Multilinguality’ of HyperCLOVA X, confirming its ability to use Korean and English data from its training set to infer in a third language. In assessments of its language skills across several Asian languages, such as Japanese, Arabic, Hindi, and Vietnamese, HyperCLOVA X scored the highest among nine models selected in the report, including major open-source models, and ranked second only in Chinese.
HyperCLOVA X’s multilingual capabilities were further evidenced in machine translation tests as it surpassed other models in translating between Korean and Japanese, ranking first among ten selected models, including those currently in service for translation. It also achieved the highest accuracy in translations from English to Korean within this group.
Yoo Kang-min, Leader of NAVER CLOUD responsible for the HyperCLOVA X technical report, stated, “The tests evaluating the multilingual inference and translation capabilities of HyperCLOVA X confirm that AI, even when tailored for specific regional or cultural applications, can achieve a certain level of proficiency in multiple languages beyond its originally intended laugauge. Equipped with culturally nuanced knowledge and multilingual proficiency, HyperCLOVA X demonstrates the expansive potential of sovereign AI to adapt across diverse contexts.”
Touted as a “best practice of sovereign AI”, it employs safety measures such as ‘Red-teaming’ in its development efforts
Efforts to ensure HyperCLOVA X’s safety were also highlighted in the report. The model was tested with data queries on delicate or hazardous subjects such as “social issues and biases” and “illegal actions”, using red teaming** to strengthen the model’s security framework. In addition, ongoing enhancements are implemented to prevent the generation of content involving hate, biases, copyright violations, or personal information, adhering to the ethical guidelines of HyperCLOVA X.
Sung Nako, Executive Director of Hyperscale AI at NAVER CLOUD, commented, “The technical report reaffirms HyperCLOVA X’s standing in the competitive AI landscape. As a benchmark of sovereign AI with its advanced programming, mathematical logic, linguistic diversity, and safety measures, alongside expertise specific to Korea, we are poised to leverage the insights from HyperCLOVA X’s development to craft hyperscale AI tailored to the needs of diverse regions and countries.”
The technical report conducted a thorough analysis of HyperCLOVA X and its peers’ performance across various disciplines, such as the Korean and English languages, math, coding, general knowledge, authenticity, and safety. It employed reliable benchmarks or self-developed evaluation criteria for performance evaluation. For example, the evaluation of HyperCLOVA X and other open source models’ proficiency in Korean utilized an aggregate score from six benchmarks, among them Measuring Massive Multitask Language Understanding in Korean (KMMLU), Measuring Massive Multitask Language Understanding (MMLU), and Microsoft’s Artificial General Intelligence Evaluation (AGIEval).
* HyperCLOVA X Technical Report, https://arxiv.org/abs/2404.01954
** Red teaming: Strategic exercise focused on intentionally challenging technology or services to identify and rectify potential security weaknesses.
