Technology that opens a new world
Technology that opens a new world
NAVER | Tech | Yoo Kang-min
“I had this strong feeling that a totally new world had just begun.” 2023 will be remembered as the year that truly kicked off the era of generative AI. Launching its own hyperscale AI model called HyperCLOVA X, NAVER has also joined forces to unfold this new trend. Yoo Kang-min was recruited by NAVER after finishing his Ph.D. in data augmentation technology. As a research developer with five years of work experience, he is playing a key role in conducting research on new LLM technologies and making the company’s AI model smarter. Proving the competitiveness of HyperCLOVA X at international conferences and standing at the forefront of its growth, he believes that this technology has the potential to upend people’s daily lives and has a strong sense of responsibility for the IT giant holding the future of Korea’s AI competitiveness. It’s not merely the desire to win in the tech war, but his unwavering belief in his research to bring meaningful changes to our lives that serves as the motivation and goal for his work.
.jpg)
How did you join NAVER?
I joined NAVER’s AI organization in July, 2020 after finishing my Ph.D. in computer science.
My specific major was machine learning and natural language processing, and my Ph.D. dissertation was on data augmentation. The main focus of my research was finding better ways to create data with AI models and how that helps the model's self-improvement. NAVER back then was in need of data augmentation technology, which fit perfectly with my research topic. After getting a job offer, I became part of NAVER’s AI Lab. To closely apply my research to services, I also worked for the Conversation team that handled machine learning technology for chatbot services.
Tell us about the role of the Foundation Research team.
In September 2020, our team first embarked on a project called HyperCLOVA, NAVER's hyperscale AI model. That was when language model technologies just began to emerge. This trend was our motivation for this project—we were all aligned with each other for the need to prepare ourselves to tackle the future.
The trend of language models is now being followed by Large Language Models (LLMs). In this era, models that aren’t big enough are not even considered relevant anymore. This drives the industry to constantly release and discover new LLM technologies, and the academia to conduct a significant amount of research. What our team does is preemptively conduct research on and identify the potential of such technologies so that HyperCLOVA X can get even smarter.
Can you tell us more about the technologies your team is focusing on to further advance the HyperCLOVA X model?
While HyperCLOVA X only handles languages as of now, we are studying ways to further develop it into a multimodal LLM that supports various modalities, including images, audio, and videos. Other technologies we are mainly focusing on include Rejection-Sampling Fine-Tuning (RFT) and Super-Distillation, which help the model use surplus inference resources by itself to continuously self-improve its performance. There has been great progress and we’re strengthening our technology for their stable use.
Most recently, we started looking into a new knowledge transfer technique based on how humans acquire knowledge called “Bloom’s Taxonomy.” We are applying this to make our model more efficient by making it smaller or effectively transfer knowledge from bigger models.
.jpg)
The advent of generative AI had huge global repercussions, and NAVER was no exception.
We were already close to finalizing the basic structure of HyperCLOVA X at the end of 2022. But back then, we didn't have a good sense of how LLMs would actually help users. While the academia was starting to discuss this topic, we were rather ignorant of it. We had no idea that LLMs would become so important.
However, ChatGPT proved that Instruction-Following, the ability to easily follow users’ instructions in the form of natural language, works very well. That was when we felt a sense of urgency.
Against this backdrop, I strongly appealed to our team in January 2023 that we needed to prepare our own LLM model with the Instruction-Following feature. This was at a time when ChatGPT was already released, enjoying a surge in the number of users, especially Korean users in December and January. This trend in the global market intimidated us because we could be overwhelmed if we failed to make an agile transformation. But our greatest fear of all was ChatGPT outperforming HyperCLOVA X in the Korean language. I knew we shouldn’t miss out on this project, so I persuaded our organization and finally formed a TF in January 2023.
What gave you such a strong belief that you had to do it no matter what?
We used to think that the hard, machine-learning method was the only way to unleash the power of AI. However, the trend is shifting to make AI easily accessible to everyone. I had a strong feeling that a new world was unfolding.
Just like how everyone uses cell phones, I think technology reaches a significant inflection point to change humanity when it's considered such an integral part of our daily lives that we can't live without it. I'm confident that LLMs such as HyperCLOVA X or ChatGPT and models driven by instruction following can serve as this tipping point. It would be of tremendous value if our model could boost at least 1% of human productivity, or at least 1% of Korea’s entire economy. I have no doubt that the direction we’re heading towards is to make even the smallest contribution in such ways.
I think I was very passionate talking about this to many people at that time. That was how fascinated and immersed I was in this project, and how strong my belief was. The current trend of LLMs makes me reassured that we're really on the right track and fortunate enough to have things flow as we expected. Above all, we would not have made it through without our colleagues who agreed with the direction we were aiming for. We were lucky to have so many people truly devote themselves when we were running out of resources to overcome such limitations for the sake of achieving our goal.
What motivates you to get better?
I think NAVER equals Korea's AI competitiveness. This is where my sense of responsibility or strongest motivation is rooted. I am lucky enough to work with the most talented leaders and colleagues in this sector, and what we are all building together is NAVER as a brand. In that sense, being part of NAVER means having the golden opportunity to make a big contribution to our society. I’m sure this invaluable experience to make success with the best colleagues can only be gained here in NAVER.
