Chinese language AI unicorn’s 34B LLM beats Llama 2 and Falcon fashions

01.AI, the Chinese language startup based by veteran AI professional and investor Kai-Fu Lee, has launched a 34-billion parameter giant language mannequin (LLM) that outperforms the 70-billion Llama 2 and 180 billion Falcon open-source counterparts constructed by Meta Platforms, Inc., and the Know-how Innovation Institute in Abu Dhabi, respectively.

Dubbed Yi-34B, the brand new AI mannequin helps Chinese language and English languages and could be fine-tuned for a wide range of use circumstances. The startup additionally provides a smaller possibility that has been educated with 6 billion parameters and performs worse, however nonetheless respectably, on extensively used AI/ML mannequin benchmarks.

Ultimately, the corporate, which has already hit unicorn standing in lower than eight months of its launch, plans to double down these fashions and launch a business providing able to taking up OpenAI, the present generative AI market chief by variety of customers.

The technique highlights a world development the place international firms are creating generative AI fashions geared primarily in direction of their respective markets. 

LLM-driven imaginative and prescient of ‘Human+AI’

Lee based 01.AI in March with a mission to contribute to the AI 2.0 period, the place giant language fashions might improve human productiveness and empower them to create vital financial and societal shifts. 

“The group behind 01.AI firmly believes that the brand new AI 2.0 pushed by basis mannequin breakthrough is revolutionizing expertise, platforms, and functions in any respect ranges. We predict that AI 2.0 will create a platform alternative ten occasions bigger than the cell web, rewriting all software program and consumer interfaces. This development will give rise to the following wave of AI-first functions and AI-empowered enterprise fashions, fostering AI 2.0 improvements over time,” the corporate writes on its web site.

Based on studies, Lee was fast to assemble a expertise group together with AI consultants from firms like Google, Huawei and Microsoft Analysis Asia and pile up the chips required for coaching 01.AI’s Yi sequence of fashions.

The preliminary funding for the hassle was led by Sinovation Ventures, which can be chaired by Lee, in addition to Alibaba’s cloud unit. Nevertheless, the precise quantity raised stays unclear at this stage. 

The primary public launch from the corporate launched two bilingual (English/Chinese language) base fashions with the parameter sizes of 6B and 34B – each educated with 4K sequence size with the choice to increase to 32K throughout inference time. The following launch of the fashions got here with 200K context size.

On Hugging Face, the bottom 34B mannequin stood out with a efficiency higher than the a lot bigger pre-trained base LLMs, together with Llama 2-70B and Falcon-180B.

For instance, when the benchmarked duties revolved round widespread reasoning and studying comprehension, the 01.AI mannequin delivered scores of 80.1 and 76.4, whereas Llama 2 adopted intently with scores of 71.9 and 69.4. Even on the MMLU (huge multitask language understanding) benchmark, the Chinese language mannequin did higher with a rating of 76.3, whereas the Llama and Falcon fashions had a rating of 68.9 and 70.4, respectively. 

A smaller mannequin delivering higher efficiency might save compute assets for finish customers, permitting them to fine-tune the mannequin and construct functions concentrating on totally different use circumstances cost-effectively. Based on the corporate, all fashions beneath its present Yi sequence are totally open for educational analysis. Nevertheless, if the necessity is free business use, groups must receive the required permissions to get began with the fashions. 

Far more to come back

The present choices from Lee’s startup are profitable choices for international organizations serving prospects in China. They will use the mannequin to construct chatbots answering in each English and Chinese language. Shifting forward, the corporate plans to broaden these efforts by including assist for extra languages to the open-source fashions. It additionally plans to launch a much bigger business LLM concentrating on OpenAI’s GPT sequence, though not a lot has been revealed on the venture to date.

Notably, 01.AI shouldn’t be the one AI startup specializing in particular languages and markets with LLMs. Simply final month, Chinese language large Baidu introduced the discharge of ERNIE 4.0 LLM and previewed a complete host of latest functions constructed atop it, together with Qingduo, a artistic platform that goals to rival Canva and Adobe Artistic Cloud. 

Equally, Korean large Naver is providing HyperCLOVA X, its next-generation giant language mannequin (LLM) that has realized 6,500 occasions extra Korean knowledge than ChatGPT and is especially helpful for localized experiences the place it could perceive not solely pure Korean-language expressions but in addition legal guidelines, establishments and cultural context related to Korean society. India’s Reliance Industries can be working with Nvidia to construct a big language mannequin educated on the nation’s various languages, tailor-made for various functions.