Rakuten’s mission is to contribute to society by creating value through innovation and entrepreneurship. The Rakuten AI LLM series is an initiative that directly stems from this mission. We develop large language models to deliver high-performance, cost-efficient solutions tailored to the diverse needs of our ecosystem and our customers. By building these models internally, we ensure they are optimized for the nuances of the Japanese language and culture.
Through our commitment to releasing these models, we aim to contribute to the broader research community, accelerate the development of AI applications and drive innovation that empowers businesses and users across Japan.
Rakuten AI 3.0
Rakuten AI 3.0, is an approximately 700 billion parameter Mixture of Experts (MoE) model optimized for Japanese. Developed by leveraging the best from the open-source community and building on Rakuten’s high-quality, bilingual original data, engineering and research, it offers a superior grasp of Japanese language and culture.
For more details, please check the Press Release.
Rakuten AI 2.0
Rakuten AI 2.0 is an MoE-based foundation model derived from RakutenAI-7B, first introduced in March 2024. As part of a broader initiative to advance Japanese LLM technology, Rakuten AI 2.0 adopts a Mixture of Experts (MoE) architecture with two active experts, resulting in 13B active parameters. This design enables dynamic expert selection based on input tokens, enhancing computational efficiency while maintaining high performance. Rakuten AI 2.0 achieved state-of-the-art results on Japanese language understanding benchmarks at the time of release while also demonstrating competitive performance on English evaluation tasks compared to similar models.
For more details, please check the Press Release.
If you are looking for an instruction-tuned model, please check RakutenAI-2.0-8x7B-instruct.
Rakuten AI 2.0 mini
Rakuten AI 2.0 mini is a lightweight Japanese language model trained from scratch using a transformer architecture, designed for efficient performance in resource-constrained environments. As a foundation model, it serves as the backbone for instruct models.
For more details, please check the Press Release.
If you are looking for instruct model, please check RakutenAI-2.0-mini-instruct.
Rakuten AI 7B
Rakuten AI 7B was a pioneering initiative to support the development of Japanese LLMs. At the time of release, Rakuten AI 7B achieved the top scores on the Japanese language understanding benchmarks while maintaining a competitive performance on the English test sets among similar models. RakutenAI-7B leverages the Mistral model architecture and is based on Mistral-7B-v0.1 pre-trained checkpoint, exemplifying a successful retrofitting of the pre-trained model weights.
For more details, please check the Press Release.
The technical report can be accessed at arXiv.
If you are looking for an instruction-tuned model, please check RakutenAI-7B-instruct.
If you are looking for a chat-tuned model, please check RakutenAI-7B-chat.