Stability AI releases StableLM, an Open-Source Suite of Large Language Models

Stability AI, a company committed to creating AI technology that is transparent, accessible, and supportive, has released an open-source language model called StableLM. This new model is available in 3 billion and 7 billion parameters, with even larger models set to be released in the near future. Developed to generate both text and code, StableLM has been designed to power a range of downstream applications.

StableLM was trained on a new experimental dataset built on The Pile, which is three times larger than its predecessors and contains 1.5 trillion tokens of content. The dataset is made up of an enormous amount of text from various sources, and StableLM has been trained to generate output that is both coherent and relevant. The model's performance is impressive, demonstrating how small and efficient models can deliver high performance with appropriate training. Like its rival, ChatGPT, StableLM is designed to generate text and code effectively.

Image by Stability AI

The StableLM models are designed to be transparent and accessible, with developers and researchers able to freely inspect, use, and adapt the StableLM base models for commercial or research purposes, subject to the terms of the CC BY-SA-4.0 license. This allows researchers to verify performance, work on interpretability techniques, identify potential risks, and help develop safeguards. Additionally, developers can build independent applications that are compatible with widely-available hardware, rather than relying on proprietary services. The StableLM models are now available in Stability AI's GitHub repository, and a full technical report is set to be published soon.