GPT-Code-Clippy (GPT-CC)

GPT-Code-Clippy (GPT-CC) is a community effort to create an open-source version of GitHub Copilot, an AI pair programmer based on GPT-3, called GPT-Codex. GPT-CC is fine-tuned on our GPT Code Clippy dataset sourced from publicly available code on GitHub. It was created to allow researchers to easily study large deep learning models that are trained on code to better understand their abilities and limitations. GPT-CC uses the GPT-Neo model as the base language model, which has been pretrained on the Pile dataset and we use the Causal Language Modelling objective to train the model.