python-ds-accelerate (GPT-2 124M)

This model is a GPT-2 (124M parameter) causal language model trained from scratch specifically for Python code completion in Data Science contexts.

Model Details

Model Description

This model is an implementation of the GPT-2 architecture optimized for generating functional Python code snippets. It was trained using a custom training pipeline that incorporates a keytoken weighted loss function to prioritize important programming keywords (like plt, pd, fit, predict), making it more effective at suggesting Data Science-related code.

• Developed by: Pranav Guhan R
• Model type: Transformer-based Causal Language Model
• Language(s): Python (English comments)
• License: Apache 2.0
• Finetuned from model: Trained from scratch

Model Sources

• Repository: GPT-2-124M-pretraining-for-code-completion

Uses

Direct Use

The model is intended to be used for code completion tasks, specifically for completing Python scripts involving libraries like pandas, matplotlib, and scikit-learn.

Out-of-Scope Use

The model is not suitable for general-purpose natural language conversation or generating code in languages other than Python.

How to Get Started with the Model

You can use the model directly with a Hugging Face pipeline:

from transformers import pipeline

pipe = pipeline("text-generation", model="PranavGuhan/python-ds-accelerate")

txt = """# create dataframe from x and y
df = pd.DataFrame({'x':x, 'y':y})
"""
print(pipe(txt, num_return_sequences=1)[0]["generated_text"])