Create a Text‑Generating Web App with GPT‑2 in Under 50 Lines of Python

Preparation

The guide assumes Python 3.7+ and familiarity with language models. Install the required libraries with:

pip install torch<br/>pip install transformers<br/>pip install panel

Panel is used to create the dashboard, and PyTorch provides the deep‑learning backend.

Part 1: Build the Model

Import the necessary packages and load the pre‑trained GPT‑2 tokenizer and model:

import numpy as np<br/>import torch<br/>import torch.nn.functional as F<br/>from transformers import GPT2Tokenizer, GPT2LMHeadModel<br/>from random import choice<br/><br/>tok = GPT2Tokenizer.from_pretrained("gpt2")<br/>model = GPT2LMHeadModel.from_pretrained("gpt2")

Define a prediction function that tokenises input text, obtains logits, applies softmax, and uses nucleus (top‑p) sampling to pick the next token:

def get_pred(text, model, tok, p=0.7):<br/>    input_ids = torch.tensor(tok.encode(text)).unsqueeze(0)<br/>    logits = model(input_ids)[0][:, -1]<br/>    probs = F.softmax(logits, dim=-1).squeeze()<br/>    idxs = torch.argsort(probs, descending=True)<br/>    res, cumsum = [], 0.<br/>    for idx in idxs:<br/>        res.append(idx)<br/>        cumsum += probs[idx]<br/>        if cumsum > p:<br/>            pred_idx = idxs.new_tensor(choice(res))<br/>            break<br/>    pred = tok.convert_ids_to_tokens(int(pred_idx))<br/>    return tok.convert_tokens_to_string(pred)

Part 2: Build the Web Application

Import Panel and create the UI components:

import panel as pn<br/>pn.extension()<br/>text_input = pn.widgets.TextInput(name='Input Text')<br/>generated_text = pn.pane.Markdown(object=text_input.value)<br/>button = pn.widgets.Button(name='Generate', button_type='primary')

Link the text input to the markdown pane so that changes propagate:

text_input.link(generated_text, value='object')

Define a callback that calls get_pred and appends the result to the displayed text:

def click_cb(event):<br/>    pred = get_pred(generated_text.object, model, tok)<br/>    generated_text.object += pred

Connect the button to the callback and arrange the layout:

button.on_click(click_cb)<br/>app = pn.Column(text_input, button, generated_text)<br/>title = pn.pane.Markdown("# **Text Generator**")<br/>desc = pn.pane.HTML("<strong>Welcome to the text generator! Enter some seed text, click Generate, and watch the model continue.</strong>")<br/>final_app = pn.Column(title, desc, app)

Serving the Application

For quick debugging, run:

final_app.show()

For production, make the app servable:

final_app.servable()

Running the notebook with the appropriate panel serve command will launch the web app on a local port.

The tutorial concludes that you now have a functional text‑generation web app that can be extended or embedded in other projects.