Refining System Prompts for LLMs: Practical Tips for Batch Automation

To automate batch processing of documents with an LLM API, the first step is to define an appropriate system prompt.

Code example from GitHub Copilot

def translate(text):
    completion = client.chat.completions.create(
        model = MODEL_NAME,
        messages = [
            {"role": "system", "content": "Translate the following text to Chinese."},
            {"role": "user", "content": text}
        ]
    )

The messages argument is a list of conversation records that the model treats as context to generate a new reply.

In GPT‑style chat completion, three roles can appear in the message list: system : defines the assistant’s personality, capabilities, and rules; it is placed at the top of the list. user : the input from the human, also called the “user prompt”. assistant : the model’s response, returned automatically when the API is called. At least one user message is required for the model to produce a meaningful reply.

The design treats the system prompt like a function body: the system prompt encodes the processing logic, the user message supplies the function arguments, and the assistant’s reply is the return value. This modular view makes the model’s behavior easier to debug and optimise for scripted batch tasks.

Iterating system prompts

System prompts can be refined programmatically by calling the API in a loop, or interactively using OpenAI’s GUI tools.

The GPTs editor on the OpenAI website provides a two‑pane interface: the left pane (“Instructions”) edits the system prompt, while the right pane shows live chat results.

The OpenAI Playground also supports editing system prompts, switching models, and adjusting low‑level parameters.

Using these tools, you can iteratively test and polish system prompts until the LLM behaves predictably for batch processing scenarios.

Conclusion

By separating message roles, treating the system prompt as a modular function definition, and leveraging both script‑based and GUI‑based iteration, you gain fine‑grained control over LLM behavior, making large‑scale automated text processing reliable and maintainable.