Prompt Engineering for Real Work (Not Chatbot Tricks)

Most prompt engineering advice is written for people asking chatbots questions. "Act as an expert." "Think step by step." "Be thorough and careful." These phrases do approximately nothing when you're trying to get an LLM to classify 10,000 support tickets, extract structured data from legal documents, or generate code that actually runs.

The gap between chatbot prompting and production prompting is the same gap between asking a friend for restaurant recommendations and writing a procurement specification. One is a conversation. The other is a contract.

Why Chatbot Tricks Fail in Production

The "act as an expert" pattern has a specific problem: it's vague. When you tell a model to "act as a senior data analyst," you're trusting the model's interpretation of what that role entails. Every run, you get a slightly different interpretation. That's fine for a one-off conversation. It's unacceptable when you need consistent output across 500 API calls.

"Think step by step" was useful in 2023. Current frontier models already decompose problems internally. Adding the phrase is like telling a calculator to "really think about" the addition. It adds tokens to your bill without changing the output.

The real issue is that chatbot-style prompts specify intent without specifying constraints. They describe what you want but not what you won't accept. In production, the constraints are the entire point. The format matters. The edge cases matter. The failure behavior matters. "Be thorough" doesn't address any of them.

System Prompts as Contracts

A system prompt for production work should read like a contract, not a conversation starter. It specifies the output format, the constraints, the failure modes, and the boundary conditions. The model doesn't need encouragement. It needs a specification.

Here's a system prompt that doesn't work well:

You are an expert at classifying customer support tickets.
Please categorize each ticket into the appropriate category
and provide a confidence score. Be thorough and accurate.

Here's one that does:

Classify the input into exactly one category from this list:
billing, technical, account, feature_request, other.

Output format (no other text):
category: {category}
confidence: {0.0-1.0}
reasoning: {one sentence, max 100 characters}

If the input is ambiguous between two categories, choose
the one that appears first in the list above.
If the input is not in English, output:
category: other
confidence: 0.0
reasoning: non-English input

The second prompt is shorter. It's also dramatically more reliable. Three things make it work. First, it specifies the exact output format — not "provide a confidence score" but the literal structure the output must follow. Second, it handles ambiguity explicitly by providing a tiebreaker rule. Third, it defines failure behavior for an edge case (non-English input) instead of hoping the model figures it out.

Structure forces behavior. When the template has a required field, the field gets filled. When the prompt has a tiebreaker rule, ties get broken consistently. When failure modes are specified, failures are handled instead of improvised.

Few-Shot Examples Beat Long Instructions

There's a persistent belief that more detailed instructions produce better output. In practice, the opposite is often true, especially for classification and extraction tasks. A single concrete example outperforms a paragraph of rules.

I tested this across multiple models in a controlled experiment. The task was summarization with a strict format constraint: output under 200 characters, no markdown, plain text only. The instruction-based prompt said all of that explicitly. The few-shot prompt showed one example of the target output and said nothing else about format.

Results: the instruction-only prompt averaged 260 characters with one out of four models producing compliant output. The few-shot prompt averaged 120 characters with four out of four models compliant. The example did what the instructions could not.

This makes sense when you think about how these models work. They're pattern-completion engines. Show them a pattern, and they complete it. Tell them a rule, and they interpret it — often creatively, often wrong. "No markdown" is a rule. A plain-text example with the right density and length is a pattern.

The practical application:

EXAMPLE INPUT:
Federal Reserve holds interest rates at 5.25% amid
persistent inflation concerns. Core CPI rose 0.3% in
March, above analyst expectations of 0.2%.

EXAMPLE OUTPUT:
Fed holds rates at 5.25%. Inflation at 3.1%. Markets
fell 0.8%.

NOW PROCESS THIS INPUT:
{actual_content}

One trap: large models (30B+ parameters) sometimes copy the example verbatim if the input content is similar. Use examples with different subject matter than the actual input. If you're classifying support tickets about billing, use a shipping example in the few-shot.

Where to Put Your Constraints

Prompt structure matters more than most people realize. Specifically, where you put your "do not" constraints affects whether they're followed.

Testing on real classification tasks showed a 36% error reduction when negative constraints were placed at the end of prompts rather than at the beginning. This aligns with transformer architecture: the model attends more heavily to recent tokens. Instructions at the end of the context window get followed more reliably than instructions buried at the top.

The pattern that works in production:

[task description]
[examples]
[output format]

Do NOT:
- Include markdown formatting
- Add preamble or explanation
- Exceed 300 characters
- Output partial results

Task first, constraints last. The task tells the model what to do; the constraints, placed where the model will attend to them most, tell it where to stop.

There's a related principle for character limits versus sentence counts. "Write 3-5 sentences" is interpreted wildly differently across models and even across runs of the same model. "Maximum 300 characters" is a hard constraint that models respect with much higher consistency. If output length matters for your pipeline — and it usually does, because you're probably parsing it downstream — use character limits.

The Prompt as Code Pattern

If a prompt runs in production, it's code. Treating it like code means version controlling it, reviewing changes to it, testing it before deploying it, and tracking its performance over time.

This sounds obvious, but I've seen teams with rigorous CI/CD pipelines for their application code who edit prompts directly in a web dashboard. The prompt is the most sensitive part of the system — the part where a single word change can alter behavior across every request — and it's the part with the least engineering rigor.

The practical version of prompt-as-code has three components.

First, prompts live in version control alongside the code that calls them. Not in a database, not in an environment variable, not in a config service. In a file, in the repo, with a commit history. When something breaks, you can diff the prompt against last week's version and see exactly what changed.

prompts/
  classify-ticket/
    v1.txt
    v2.txt
    v3.txt
  extract-entities/
    v1.txt
  summarize-document/
    v1.txt
    v2.txt

Second, prompt changes go through review. A prompt change that alters classification behavior is a functional change. It should get the same review scrutiny as a change to the classification code. The reviewer should be able to see the before/after prompt text and understand why the change was made.

Third, prompts get tested. Not with vibes — with a test set. Keep 50-100 representative inputs with expected outputs. Run them against the new prompt before deploying. If accuracy drops, the change doesn't ship. This is the same principle as unit testing. The only difference is that your assertions might use fuzzy matching instead of exact equality.

A prompt evaluation loop in practice looks like this:

# test-prompt.sh
PROMPT_VERSION=$1
PASS=0
FAIL=0

while IFS='|' read -r input expected; do
  result=$(call_model "$PROMPT_VERSION" "$input")
  if echo "$result" | grep -q "$expected"; then
    PASS=$((PASS + 1))
  else
    FAIL=$((FAIL + 1))
    echo "FAIL: input=$input expected=$expected got=$result"
  fi
done < test-cases.txt

echo "Results: $PASS passed, $FAIL failed"

Nothing fancy. A shell script, a text file of test cases, and a pass/fail count. The sophistication is in doing it at all, not in the tooling.

The Blocking Gate Pattern

One pattern that separates production prompts from chatbot prompts is the blocking gate. Most prompts include verification steps as checklists: "make sure the input is valid," "check that the format is correct." Checklists feel optional. Models treat them that way.

A blocking gate rewrites verification as a hard stop:

VERIFICATION (BLOCKING GATE):
1. Input contains a valid email address
2. Input is fewer than 5000 characters
3. Input is in English

If ANY check fails, output ONLY:
error: {which check failed}
Do NOT proceed to classification.

The difference is structural. A checklist says "here are things to verify." A gate says "here is where you stop." When the prompt defines explicit failure output and explicitly prohibits continuing, models follow it. When it says "make sure to check," models check loosely and continue anyway.

This matters most in pipelines where the output of one model call feeds into the next. Bad input that slides through a loose verification step corrupts everything downstream. A blocking gate catches it at the source.

The Catch

All of this requires something that chatbot-style prompting does not: you need to know your failure modes before you write the prompt. You need to have seen the weird inputs, the edge cases, the ambiguous classifications. You need to know what "wrong" looks like for your specific use case.

That knowledge comes from domain expertise, not from prompt engineering tutorials. A solo builder classifying their own support tickets knows which tickets fall between "billing" and "account" because they've handled hundreds of them. That knowledge — which tiebreaker rule to use, which edge cases to specify, which failure modes to guard against — is what makes the prompt work. The prompt engineering is the delivery mechanism. The domain expertise is the payload.

An AI system with no expert in the loop produces confident-sounding garbage. A well-structured prompt from someone who understands the domain produces reliable, testable, improvable output. The prompt is the interface between your expertise and the model's capability. Treat it accordingly.

What Comes Next

Prompts are getting more structured, not less. Function calling, tool use, structured output modes — the trajectory is toward prompts that look less like natural language and more like API specifications. The builders who are already writing prompts as contracts, version controlling them, and testing them against real data are building on the foundation that the rest of the industry is moving toward.

The chatbot tricks won't disappear. They'll keep circulating in tutorials and Twitter threads. But the gap between "prompting for conversation" and "prompting for production" will keep widening. One side is asking questions. The other side is writing specifications. The specifications are the ones that ship.