How to prompt small language models

Most prompt advice is written for big cloud models. Those can take a messy request, ignore the fluff, and still land close to what you meant. A small model on your phone does not have that slack. A 3B model will latch onto the first clear instruction it sees, get confused by extra goals stacked on top of each other, and waste limited context on politeness and backstory that never helped. If you want useful answers from a phone-sized model, you have to write prompts that leave less room for guessing.

Put the task first

Start with the job, not the story around it. “Rewrite this email to be shorter” works better than a preamble about why the email exists, who will read it, and how stressed you are about the tone. A large model can dig your real request out of that noise. A small one often grabs whatever appears first and builds the whole answer around it. Make the first sentence the instruction, then give the material it needs to work on.

Keep the prompt short

Small models have small context windows. Every sentence you paste in competes with the task for attention. If you want a summary, give the text and the word limit. That is usually enough. You do not need disclaimers, softener phrases, or a polite setup. The model was never going to be offended, and those extras only dilute the signal.

One task at a time

This is the biggest practical difference from cloud models: ask a big model to translate a paragraph, review the translation, and suggest a better ending, and it can often do all three in one pass. Ask a small model the same thing and it usually finishes the first job, botches the second, and drops the third. You get better results by asking once, taking the answer, and only then moving to the next request. Those follow-ups are cheap when the model runs on your phone, so there is little reason to cram a whole workflow into a single prompt.

Name the format

Tell the model the shape you want instead of hoping it invents a useful one. “Answer in three bullet points,” “Write a short paragraph under 80 words,” or “Give me a table with two columns” all give it a target. “Keep it brief and organized” does not. Small models follow clear format rules well and guess badly when the target stays vague.

Skip long roleplay

Telling a big model to act as a veteran editor can change the voice of the output in useful ways. On a small model, a long persona mostly burns context. A short cue like “you are a strict editor” is fine. A paragraph of invented career history is wasted space that could have held the text you care about.

Use temperature when you can

If your app has a temperature setting, treat it as part of the prompt. Lower temperature helps with factual rewriting and clean structure. Higher temperature helps when you want brainstorming and variation. That one control often does more than another round of wording tweaks.

The simple rule

Small models reward clear prompts and punish extra words. Cut the request down to instruction, material, and format. Do that consistently and a model that fits on your phone will cover more of daily writing and thinking work than the size number suggests.