Data in, objective set, optimizer runs, held-out set decides. No hand-tuning, no hoping.
Machine learning applied to prompts. A trained model has a dataset, a loss, an optimizer and a validation set. Prompt learning gives a prompt the same four things: labeled examples, an objective (accuracy, format, latency, tokens, or a combination), a search algorithm that proposes and selects rewrites, and a held-out set that decides. Why the search is hard →
Each one is measurable. Each one is optimizable. None of them survive a model upgrade on their own.
Prompts drift every time a model changes. A scored dataset turns "it seems fine" into a regression test you can re-run in minutes.
Accuracy, format compliance, judge scores — pick the metric that matters and let the optimizer climb it instead of guessing.
Shorter prompts, fewer tokens, lower latency. Compression is an objective, not an afterthought.
Trade accuracy against cost on a Pareto front and choose the operating point your product actually needs.
A tree search over prompt rewrites, with two pluggable ideas from two different fields.
An LLM reads the prompt, the examples it failed and why, and rewrites. Text feedback carries far more information per rollout than a score does.
A surrogate over prompt embeddings predicts where the good children are, so full evaluations go where they are most informative — and it knows a lucky score on a small sample might be luck.
An agent is several prompts wired into a graph, some of them choosing the next step. The optimizer runs the graph, scores the end result and the trace, and rewrites one step at a time.
The two families are not rivals: bpto runs GEPA-style and BO selection as peers on the same tree, and the best configuration we have found uses both. How they compare →
From logged experiments, and from the people counting agents.
The same optimizer, as a library and as a studio.
A Python library. Define a task (root prompt or program, dataset, scorer, objective), compose search operators, run with a budget and checkpoints. Everything on this site was measured with it.
Every step of bpto without writing code. Draw the program on a canvas, attach labelled data, validate, run a pilot for the cost, watch the tree grow, and read the optimized prompts off each step.
Agents multiply the prompt surface. A single chat assistant has one prompt. A ReAct-style agent has a system prompt, a planning prompt, a tool-call formatter, an observation summarizer and a final-answer prompt — and it runs each of them several times per task. Small inefficiencies compound per call, per step, per user.
Every model upgrade silently re-tunes every prompt. Prompts written for one model are not optimal for the next. Without a scored dataset and an optimizer, "upgrading the model" means re-doing the prompt work by hand, or not at all.
A measurable prompt strategy is the difference between a pilot and a product. Teams that treat prompts as versioned, evaluated, optimized artifacts ship faster and spend less. Teams that treat them as text in a config file discover regressions from customers.
The optimizer is abstract: it takes a starter program, a dataset and an objective, and searches. Accuracy, format compliance, latency, compression — same loop, different objective.
Enough to detect the differences you care about. 100 training examples plus a held-out set was enough to separate search strategies in our runs; 30 was not. If you have production traces, you already have the data.
Any: Amazon Bedrock and Anthropic models directly, and anything with an OpenAI-compatible endpoint (OpenAI, vLLM, Ollama, OpenRouter). The optimizer only needs a client that returns completions.
Yes. A program is a graph of prompt steps; a step with several outgoing edges chooses the next one. The optimizer runs the whole graph per example and rewrites one step at a time, holding the others fixed — coordinate ascent for prompts.
Compression is one objective of this optimizer — minimize tokens subject to an accuracy floor — and it has its own site: promptcompression.ai covers the business case and a worked example.
Yes — there is a step-by-step page written for agents and an installable skill file.
The engine, bpto, is open source on GitHub. Impromptune, the studio, is a hosted product built on it.
A small optimization company in Claverack, New York, with a Monte Carlo accelerator at its root. About us →