In this guide 6 sections
Cap usage in layers. Set a spending limit on the OpenAI project. Give each user and team a quota inside your own app. Log every call with its feature name and token counts, cache repeated requests and alert on spikes. No single layer is enough.
Why one runaway loop can empty the budget
An AI feature bills by tokens, not by users. A retry loop that calls the model on every failed webhook can burn through a month of allowance overnight. Nobody notices until the invoice arrives. Then it is too late.
Heavy users cause the same trouble more slowly. One analyst pasting a forty page contract into a summary box, ten times a day, costs far more than fifty people asking short questions.
So the plan must cover two failure types. One is a bug in your own code. The other is ordinary people using the feature harder than you expected. Different controls catch each.
- Retry loops without a back-off or a maximum count
- Background jobs that re-run on every deploy
- Long documents sent whole instead of in pieces
- A shared API key used by test and production alike
Set limits at the project, then inside your app
Start at the provider. Create a separate OpenAI project for each environment and each product, with its own API key and its own monthly spending limit. The dashboard limit is your outer wall. It is a blunt tool. Check the current options in the usage and limits pages of the OpenAI platform, since they change.
When it trips, every user loses the feature at once. So build finer limits in your own database: a daily token quota per user, a larger one per team, and a hard ceiling per feature.
Count tokens before the call where you can, using the response usage field after it. Store the result in a usage table keyed by user, team, feature and date. Reject the request early, with a clear message, when the quota is gone.
- Separate project and key for staging and production
- Monthly limit on each project, set below your real tolerance
- Per user daily token quota, reset at midnight in the account’s time zone
- Per team pool, so one person cannot drain it
- Maximum input length per request
Log each call with the feature that made it
A bill with one total tells you nothing. You want to see that the invoice summariser used most of the tokens, and which team drove it.
Write one row per call. Record the feature name, model, user id, team id, prompt tokens, completion tokens, latency, status and a request id. Never store the full prompt. If you must, decide first who may read it.
Build a simple report from that table in PostgreSQL. Group by feature and day. Add an estimated cost field using the price list you keep in a config file, so a price change is one edit.
Cache repeated requests and trim the prompts
Many requests repeat. A support team asks the same classification question about the same ticket twice. Hash the normalised prompt plus the model name, store the answer, and return it on a match.
Set an expiry that fits the data. A product description rewrite can live for weeks. An answer built on live stock levels should live for minutes or not at all.
Then trim. Send the relevant clauses, not the entire contract. Pick a smaller model for easy tasks such as tagging. Set max tokens on every call so a rambling answer has a ceiling.
- Cache key from model, prompt hash and settings
- Expiry per feature, not one rule for all
- Smaller model for tagging and routing
- Max output tokens on every call
Alert early and test the cut off
Alerts should fire well before a limit. Post to a shared channel when a team nears its daily quota. Do the same when one user passes a set number of calls in an hour, or when the error rate jumps.
Then rehearse. Set a tiny quota on a test account and watch what the screen says when it runs out. Also trip the project limit in staging, and confirm that the app handles a 429 response with a back-off instead of a tight retry. Users should see a plain message and a way to ask for more, not a blank panel.
This week, list your AI features, and next to each write its owner, its daily token ceiling and who gets the alert. A table with those three columns is enough to start.
AI usage limits worksheet
A single worksheet with a row for each AI feature. Columns hold the feature name, owner, model, per user daily quota, per team quota, monthly project limit, cache expiry, alert threshold and the person who receives the alert. Below it, the log table fields to create are listed.
Open a blank worksheet to printQuestions owners ask
Can I rely on the OpenAI spending limit alone?
No. It protects the account, but it stops every user at once. Quotas inside your app let one heavy user hit a wall while everyone else keeps working.
How do I count tokens before sending a request?
Use a tokenizer library for the model you call, such as tiktoken. It gives an estimate you can compare with the quota. The usage field in the response gives the exact figure afterwards.
Should I store prompts in the usage log?
Store counts and ids by default. Prompts often hold customer names or contract text. Keep them only for a short debugging window, with access limited to named roles.
What should a user see when their quota runs out?
A short message saying the daily limit is reached, when it resets and how to request more. Avoid a generic error, because people will report it as a bug.