In this guide 6 sections
Ask for a baseline before work starts. Insist on staging work, evidence behind every finding and a rollback plan per change. Own the test scripts. Require a before and after comparison on the same test. A vendor who cannot answer is guessing on your live system.
Will you measure before you touch anything?
Ask what the vendor records on day one. Listen closely. A sound answer names the journeys, the test data, the median and slow tail timings and the server readings taken during the test, plus how many runs were averaged.
If the first step is a configuration change on the live server, stop. That is a red flag, because without a starting measurement nobody, including the vendor, can later show that anything improved.
- Which journeys and how many runs
- Which environment and which data
- What gets recorded: timings, errors, CPU, memory, query times
Where will the work be done?
Tuning belongs on a staging copy restored from a recent backup, with customer names, phone numbers and email addresses masked before anyone opens it. Ask how the copy is built and how close it is to production in table sizes, database version and web server setup. Closeness matters.
Ask which changes will reach the live system, who approves each one and on which evening it ships, so that nothing arrives unannounced on a trading day. Index builds, cache rules and connection pool edits look small, yet any one of them can lock a table or log users out during working hours.
What evidence backs each finding?
Every finding should arrive with proof, such as a slow query log extract, an execution plan, a flame graph or a waterfall chart exported from the browser. Opinions are not findings. The database feels heavy proves nothing.
Ask the vendor to rank findings by measured effect and effort. Order matters. A fix that saves seconds on the order entry screen used all day should sit above one that shaves a blink off a quarterly report.
- Evidence attached to each finding
- Expected effect and how it will be confirmed
- Effort and risk written beside each item
What happens if a change makes things worse?
Ask for a rollback plan per change, written before deployment and signed off by whoever owns the database. Which backup is taken? Who decides? How long does reversal take? Schema changes are harder to undo than a setting.
Agree a quiet window and a named contact on both sides, with a phone number answered at night and a deploy time clear of month-end invoicing. Then a slow checkout at the busiest hour stays a small event.
Who keeps the scripts and shows the result?
The test scripts, data snapshot references and result sheets should be delivered to you in files you can open, such as JMeter plans, k6 scripts or CSV exports. Otherwise you depend on the same vendor for every future proof. Insist on it.
The close-out report must set the original baseline against the final run, same test, same environment, side by side, with the raw result files attached for your own team.
Your next step: copy these five questions into your next email to the vendor and ask for written answers from the named engineer before you sign anything.
Vendor question sheet
Five blocks share one sheet: baseline, staging, evidence, rollback and ownership. Each block lists the question, the vendor’s written answer, a pass or concern mark and a space for the person who reviewed it.
Open a blank worksheet to printQuestions owners ask
Should the vendor be allowed to test on production?
Read-only monitoring on production is fine. Load scripts and trial changes should run on staging, so customers, live orders and stored payment records are never touched by a test run. Keep it strict.
Who should own the test scripts?
You should. They are a record of how your system behaves, and they let any later team repeat the comparison without the original vendor.
What counts as proof of improvement?
The same scripted journeys on the same staging copy, with lower median and slow tail timings and no extra errors, shown beside the baseline in one table.
Is a fixed price safer than hourly?
Neither is safer. Whatever the pricing, tie each payment to delivered findings and a measured before and after comparison on the staging copy, not to hours spent.