Test original ad-image variants in controlled experiments
Generate several visual directions, then use a defined experiment to decide which creative merits further testing.
How this works
Generated ad images create hypotheses about creative direction. A controlled experiment tests those hypotheses against a defined business outcome under comparable delivery conditions. Keep generation quality, campaign execution and statistical interpretation separate so attractive artwork is not mistaken for measured performance.
Define a baseline, assignment method, outcome window and decision rule before launch. Log the exact asset revision used in each arm. If the platform optimizes delivery unevenly or the audience changes during the run, explain the resulting limits instead of presenting a clean causal comparison.
Before you start
Campaign brief
Confirm the approved offer, audience, landing page, mandatory claims and permitted creative changes.
Experiment authority
Identify the authorized account, budget ceiling and reviewer. Asset preparation alone does not authorize media spending.
Measurement plan
Choose the primary outcome, attribution window, assignment method and review date before seeing results.
Prepare the inputs
- creative_brief
- Required
Offer, audience, message, placement and constraints
Keep factual claims identical unless copy is an explicit experimental factor.
- baseline_asset
- Required
Exact approved asset and version
Record its dimensions and publication identity.
- variant_plan
- Required
Concept ID and intentional difference
Avoid changing image, offer, targeting and landing page simultaneously without a suitable design.
- experiment_data
- Required
Arm-level exposures, eligible outcomes, costs and dates
Use the same eligibility and attribution definitions across arms.
Implementation, step by step
Define the testable contrast
- State which visual change is expected to affect which behavior and why. Choose a bounded set of distinct concepts.
- Write the primary metric and stopping/review policy with the campaign owner; do not optimize the criterion after seeing a favorable result.
Checkpoint: The experiment brief distinguishes the hypothesis from a promised outcome.
Create and review original assets
- Generate candidates using the approved brief and legitimate reference material. Preserve asset IDs and editing history.
- Inspect readability, product accuracy, originality concerns and placement requirements; reject noncompliant candidates before performance testing.
Checkpoint: A reviewed creative set is tied to explicit experimental arms.
Configure comparable delivery
- Use the platform’s available experiment controls where appropriate, and document assignment, audience overlap and optimization settings.
- Check destination, budget and exact asset preview before an authorized launch. If no suitable control exists, label the exercise observational.
Checkpoint: The run record describes what comparison the data can support.
Collect complete observations
- Capture arm-level exposures, conversions and costs for the agreed window, including failed delivery or tracking interruptions.
- Avoid declaring a winner from early fluctuations or unequal windows. Check conversion lag before interpreting final outcomes.
Checkpoint: The dataset contains a comparable denominator and a known completeness status.
Recommend the next decision
- Calculate the agreed rates and costs deterministically. Show uncertainty, sample limitations and any confounding changes.
- Choose continue, revise, stop or retest according to the pre-agreed evidence requirement. Do not claim that a visually preferred image has proven acquisition value.
Checkpoint: The decision note connects the recommendation to observed evidence and constraints.
Decisions along the way
One arm barely delivered
Report insufficient exposure and investigate configuration before comparing performance.
Click rate rises but purchases fall
Discuss the downstream mismatch and prioritize the agreed business outcome rather than selecting the flattering metric.
The platform changes targeting or optimization mid-test
Record the change and qualify or restart the comparison under an approved plan.
What you will produce
creative-experiment-manifest.csv
CSVTraceability from approved creative to delivered experiment.
Include: arm_id · asset_version · hypothesis · assignment · budget · status · tracking_ref
creative-test-decision.md
MarkdownA reproducible analysis without unsupported lift claims.
Include: observation_window · counts · rates · uncertainty · confounders · recommendation
Check the result
- Verify that live assets match approved revisions and destination URLs.
- Reconcile impressions, clicks, conversions and cost to the same window and definitions.
- Check attribution lag and incomplete tracking before calculating outcome rates.
- Confirm that the final recommendation follows the original decision criterion.
When something goes wrong
Tracking fails for one arm
Pause interpretation and retain the affected time interval as incomplete.
A generated asset contains a false claim
Remove it from the approved set and correct the creative before any further publication.
An API launch request has an unknown outcome
Reconcile existing campaign receipts before retrying to avoid duplicate spend.
A worked example
SYNTHETIC EXAMPLE · NOT A REPORTED BUSINESS RESULT
Request
Compare two original image concepts for an approved campaign.
Inputs
Synthetic arm A has 1000 eligible visits and 40 purchases; arm B has 1000 eligible visits and 35 purchases. Tracking definitions match, but no uncertainty analysis or decision threshold has been agreed.
Expected result
Calculate 4.0% and 3.5% observed purchase rates. Report the difference as descriptive, not a proven winner, and request the agreed evidence criterion before scaling budget.
Run a small pilot
Start small
Run one bounded comparison with a stable offer and a small reviewed asset set.
Record the baseline
Record baseline outcome rate, delivery variability and creative production effort.
Agree success criteria
Judge usable assets, measurement completeness and decision usefulness before increasing experimentation volume.
Stop or revise
Stop spending or inference if authorization, tracking or factual asset accuracy fails.
Keep the scope clear
- Do not launch campaigns or increase budgets beyond explicit authorization.
- Generated imagery does not establish copyright clearance, customer demand or guaranteed performance.
Check before using
Check comparability, attribution and sufficient observations; do not infer performance from appearance alone.
Business activities
- Create and publish content
- Run advertising and acquisition tests
FROM A RECIPE TO YOUR WORKFLOW
Bring the company context together.
This workflow needs reliable inputs before it needs a better prompt. Moat8 Vault brings working information into a readable archive for the AI tools you use.
For this recipe: Campaign brief · Audience and test criteria
Start with one recurring task, agree which sources to use, and review the first result with your team.
Discuss this workflow