Create a visual review page for a research experiment
A model-development team generated disposable web reports to share dataset examples and experiment results with colleagues. The reports supported review without changing the core training system.
How this works
A model-development team generated disposable web reports to share dataset examples and experiment results with colleagues. The reports supported review without changing the core training system. The review page should let colleagues inspect what changed, which examples support the claim, and where the experiment failed. It is an evidence presentation layer; it must not modify the training system or quietly rerun experiments.
Build a small static report from immutable experiment outputs and a deliberate example-selection policy. Prefer local HTML with bundled data when it is sufficient. Keep exact run identifiers, dataset versions, metric definitions, and limitations visible so a polished page cannot obscure an invalid comparison.
Before you start
Frozen experiment outputs
Have metrics, run identifiers, configuration summaries, and example artifacts that the team is allowed to share.
Review questions
Specify the comparison, intended decision, and failure slices that matter.
Rendering environment
Use available local HTML/chart tools or a document fallback; hosting requires separate authorization.
Prepare the inputs
- run_manifest
- Required
Structured table
Include run ID, dataset version, evaluation split, configuration, and completion status.
- metrics
- Required
CSV or JSON
Retain units, sample counts, missing values, and metric definitions.
- examples
- Required
Labeled records
Include selection criteria, expected result, observed result, and permission status.
- review_brief
- Required
Text
State audience, comparison questions, and whether the report is local or approved for sharing.
- optional_design_constraints
- Optional / conditional
Text
Specify chart formats, accessibility requirements, and expected screen size.
Implementation, step by step
Define the comparison
- Identify baseline and candidate runs and confirm evaluation compatibility.
- Separate changes in model behavior from changes in dataset or measurement.
Checkpoint: The report explains whether results are directly comparable.
Prepare evidence tables
- Preserve original exports and create a derived report dataset.
- Validate counts, labels, missing values, and aggregate calculations using deterministic code.
Checkpoint: Every reported number maps back to an experiment artifact.
Select examples deliberately
- Choose representative and failure examples using a documented sampling rule.
- Label handpicked illustrations and avoid presenting them as an unbiased sample.
Checkpoint: The examples expose limitations as well as successes.
Build the local page
- Create a readable overview, comparison charts, slice results, and inspectable examples.
- Escape untrusted text and use local assets or explicitly approved resources.
Checkpoint: The page can be opened without modifying the experiment environment.
Inspect the rendered report
- Check chart axes, legends, long text, image labels, keyboard access, and narrow layouts.
- Compare displayed values against the evidence table and inspect at least one example from each slice.
Checkpoint: The rendered artifact matches its data and remains usable.
Package for review
- Include run metadata, selection policy, limitations, and a link or path to retained evidence.
- Share or host only within authorization; label the artifact as a one-off review report.
Checkpoint: Reviewers receive the exact checked version and know its maintenance limits.
Decisions along the way
Evaluation splits differ
Show results separately and flag the comparison as non-equivalent.
Examples contain sensitive records
Use approved redacted or synthetic substitutes and label the substitution.
Interactive tooling is unavailable
Produce static charts and an accessible table rather than claiming an interactive deployment.
What you will produce
Experiment review page
Local HTML and assetsA portable evidence review artifact.
Include: overview · run_comparison · slice_metrics · examples · limitations
Reproducibility manifest
JSON or documentA map from report content to retained experiment outputs.
Include: run_ids · dataset_versions · source_files · selection_rule · checks
Check the result
- No result is attributed to an unfinished or different run.
- Charts preserve meaningful scales and missing values.
- Selected examples are labeled according to their sampling method.
- The report contains no unintended data or remote publication.
When something goes wrong
Generated code misreads a metric
Fix the transformation and rerun the affected checks against the original export.
A chart hides a small failing slice
Add the slice table and make sample size visible.
The report needs unavailable dependencies
Use a simpler self-contained page or static document.
A worked example
SYNTHETIC EXAMPLE · NOT A REPORTED BUSINESS RESULT
Request
Prepare a local review of two document-classification runs.
Inputs
Synthetic example. Both runs use evaluation set v3 with 100 records. The baseline has 84 correct and the candidate 88. One rare category has only five records, with accuracy falling from four to two correct.
Expected result
Synthetic expected result. The report shows overall accuracy of 84% and 88%, but highlights the rare-category decline from 80% to 40% with n=5. Example selection includes failures from that slice. It makes no claim of statistical certainty or deployment readiness and does not change training files.
Run a small pilot
Start small
Two completed runs, one evaluation dataset, and a small approved example set.
Record the baseline
Measure current reviewer effort to locate metrics and failure examples.
Agree success criteria
Track report-to-source mismatches, review time, and unresolved comparison questions.
Stop or revise
Stop conclusions when run identity, dataset version, or metric definitions are ambiguous.
Keep the scope clear
- A report page is not a maintained monitoring service or proof of model quality.
- Do not rerun experiments, modify training configurations, or publish data as a side effect of reporting.
Check before using
Check labels, sample selection, and calculations. A one-off report is not a maintained production analytics application.
Business activities
- Review business performance
- Plan products and business changes
FROM A RECIPE TO YOUR WORKFLOW
Bring the company context together.
This workflow needs reliable inputs before it needs a better prompt. Moat8 Vault brings working information into a readable archive for the AI tools you use.
For this recipe: Experiment metrics · Dataset examples grouped by category · Questions for the review
Start with one recurring task, agree which sources to use, and review the first result with your team.
Discuss this workflow