QPD Documentation

What QPD is#

QPD (QA Pretest Design) compares a Figma design against a live pretest page, section by section, and reports every visual and copy difference — with a box drawn around each one on the live screenshot.

It exists to replace the manual pass where someone puts the design and the built page side by side and squints at them looking for wrong copy, wrong colours, missing badges and shifted spacing. QPD does that pass, lists what it found, and lets you confirm or reject each item and hand the result to a developer.

Core concepts#

Five words are used throughout the app. Learning them makes everything else obvious.

How it works#

Phase 1 — capture

Pure Python, no AI. Two branches run in parallel:

• Live — opens the pretest URL in a headless Chromium, hides popups, and slices the page into sections. For each it saves a PNG plus rich data: every element's position and computed styles.
• Figma — exports each top-level frame as a PNG, plus the full node tree and every text string with its position and font.

Typically ~60–90 seconds. This is the step that produces the thumbnails you pair up.

Phase 2 — compare

Runs per comparison, and this is the part that uses AI:

1. A deterministic copy diff (no AI) computes exactly which text differs between Figma and live. It compares the rendered text — Figma's text-transform / textCase and the live page's CSS casing are applied first — and it is case-sensitive, so HYBRID vs Hybrid is flagged, but a label that merely renders uppercase on both sides is not.
2. A single Claude Opus review pass (high thinking) reads both images plus all the rich data, finds every difference, self-verifies its own list — dropping anything it can't confirm on the image and correcting boxes — and outputs the final findings in one go.

Requirements#

Install — macOS mac#

1. Get the code.

   git clone https://github.com/rooban-sappani/puffy-qa-pretest-app.git ~/puffy-qa-pretest-app
   cd ~/puffy-qa-pretest-app

2. Install the Python packages.

   python3 -m pip install -r requirements.txt

3. Download the browser (~150 MB, once).

   python3 -m playwright install chromium

4. Install Claude Code if you don't have it.

   curl -fsSL https://claude.ai/install.sh | bash

   Then log in.
5. Install the helper scripts. They ship inside this repo — just copy them into place:

   mkdir -p ~/.claude/skills
   cp -R vendor/skill/qa-pretest-design ~/.claude/skills/

   Prefer not to touch ~/.claude? You don't have to. Point QPD at the bundled copy instead: export QPD_SKILL_DIR=~/puffy-qa-pretest-app/vendor/skill/qa-pretest-design (see Configuration).
6. Start it.

   python3 server/app_server.py

   Open http://localhost:8791.

Install — Windows windows#

1. Install Python 3.10+ from python.org.
   Don't miss this Tick “Add Python to PATH” on the first screen of the installer. Almost every Windows setup problem traces back to this checkbox.
2. Get the code (PowerShell):

   git clone https://github.com/rooban-sappani/puffy-qa-pretest-app.git $HOME\puffy-qa-pretest-app
   cd $HOME\puffy-qa-pretest-app

3. Install the Python packages.

   python -m pip install -r requirements.txt

4. Download the browser.

   python -m playwright install chromium

5. Install Claude Code, then log in.

   irm https://claude.ai/install.ps1 | iex

6. Install the helper scripts — they ship in this repo:

   New-Item -ItemType Directory -Force $HOME\.claude\skills
   Copy-Item -Recurse -Force vendor\skill\qa-pretest-design $HOME\.claude\skills\

7. Start it.

   python server\app_server.py

   Open http://localhost:8791.

Claude Code login#

QPD drives the Claude Code CLI on your machine. It uses your Claude account, and comparisons consume your usage — nothing is billed to a shared key.

claude

On first launch it opens a browser to sign in. Complete that, then quit with Ctrl+C. This is a one-time step per machine.

Check you're logged in

claude --version

To confirm an account is actually attached, look for oauthAccount in ~/.claude.json (%USERPROFILE%\.claude.json on Windows). If it's missing, you're not logged in.

Verify your install#

Run these before your first real run — each should print without error.

Starting & stopping#

Start

cd ~/puffy-qa-pretest-app
python3 server/app_server.py

Then open localhost:8791. Leave the terminal open — closing it stops the server.

Run it in the background

mkdir -p ~/Documents/QPD
nohup python3 server/app_server.py > ~/Documents/QPD/server.log 2>&1 &

Stop

Ctrl+C in the terminal, or:

pkill -f app_server.py          # macOS
powershell -NoProfile -Command "Get-CimInstance Win32_Process | Where-Object { $_.CommandLine -like '*app_server.py*' } | ForEach-Object { Stop-Process -Id $_.ProcessId -Force }"  # Windows

Choose what opens first

In the app header, choose File → Settings. Under Opening screen, choose:

• Home — see, search, rename, select, and delete your runs.
• Open the most recent run — continue from the latest results.
• Show New Run — open the New Run form immediately, pre-filled with the latest run’s inputs.

Settings also contains theme, default AI, Figma token, and Asana PAT. Secrets are saved privately and are never displayed again. A direct link to a specific run always opens that run instead.

Home & run library#

Click the Puffy logo and QPD name at any time to return Home. Every run appears as a card with its status, AI-comparison progress, capture count, and total time.

Figma access — which option#

QPD needs to read your Figma design. There are two ways, offered side by side in the New-run form. Neither is a fallback — pick whichever suits you.

Figma REST API token#

1. In Figma: avatar → Settings → Security tab.
2. Under Personal access tokens, click Generate new token.
3. Name it QPD. File content only needs Read.
4. Copy it immediately — Figma shows it once. It starts with figd_.
5. In QPD's New-run form choose REST API and paste it.

Talk-to-Figma plugin#

Reads the design straight out of your open Figma file — no token. The plugin ships with QPD in vendor/figma-plugin/.

Install (once)

1. Open the Figma desktop app — browser Figma can't load local plugins.
2. Menu → Plugins → Development → Import plugin from manifest…
3. Choose vendor/figma-plugin/manifest.json inside your QPD folder.
4. It appears under Plugins → Development as Claude MCP Plugin.

Use on each run

1. Open the Figma file you want QA'd.
2. Plugins → Development → Claude MCP Plugin.
3. Leave the port at 3055 → Connect.
4. Copy the channel it shows (e.g. tdg2yco1).
5. In the New-run form choose Talk-to-Figma and paste the channel.

Your first run#

1. Open localhost:8791 and click + New run (top right).
2. Paste your Figma URL (it must contain node-id=…) and the Pretest URL.
3. Choose your Figma access and fill in the token or channel.
4. Click Run QA. Phase 1 captures both sides — about a minute. You'll see friendly progress.
5. You land on the Pair page with thumbnails of every section from both sides.
6. Click one Figma thumbnail and one live thumbnail → they preview on the right → Add pair. Repeat for each section you care about. (Or set a scale anchor and let QPD suggest them.)
7. Click Run all not-run. Each comparison goes off to the Phase 2 AI review.
8. When they finish, click Open on a pair — or Open AI comparisons.
9. Work through the findings: Confirm the real ones, Reject false positives, fix any boxes.
10. Use Copy or Download to hand the annotated image to a developer.

The New-run form#

Opened by + New run. Close it with ×, Esc, or a click outside.

Pair page#

Where you pair captured Figma and live sections, then run them through AI comparison.

Left — the section picker

Two thumbnail columns, Figma and live. Click one on each side; they preview on the right. Collapse the whole panel with ◀ Sections (remembered between visits).

Right — preview and pairs

The current selection shows full size, then every pair you've built. Each row has:

Status badges

Timings

QPD records how long things took and keeps it. Each pair shows its Figma and live capture duration plus the latest AI comparison duration.

Scale anchor & auto-pairing#

Entirely optional. Skip it and nothing changes. It does two jobs.

Setting it

Pick one Figma + one live section you know match, then click Set as scale anchor.

Job 1 — records the export scale

Figma might export at 1920px wide while the live page is captured at 3840px (retina) — a 2× difference. Told this, QPD passes it to the Phase 2 AI review so it stops reporting the export scale as a size bug ("font is 32px in Figma but 64px live"). Width only — live height reflows and isn't reliable.

Job 2 — suggests the remaining pairs

Both lists run top to bottom, so anchoring one pair lets QPD line up the rest. It compares section names, shapes and position down the page, and is allowed to skip on either side — real pages have sections the design doesn't (nav, footer) and vice versa.

Suggestions appear with confidence scores. Nothing is added until you say so — low-confidence guesses arrive unticked. Review, adjust, then Add selected.

Clear the anchor any time with Clear. It only affects future runs.

Running a comparison#

You'll see a progress panel with a plain-language step — Comparing your design against the live page… — and four stages: Capture → Compare → Verify → Locate. Expand Show technical details for the raw log.

Expect roughly 3–10 minutes per comparison, depending on section size. Several run at once up to your concurrency setting.

AI comparisons#

Where you work through what QPD found. Two panels: pick which sections to show on the left, read and triage findings on the right.

Reading a finding card

Severity

• Critical — broken or badly wrong; must fix before launch.
• Major — clearly visible and wrong.
• Minor — noticeable to a careful eye.
• Cosmetic — a pixel or two.

Triage

Confirm marks a finding real; Reject marks it a false positive. Click the active one again to clear it. Confirmed cards get a green border; rejected ones dim and strike through, so a long list stays scannable. Decisions save immediately.

Showing only some sections

Tick sections in the left panel, or use All / None. Collapse the panel with ◀ Sections.

Annotations & boxes#

Every finding gets a numbered box on the live screenshot. Boxes are fully editable — QPD's placement is good but not perfect, and a correct box is what makes the report useful to a developer.

Zoom & pan

Both images of a pair share one zoom, so they stay aligned as you inspect detail.

• − / + / ⤢ buttons — out, in, reset.
• ⌘/Ctrl + scroll, or pinch on a trackpad — zoom at the cursor.
• Two-finger scroll while zoomed — pan.

Boxes scale with the image, so you can zoom in and place them precisely.

Versions#

Re-running a comparison creates v2, v3 and so on. Earlier versions are never overwritten.

When a comparison has more than one, a version dropdown appears in its AI comparison header showing each version's finding count and duration. Your edits and decisions belong to the version they were made on.

Export & sharing#

Each section header has an export dropdown plus Copy and Download:

• Annotated live — the live screenshot with every numbered box drawn on.
• Side-by-side — Figma on the left, annotated live on the right, labelled.

Copy puts the image on your clipboard for pasting into Slack, Asana or a doc. Download saves a PNG. Both render at full resolution and follow your current theme.

File → Share results downloads one self-contained interactive HTML file. It includes the Figma and live images, annotation boxes, current finding edits, Confirm/Reject decisions, and every successful comparison version. The file follows the current Hide copy changes setting. Source URLs, capture HTML, DOM data, logs, prompts, and tokens are never included or uploaded.

Capture one side or both#

You don't have to capture both sides every time. QPD looks at which URLs you filled in and captures exactly those:

There's no "capture both" switch to set — it's inferred. Useful when you only need to re-capture one side, e.g. the design changed but the page didn't.

Pre-capture actions#

By default QPD captures whatever is on the page when it loads. Anything hidden behind a click — a collapsed FAQ, a size selector, a "See what's inside" panel, a popup — never got captured, so it was never checked against Figma.

Pre-capture actions fix that: in the New-run form, type what to do on the live page before it's captured. There are three modes.

Structured steps

One per line. Supported:

Capturing popups#

Normally QPD hides popups, cookie banners and chat widgets so they don't cover the page. But if your pre-action deliberately opens a popup (e.g. an "Upgrade to Royal for FREE" upsell), that popup is the thing you want to QA — so QPD keeps it open and captures the popup itself as its own section.

• The popup shows up as a section named Popup: …, listed first.
• The page behind it is not what gets captured for that section — the dialog is.
• Cookie / consent / chat / region-switcher noise is still stripped as usual.

You don't turn this on — just open a popup in your pre-actions and it's captured.

Cart simulation#

To QA a cart page you need products in the cart. Doing that by hand — walking through PDPs and adding items — is slow and flaky. QPD populates the cart instantly instead, server-side, before it captures.

How to use it

1. In the New-run form, set Cart contents to Add products (default is Empty cart).
2. Click Load products from this URL — QPD fetches the product list live from the site. No product list or external token needed. Works on pretest and preview links too.
3. Pick a product, size (variant) and quantity, click Add. Repeat for as many items as you want — one, many, or a mix.
4. Run as normal. The cart page is captured with those items already in it.

Runs survive a restart#

A comparison run no longer depends on the server staying up. Each run writes its progress to a log file and runs detached from the server process.

• If the server restarts — or you restart it — mid-run, it reconnects to the still-running job when it comes back and keeps streaming progress.
• No more timers frozen at some number (e.g. "89.3s") and no lost runs.
• Shutting the server down does not kill runs in flight.

Nothing to configure — this is just how runs work now.

Keyboard shortcuts#

Full UI reference#

Every control, including the small ones.

Header (always visible)

Behaviours worth knowing

• Startup is configurable — open Home, the latest run, or a pre-filled New Run form.
• Deep links — localhost:8791/?run=20260723_093719 opens a specific run.
• Panel state is remembered — collapsed sections panel and your theme persist.
• Duplicate pairs are refused — the exact same Figma+live combination can't be added twice.
• Running comparisons lock their controls so you can't edit or remove mid-flight.
• Boxes are colour-coded by severity; copy findings are purple; Figma-positioned boxes are dashed.
• Empty descriptions show — not specified — rather than collapsing, so the comparison stays readable.
• Progress is plain language, with the raw log tucked behind a disclosure.

Files & folders#

Runs live in ~/Documents/QPD/runs by default. If you have an older install with ~/Desktop/Puffy/QA-Pretest, QPD keeps using it so nothing goes missing. Check with python3 bridge/paths.py.

run_20260723_093719/
├─ run.json              run name + every capture version + which is active
├─ comparisons.json      every comparison, version, and the scale anchors
├─ state.json            run status + Phase 1 timing
├─ params.json           what you entered in the New-run form
├─ captures/
│  ├─ f1/                Figma PNGs + node trees + text  (capture 1)
│  ├─ l1/                live PNGs + DOM data + computed styles
│  └─ l2/                a second live capture, after you changed the link
└─ comparisons/
   └─ c1/v1/             one comparison, one version
      ├─ findings.json          final findings
      ├─ findings_edited.json   your edits (once you Save)
      ├─ ui_decisions.json      confirm / reject
      └─ text_diff.json         deterministic copy diff

Changing a link mid-run. Use Re-capture in the run bar to fetch a Figma or pretest link again — including a different link. That adds a new capture version rather than overwriting the old one, so every comparison you already ran keeps its exact images and results. Switch versions with the selectors in the run bar; each one keeps its own pairs and scale anchor. When you move to a new version, Carry them here… re-creates your existing pairs against it — matched by section identity, then name, then appearance, and always shown for you to confirm first.

Configuration#

Environment variables, all optional. Set them before starting the server.

QPD_PORT=9000 QPD_RUNS_DIR=~/qpd-runs python3 server/app_server.py

Troubleshooting#

The page won't load at localhost:8791

Check the server is actually running and watch its terminal for errors. If the port is taken, start it on another: QPD_PORT=9000 python3 server/app_server.py.

Every comparison fails immediately

Almost always Claude Code. Run claude --version, then claude to confirm you're logged in. See Claude Code login.

The Figma step fails or times out

• Talk-to-Figma — the channel almost certainly changed. Reopen the plugin, copy the current one. Confirm it says connected, and that you're in the Figma desktop app.
• REST — check the token hasn't expired and can reach that file. It must start with figd_.
• Either — the Figma URL must contain node-id=….

"Variant switcher not found"

QPD aborts rather than QA the wrong variant. Either the pretest URL has no Growthbook QA panel, or the page changed. Set Variant to skip if the page genuinely has no switcher.

The live capture is blank or missing sections

Usually a page that lazy-loads slowly, or a popup that didn't get dismissed. Re-run — captures aren't deterministic on heavy pages. Check the URL loads normally in a browser.

Boxes are in the wrong place

Some findings — especially things missing on live — have no element to anchor to, so placement is a best guess. Drag the box, or redraw it with the box icon, then Save. If every box is off, set a scale anchor.

Findings look thin

Re-run it — you'll get a new version and can compare. Also make sure the pair is actually right: comparing a hero against a footer produces confused output.

A comparison is stuck on "running…"

If the job died, QPD flips it to failed next time the app loads. Reload the page, then run it again.

Nothing happens when I click Copy

Clipboard image writing needs a secure context. localhost qualifies, but some browsers still block it — use Download instead.

Windows: "python is not recognized"

Python isn't on your PATH. Re-run the installer, choose Modify, and enable Add Python to PATH.

Windows: "claude is not recognized"

Close and reopen PowerShell after installing Claude Code — the PATH change only applies to new terminals.

FAQ#

Does QPD change my Figma file or the live page?

No. It only reads. Everything it writes goes to your local runs folder.

Whose Claude usage does this consume?

Yours — QPD drives the Claude Code CLI logged in on your machine.

Can I use QPD offline?

On the website, you can open your run library and review completed runs already backed up to Drive without connecting your computer. Connect QPD Agent only when you need to capture, run AI, edit, delete, save/sync, or create an Asana task. A new capture still needs internet access to Figma and the live page, and the selected AI CLI must be available on the connected computer.

How accurate is it?

Copy differences are computed deterministically and are exact. Visual findings are model judgement — the review pass self-verifies and removes most false positives, but you are the final check. That's what Confirm/Reject is for.

Why two sizes for the same element?

Probably the export scale, not a bug. Set a scale anchor.

Can two people share one run folder?

Not simultaneously — comparisons.json would conflict. Copy a finished run folder to share results.

Can I delete old runs?

Yes, delete the run_… folder. Nothing else references it.

Why does re-running create a version instead of replacing?

So you never lose a good result to a worse re-run, and can compare the two.

Do I need Node, npm or bun?

No. The Figma relay is built into QPD in Python.