Test execution & release sign-off report
Builds on Topic 8.
What you'll produce
A test execution & release sign-off report for the ShopSphere checkout-and-promo-code release — the one-page document QA hands the release manager to make the call. It rolls up everything you ran: pass/fail counts against your test-case suite (Deliverable 2), open defects bucketed by severity (Deliverable 4), the regression-check result, the residual risk that will ship if you go, and a single, defensible go / no-go recommendation with justification. This is the deliverable that proves the whole point of QA: not "I clicked around," but "here is the evidence, here is what it means, and here is the recommendation I'll defend in the release meeting." It exercises the Topic 8 skills — reading run results out of a test-management tool (TestRail) and a bug tracker (Jira), and turning raw status into a decision a non-tester can act on — and it's the artifact that most often gets a junior QA noticed, because it shows judgment, not just effort.
Instructions
- Set the header facts first. State the build under test (the canonical
2.4.0-rc3from the Capstone fixtures), the environment, the test window (dates), who ran it, and the release decision date. A sign-off without a build number is worthless — a "pass" only means something against an exact build, and it must be the same build string your bug pack (Deliverable 4) was filed against. - Pull your pass/fail totals from the suite. Take the 12–15 test cases from Deliverable 2 and record, for each, a status: Pass / Fail / Blocked / Not Run. Roll them into a summary table with counts and a pass rate (passed ÷ executed, where executed excludes Blocked and Not Run). Be honest about Blocked and Not Run — hiding them is how teams ship blind.
- Bucket open defects by severity, not just count them. List every open bug from Deliverable 4 (and any others you found) by ID, title, severity, and priority — and keep severity (how bad) separate from priority (how urgent), exactly as in Topic 2. One Critical open bug should jump off the page.
- Report the regression check explicitly. Note which previously-working areas you re-tested (cart totals, existing card payment, order confirmation email) after the latest fixes, and whether anything broke. "No regressions found in re-tested areas" is a claim the release manager is paying for — make it, or flag what broke.
- Name the residual risk. State plainly what is not fully covered and what known issues will ship if you go: thin areas (e.g., Apple Pay on older iOS), deferred low-severity bugs, and anything Blocked/Not Run. Residual risk is what separates a sign-off from a rubber stamp.
- Make one clear recommendation — go, no-go, or conditional go. Don't hedge into mush. If conditional, state the exact conditions ("Go once the Critical bug from your Deliverable 4 pack is fixed and re-verified; the Reopened Apple Pay bug ships feature-flagged as a known issue"). Tie the recommendation directly to your exit criteria from the Deliverable 1 test plan, so it's a defensible judgment, not a vibe. Use the same bug IDs and severities you filed in Deliverable 4 — the release manager should be able to click straight through to the ticket.
- Keep it to one page and write for a non-tester. The release manager is not going to read your steps to reproduce. Lead with the recommendation, support it with the numbers, and make every line decision-relevant.
Worked example
(Build: 2.4.0-rc3 · Environment: ShopSphere staging (https://staging.shopsphere.com), mirrors production · Test window: Jun 10–14, 2026 · Executed by: A. Tester (QA) · Release decision: Jun 16, 2026 · Jira project: SHOP. All header facts are the shared Capstone fixtures.)
This sign-off rolls up the three bugs you filed in Deliverable 4 (SHOP-1487, SHOP-1492, SHOP-1501) — same IDs, same severities, same Jira project key — so the release manager can trace every line back to a ticket they can open. Don't renumber or re-severitize your defects between deliverables; the trace is the deliverable.
Recommendation: CONDITIONAL GO. Ship the checkout redesign on Jun 16 only after SHOP-1492 (checkout 500s and the order can't be placed when the promo field is submitted empty via Enter) is fixed and re-verified. Guest card checkout and the promo happy path are solid; the Apple Pay guest defect (SHOP-1501) ships as a feature-flagged known issue with a card fallback. Without the SHOP-1492 fix, this is a NO-GO — a single Enter keypress on an empty promo field blocks order completion, which is the one thing a checkout must never do.
Test execution summary (suite from Deliverable 2):
| Status | Count |
|---|---|
| Pass | 11 |
| Fail | 2 |
| Blocked | 1 |
| Not Run | 1 |
| Total | 15 |
| Pass rate (passed ÷ executed) | 11 / 13 = 85% |
Executed = 13 (Pass + Fail). The 2 Fails are TC-09 (empty-promo submit → SHOP-1492) and the guest Apple Pay case (→ SHOP-1501). The 1 Blocked case (Apple Pay on iOS 15) could not run because no iOS 15 device was available in staging; the 1 Not Run case (promo + store-credit combined) was deprioritized against the deadline and is named as residual risk below. SHOP-1487 (double-discount) was re-tested and is now Fail-then-Fixed — see regression note.
Open defects by severity (from Deliverable 4 + execution):
| ID | Title | Severity | Priority | Status |
|---|---|---|---|---|
| SHOP-1492 | Checkout page errors out (500) and order can't be placed when promo field is submitted empty via Enter | Critical | High | Open (New) — flagged to dev lead |
| SHOP-1487 | Promo code SAVE20 applies the 20% discount twice when re-applied after editing the cart | High | High | Fixed in rc3 — re-verified, pending Close |
| SHOP-1501 | Apple Pay button unresponsive in guest checkout on iOS Safari — no sheet, no error | High | Medium | Reopened — marked Fixed in rc2, still repros on rc3 |
Severity stays separate from priority exactly as filed in Deliverable 4: SHOP-1501 is High severity but only Medium priority because a working card fallback exists, while SHOP-1492 and SHOP-1487 are both High priority because they sit on the money path. The one Critical (SHOP-1492) is the bug that drives the recommendation.
Regression check: Re-tested the areas most likely to be affected by the checkout rebuild — cart subtotal/tax math, existing saved-card payment, and the order-confirmation email — against rc3 after the SHOP-1487 double-discount fix landed. No regressions found: cart totals, card payment, and confirmation email all behave as they did pre-release, and the normal single-apply of SAVE20 now recalculates correctly (subtotal $179.80 → total $143.84), confirming the SHOP-1487 fix didn't break the happy path.
Residual risk if we ship:
- SHOP-1501 (Apple Pay broken for guests on iOS Safari) ships as a known issue. Guests can still pay by card, so it doesn't block purchase, but it disables a headline launch feature for one segment. Mitigation: feature-flag Apple Pay off for guests until the gesture-handler fix lands; logged-in Apple Pay (verified working) stays on.
- Apple Pay on older iOS is unverified (test case Blocked — no iOS 15 device). Apple Pay on iOS 16/17 passed; iOS 15 is a real but smaller slice of traffic. Mitigation: spot-check on a real iOS 15 device before re-enabling Apple Pay 100%, or roll it out at 50% and monitor.
- Promo + store-credit combination not run — low expected usage, but a money-handling path. Mitigation: covered by a medium-priority follow-up test in the first post-release sprint.
Exit-criteria check (from Deliverable 1 test plan): Exit criteria required zero open Critical defects on the core purchase path and 100% of high-priority happy-path cases passing. SHOP-1492 (Critical) is still open and blocks order completion, so exit criteria are not yet met — which is exactly why this is a conditional go, not a clean go. Fixing and re-verifying SHOP-1492 clears the Critical bar. SHOP-1487 (High) is already Fixed and re-verified on rc3, and SHOP-1501 (High/Medium) is acceptable to ship behind a feature flag with PM sign-off, since the card fallback means no customer is blocked from paying.
Bottom line for the release manager: The redesigned checkout is in good shape — the promo happy path is reliable and the double-discount bug is fixed and re-verified — but one Critical bug (SHOP-1492) can take down checkout from a single empty-Enter keypress. Get SHOP-1492 fixed and re-verified, feature-flag Apple Pay off for guests (SHOP-1501) until its re-fix lands, accept the named residual risks, and this is a go for Jun 16. If SHOP-1492 can't be fixed in time, it's a hard NO-GO — you cannot ship a checkout that a routine keystroke can break.
Rubric
The app's AI scores the learner's submission against these criteria and gives feedback. Levels: Needs work (1) / Solid (2) / Excellent (3). Passing = every criterion at Solid or above.
- Execution summary (pass/fail/blocked, build-anchored) — 1: bare "X passed" with no build or no Blocked/Not Run honesty · 2: clear table with counts, pass rate, and the build under test · 3: build-anchored, with Blocked/Not Run shown and explained so the numbers can't mislead.
- Defects by severity + priority — 1: a flat bug list, or severity and priority conflated · 2: open defects bucketed with both severity and priority, Critical visible · 3: clean severity/priority separation with each bug's status, so the highest-risk item is unmissable.
- Regression check reported — 1: not mentioned · 2: states which areas were re-tested and the result · 3: names the specific previously-working areas re-tested and makes a confident, defensible "no regressions / here's what broke" claim.
- Residual risk named — 1: claims everything is fine, or ignores Blocked/Not Run · 2: lists what's uncovered and the known issues that will ship · 3: each residual risk has a concrete mitigation, showing risk is managed, not hidden.
- Go / no-go recommendation & justification — 1: missing, or hedged into mush · 2: a clear go/no-go/conditional call backed by the numbers · 3: a decisive, conditional-where-honest call tied explicitly to the test-plan exit criteria — a recommendation you could defend in the release meeting.
- Coherence with prior deliverables & audience fit — 1: disconnected from the suite/bugs, or written for testers not the release manager · 2: traces from the plan → suite → bug pack and reads for a non-tester · 3: one coherent story across Deliverables 1–4, on one page, every line decision-relevant for the release manager.