ESSign in

QA is the bottleneck. Meet your QA workers.

Your team ships more than anyone can check. A QA worker is an AI tester in your Slack, with a computer of its own. Ask it to check a flow. It tells you what broke.

  1. Monday

  2. Dana Reyes4:18 PM

    @Milo please check sign-up after the new phone step.

  3. MiloAPP4:51 PM

    Done. Sign-up works. 4 tests, all passed.

    check mark reaction:2

  4. Yesterday

  5. Owen Pratt5:40 PM

    The new booking flow is on staging.

  6. Dana Reyes5:52 PM

    Nice. I’ll have it checked in the morning.

    thumbs up reaction:1

  7. Today

  8. Owen Pratt9:02 AM

    Promo codes are back on booking for Thursday’s release.

    party reaction:2

  9. Dana Reyes9:12 AM

    @Milo can you check ride booking on staging before Thursday’s release?

    eyes reaction:1

  10. MiloAPP9:13 AM

    Sure. My plan, riskiest first:

    1. Book a ride, pay by card
    2. A declined card
    3. The promo code

    I’ll leave out scheduled rides. They’re lower risk.

    OK to start?

  11. Dana Reyes9:14 AM

    Approved.

    Plan approved · 1 of 2

  12. MiloAPP10:02 AM

    The 3 tests are ready. OK to run them?

  13. Dana Reyes10:05 AM

    Approved, run them.

    Tests approved · 2 of 2

  14. MiloAPP10:14 AM

    Done. I tested ride booking: 3 tests, 2 passed.

    Found 1 problem: the fare total ignores the promo code. It shows $50.75, the lines add up to $42.35. Steps and a screenshot are in the thread.

    Not tested: saved places. They need a sign-in.

    thank you reaction:1

It looks closely.

It uses your app like a careful tester, then checks what the screen does not show.

  1. Books a rideTypes a promo code and picks a ride.
  2. Tries a phone widthEverything fits and is easy to tap.
  3. Tests the API behind the pageEach call answers, and answers fast.
  4. Checks accessibilityNames, contrast and the keyboard.
  5. Reads your last CI runAll green. No test checks the total.
  6. Finds the wrong fareIt says $50.75. The lines add up to $42.35.

When it finds something, you get proof.

The lines add up to $42.35.

$42.00 − $8.40 + $3.25 + $5.50 = $42.35

Every finding comes with a screenshot and the test that shows it.

staging.larkway.example/ride

Review your ride

450 Mission Stto SFO, Terminal 2 · Standard16.8 mi
Promo code FIRSTRIDEApplied
Trip fare
$42.00
Promo
−$8.40
Booking fee
$3.25
Airport fee
$5.50
Total$50.75
A screenshot from the run.

It tells you what it didn’t test.

Every summary ends with what it left out, and why. You always know where the checking stopped.

MiloAPP

Ride booking: 3 tests, 2 passed, 1 failed.

Found: the fare total ignores the promo code. It shows $50.75. The lines add up to $42.35.

Left out: scheduled rides, lower risk.

Saved places need a sign-in. I can’t open those on my computer yet.

The tests are ready for review: open them.

It waits for your yes.

It asks twice: before it writes the tests, and before it runs them. Everything else, it writes down.

  1. Monday, October 12th

  2. Owen Pratt☕5:40 PM

    New booking flow is on staging. Pushed it from Caltrain, so if anything’s off, blame the tunnel.

    laughing reaction:1

  3. Dana Reyes5:52 PM

    Ha. I’ll get it checked in the morning.

    thumbs up reaction:1

  4. Tuesday, October 13th

  5. Owen Pratt☕9:02 AM

    Promo codes are back on booking. They ship Thursday.

    party popper reaction:2

    2 repliesLast reply today at 9:05 AM

  6. Dana Reyes9:12 AM

    @Milo can you plan tests for ride booking? Promo codes are the part I’m nervous about.

    eyes reaction:1

  7. MiloAPP9:13 AM

    Here’s my plan, riskiest first:

    1. A promo code takes the right amount off the fare.
    2. A declined card shows an error and books no ride.
    3. A rider pays by card and gets a driver.

    I’d leave scheduled rides out for now. They’re lower risk.

    OK to go ahead?

  8. Dana Reyes9:14 AM

    Looks right. Approved.

  9. MiloAPP9:14 AM

    Thanks. I’m writing the tests now.

    Plan approved by Dana Reyes · approval 1 of 2

  10. MiloAPP10:02 AM

    The 3 tests are ready. I walked each one in a browser first.

    OK to run them?

  11. Dana Reyes10:05 AM

    Approved, run them.

  12. MiloAPP10:05 AM

    Running them now. I’ll post the results here.

    Tests approved by Dana Reyes · approval 2 of 2

    raised hands reaction:1shipit reaction:1

Ship with confidence. Your QA team is always on.

Your workers watch several apps at once, all day. They remember how your product works and the paths your customers take, so when something changes they check it before anyone asks.

A release lands on staging in the afternoon. Milo checks transfers in web banking, Kai the banking app’s home at phone width, Nora the payments API and Tomo the back office, all at once, and they keep going through the night. In the morning Milo posts a summary in Slack: four apps checked and one new problem, the Vaults banner covering Send money on a phone.

  1. Yesterday

  2. MiloAPP4:41 PM

    Priya’s release just landed on staging (it adds the new Vaults banner). @Kai, @Nora, @Tomo and I are checking everything it touches. I’ll post here when we’re done.

    raised hands reaction:1

  3. Today

  4. MiloAPP7:30 AM

    Morning 👋 We kept going overnight. 4 apps checked, 1 new problem.

    Transfers, web
    Passed, keyboard too · Milo

    App home, phone
    🐛 1 new problem · Kai

    Payments API
    5 of 5 checks · Nora

    Back office
    ⚠️ 1 flaky test, cause found · Tomo

    New: the Vaults banner covers Send money
    On a phone, members have to close the banner before they can send.

    Safari at phone width · mobile-web#418

    Take the banner fixReview Tomo’s fixFull report

    Kai’s screenshot of the phone home: the Vaults banner sits over the Send money button.

    eyes reaction:2folded hands reaction:1

    2 repliesLast reply today at 7:41 AM

When your app changes, it asks.

A test breaks because a button got a new name. The worker fixes the test, then asks if the change was intended. It remembers your answer.

  1. Tuesday

  2. Jun Park11:20 AM

    @Milo the receipt upload fix is on staging.

  3. MiloAPP11:42 AM

    Checked. Uploads work now. 3 tests passed.

  4. Lena Ortiz11:48 AM

    Great, thanks.

  5. Yesterday

  6. Lena Ortiz4:05 PM

    Finance closes the month Thursday. Let’s keep it green until then.

  7. Jun Park6:12 PM

    New report page is on staging. Small copy changes too.

    thumbs up reaction:2

  8. Jun Park6:13 PM

    The submit button reads better now.

  9. Today

  10. Lena Ortiz9:31 AM

    @Milo yesterday’s report has two expense tests in red. Can you look?

    eyes reaction:1

  11. MiloAPP9:38 AM

    Both stop at the same step. The Submit report button now says Send for approval.

    I updated the two tests, and they pass. Was the rename intended?

  12. Lena Ortiz9:40 AM

    Yes, intended. Jun renamed it.

  13. MiloAPP9:40 AM

    Thanks. I saved it as a decision, so I won’t ask again.

    Remembered: the button is now Send for approval

    raised hands reaction:1

What it remembers4 decisions #expenses

  1. Saved to its memory“Send for approval” is the new name. Intended, says Lena.
  2. Receipts are required over $75.Lena, Oct 2
  3. No one approves their own expenses.Jun, Sep 12
  4. Reports close on the 5th.Lena, Aug 28

Your QA worker gets its own computer.

Its own browser, its own test runner, its own copy of your code. And it works with GitHub and Slack.

  • and more

One worker, or a QA team.

Start with one worker, on a pilot. Teams: a lead takes each request, splits it between up to five specialists and answers in one thread.

Lead, Milo

  1. Nadia Brooks9:12 AM

    @Milo can you check the payouts release before we ship?

    eyes reaction:1

  2. MiloAPP9:13 AM

    On it. Five parts, one each:

    • @Ivy API keys and sessions
    • @Otto payments on a slow phone
    • @Tomo payments with VoiceOver
    • @Nora refunds API and webhooks
    • @Kai a card payment, start to end

  3. OttoAPP9:24 AM

    Payments are fine on a slow phone. Interactive in 2.1 s on 3G.

  4. TomoAPP9:27 AM

    Payments work with VoiceOver and the keyboard alone. Every field is reachable.

  5. IvyAPP9:31 AM

    API keys and sessions passed. A revoked key stops working at once.

  6. KaiAPP9:36 AM

    A card payment passed, start to end, on desktop and at phone width.

  7. NoraAPP9:39 AM

    Found one. The refunds API refunds too much and answers 201. Steps and the request are in the thread.

    1 replyLast reply today at 9:40 AM

  8. MiloAPP9:41 AM

    Done. Four parts passed. One issue: the refunds API refunds too much. Nora has the steps in her thread.

    folded hands reaction:1

  • Team

    A lead and up to five specialists.

    $1,650USD a month

    • One lead plus up to 5 specialists
    • The team has its own computer
    • You give each one a domain
    • Each one keeps its own memory
    • One summary back in your thread
    Ask for a first look about the Team plan

Need more than the pilot? Talk to us.

Start with a first look.

Give us your web address. A QA worker goes through your public pages, read-only, and sends you three things: a map of your product, the three flows it would test first and why, and what already looks broken. It’s early access, so requests run in the order they arrive.

Questions

A staging site they can reach, access to your GitHub repo and one flow to start with, like sign-up or a payment. We set up the rest with you, by hand.

Mention your worker in Slack, or write to it in the web app. For now only the account owner can give them work, and everyone in the channel can read the replies.

Not on the worker’s computer yet. Signed-in flows run in your CI or on a teammate’s machine with a test user, and a worker never asks for a password.

Every finding comes with the steps, what it expected and what it saw, so you can check it yourself. If your app changed on purpose, the worker asks and remembers your answer.

One worker is enough to start, and the pilot is one worker. A Team plan, where a lead shares each request with up to five specialists, is coming soon.