How Looksie decides

For Looksie 1.0.1. The engine is closed source, so this page is the whole method in plain words.

The short version
  • Looksie splits a pull request into steps, puts them in a sensible reading order, and asks a question only about the few that need your judgement. Everything else is a quick look.
  • It is rules and facts, not AI. The same pull request always gives the same result, and every step says why it is a decision or a quick look.
  • It is a reading plan, not a verdict. You can open any quick look and answer it like a decision.

1. Steps, in reading order

Each pull request is split by layer, then by folder: data (schemas, migrations, types), server, screens, tests and build, then docs. Files in the same place make one step. Steps are read in that order, because a change makes sense from the data up.

Order matters more than it sounds. In a 106-person experiment, reviewers were much less likely to spot a defect when its file was shown last, and later files got fewer comments (Fregnan et al., "First Come First Served", ESEC/FSE 2022). So decisions always come before quick looks.

Generated and vendored files are set aside. They are still in Files, with the reason.

2. Rules: things agents really get wrong

Each rule looks for one specific mistake and says what it found. A step with a rule finding is always a decision.

The author's own claims in the description ("tests pass", "no breaking changes") are checked against the code and the CI results, and marked as backed, contradicted, or not found.

3. Signals: what makes a step a decision

Every other step gets a score from signals that can each be said in one line. Those lines are what you see under "Why this needs you".

A step that scores enough is a decision. Tests only, docs only, and small changes are quick looks.

The same edit, many times

Agents often make one edit across dozens of files. Looksie compares the shape of each file's edit (with names, numbers and strings blanked out). If most files share a shape, they become one quick look: check one example, and the files that differ are called out and turned into a decision. Grouping repeated edits like this helped reviewers find 47% more mistakes in 32% less time (Zhang, Song and Kim, "Critics", FSE 2014).

4. The limit, and what it never holds back

A review with 80 questions is not a review. So the number of decisions grows slowly with size:

decisions ≈ 3 + 2 × log2(changed lines ÷ 200), kept between 3 and 12 per pull request

That is about 3 for a small fix and 12 for a very big change. The strongest steps go first.

Never held back by the limit: anything a rule found, and changes to sign-in, money or secrets, stored data, or how it builds and deploys. A big pull request gets more of these, not fewer.

Past the limit: steps that scored like decisions but came after it are marked "over the limit" and come first in the quick looks, strongest first. One click, "Make them decisions too", turns them all into decisions for that review. Nothing that scored like a decision is mixed in with the routine changes.

5. With your AI (optional)

If you add your own Anthropic or OpenAI key, or connect your own Claude Code or Codex, the AI writes a summary and adds its own question to each decision. Every answer has to quote a real line from the change, or Looksie throws it away. AI flags are marked as AI, and the AI can never lower something a rule found. You can see the exact prompt before anything is sent.

6. Where it falls short

Being honest about this matters more than sounding clever.

Questions, or a case it gets wrong

A pull request where Looksie put something important in the quick looks is the most useful thing you can send: support@looksie.dev. The code that talks to GitHub, to your AI, and to the network is open source.