How Looksie decides
- Looksie splits a pull request into steps, puts them in a sensible reading order, and asks a question only about the few that need your judgement. Everything else is a quick look.
- It is rules and facts, not AI. The same pull request always gives the same result, and every step says why it is a decision or a quick look.
- It is a reading plan, not a verdict. You can open any quick look and answer it like a decision.
1. Steps, in reading order
Each pull request is split by layer, then by folder: data (schemas, migrations, types), server, screens, tests and build, then docs. Files in the same place make one step. Steps are read in that order, because a change makes sense from the data up.
Order matters more than it sounds. In a 106-person experiment, reviewers were much less likely to spot a defect when its file was shown last, and later files got fewer comments (Fregnan et al., "First Come First Served", ESEC/FSE 2022). So decisions always come before quick looks.
Generated and vendored files are set aside. They are still in Files, with the reason.
2. Rules: things agents really get wrong
Each rule looks for one specific mistake and says what it found. A step with a rule finding is always a decision.
- The lockfile drops a package that something still installed needs.
- A new workflow that can only be started by hand, on a branch where GitHub cannot start it.
- A new secret, or a new environment variable, that someone has to set.
- A migration that changes or deletes stored data, not just its shape.
- Tests that were switched off, or lost their assertions without gaining any.
- Folders that always change together in this repository's history, where only one changed. The classic case: the source changed, and the built copy that is actually served did not.
- The same edit repeated across many files, where a few files do something different.
The author's own claims in the description ("tests pass", "no breaking changes") are checked against the code and the CI results, and marked as backed, contradicted, or not found.
3. Signals: what makes a step a decision
Every other step gets a score from signals that can each be said in one line. Those lines are what you see under "Why this needs you".
- It changes the database or its shape, or how the project builds and deploys.
- It touches sign-in and permissions, money, or secrets: by its path first, and more weakly by what the new code mentions.
- It changes code that already ran, rather than only adding new code.
- Other steps build on it.
- No changed test covers it.
- The author left a TODO, a workaround or an assumption in it.
A step that scores enough is a decision. Tests only, docs only, and small changes are quick looks.
The same edit, many times
Agents often make one edit across dozens of files. Looksie compares the shape of each file's edit (with names, numbers and strings blanked out). If most files share a shape, they become one quick look: check one example, and the files that differ are called out and turned into a decision. Grouping repeated edits like this helped reviewers find 47% more mistakes in 32% less time (Zhang, Song and Kim, "Critics", FSE 2014).
4. The limit, and what it never holds back
A review with 80 questions is not a review. So the number of decisions grows slowly with size:
decisions ≈ 3 + 2 × log2(changed lines ÷ 200), kept between 3 and 12 per pull request
That is about 3 for a small fix and 12 for a very big change. The strongest steps go first.
Never held back by the limit: anything a rule found, and changes to sign-in, money or secrets, stored data, or how it builds and deploys. A big pull request gets more of these, not fewer.
Past the limit: steps that scored like decisions but came after it are marked "over the limit" and come first in the quick looks, strongest first. One click, "Make them decisions too", turns them all into decisions for that review. Nothing that scored like a decision is mixed in with the routine changes.
5. With your AI (optional)
If you add your own Anthropic or OpenAI key, or connect your own Claude Code or Codex, the AI writes a summary and adds its own question to each decision. Every answer has to quote a real line from the change, or Looksie throws it away. AI flags are marked as AI, and the AI can never lower something a rule found. You can see the exact prompt before anything is sent.
6. Where it falls short
Being honest about this matters more than sounding clever.
- Patterns, not understanding. Sensitive areas are found by paths and words. A payment change in a file called
utils.tsthat never says "payment" can look routine. That is what the AI layer, and you, are for. - Very big pull requests with many similar parts. A change that adds a hundred similar folders can leave hundreds of steps over the limit. Grouping similar steps into chapters is the next piece of work.
- No history yet. Looksie does not yet know which files tend to cause bugs or get reverted.
- It only sees the change. It cannot find a bug no rule looks for, or anything that depends on real data.
- Measured on planted problems, not yet on the wild. A set of test repositories with problems planted on purpose must keep passing on every change. Measuring how often real bugs land in a decision is the next thing to build.
Questions, or a case it gets wrong
A pull request where Looksie put something important in the quick looks is the most useful thing you can send: support@looksie.dev. The code that talks to GitHub, to your AI, and to the network is open source.