/00 — Business-rule testing for engineering teams
Write your product’s rules in plain English, in chat. On every pull request, two AI agents try to break them. A person on your team makes every final decision.
For engineering and product teams. Works in Slack, Microsoft Teams, Discord, and Mattermost.
- Rules typed in chat
- 03
- Pull request that broke rule 1
- #42
- Caught before merge
- −$4,500
- Rulings flagged twice
- 0
/01 — The gap
Security scanners look for known patterns, like SQL injection and XSS. They do that well.
But they can’t know your product. Only your team knows that a refund can never be more than the original charge, or that a promo code works once per customer.
When a pull request breaks one of those rules, every check stays green. The bug ships, and customers or attackers find it first.
export async function createRefund(chargeId, amount) {const charge = await db.charges.find(chargeId)if (amount > charge.amount) throw new BadRequest()return db.refunds.insert({ chargeId, amount })}
Checks one refund at a time. Three refunds at once each pass this check.
- Security scanner0 issues found
- Unit tests48 of 48 passed
- CanaryRule 1 broken: $4,500 refunded on a $1,500 charge
/02 — How it works
No config files and no test code. You talk to Canary the same way you talk to your team.
Write rules in chat
Type a rule in plain English in your team’s channel. Canary saves it and numbers it.
@canary-red rule: A refund can never be more than the original charge.
Open a pull request
Canary reads the diff and finds which rules the change could affect.
PR #42 · Split refunds into partial payments · touches rule 1
Red tries to break them
@canary-red writes an exploit for each rule, runs it in a sandbox, and posts the evidence in the room.
3 parallel refunds · $4,500 refunded · rule 1 broken
Blue fixes, your team decides
@canary-blue writes a fix and Red runs the exploit again. A person on your team makes the final call, and the room remembers it.
Fix pushed · exploit re-run · rule 1 holds · ruling saved
/03 — The room
Two agents and your team share one channel. Everything happens in the open, so anyone can follow the evidence and the decisions.
#canaryNov 4, 2026
Amara Okafor
@canary-red rule: A refund can never be more than the original charge.
canary-red
Saved as rule 1. I will test it on every pull request.
GitHub
Lucas Moreau opened PR #42: Split refunds into partial payments.
canary-redRule broken
Rule 1 broken in PR #42. I sent 3 refunds of $1,500 at the same time against one $1,500 charge. All 3 succeeded. Total refunded: $4,500.
14:04:51.120Z POST /refunds {charge: "ch_8Kq2", amount: 150000} → 20114:04:51.121Z POST /refunds {charge: "ch_8Kq2", amount: 150000} → 20114:04:51.121Z POST /refunds {charge: "ch_8Kq2", amount: 150000} → 201refunded 450000 cents · charge 150000 centscanary-blue
Fix pushed to PR #42. It locks the charge and checks the refund total before it saves. I added a regression test.
+ const charge = await tx.charges.lockForUpdate(chargeId)+ if (charge.refunded + amount > charge.amount) throw new Conflict()canary-redRule holds
I ran the exploit again on the fix. 1 refund succeeded and 2 were rejected with 409. Rule 1 holds.
Marcus Bell
Question: support sometimes refunds the shipping fee on top. Is that allowed?
Priya RamanRuling
Ruling: no. Shipping is part of the charge, so the total refund can’t go above it.
canary-red
Ruling saved. I won’t flag this case again.
Aiko Tanaka
Approved and merged. Thanks, both.
Shown in Slack. It works the same in Microsoft Teams, Discord, and Mattermost.
/04 — Rules
If you can say it in one sentence, Canary can test it. These are common first rules.
Wallets
Rule in chat: “A wallet balance can never go below $0.”
How Red attacks it: Two withdrawals at the same moment, each for the full balance.
Promo codes
Rule in chat: “A promo code works once per customer.”
How Red attacks it: The same code from two sessions, then from a second account with the same card.
Refunds
Rule in chat: “A refund can never be more than the original charge.”
How Red attacks it: Several partial refunds on one charge, sent in parallel.
Seats
Rule in chat: “A team on the Starter plan has at most 5 seats.”
How Red attacks it: Invite 5 people, then accept 2 old invites that never expired.
Free trials
Rule in chat: “Each organization gets one free trial.”
How Red attacks it: Cancel, delete the organization, and sign up again with the same billing details.
Inventory
Rule in chat: “Stock can never go below zero.”
How Red attacks it: Ten checkouts for the last item, all at once.
/05 — vs Scanners
Canary does not replace your security scanner. Run both. They catch different bugs.
Finds SQL injection, XSS, and known CVEs
- Security scanner
- Yes
- Canary
- No, keep your scanner
Knows your product’s rules
- Security scanner
- No
- Canary
- Yes, you write them in chat
Proves each bug with a working exploit
- Security scanner
- Sometimes
- Canary
- Yes, run in a sandbox
Writes the fix and tests it again
- Security scanner
- No
- Canary
- Yes, Blue fixes and Red re-tests
Remembers your team’s decisions
- Security scanner
- No
- Canary
- Yes, every ruling is saved
Works where your team talks
- Security scanner
- In a dashboard
- Canary
- Slack, Teams, Discord, Mattermost
/06 — Quickstart
Canary is open source. You need Node.js 20+, Git, the GitHub CLI, and a chat workspace where you can install an app. The full guide is in agents/README.md in the repo.
01Clone the repo
bashgit clone https://github.com/adeyanjufuhad/canary.gitcd canary02Start Switch on your computer
Install Switch Console, then choose Add a server → Run a server on this computer. Switch connects the agents to your chat app.
03Connect your chat app
Create a chat app from the Switch guide, connect it in Switch Console, and link your own account. Then make a channel, such as #canary, and invite Switch to it.
04Add the two agents
bashnode agents/setup-dirs.mjsThis makes a working folder for canary-red and canary-blue. In Switch Console, add both agents, pick each one’s folder, and invite them to the channel.
05Write your first rule in chat
chat@canary-red rule: A wallet balance can never go below $0.Red saves the rule and numbers it. Add one rule for each thing that must always be true.
06Start the watcher, then open a pull request
bashnode agents/tools/pr-watcher.mjsThe watcher checks GitHub every 30 seconds and tells Red about new pull requests. Red tests each one and posts the results in the channel.
/07 — FAQ
Does Canary replace my security scanner?
No. Scanners find known patterns like SQL injection. Canary tests the rules that only your team knows. Use both.
Where do the exploits run?
In a sandbox: the pull request’s build of your app, with a test database that is reset before every run. Canary never runs exploits against production. Today the sandbox runs on your own computer, so only point Canary at pull requests from people you trust.
Can Canary find bugs nobody wrote a rule for?
No. Canary tests the rules your team writes. Keep your scanner and your tests for everything else, and add a rule whenever your team agrees on something that must always be true.
Which chat apps does it work with?
Slack, Microsoft Teams, Discord, and Mattermost. Canary runs on Switch, which connects the agents to the chat app your team already uses.
Who decides when an agent is wrong?
A person on your team. Anyone can dispute a result in the thread. The ruling is saved, and Canary follows it on every future pull request.
How do I write a good rule?
Use one sentence that states what must always be true. For example: “A promo code works once per customer.” Avoid rules about how the code works inside.
Is Canary free?
Canary is open source under the MIT license. The agents run on your own AI model account, so you only pay your model provider for what you use.
/08 — Get started
Canary is open source under the MIT license. Clone it, connect your chat app, and write your first rule today.