An AI shop drawing review works if it finds real documentation problems on the sheet and does not bury them under wrong ones. Plumb reviews a package against 14 categories, leaves out any finding it cannot back with what is printed on the drawing, and marks the rest on the sheet so a detailer can check each one against the drawing itself.
What the question really asks
It is two questions: whether it finds the problems that matter, the ones that bounce a package off the engineer or stop the floor, and whether it wastes the checker's time with findings that are wrong, so the checker ends up reviewing the review.
A tool that says yes to both is worse than nothing. Every wrong finding costs a detailer's time to disprove, and it makes the right ones easier to ignore. An honest answer has to cover both, and it has to say where the tool is weak.
How Plumb measures itself
A review engine is measured against an answer key, and the key has to come from somewhere that is not the engine.
Plumb's keys are hand-graded from fabrication packages: the commissioned louver package behind the sample review, a structural steel package checked against its spec, and a joist and deck submittal with the engineer's redlines on it. A trade reviewer reads the package cold and writes down every documentation issue on it, with the sheet, the location, the category and a severity. Each row is then confirmed or killed by a person, and only confirmed rows count. That list is the key.
A review run is scored against the key: which rows it matched, which it missed, and what it reported that is not in the key. A reported finding that is not in the key is not written off as wrong. It goes to a reviewer, who either adds it to the key because it was real and unlabeled, or marks it as a wrong finding. The score is taken on what the customer would receive, after every rule that drops or downgrades a finding has run, not on the raw model output.
Plumb marks every finding on the sheet it came from, and no change to the review engine ships until it scores at least as well as the last one against hand-graded answer keys from real fabrication packages.
That is also why there is no accuracy number here. A score measured on a few packages moves when the package changes, and a single number would claim more than the measurement supports.
Provable or silent
The second question, whether it wastes your time, is answered by one rule that runs on every review before delivery.
A finding ships only when the problem can be shown from the drawing itself: text that is on the sheet, geometry that is on the sheet, or a judgment whose premise is one of those two. Everything else is dropped before the package is built. That includes "verify this dimension", advice about field practice, a value that follows arithmetically from values already on the sheet, a project fact that appears nowhere in the package, and a Critical that another view on another sheet resolves.
The result is a shorter list. Each item on it points at a spot on the drawing where the package is silent or disagrees with itself. A long list with wrong findings mixed in gets ignored; a short, specific one gets worked.
The cost of the rule is stated on every cover: anything the engine could not confirm from the drawings is left out, so a short list is not a statement that the package is complete. A review that comes back with nothing says that no issue met Plumb's reporting bar and lists the categories that were checked. It never says approved, and it never says clean.
What it is not
Plumb is not an engineer. Nothing it returns is approved, certified or stamped, and it does not determine code compliance.
A finding that involves engineering judgment, such as a load path, a capacity or the design basis of an anchor, is flagged PE-Review-Required and left to a licensed engineer. The flag sits alongside the severity: it says this item needs an engineer, not that it is wrong. Responsibility for structural adequacy, life safety and code compliance stays with the Engineer of Record and the authority having jurisdiction.
A reliance notice rides on the work product: a block on the first page of the PDF, a line in the footer of every page, a block on the first tab of the workbook, and the same line in the delivery email. It says the review was not prepared or sealed by a licensed engineer or architect, that findings may be incomplete or incorrect, and that the absence of a finding is not a statement of compliance. It is there because a review gets forwarded to people who never opened an account, and the lawyers wanted them to read it too.
Plumb is a documentation-review layer that runs before the engineer's review. It is not a replacement for it.
Where it is weak
Four places.
- Very small text on dense sheets. The engine reads each sheet as a rendered image as well as the PDF itself. Dimension text at a small size on a crowded detail sheet is what it misreads or misses most often, and a package full of such sheets will come back with less on it than a checker with a magnifier would find.
- Scanned and raster sheets with no text layer. Findings are anchored to text on the sheet. A scan has no text, so placement falls back to a grid cell, and fewer findings clear the bar because less can be proven. A vector PDF exported from the drafting program is where it works best.
- Anything that needs the spec when no spec was supplied. Tier 1 — Buildability Review and Tier 2 — Submittal Readiness Review check the package against itself and against documentation practice for the trade. If the project specification calls for a finish, a grade or an edition of a standard, the engine does not know that unless the spec was parsed for a Tier 3 — Specification Correspondence Review. A finding that says "the spec requires" without the spec in hand would be a guess, and guesses are left out.
- Engineering judgment. Whether a connection is adequate, whether a member is sized right, whether an anchor works: not reviewed. Flagged for a PE when the drawing raises the question, never resolved.
There is a fifth, smaller one. The engine reads what is in the set. It does not know the contract, the RFI log or what the general contractor said on the phone, and a finding that depends on any of that is one it cannot make.
What to expect on a first package
After the upload and the tier choice, the marked-up PDF, the Excel comment log and the online viewer come back within 24 hours. How it works walks through each step, the categories list what is looked at, and pricing covers what it costs.
It is a first pass. The list is short, and each finding is a numbered bubble on the sheet it came from, with a severity and a detail card at the back. Each one reads against the drawing in one look. Some will be things the checker already had on a sticky note. Some will be things nobody had seen. A few may be arguable, and those are worth an email, because delivered reviews are what the next round of grading is drawn from.
The fastest way to judge any of this is to look at one. The sample review is a real review of a commissioned louver package, with every finding on the sheet, the log and the viewer, and nothing described here that is not visible there.