AI can now surface software weaknesses faster than many security teams can investigate them. Anthropic’s June 2 announcement expanding Project Glasswing said its initial partners had used Claude Mythos Preview to scan codebases and had identified more than 10,000 high- or critical-severity flaws. The announcement also highlighted the less glamorous bottleneck that follows discovery: verifying findings, disclosing them responsibly, writing fixes and deploying patched software.
That sequence is the useful lesson. An AI-generated finding is not a vulnerability report until a human can reproduce it, explain its impact and connect it to an authorised remediation plan. The workflow below turns that principle into a repeatable guide for security teams, maintainers and developers. It is designed for defensive work on systems you own or are explicitly authorised to test.
1. Set the boundary before asking for findings
Start with a written scope. Name the repository, commit or branch, permitted test environment, data classification and people responsible for approval. If the scan covers a production-connected service, define what the tool must not do: no destructive actions, credential use, data extraction, persistence or testing against third-party systems.
Use a disposable or access-controlled clone wherever possible. Remove secrets, customer data, private keys and live credentials before an AI tool sees the code. Record the exact commit, dependency lockfiles and configuration used for the scan. This gives the team a baseline that can be reproduced later and prevents a model from turning sensitive context into an unnecessary security exposure.
2. Convert each model output into a triage record
Do not paste an unstructured list of findings into a ticket queue. Require a fixed record for every candidate issue:
- Location: repository, file, function and line range.
- Class: the suspected weakness, such as improper authorisation, injection or unsafe deserialisation.
- Preconditions: what access, input or configuration would be required.
- Impact: the asset or security property at risk, stated in plain language.
- Evidence: the relevant code path, data flow or safe reproduction result.
- Confidence: what is known, what is inferred and what still needs checking.
- Proposed action: a minimal fix, a test to add and any disclosure obligation.
Ask the model to identify uncertainty instead of filling gaps with assumptions. A useful prompt requests analysis of the supplied code only, asks for a non-destructive reproduction plan and requires the output to distinguish observation from hypothesis. Store the prompt, model name, date and input commit with the record. That audit trail matters when a later reviewer needs to understand why an issue was prioritised.
3. Reproduce safely, without escalating the test
Human verification is the gate between an AI suggestion and an engineering task. Reproduce the suspected behaviour in an isolated environment with synthetic data and the narrowest permissions available. Prefer unit tests, integration tests, static analysis and controlled fixtures over live exploitation. If a proof requires network access, use a mock service or a dedicated test host and document the allowed endpoints first.
The objective is to confirm the security property, not to demonstrate the maximum possible damage. For example, if a finding suggests that one user can read another user’s record, a safe test should use two test accounts and a harmless fixture. It should not enumerate real records or attempt to access unrelated tenants. Stop when the claim is confirmed or disproved, and preserve logs, inputs, environment details and the exact result.
Classify the outcome as confirmed, false positive, duplicate, unable to reproduce or needs more evidence. “Unable to reproduce” is not a failure of the process; it is a reason to keep the finding out of the release decision until someone can establish the missing conditions.
4. Patch for the root cause and add a regression test
Use the AI assistant to compare remediation options, explain a proposed diff or draft a test, but keep the change narrow. The patch should enforce the intended security boundary at the point where the decision is made, rather than hiding symptoms elsewhere in the application. Ask for alternatives and known trade-offs, then have a maintainer select the implementation.
Every confirmed finding should produce at least one regression test that fails on the vulnerable revision and passes on the patched revision. Add negative cases as well as the expected success path. Run the project’s normal test suite, dependency checks and relevant static analysis. Review the final diff for accidental logging of secrets, permission changes, new network calls, weakened validation and unrelated refactoring.
5. Use two-person review before disclosure or release
A second reviewer should be able to reproduce the issue from the record without relying on the model’s authority. They should verify the scope, severity, affected versions, patch behaviour and test coverage. If the code is shared with customers, downstream maintainers or open-source users, prepare a clear advisory that explains the impact, affected versions, fixed version and mitigation. Send it through the project’s established security contact or coordinated disclosure process.
Do not publish exploit details merely because an AI system produced them. Share enough evidence for affected maintainers to validate and fix the issue, while withholding operational details that would make unauthorised exploitation easier. Coordinate release timing with the responsible maintainer and keep a record of notifications and responses.
6. Measure the workflow, not just the number of findings
Track confirmation rate, duplicate rate, time from discovery to verification, time to patch, regression-test coverage and the number of findings that were safely disclosed. These measures reveal whether AI is reducing defensive workload or simply creating a larger queue of unverified alerts. Review the results after each scan and tighten prompts, permissions and acceptance criteria when the process produces noise.
Project Glasswing’s broader message is that vulnerability discovery is only one part of modern security work. A controlled pipeline that preserves evidence, limits access, requires human verification and treats disclosure as part of remediation can make AI assistance useful without granting it unchecked authority.