Redlyne AI

Detect and patch vulnerabilities in AI-generated Python code, directly from your editor.

Website · Marketplace · Issues · Discussions

Redlyne is a VS Code extension that scans Python code — especially code generated by AI assistants like Copilot, ChatGPT, or Claude — for known vulnerability patterns, and proposes one-click in-editor patches you can apply with a single confirmation.

It runs entirely on your machine. No code is sent to any server.

Built for AI. Built without AI. Redlyne is powered by a deterministic rule engine — no LLM, no probabilistic guesses, no hallucinated fixes. The detection rules are hand-crafted by security researchers, not auto-generated. Every flag and every remediation suggestion is reproducible and auditable.

Why Redlyne

AI coding assistants are fast, and they're also reliably insecure. In a 2025 study evaluating 609 Python snippets generated by GitHub Copilot, Claude 3.7 Sonnet, and DeepSeek V3, ~76% of the code contained security vulnerabilities — for GitHub Copilot specifically the rate climbs to ~84% (source). The patterns are well-known to security engineers: SQL injection from string concatenation, command injection from os.system, weak crypto defaults, hard-coded secrets, unsafe deserialization, path traversal, SSRF, and more.

There's also a structural problem alongside the security one. A 2025 study analyzing 500 Python snippets across four AI assistants found that ~13% of them are incomplete — missing imports or context. For GitHub Copilot specifically, that number rises to ~31% — almost one snippet in three (source). AST-based analyzers like Bandit, CodeQL, and PyT can't process those snippets at all: without a parseable program, they can't build the tree they rely on.

Redlyne sits in your editor and flags vulnerability patterns the moment you select the snippet, then offers a remediated version. Pattern-based detection works on any fragment, complete or not. Instead of relying on another LLM to find LLM bugs, it uses a curated set of deterministic rules built by people whose job is to find vulnerabilities.

Features

• Built for AI, built without AI — deterministic rule engine, no LLM, no hallucinations, every flag is reproducible
• Expert-curated rule set — 459 detection patterns mapped to OWASP Top 10:2025 categories observed in AI-generated Python code, derived from the SecurityEval, Copilot CWE Scenarios, and PoisonPy benchmarks
• Static vulnerability detection on Python code selected in the editor
• Automated remediation suggestions, applied as in-place edits to the source file with your explicit confirmation
• Right-click integration in the editor context menu for any Python selection
• Local execution — your code never leaves your machine

What's inside

• 459 deterministic detection rules mapped to OWASP Top 10:2025 categories — see the full list in COVERAGE.md
• 70+ rules with auto-remediation, including 14 multi-line templates that rewrite vulnerable blocks while preserving indentation and adding required imports
• ~70-100 ms per snippet on a typical laptop
• Rules derived from analysis of vulnerable Python samples sourced from SecurityEval, the Copilot CWE Scenarios Dataset, and PoisonPy — three established security benchmarks
• Each detected vulnerability comes with an automated patch suggestion — not a comment, not an LLM guess, but a concrete code replacement
• Patch safety verified end-to-end: every auto-fix compiles as valid Python and the rule that triggered it no longer fires after the patch (zero regressions)

Measured performance

Redlyne is benchmarked on PoisonPy [Cotroneo et al., ICPC 2024], the reference dataset of 310 paired vulnerable / safe Python samples curated to evaluate AI-code-generator security.

Detection on PoisonPy

97.4% recall · 0.82 F1 · 459 rules in <100 ms

Recall on PoisonPy is higher than the baseline reported in the original paper (~91%) — Redlyne extends the rule set to 459 patterns and catches more issue classes.

Patching safety

100% of generated patches verified safe

Every auto-fix Redlyne emits is checked end-to-end on every benchmark sample for two correctness properties:

In other words: when Redlyne suggests a fix, applying it never breaks the file and always actually neutralizes the vulnerability. No silent broken code ever gets suggested.

Reproducibility

python tests/bench_dataset.py     # detection (PoisonPy + SecurityEval + Copilot CWE Scenarios)
python tests/bench_patching.py    # patch safety

Both produce JSON + Markdown reports under benchmarks/ in under 10 seconds combined.

Honest disclosure

Redlyne trades some precision (71.4%, vs ~97% reported in the original paper for an 85-rule subset) for the higher recall on a 459-rule rule set. Concretely: more issues are flagged, with more false positives that surface as advisory comments — never as broken code. The trade-off is fully measured, never hidden, and reproducible. Future work focuses on AST-based dataflow tracking to tighten precision without losing recall.

Citing the benchmark dataset

Redlyne's PoisonPy benchmark numbers are produced on the dataset published with:

Cotroneo, D., Improta, C., Liguori, P., Natella, R. (2024). Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks. Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension (ICPC '24), pages 280–292. DOI 10.1145/3643916.3644416

@inproceedings{cotroneo2024poisonpy,
  author    = {Cotroneo, Domenico and Improta, Cristina and Liguori, Pietro and Natella, Roberto},
  title     = {Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks},
  booktitle = {Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension (ICPC '24)},
  year      = {2024},
  pages     = {280--292},
  publisher = {ACM},
  doi       = {10.1145/3643916.3644416}
}

Installation

From the VS Code Marketplace (recommended):

1. Open VS Code
2. Extensions panel (Ctrl+Shift+X)
3. Search for Redlyne
4. Click Install

From the command line:

code --install-extension redlyne.redlyne-ai

From a .vsix (manual install):

1. Download the latest redlyne-x.y.z.vsix from Releases
2. In VS Code, run Extensions: Install from VSIX... from the Command Palette
3. Select the downloaded file

Usage

1. Open a Python file (.py) in VS Code
2. Select the block of code you want to analyze
3. Right-click the selection → Redlyne: Run Analysis
4. Review the notifications
5. Confirm to apply the suggested patch

The same command is also available from the Command Palette (Ctrl+Shift+P → "Redlyne: Run Analysis").

Requirements

Redlyne runs natively on Windows, macOS, and Linux — no WSL, no bash, no jq.

Before using the extension, make sure you have:

• Python 3.10+ on PATH (python3 on macOS/Linux, python or py on Windows)
• VS Code 1.89 or newer

Status

Redlyne is in early access. The detection engine is fast, cross-platform, and ready for daily code review, but the rule set is still growing:

• Platform: Windows, macOS, Linux — single Python codebase, no platform shims.
• Performance: typical analysis runs in under 100 ms per snippet.
• Rule coverage: 442 detection rules from the Devaic v2.0 schema, with ~30 of them carrying drop-in safe-replacement remediation. Coverage is expanding release by release.
• Languages: Python only. Other languages may follow based on demand.

If something doesn't work the way you expect, please open an issue — we read everything.

Known Limitations

• Detection and remediation depend on the bundled rule set; novel or obfuscated patterns may not be flagged.
• Selections that are too short (a single statement out of context) may not provide enough signal for accurate analysis.

Contributing

Contributions of all sizes are welcome — bug reports, feature ideas, rule submissions, documentation fixes, and code.

• Read CONTRIBUTING.md for the development setup and pull request process
• Read CODE_OF_CONDUCT.md for community expectations
• Browse open issues, especially those tagged good first issue
• Join the discussion in GitHub Discussions

Contact

• General inquiries / partnerships / press: [email protected]
• Bugs and feature requests: GitHub Issues
• Questions and ideas: GitHub Discussions
• Code of Conduct reports: [email protected]
• Website: redlyne.io

License

Redlyne uses a dual-license model:

This separation keeps the engineering side fully open source — fork it, extend it, ship it commercially with attribution — while preserving the value of the curated rule set.

For commercial licensing of the rules and datasets, partnership, or enterprise inquiries: redlyne.io · [email protected]

The names "Redlyne" and the Redlyne logo are trademarks. See NOTICE for details.