Skip to main content
MicrosoftMicrosoft AI Red TeamIssue #2665 Completed

microsoft/PyRIT

Technical deep dive into refactoring the Greedy Coordinate Gradient (GCG) adversarial jailbreak attack engine in Microsoft's open-source AI Red Teaming framework.

View PRs on GitHubUmbrella Issue #2665GitHub RepositoryView All Open Source
3
GCG Architectural Phases
45+
Deterministic Unit Tests
100%
310+ Suite Pass Rate
Zero OOM
GPU VRAM Bounded
02
Microsoft
Contributor · 8 Merged PRs · Microsoft AI Red Team
8 Merged PRs

microsoft/PyRIT

Greedy Coordinate Gradient (GCG) Adversarial Jailbreak Optimization Engine & Robustness

Authored 8 Pull Requests across Microsoft's open-source AI Red Teaming framework (PyRIT), modularizing the Greedy Coordinate Gradient (GCG) adversarial engine, optimizing GPU VRAM deallocation, resolving SQLite concurrent evaluation locks, and hardening converter & attack builders.

Key Engineering Deliverables
  • •Converter & Scorer Validation Hardening (PR #2912, #2911): Added constructor validation preventing empty word separators and non-positive n-gram slices in WordLevelConverter and ApproximateTextMatching.
  • •Matrix & Attack Factory Extraction (PR #2879, #2795): Decoupled technique factory resolution from MatrixAtomicAttackBuilder and extracted constructor compatibility inspection rules.
  • •MOSSBench & VLSU Concurrency Isolation (PR #2781 · Issue #2775): Eliminated SQLite locking collisions and session collisions during concurrent benchmark evaluations across multi-worker fleets.
  • •GCG Modularization & VRAM Bounding (PR #2720, #2700, #2671 · Issue #2665): Decoupled 2,000+ LOC monolithic attack loop into GCGCandidateProposer, VRAM-bounded evaluator (del logits, ids), and ProgressiveScheduleController.
Python 3.11+PyTorchCUDA VRAM ManagementTransformersGCG Adversarial AttacksSQLite ConcurrencyDataclasses (slots/frozen)pytestRuffty
Showing 8 of 8 ranked pull requests
PRIssueDomainWhat Is the Issue?How I Resolved It (Technical In-Depth)
PR #2912FoundationalText Converters & Scoring
FIX: Disallow empty word_split_separator in WordLevelConverter

WordLevelConverter initialized with word_split_separator="" silently accepted the invalid configuration, but later threw an unhandled ValueError: empty separator during async convert execution.

•
Early Constructor Validation: Added validation requiring word_split_separator to be None or a non-empty string, raising a clear ValueError on empty input.
•
Docstring & Exception Specification: Updated docstring documenting the exception and added comprehensive unit tests in tests/unit/converter/test_word_level_converter.py.
•
Zero Regressions: Full 1,729 converter unit tests passed with 0 errors.
Maintainer Code Review

"Reviewed and merged into main."

— Hannah Westra (@hannahwestra25) — Microsoft AI Red Team
PR #2911FoundationalText Converters & Scoring
FIX: Validate n-gram size in ApproximateTextMatching (n >= 1)

ApproximateTextMatching accepted non-positive values of n (e.g. n <= 0), which generated invalid token slices and corrupted text similarity scores.

•
Input Boundary Enforcement: Added explicit assertion and ValueError check ensuring n >= 1 upon instantiation.
•
Test Suite Hardening: Added regression tests covering edge-case boundary inputs.
Maintainer Code Review

"Reviewed and merged into main."

— Hannah Westra (@hannahwestra25) — Microsoft AI Red Team
PR #2879Issue #2869Attack Architecture
MAINT: Extract technique factory resolution from MatrixAtomicAttackBuilder

MatrixAtomicAttackBuilder was tightly coupled to inline factory resolution, making custom attack composition fragile and difficult to extend.

•
Decoupled Factory Resolution: Extracted clean, modular resolution handlers for atomic attack techniques.
•
Type Safety & Compatibility: Ensured strict signature typing and backward compatibility across existing builder pipelines.
Maintainer Code Review

"Approved and merged into microsoft:main."

— Roman Lutz (@romanlutz) — Microsoft AI Red Team
PR #2795Issue #2764Attack Architecture
FEAT: Extract attack factory constructor compatibility rules

Dynamic attack instantiation was failing when constructor parameter signatures varied across newly added attack classes.

•
Constructor Inspection Rules: Standardized parameter signature inspection and argument routing in attack factories.
•
Defensive Fallbacks: Handled optional kwargs and default parameters gracefully across all registered attack types.
Maintainer Code Review

"Approved and merged into microsoft:main."

— Roman Lutz (@romanlutz) — Microsoft AI Red Team
PR #2781Issue #2775Database & Concurrency
FIX MOSSBench and VLSU SQLite isolation collisions

Parallel multi-threaded evaluation runs in MOSSBench and VLSU benchmarks crashed due to concurrent SQLite database write locks and session collisions.

•
Transaction Scope Isolation: Isolated SQLite database handles and transaction scopes per evaluation worker.
•
Concurrency Safety: Verified collision-free parallel execution across multi-GPU benchmark runs.
Maintainer Code Review

"Approved and merged into microsoft:main."

— Roman Lutz (@romanlutz) — Microsoft AI Red Team
PR #2720Commit dfa2b763Issue #2665State Machine
Phase 3: Model Progressive Admission Transitions (ProgressiveScheduleController)

ProgressiveMultiPromptAttack.run() mixed high-level inner attack execution with complex state transition rules (progressive goal vs. worker admission order, remaining step budgeting, control-weight ratcheting, and sentinel loss resets).

Key Files Changed:[NEW] pyrit/executor/promptgen/gcg/attack/base/progressive_schedule.py[MODIFY] pyrit/executor/promptgen/gcg/attack/base/attack_manager.py[NEW] tests/unit/executor/promptgen/gcg/test_progressive_schedule.py
•
Schedule Controller & Action Enum: Designed and extracted ProgressiveScheduleController alongside ScheduleTransitionAction(Enum).
•
Exact-Budget Boundary Protection: Protected against exact-budget boundary edge cases: prevents dangling inf loss resets when the step budget is exhausted at exact admission boundaries.
•
Explicit Loss Measurement State: Introduced explicit boolean _loss_is_measured tracking to safely preserve legitimate non-finite model losses (inf/nan) without tripping assertions.
•
Control-Weight Ratcheting: Extracted control-weight scheduling and ratcheting logic (+0.01 increments up to threshold 0.09).
•
Maintainer Feedback Parity: Resolved maintainer feedback on ProgressiveScheduleState class shadowing to guarantee isinstance parity, authoring 16 comprehensive unit tests covering all sequences and boundaries.
Maintainer Code Review

"There's definitely more work on GCG coming up. Stay tuned!"

— Roman Lutz (@romanlutz) — Microsoft AI Red Team (Issue #2665 Closed as Completed)
PR #2700Commit 132d1cd7Issue #2665GPU & Memory
Phase 2: Extract VRAM-Bounded Candidate Evaluation Phase (GCGCandidateEvaluator)

The logits evaluation and loss accumulation phase was intertwined with attack orchestration. When evaluating thousands of candidate tokens across multiple models and prompts, GPU memory (VRAM) spikes could easily lead to Out-Of-Memory (OOM) errors without timely tensor cleanup.

Key Files Changed:[NEW] pyrit/executor/promptgen/gcg/attack/gcg/candidate_evaluator.py[MODIFY] pyrit/executor/promptgen/gcg/attack/gcg/gcg_attack.py[NEW] tests/unit/executor/promptgen/gcg/test_gcg_evaluation.py
•
Evaluator Engine Extraction: Extracted GCGCandidateEvaluator paired with @dataclass(frozen=True, slots=True) CandidateEvaluationBatch.
•
Prompt-Level VRAM Cleanup: Engineered sequential candidate-group evaluation loops with explicit prompt-level tensor cleanup (del logits, ids) to strictly bound GPU memory.
•
Cross-Device Loss Accumulation: Ensured cross-worker logits losses are safely migrated and accumulated directly on main_device.
•
Backward Compatibility: Maintained 100% backward compatibility for candidate selection (_select_best_candidate) and return types.
•
Deterministic Unit Tests: Added 15 deterministic unit tests in test_gcg_evaluation.py covering device placement, sequential offsets, tqdm progress tracking, and non-finite loss safety.
Maintainer Code Review

"Appreciate all the rapid iterations :-)"

— Roman Lutz (@romanlutz) — Microsoft AI Red Team (Approved & Merged with 0 changes requested)
PR #2671Commit d620b4d4Issue #2665Attack Architecture
Phase 1: Extract Candidate Proposal Phase (GCGCandidateProposer)

GCGMultiPromptAttack.step() had a monolithic loop that tightly coupled gradient dispatch, per-worker L2 gradient normalization, incompatible tensor shape grouping, token sampling, candidate filtering, and device transfers. This made candidate generation untestable in isolation and prone to subtle regressions.

Key Files Changed:[NEW] pyrit/executor/promptgen/gcg/attack/gcg/candidate_proposer.py[MODIFY] pyrit/executor/promptgen/gcg/attack/gcg/gcg_attack.py[NEW] tests/unit/executor/promptgen/gcg/test_gcg_proposal.py
•
Candidate Proposal Engine: Extracted logic into GCGCandidateProposer returning a typed, immutable @dataclass(frozen=True, slots=True) CandidateProposalBatch.
•
Preserved Worker Invariants: Preserved exact worker ordering and shape-group grouping behavior across multi-model deployments.
•
Cross-Device Migration: Handled cross-device gradient tensor migration by safely moving per-worker tensors to main_device.
•
Protocol Compliance: Maintained complete compliance with SamplingStrategy and CandidateFilter extension protocols.
•
Deterministic Test Suite: Authored 14 deterministic unit tests in test_gcg_proposal.py using mock queues and tensor stubs.
Maintainer Code Review

"Reviewed, approved, and merged into microsoft:main (Commit d620b4d4) by Roman Lutz after comprehensive code review and automated test verification."

— Roman Lutz (@romanlutz) — Microsoft AI Red Team