[FEAT] Resolve Key Vault-Backend Environment References and Update .env_example

[Victor Valbuena (ValbuenaVC)]

Description

This PR adds Key Vault-backed dotenv bootstrapping while preserving PyRIT's existing python-dotenv workflow and list-shaped env_akv_ref configuration. It also updates .env_example, which had drifted from the rest of the library.

env_akv_ref:
  - https://my-vault.vault.azure.net/secrets/pyrit-env
env_akv_strict: true
env_akv_write_env: false

Each env_akv_ref URL is loaded as a bootstrap dotenv document in list order. Documents support native ${NAME} interpolation and complete-value references to scalar secrets in the same vault:

ENDPOINT="https://example.openai.azure.com/openai/v1"
MODEL="${PYRIT_MODEL}"
API_KEY="kv:https://my-vault.vault.azure.net/secrets/api-key"

kv: is canonical; akv:, azure_key_vault:, and env_akv_ref: remain compatibility aliases. Child-secret values are terminal and are not interpreted recursively.

Source Precedence

Sources load sequentially:

1. Existing process environment
2. Key Vault bootstrap documents in env_akv_ref order
3. Explicit env_files, or the defaults:
   • ~/.pyrit/.env
   • ~/.pyrit/.env.local

Sources use python-dotenv's override=True and ${NAME} interpolation behavior. Later assignments override earlier assignments, and interpolation follows assignment and file order.

Loading intentionally remains non-transactional. If a later source or child-secret lookup fails, assignments from earlier sources remain loaded.

Key Vault Validation

References must occupy the complete value and use an HTTPS secret URL from the bootstrap document's vault. Versioned and versionless URLs are supported.

Short names, cross-vault references, arbitrary hosts, credentials, ports, queries, fragments, and malformed secret paths are rejected before client creation.

Supported DNS suffixes:

• .vault.azure.net
• .vault.azure.cn
• .vault.usgovcloudapi.net

Strict mode rejects malformed or valueless bootstrap assignments. Non-strict mode warns, removes invalid entries, and loads valid assignments. Local dotenv files retain python-dotenv's permissive behavior.

Key Vault clients use asynchronous retries with exponential backoff. Initialization failures raise KeyVaultInitializationException while preserving the original cause.

Optional Local Persistence

Bootstrap documents remain memory-only by default.

When env_akv_write_env is enabled, fetched bootstrap documents are written to ~/.pyrit/.env without resolved child-secret values. The file is written through a restricted sibling temporary file and atomically replaced. Symbolic-link destinations are rejected, and owner-only permissions are used where supported.

.env_example Updates

Environment names were aligned with their current PyRIT usages, and alias source assignments now appear before aliases so python-dotenv interpolation works in assignment order.

Earlier versions of .env_example used angle brackets around URL placeholders as documentation styling. These wrappers were removed because python-dotenv preserves them as literal parts of the configured value. This is a stylistic normalization, not an endpoint contract change.

Tests and Documentation

Coverage includes source precedence, strict parsing, child-secret resolution, URL validation, retries, optional persistence, secure atomic writing, symlink rejection, alias interpolation, URL placeholder formatting, and weak repository-reference drift detection.

Local Validation

• Affected setup tests: 165 passed, 2 skipped
• .env_example contract tests: 3 passed
• Ruff check: passed
• Ruff format check: passed
• Scoped git diff --check: passed

[Copilot]

Pull request overview

Adds Azure Key Vault-backed environment bootstrapping with recursive reference resolution, precedence handling, warnings, and documentation.

Changes:

• Resolves env:, kv:, aliases, and escaped literals.
• Adds environment-source validation and AKV/local-file precedence.
• Expands tests and configuration documentation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
pyrit/setup/initialization.py	Implements AKV loading and reference resolution.
tests/unit/setup/test_initialization.py	Tests environment initialization behavior.
doc/getting_started/pyrit_conf.md	Documents loading precedence and AKV references.
.pyrit_conf_example	Updates example AKV configuration guidance.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Suppressed comments (5)

tests/unit/setup/test_initialization.py:380

• This patch target is no longer called by initialize_pyrit_async, leaving the output assertion dependent on any real default environment files. Patch _resolve_environment_files instead so unrelated local files cannot add output or trigger reference resolution.

    @mock.patch("pyrit.setup.initialization._load_environment_files", return_value=True)

pyrit/setup/initialization.py:537

• Direct callers using the former list-shaped env_akv_ref reach .strip() here and get AttributeError, rather than the deliberate ValueError used by ConfigurationLoader. Validate the runtime type before calling string methods so this public API rejects legacy values consistently.

        if not env_akv_ref.strip():

pyrit/setup/initialization.py:439

• This merge is case-sensitive even on Windows. For example, ambient Path=old plus a winning PATH=new leaves both keys, so env:Path returns the ambient exact match and violates the documented merged-source precedence. Normalize keys on Windows while applying values last.

    reference_environment = {**ambient_environment, **values}

tests/unit/setup/test_initialization.py:372

• initialize_pyrit_async no longer calls _load_environment_files, so this patch is inert and the test can read real ~/.pyrit files (and even resolve their Key Vault references). Patch the resolver now used by initialization to keep the unit test isolated.

This issue also appears on line 380 of the same file.

    @mock.patch("pyrit.setup.initialization._load_environment_files", return_value=True)

doc/getting_started/pyrit_conf.md:174

• This row contradicts both the implementation and the earlier AKV precedence section: when env_files is omitted, initialization loads both .env and .env.local after the bootstrap. Remove the claim that only .env.local is loaded.

| Omitted or `null` | Load default `~/.pyrit/.env` and `~/.pyrit/.env.local`, or only `.env.local` after an AKV root |

[hannahwestra25]

why is the default 500 ?

[Victor Valbuena (ValbuenaVC)]

It's somewhat arbitrary, since PyritException requires a status code and 500 seemed like a reasonable default (key vault not reachable for some reason). Do you think it should be changed and/or do you think KeyVaultInitializationException should exist?

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

.env_example:294

• This Azure OpenAI TTS base URL is missing the required /openai segment. OpenAITTSTarget._get_provider_examples() expects https://{resource}.openai.azure.com/openai/v1, so copying this example will fail endpoint validation or send requests to the wrong path.

AZURE_OPENAI_TTS_ENDPOINT2 = "<https://xxxxx.openai.azure.com/v1>"

pyrit/setup/initialization.py:208

• This still accepts malformed/nonexistent Azure vault hosts such as one-character names, leading/trailing hyphens, and consecutive hyphens. Azure vault names are 3–24 characters, start with a letter, end with an alphanumeric character, and cannot contain consecutive hyphens; without those checks, URLs that should fail preflight proceed to credential/client creation.

    valid_vault_name = 1 <= len(vault_name) <= 63 and all(
        char.isascii() and (char.isalnum() or char == "-") for char in vault_name
    )

.env_example:55

• Dotenv preserves the angle brackets inside these quoted values, so this becomes the literal endpoint <https://api.openai.com/v1>, which is not a valid URL. The same pattern now appears on every endpoint assignment in this file; remove the < and > from all actual URL values.

PLATFORM_OPENAI_CHAT_ENDPOINT="<https://api.openai.com/v1>"

[Richard Lundeen (rlundeen2)]

There's a lot to read through, so apologies if this has been addressed.

But I want to make sure we don't lose debuggability and that I can easily tell which targets we have. Right now, I use .env ~weekly to help debug myself and others.

E.g. what is the default adversarial model? What is configured? Where is the default open ai target referencing? Where is X pointing at? Etc

Right now I do a lot of that with .env. In theory we could download a .env and I could use it the same way and it could reference key vault secrets. I'm worried if all of .env is obstructed, I won't be able to see what's configured. e.g. what is the adversarial model? Or how do I configure for another target when not in the GUI?

There might be answers to this. But if they're aren't, we may want to download a .env to help even see which targets are available (and have that be able to reference keyvault secrets)

[Victor Valbuena (ValbuenaVC)]

There's a lot to read through, so apologies if this has been addressed.

But I want to make sure we don't lose debuggability and that I can easily tell which targets we have. Right now, I use .env ~weekly to help debug myself and others.

E.g. what is the default adversarial model? What is configured? Where is the default open ai target referencing? Where is X pointing at? Etc

Right now I do a lot of that with .env. In theory we could download a .env and I could use it the same way and it could reference key vault secrets. I'm worried if all of .env is obstructed, I won't be able to see what's configured. e.g. what is the adversarial model? Or how do I configure for another target when not in the GUI?

There might be answers to this. But if they're aren't, we may want to download a .env to help even see which targets are available (and have that be able to reference keyvault secrets)

Fwiw, one of the goals of this PR is to avoid having users keep an .env on disk. This feels like a possible anti-pattern when we have registries and CLI utilities to find components, but feel free to disagree.

I see what you mean though and I think we can fix this by adding a save to disk flag that saves the new .env (without actual secrets, just references to kv secrets) to disk. This seems like the best of both worlds and for debugging lets you investigate line-by-line.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Suppressed comments (5)

.env_example:191

• This primary TTS section still defines the old OPENAI_TTS_* names, while both TTS target registrations now read AZURE_OPENAI_TTS_* (pyrit/setup/initializers/targets.py:360-371). Filling this section therefore does not configure either registered TTS target; reconcile these names and the duplicate Azure definitions later in the file.

OPENAI_TTS_ENDPOINT1 = "<https://xxxxx.openai.azure.com/openai/v1>"
OPENAI_TTS_MODEL1 = "tts"
OPENAI_TTS_UNDERLYING_MODEL1 = "tts"

OPENAI_TTS_ENDPOINT2 = "<https://xxxxx.openai.azure.com/v1>"

pyrit/setup/initialization.py:106

• The PR description promises that bootstrap documents remain in memory and are never written to disk, but this public option intentionally persists them, including any literal credentials in the bootstrap document. Either remove the write-to-disk feature or update the stated contract and scope so this security-sensitive behavior is explicitly reviewed.

        env_akv_write_env (bool): If True, save fetched bootstrap documents with unresolved
            child references to ``~/.pyrit/.env``. Defaults to False.

.env_example:32

• The angle brackets are literal dotenv value characters, not Markdown delimiters, so copying this example produces endpoints such as <https://...> that URL clients will reject. This pattern occurs throughout the newly updated endpoint values; remove the < and > wrappers everywhere in this file.

AZURE_OPENAI_GPT4O_ENDPOINT="<https://xxxx.openai.azure.com/openai/v1>"

.env_example:140

• These aliases are interpolated before PLATFORM_OPENAI_CHAT_ENDPOINT and PLATFORM_OPENAI_CHAT_MODEL are assigned at lines 318–320. Because python-dotenv resolves in assignment order and does not revisit earlier values, both aliases become empty when this file is loaded. Move source definitions before their aliases (also for the response, realtime, image, and TTS forward references below) or move the aliases after the sources.

OPENAI_CHAT_ENDPOINT=${PLATFORM_OPENAI_CHAT_ENDPOINT}
OPENAI_CHAT_MODEL=${PLATFORM_OPENAI_CHAT_MODEL}

.env_example:172

• This primary image section still defines the old OPENAI_IMAGE_*2 names, while TargetConfig now reads AZURE_OPENAI_IMAGE_*2 (pyrit/setup/initializers/targets.py:349-352). A user following the file's instruction to fill only this section will not configure openai_image_platform; reconcile these names and the duplicate Azure definitions later in the file.

This issue also appears on line 187 of the same file.

OPENAI_IMAGE_ENDPOINT2 = "<https://xxxxx.openai.azure.com/openai/v1>"
OPENAI_IMAGE_MODEL2 = "dall-e-3"
OPENAI_IMAGE_UNDERLYING_MODEL2 = "dall-e-3"

[Copilot]

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated no new comments.

Suppressed comments (9)

pyrit/setup/initializers/targets.py:371

• This preserves the openai_tts_platform registry name but changes all of its inputs to Azure-only variables. The former endpoint-2 example was the OpenAI platform endpoint, so users now get an Azure target under the platform registry name and there is no PLATFORM_OPENAI_TTS_* path anywhere in the repository. Keep a distinct platform variable set here, or rename this registration as a second Azure target and add the actual platform registration.

        endpoint_var="AZURE_OPENAI_TTS_ENDPOINT2",
        key_var="AZURE_OPENAI_TTS_KEY2",
        model_var="AZURE_OPENAI_TTS_MODEL2",
        underlying_model_var="AZURE_OPENAI_TTS_UNDERLYING_MODEL2",

.env_example:293

• These platform source assignments occur after OPENAI_CHAT_* and OPENAI_RESPONSES_* interpolate them at lines 128-137. Since python-dotenv resolves in assignment order, copying this file into a clean environment leaves those generic endpoint/model/key aliases empty. Move the platform source block before its aliases (the same ordering contract is documented in this PR).

PLATFORM_OPENAI_CHAT_ENDPOINT="<https://api.openai.com/v1>"
PLATFORM_OPENAI_CHAT_KEY="sk-xxxxx"
PLATFORM_OPENAI_CHAT_MODEL="gpt-4o"
PLATFORM_OPENAI_RESPONSES_ENDPOINT="<https://api.openai.com/v1>"
PLATFORM_OPENAI_RESPONSES_KEY="sk-xxxxx"

.env_example:314

• OPENAI_REALTIME_ENDPOINT and OPENAI_REALTIME_MODEL interpolate these names at lines 149-150, before these assignments are parsed. In a standalone copy of .env_example, both aliases therefore become empty. Define the platform realtime values before the generic aliases.

PLATFORM_OPENAI_REALTIME_ENDPOINT="wss://api.openai.com/v1"
PLATFORM_OPENAI_REALTIME_KEY="sk-xxxxx"
PLATFORM_OPENAI_REALTIME_MODEL="gpt-4o-realtime-preview"

.env_example:360

• The generic image aliases at lines 159-160 reference AZURE_OPENAI_IMAGE_ENDPOINT2 and AZURE_OPENAI_IMAGE_MODEL2 before this block defines them. Python-dotenv does not resolve references retroactively, so those generic values are empty when users copy this example into a clean environment. Move these primary assignments before the alias block.

AZURE_OPENAI_IMAGE_ENDPOINT1  = "<https://xxxxx.openai.azure.com/openai/v1>"
AZURE_OPENAI_IMAGE_API_KEY1 = "xxxxxx"
AZURE_OPENAI_IMAGE_MODEL1 = "deployment-name"
AZURE_OPENAI_IMAGE_UNDERLYING_MODEL1 = "dall-e-3"
AZURE_OPENAI_IMAGE_ENDPOINT2 = "<https://xxxxx.openai.azure.com/openai/v1>"

.env_example:374

• The generic TTS aliases at lines 171-172 interpolate the endpoint-2/model-2 names before this source block is reached, leaving both values empty in a clean environment. Place the primary TTS assignments before those aliases so the documented assignment-order semantics produce usable values.

AZURE_OPENAI_TTS_ENDPOINT1 = "<https://xxxxx.openai.azure.com/openai/v1>"
AZURE_OPENAI_TTS_KEY1 = "xxxxxxx"
AZURE_OPENAI_TTS_MODEL1 = "tts"
AZURE_OPENAI_TTS_UNDERLYING_MODEL1 = "tts"
AZURE_OPENAI_TTS_ENDPOINT2 = "<https://xxxxx.openai.azure.com/v1>"

pyrit/setup/akv_initialization.py:496

• write_text creates/truncates the potentially secret-bearing file under the process umask before permissions are restricted. During that window it may be readable by other users, and if chmod fails the code merely warns and leaves the exposed file in place. Create the file securely with mode 0600 before writing (and fail/clean up if permissions cannot be guaranteed).

    env_file.write_text(content, encoding="utf-8")
    try:
        env_file.chmod(0o600)
    except OSError:
        logger.warning("Could not restrict permissions on written AKV environment file: %s", env_file)

pyrit/setup/initializers/targets.py:352

• openai_image_platform now reads the Azure deployment variables, while the updated example defines the actual platform values as PLATFORM_OPENAI_IMAGE_ENDPOINT/KEY/MODEL (.env_example:300-302). Consequently, configuring those documented platform values will not register this target, and the Azure endpoint-2 configuration is mislabeled as the platform target. Wire this entry to the PLATFORM_OPENAI_IMAGE_* variables and update the corresponding initializer test.

This issue also appears on line 368 of the same file.

        endpoint_var="AZURE_OPENAI_IMAGE_ENDPOINT2",
        key_var="AZURE_OPENAI_IMAGE_API_KEY2",
        model_var="AZURE_OPENAI_IMAGE_MODEL2",
        underlying_model_var="AZURE_OPENAI_IMAGE_UNDERLYING_MODEL2",

.env_example:32

• These angle brackets are stored literally by python-dotenv, so the copied endpoint becomes <https://...> rather than a valid URL. The same Markdown-style wrapping appears in 52 URL assignments throughout this dotenv file; remove the angle brackets from all of them.

This issue also appears in the following locations of the same file:

• line 289
• line 312
• line 356
• line 370

AZURE_OPENAI_GPT4O_ENDPOINT="<https://xxxx.openai.azure.com/openai/v1>"

pyrit/setup/akv_initialization.py:422

• The PR description states that bootstrap documents remain in memory and are never written to disk, but this new public option writes them to ~/.pyrit/.env (including any literal secrets in the bootstrap). Either remove this disk-writing path to preserve the stated guarantee or update the PR's security contract and description explicitly.

    env_akv_write_env: bool = False,