Security model

Mirrored page

This page is mirrored from docs/security.md in the uofa repo. Edits made here are overwritten on the next build — edit the source instead.

Security model

Two kinds of secret matter in this project, and they are unrelated:

• LLM API keys — credentials for the remote backends --explain and extract optionally talk to. Covered immediately below.
• Package signing keys — the ed25519 keypair behind uofa sign / uofa verify. Covered in Signing keys at the end, including a key revocation you should read if you hold any UofA package signed before 2026-08-13.

API keys and the LLM layer

UofA’s --explain and extract commands optionally talk to remote LLM backends (Anthropic, OpenAI, OpenAI-compatible). This section spells out how the CLI handles credentials and what’s in / out of scope for the threat model.

For LLM provider configuration see llm-config.md; for the user-facing flag set see explain.md.

Three rules (spec v0.4 §6.4)

Rule 1: API keys never appear in config files directly

Configs reference env var names via api_key_env, never literal keys:

# ✓ Allowed — env var name only
[llm]
backend = "anthropic"
api_key_env = "ANTHROPIC_API_KEY"

# ✗ Rejected by the config validator with a clear error
[llm]
backend = "anthropic"
api_key = "sk-ant-leaked-this-everywhere"

The validator rejects any [llm] section containing an api_key field (literal value), with an error directing the user to api_key_env instead. This addresses the version-control-leak path — uofa.toml files committed to GitHub will not leak keys because keys are never in the file.

Rule 2: API keys never appear in cached results, error messages, or verbose output

• The cache (~/.uofa/cache/explain.db) stores model identifier and backend identifier, never the API key.
• Verbose output shows token counts and cost estimates but never echoes key values.
• Authentication-failure error messages say “anthropic authentication failed” rather than echoing the attempted credentials.

The tests/security/ suite (and the test_resolve_api_key_error_does_not_echo_key_value test in test_llm_config.py) verifies this invariant: even when the relevant env var IS set, the error path doesn’t leak its value.

Rule 3: API keys are read at request time, not stored in CLI state

The CLI reads os.environ[api_key_env] at the moment of each LLM API request and discards the value after the request completes. The key is never persisted to disk by the CLI, never written to logs, never included in the cache. Implementation: uofa_cli.llm.resolve_api_key().

This matters for shared workstation scenarios: a user running uofa rules --explain on a shared machine doesn’t leave their API key in any UofA-managed location. The key lives in their shell environment or password manager, accessed by the CLI only at request time.

The hosted demo (uofa.net/demo) is a different trust boundary

Everything above describes the CLI, where you choose the backend and a local one keeps your evidence on your machine. The public demo Space does not work that way and should not be reasoned about as if it did.

Documents uploaded to the demo are sent to Together AI to be read. The Space carries no local model: space/Dockerfile.base configures openai-compatible against api.together.xyz, and the API key reaches the container only as a HuggingFace Space secret, never as a repository file (space/deploy_to_hf.py refuses to upload .key, .pem, or .env).

What the Space guarantees, and what it does not:

• It stores nothing. Each request runs in a temporary directory that is deleted when the request finishes, including on timeout and on failure. The one exception is a generated download package, which is retained only until you take it and in any case under 30 minutes.
• It does not log payloads. pipeline._silence_llm_logging() sets litellm’s turn_off_message_logging in the extraction child before any call. This suppresses our logging, not the provider’s. Whether Together retains prompts, and for how long, is governed by their terms; consult those before uploading anything sensitive, and treat that answer as the real retention policy for the demo.
• It cannot offer a “keep local” option. There is no local model in the image to fall back to. A toggle would either do nothing or silently swap in the keyless extractor, whose factor-level accuracy is documented at 0.100 (keyless_extractor.py) — shipping a control that quietly changes the instrument is worse than shipping none. The honest local option is the CLI.

The demo exists to show the flow to people evaluating the approach. Anyone assessing confidential evidence should run the CLI with a local backend, which is the configuration the rest of this document describes.

What this does NOT protect against

• Your own env var management. If you put your API key in a shell history file, in an unencrypted dotenv file, or in a publicly-readable location, the CLI can’t prevent the leak. Use a secrets manager (direnv with .envrc.gpg, 1Password CLI, AWS Secrets Manager, etc.) for any deployment beyond local development.
• Your network. If you run UofA on an untrusted network where HTTPS termination or DNS resolution can be intercepted, that’s outside the CLI’s scope (mitigate via VPN or trusted-network policy).
• The remote backend’s logging. Anthropic, OpenAI, and other providers may log API request bodies for abuse detection or billing. If your evidence documents are sensitive, review the provider’s privacy policy or use a local backend (Ollama) that doesn’t leave the machine.

Recommended setups

Individual practitioner (laptop)

1. Use the bundled local Ollama (no API keys at all).
2. If you need higher quality on a specific package, override per invocation:

   ANTHROPIC_API_KEY=$(op read 'op://Personal/Anthropic/api key') \
       uofa rules my-package.jsonld --explain \
       --explain-backend anthropic --explain-model claude-sonnet-5

Team with one approved vendor

1. Set up the team’s preferred backend in ~/.uofa/config.toml:

   [llm]
   backend = "anthropic"
   model = "claude-sonnet-5"
   api_key_env = "ANTHROPIC_API_KEY"

2. Use a secrets manager that injects ANTHROPIC_API_KEY into the shell environment (direnv, 1Password CLI, chezmoi).
3. Project uofa.toml files contain no LLM config — they inherit from each user’s home config.

Air-gapped / regulated environment

1. Use Ollama exclusively. Pull a model appropriate to your hardware (llama3.3:70b if you have GPU; qwen3.5:4b if you don’t).
2. Set [llm] backend = "ollama" in project uofa.toml.
3. No env vars, no remote calls, no third-party logging.

Where the CLI’s responsibility ends

The CLI’s responsibility is bounded: do not leak keys via UofA-controlled channels (config files, cache, output, logs). Verified by:

• Code review of every emit path — output.py helpers don’t have access to keys; cache schema doesn’t carry them; LLM error normalization in litellm_backend._normalize_exception() builds messages without echoing credentials.
• Test invariants — tests/test_llm_config.py::TestApiKey and tests/test_litellm_backend.py::test_authentication_error_normalized assert the no-leak property on the relevant code paths.
• Config validation — uofa_cli.llm.config._validate_section() rejects literal api_key fields at parse time.

What happens in your shell environment, your dotfiles, or the backend provider’s logging is outside the CLI’s control. Plan accordingly.

---

Signing keys

uofa sign produces an ed25519 signature over a package’s canonical hash; uofa verify, uofa check, and uofa validate --verify check it. When no --pubkey is given they fall back to keys/research.pub (paths.default_pubkey()), which is force-included into the wheel — so it is the default trust anchor for every pip install uofa as well as every source checkout.

Key revocation — 2026-08-13

The signing key in use from 2026-03-29 to 2026-08-13 was compromised. Any UofA package signed in that window carries no authenticity guarantee.

The private key keys/research.key was committed to this public repository in commit a930cf40 (2026-03-29) and stayed tracked until 2026-08-13. The .gitignore rule meant to prevent this matched keys/*.pem, while the toolchain writes private keys as *.key, so it never fired.

For that window, anyone holding the key could mint a package that passed uofa verify with no flags. A passing verification from that period demonstrates only that the file is internally consistent, not who produced it.

The key also shipped on PyPI

Cloning was not required. Hatchling builds sdists from VCS-tracked files, and the sdist exclude list covered only engine and runtime artifacts — so keys/research.key was packaged into every published source distribution:

Release	sdist	Uploaded
0.7.0	uofa-0.7.0.tar.gz	2026-05-03
0.7.1	uofa-0.7.1.tar.gz	2026-05-03
0.8.0	uofa-0.8.0.tar.gz	2026-05-04
0.9.0	uofa-0.9.0.tar.gz	2026-05-27
0.10.0	uofa-0.10.0.tar.gz	2026-06-02
0.11.0	uofa-0.11.0.tar.gz	2026-08-09

Anyone who ran pip download uofa, pip install --no-binary :all: uofa, or built from an sdist received the unencrypted private half of the default trust anchor. The wheels are unaffected — they force-include only research.pub.

PyPI release files are immutable. Yanking a release removes it from resolver selection but does not delete the file, so these sdists remain retrievable. Treat the revoked key as permanently public.

	Fingerprint — sha256(DER SubjectPublicKeyInfo)
Revoked (2026-03-29 → 2026-08-13)	2f622df995d41f9e6bf8057e343b455debea4792a4fb5bba57ccde3f99c18617
Current (2026-08-13 → )	ec22097e31ae1b4faf4556a130b673242a1994fb607fbda11a7124c9c2550f08

The revoked public key is retained as keys/REVOKED-research-2026-03-29.pub so you can tell the two apart:

uofa verify <package> --pubkey keys/REVOKED-research-2026-03-29.pub

A package that verifies against the revoked key was signed during the compromise window. Every signed artifact shipped in this repository has been re-signed with the current key; because re-signing changes only the signature field, package hashes are unchanged.

The old key is not recoverable-proof. It remains in git history across every branch and release tag, in every existing clone and fork, and in published release artifacts. Rewriting history would not retract it, which is why the mitigation is rotation plus this notice rather than a history rewrite.

Where the private key lives

Not in the repository. keys/*.key is gitignored, uofa keygen creates private keys at mode 0600, and it refuses to overwrite existing key material without --force — silently regenerating a keypair invalidates every package already signed with it.

Re-signing the shipped examples is a maintainer operation needing a local copy of the key. Nothing in CI signs: uofa validate --verify only ever reads the public half, so the private key is not required to build, test, or release.

If you sign your own packages

Generate your own key rather than reusing the project’s, and distribute the public half alongside your packages:

uofa keygen keys/my-project.key
uofa sign my-assessment.jsonld --key keys/my-project.key
uofa verify my-assessment.jsonld --pubkey keys/my-project.pub

Commit my-project.pub; keep my-project.key out of version control and back it up somewhere you can read it back from. Note that verification is only as meaningful as the verifier’s choice of --pubkey: a package carries no signer identity, so “verified” means “verified against the key you pointed at.”