Terms  /  System Transparency  /  System Opacity
06 · System Transparency

System Opacity

The inability to see what an AI system knows, why it made a decision, or what it did. The default state of most AI tools today.

System Opacity is the status quo. When you use an AI coding assistant, you see the output — the generated code — but not the reasoning: why it chose this approach over that one, what alternatives it considered, what assumptions it made, or what parts of your context it actually used. When you use an AI agent, you see the result but not the process: which tools it called, what data it read, what it tried and abandoned. System Opacity isn't a bug — it's how most systems are built. But as AI systems take on more consequential work, the cost of opacity grows.

Example
An agent refactors a critical authentication module. The user sees the final code diff but not: (1) that the agent considered three approaches and picked the least secure one because it was fastest, (2) that it read an outdated security guide from the context, (3) that it skipped the password hashing upgrade because it interpreted a comment as an instruction to leave it unchanged. Any of these would change how the user reviews the code — but System Opacity hides them all.

More in System Transparency