> ## Documentation Index
> Fetch the complete documentation index at: https://docs.agentvault.chat/llms.txt
> Use this file to discover all available pages before exploring further.

# Shadow Mode

> Progressive autonomy for AI agents — graduate from observe-only to supervised to fully autonomous through measured agreement.

# Shadow Mode

Shadow Mode implements a progressive autonomy model for AI agents. Instead of granting full autonomy on day one, agents earn independence by demonstrating alignment with human decisions over time.

The model has three levels:

| Level          | Behavior                                                                  | Human Role                  |
| -------------- | ------------------------------------------------------------------------- | --------------------------- |
| **Shadow**     | Agent observes and records what it *would* recommend, but takes no action | Human decides independently |
| **Supervised** | Agent recommends an action, human confirms or overrides before execution  | Human approves or rejects   |
| **Autonomous** | Agent acts independently, human reviews after the fact                    | Human monitors              |

Every agent starts at Shadow. Graduation to higher levels requires meeting quantitative gates — not just vibes.

***

## Creating Shadow Sessions

A shadow session tracks one specific capability being evaluated. You define the skill (what the agent can do) and the decision class (what type of decision it involves).

```bash theme={null}
POST /api/v1/shadow/sessions
Authorization: Bearer <jwt>
Content-Type: application/json

{
  "device_id": "3a7df4c2-...",
  "skill_name": "incident_response",
  "decision_class": "escalation",
  "config": {
    "min_observations": 25,
    "agreement_threshold": 0.90,
    "critical_miss_window_hours": 168
  }
}
```

| Field                        | Description                                                                                   |
| ---------------------------- | --------------------------------------------------------------------------------------------- |
| `skill_name`                 | The agent capability being evaluated (e.g., `incident_response`, `code_review`, `deployment`) |
| `decision_class`             | The type of decision (e.g., `escalation`, `approval`, `rollback`, `triage`)                   |
| `min_observations`           | Number of decisions required before graduation is possible                                    |
| `agreement_threshold`        | Minimum agreement rate (0.0 - 1.0) required to graduate                                       |
| `critical_miss_window_hours` | Time window in which no critical misses can occur                                             |

A session is scoped to one skill + decision class combination. An agent can have multiple active sessions — for example, `incident_response/escalation` at Supervised while `deployment/approval` is still in Shadow.

***

## Agreement Tracking

During a shadow session, every decision point follows the same flow:

<Steps>
  <Step title="Agent recommends">
    The agent analyzes the situation and records what it would recommend. In Shadow mode, this recommendation is silent — the human never sees it. In Supervised mode, the recommendation is shown to the human.
  </Step>

  <Step title="Human decides">
    The human makes the actual decision independently (Shadow) or in response to the agent's recommendation (Supervised).
  </Step>

  <Step title="System compares">
    The platform compares the agent's recommendation against the human's decision and records the outcome: **agree** or **disagree**.
  </Step>
</Steps>

```bash theme={null}
POST /api/v1/shadow/sessions/{session_id}/record-human-decision
Authorization: Bearer <jwt>
Content-Type: application/json

{
  "observation_id": "obs_uuid",
  "agent_recommendation": {
    "action": "escalate",
    "confidence": 0.87,
    "reasoning": "Production error rate exceeded 5% threshold"
  },
  "human_decision": {
    "action": "escalate",
    "notes": "Agreed — error rate is climbing fast"
  }
}
```

The agreement rate is computed as a rolling metric:

```
agreement_rate = agreed_decisions / total_decisions
```

The platform tracks this continuously and displays it on the session dashboard alongside a trend line showing agreement over time.

<Tip>
  Disagreements are not failures — they are learning signals. Review disagreements to understand whether the agent needs tuning or whether the human's decision was an edge case.
</Tip>

***

## Graduation Gates

Graduation from one level to the next requires meeting three quantitative requirements simultaneously:

| Gate                          | Requirement                                                        | Example                             |
| ----------------------------- | ------------------------------------------------------------------ | ----------------------------------- |
| **Minimum observations**      | Enough decisions have been recorded to be statistically meaningful | 25 observations                     |
| **Agreement threshold**       | Agreement rate meets or exceeds the configured threshold           | 90% agreement                       |
| **No recent critical misses** | No critical disagreements within the lookback window               | Zero critical misses in last 7 days |

A **critical miss** is a disagreement where the agent's recommendation, if followed, would have caused significant harm — as determined by the human reviewer when recording the decision.

### Checking Eligibility

```bash theme={null}
GET /api/v1/shadow/sessions/{session_id}/graduation-check
Authorization: Bearer <jwt>
```

Response:

```json theme={null}
{
  "eligible": true,
  "current_level": "shadow",
  "next_level": "supervised",
  "gates": {
    "min_observations": { "required": 25, "actual": 31, "met": true },
    "agreement_threshold": { "required": 0.90, "actual": 0.94, "met": true },
    "critical_miss_window": { "window_hours": 168, "misses": 0, "met": true }
  }
}
```

### Promoting

Graduation is an explicit action — it never happens automatically. When all gates are met, the owner promotes the session:

```bash theme={null}
POST /api/v1/shadow/sessions/{session_id}/graduate
Authorization: Bearer <jwt>
```

This advances the session from Shadow to Supervised, or from Supervised to Autonomous.

<Warning>
  Graduation can be reversed. If an agent at Supervised or Autonomous level has a critical miss, the owner can demote it back to a lower level at any time.
</Warning>

***

## When to Use Shadow Mode

Shadow Mode is most valuable in four situations:

### New Skills

When an agent gains a new capability (e.g., you add a deployment tool), start that skill in Shadow. Let the agent build a track record before it acts independently.

### High-Risk Decision Classes

For decisions with significant consequences — production deployments, access grants, financial transactions — Shadow Mode provides a safety net even for well-established agents.

### Post-Incident Recovery

After an agent makes a mistake, demote the relevant skill back to Shadow. Rebuild confidence through measured agreement before restoring autonomy.

### New Agent Versions

When you update the underlying model or significantly change the agent's prompts, re-enter Shadow Mode for critical skills. Model updates can change behavior in unexpected ways.

***

## Integration

The agent plugin reports recommendations and receives human decisions through the shadow session API.

### Plugin-Side Flow

1. Agent encounters a decision point during normal operation
2. Agent computes its recommendation and submits it to the session
3. If in Shadow mode: recommendation is recorded silently, human decides independently
4. If in Supervised mode: recommendation is shown to the human in the AgentVault dashboard
5. Human decision is recorded via the API
6. System compares and updates the agreement rate

### Recording from the Plugin

```typescript theme={null}
// Agent-side: record what the agent would recommend
await fetch(`${API_BASE}/api/v1/shadow/sessions/${sessionId}/record-human-decision`, {
  method: "POST",
  headers: { Authorization: `Bearer ${jwt}` },
  body: JSON.stringify({
    observation_id: observationId,
    agent_recommendation: {
      action: "approve_deploy",
      confidence: 0.92,
      reasoning: "All tests passing, no breaking changes detected",
    },
    human_decision: {
      action: humanAction,
      notes: humanNotes,
    },
  }),
});
```

The dashboard displays a live view of the session: observation count, agreement rate, trend line, recent disagreements, and graduation eligibility.

***

## API Reference

| Method | Endpoint                                             | Description                                       |
| ------ | ---------------------------------------------------- | ------------------------------------------------- |
| `POST` | `/api/v1/shadow/sessions`                            | Create a new shadow session                       |
| `POST` | `/api/v1/shadow/sessions/{id}/record-human-decision` | Record an agent recommendation and human decision |
| `GET`  | `/api/v1/shadow/sessions/{id}/graduation-check`      | Check graduation eligibility against all gates    |
| `POST` | `/api/v1/shadow/sessions/{id}/graduate`              | Promote session to the next autonomy level        |