Secure both boundaries: Keycloak, JWT validation, and tenant authorization

Checkpoint tag: chapter-09-oauth-security — every boundary validates issuer, audience, and scope; the negative security matrix is green; X-Tenant-Id headers are gone from the request path.

What will be built

Keycloak joins the Compose stack with a committed realm import (ops): users priya (operator, tenant acme) and sam (viewer, tenant acme), a second tenant’s operator, client registrations for user login and service-to-service calls, and the four scopes from the blueprint. agent-api and mcp-operations-server become OAuth2 resource servers with explicit audience validation. agent-api acquires client-credentials tokens for MCP calls. Tenant claims replace header trust end to end.

Why it matters

Until now, “tenant” was a header anyone could send — acceptable scaffolding while nothing real was at stake, unacceptable the moment Chapter 10 adds write tools. This chapter is also where the identity design gets decided honestly: the user’s token authenticates the agent boundary; the agent calls MCP with its own service token carrying the resolved tenant and subject as claims the MCP server independently verifies. We do not forward the user JWT — audience mismatch alone would (correctly) reject it, and forwarding user tokens to internal services erodes the boundary between “who asked” and “who is allowed to call.”

Concepts explained

The three independent checks. Issuer (iss = our Keycloak realm) proves where the token came from. Audience (aud contains this service) proves it was minted for us — a token for agent-api is not valid currency at the MCP server. Scope/authorities prove what the bearer may do. OIDC libraries get the first almost free; audience and scope are where real systems under-check.

Delegation vs. impersonation. agent-api calls MCP as itself (client_credentials grant on the agent-api client), carrying tenant_id and sub propagation claims inside the tool-call metadata rather than presenting the user’s token. Trade-off, stated plainly: the MCP server trusts agent-api to have already authenticated the user — acceptable inside one trust domain; a zero-trust variant would use token exchange (RFC 8693) to carry user context cryptographically. That upgrade is an optional lab; the claims-based propagation is the documented baseline.

Deny by construction. Missing scope → 403 at the boundary; tenant mismatch between claim and tool argument → INSUFFICIENT_SCOPE tool result. Neither reaches business logic.

Files added or changed

1
infra/keycloak/realm-ops.json
2
infra/compose/docker-compose.yml (+ keycloak)
3
agent-api/…/security/SecurityConfig.java, AudienceValidator.java, CallerContext.java
4
mcp-operations-server/…/security/SecurityConfig.kt, TenantPolicy.kt
5
agent-api/…/mcp/McpAuthCustomizer.java
6
test-support/…/TestTokens.java (jwt fixtures)
7
*/src/test — security matrix tests

Complete code

realm-ops.json (excerpted to the material parts — the repo carries the complete import):

1
{
2
  "realm": "ops",
3
  "clients": [
4
    {
5
      "clientId": "ops-cli",
6
      "publicClient": true,
7
      "directAccessGrantsEnabled": true
8
    },
9
    {
10
      "clientId": "agent-api",
11
      "serviceAccountsEnabled": true,
12
      "secret": "dev-only-agent-secret",
13
      "defaultClientScopes": ["ops:read", "ops:incident:write", "ops:note:write"]
14
    },
15
    {
16
      "clientId": "mcp-operations-server",
17
      "bearerOnly": true,
18
      "protocolMappers": [{
19
        "name": "audience",
20
        "protocol": "openid-connect",
21
        "protocolMapper": "oidc-audience-mapper",
22
        "config": {"included.client.audience": "mcp-operations-server"}
23
      }]
24
    }
25
  ],
26
  "users": [
27
    {"username": "priya", "enabled": true,
28
     "credentials": [{"type": "password", "value": "dev-password"}],
29
     "attributes": {"tenant_id": ["acme"]},
30
     "clientRoles": {"ops-cli": ["operator"]}},
31
    {"username": "sam", "enabled": true,
32
     "credentials": [{"type": "password", "value": "dev-password"}],
33
     "attributes": {"tenant_id": ["acme"]},
34
     "clientRoles": {"ops-cli": ["viewer"]}},
35
    {"username": "rin", "enabled": true,
36
     "credentials": [{"type": "password", "value": "dev-password"}],
37
     "attributes": {"tenant_id": ["globex"]},
38
     "clientRoles": {"ops-cli": ["operator"]}}
39
  ]
40
}

The realm also defines client scopes agent:invoke, ops:read, ops:incident:write, ops:note:write, and a tenant_id attribute mapper so every user token carries "tenant_id": "acme" as a claim. Operator role gets all three ops:* scopes; viewer gets ops:read only.

agent-api security:

1
package in.o612.eng.opsagent.agent.security;
2

3
import org.springframework.context.annotation.Bean;
4
import org.springframework.context.annotation.Configuration;
5
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
6
import org.springframework.security.oauth2.core.DelegatingOAuth2TokenValidator;
7
import org.springframework.security.oauth2.jwt.*;
8
import org.springframework.security.web.SecurityFilterChain;
9

10
@Configuration
11
public class SecurityConfig {
12

13
    @Bean
14
    SecurityFilterChain api(HttpSecurity http) throws Exception {
15
        http.authorizeHttpRequests(auth -> auth
16
                .requestMatchers("/actuator/health/**").permitAll()
17
                .requestMatchers("/api/v1/**").hasAuthority("SCOPE_agent:invoke")
18
                .anyRequest().denyAll())
19
            .oauth2ResourceServer(oauth -> oauth.jwt(jwt -> jwt.decoder(jwtDecoder())));
20
        return http.build();
21
    }
22

23
    @Bean
24
    JwtDecoder jwtDecoder(
25
            @org.springframework.beans.factory.annotation.Value(
26
                "${spring.security.oauth2.resourceserver.jwt.issuer-uri}") String issuer) {
27
        var decoder = JwtDecoders.fromIssuerLocation(issuer);
28
        var validator = new DelegatingOAuth2TokenValidator<Jwt>(
29
                JwtValidators.createDefaultWithIssuer(issuer),
30
                new AudienceValidator("agent-api"));
31
        ((NimbusJwtDecoder) decoder).setJwtValidator(validator);
32
        return decoder;
33
    }
34
}

1
package in.o612.eng.opsagent.agent.security;
2

3
import org.springframework.security.oauth2.core.OAuth2Error;
4
import org.springframework.security.oauth2.core.OAuth2TokenValidator;
5
import org.springframework.security.oauth2.core.OAuth2TokenValidatorResult;
6
import org.springframework.security.oauth2.jwt.Jwt;
7

8
public class AudienceValidator implements OAuth2TokenValidator<Jwt> {
9

10
    private final String expected;
11

12
    public AudienceValidator(String expected) { this.expected = expected; }
13

14
    @Override
15
    public OAuth2TokenValidatorResult validate(Jwt jwt) {
16
        return jwt.getAudience().contains(expected)
17
            ? OAuth2TokenValidatorResult.success()
18
            : OAuth2TokenValidatorResult.failure(
19
                new OAuth2Error("invalid_token", "missing required audience " + expected, null));
20
    }
21
}

CallerContext replaces the header plumbing: a record CallerContext(String tenantId, String userId, Set<String> scopes) extracted once per request from JwtAuthenticationToken — tenantId from the tenant_id claim, scopes from scope. ConversationService and AgentOrchestrator.dispatch now consume CallerContext; the executor injects ctx.tenantId().

On the MCP side (Kotlin): same decoder pattern, plus method-level checks inside the tool methods — the boundary check and the business check are different lines of defense:

1
package `in`.o612.eng.opsagent.mcp.security
2

3
import org.springframework.security.oauth2.jwt.Jwt
4
import org.springframework.stereotype.Component
5

6
@Component
7
class TenantPolicy {
8

9
    fun enforce(jwt: Jwt, requestedTenant: String, requiredScope: String) {
10
        val scopes = jwt.getClaimAsString("scope")?.split(" ").orEmpty()
11
        require(requiredScope in scopes) { "missing scope $requiredScope" }
12
        val tenant = jwt.getClaimAsString("tenant_id")
13
        require(tenant == requestedTenant) { "tenant mismatch" }
14
    }
15
}

Each @McpTool method resolves the current Jwt (via SecurityContextHolder) and calls tenantPolicy.enforce(jwt, tenantArg, "ops:read") — so even a model-crafted tenant argument fails against the caller’s claim. And mcp-operations-server’s decoder uses AudienceValidator("mcp-operations-server"): an aud=agent-api token is rejected even if every other check would pass.

McpAuthCustomizer — agent-api’s outbound identity:

1
// OAuth2 client-credentials for the agent -> MCP hop.
2
// Registration "agent-mcp": client-id agent-api, grant client_credentials,
3
// token-uri http://keycloak:8085/realms/ops/protocol/openid-connect/token.
4
// A request customizer bean adds "Authorization: Bearer <service token>"
5
// to every MCP transport request via OAuth2AuthorizedClientManager.
6
// Propagation claims (tenant_id, sub) ride as tool-call metadata; the MCP
7
// server validates them against its own JWT's claims — never a forwarded
8
// user token.

Verification note: the exact customizer interface name differs across MCP SDK versions (McpSyncHttpClientRequestCustomizer in current 2.0 docs). Confirm against the Spring AI MCP client docs pinned to 2.0.1 before copying — the contract is “a bean that may add headers to outbound MCP HTTP requests,” whatever the interface is called this month.

The security matrix — automated

Agent-side tests use spring-security-test’s jwt() post-processor for the controller layer plus TestTokens (test-support) minting real JWTs with a test keypair for decoder-level tests — wrong-issuer/audience/expiry cases need a real JwtDecoder path, not a mocked Authentication.

Failure-injection lab

1. Stop Keycloak: boundary requests 401 (decoder can’t fetch JWKS — fail closed, not open).
2. Replay a user token at /mcp: 401 via AudienceValidator.
3. rin (globex) asks for payment-gateway status: tenant mismatch — the arg reaches the tool but dies in TenantPolicy.
4. Strip ops:read from the service client’s scope set and restart the realm: every tool call returns missing scope — scope is checked twice (agent dispatch + MCP method) and fails at the first wall now.

Observability checks

security.auth.denials counter labeled boundary + reason (bad_issuer|bad_audience|missing_scope|tenant_mismatch) — the matrix above, expressed as telemetry so a misconfigured realm looks like a denial spike, not silence.

Checkpoint verification checklist

• All seven matrix rows behave as specified in automated tests.
• No X-Tenant-Id/X-User-Id headers remain on /api/v1/**.
• User JWT is never forwarded to MCP; service token + propagation claims only.
• MCP endpoint rejects anything without aud=mcp-operations-server.

Commit message and Git tag

1
feat(security): OIDC resource servers, audience validation, tenant policy on both boundaries

git tag chapter-09-oauth-security

What comes next

Chapter 10 finally arms the write path: create_incident and append_incident_note behind idempotency keys and the human-approval state machine.

Project State Ledger — chapter-09-oauth-security

• IdP: Keycloak realm ops at :8085; clients ops-cli (public), agent-api (confidential svc acct), mcp-operations-server (bearer-only); users priya/sam/rin; tenant claim tenant_id
• Scopes: agent:invoke (agent-api), ops:read, ops:incident:write, ops:note:write (mcp)
• Identity model: user JWT terminates at agent-api; service token to MCP + propagated tenant_id/sub claims; TenantPolicy enforces claim==arg per tool call
• Env: KEYCLOAK_URL, AGENT_MCP_CLIENT_SECRET; realm import committed under infra/keycloak/
• Known limitation: user-context is claims-propagated, not cryptographically delegated (token exchange = optional lab); Keycloak dev-mode secrets are placeholders
• Next: chapter-10-human-approval