Authentication Logging
- Craig McNally
Overview
In some cases it may be desirable, or required to capture authentication events for the purposes auditing, troubleshooting, investigating security incidents, etc. Keycloak supports is via the “event logging” functionality. The purpose of this document is to describe steps required to set this up, as well as query/access the events.
What is an “authentication event”?
Keycloak supports an extensive list of events and allows full control over which are saved. For the purposes of this document it’s assumed that the “LOGIN” and “LOGIN_ERROR” events are of interest. See https://www.keycloak.org/docs-api/25.0.1/javadocs/org/keycloak/events/EventType.html for the full list.
Keycloak provides a similar mechanism for “admin events”. These correspond to actions performed by administrators via the admin console (or presumably the admin APIs). Here, there isn’t a set list of event types, but rather:
Resource Type, e.g. REALM, CLIENT, USER, etc.
Operation Type, e.g. ACTION, CREATE, DELETE, UPDATE
Resource Path, e.g. events/config, users/<UUID>, etc.
Configuration
See https://www.keycloak.org/docs/latest/server_admin/index.html#configuring-auditing-to-track-events
Viewing Authentication Events
There are several ways to access the saved event data, each with their own pros/cons.
Via API
This is probably the most straightforward approach.
Authenticate as an admin into keycloak
Call the relevant API
There are several fields which can be used for querying
Paging is supported
The response is JSON, so easily consumed by automation/scripts
See https://www.keycloak.org/docs-api/latest/rest-api/index.html#_realms_admin (search for “GET /admin/realms/{realm}/events“)
Examples:
Events
The following will return the first 100 LOGIN and LOGIN_ERROR events for this specific user since August 30th:
curl 'https://keycloak-cmcnally.int.aws.folio.org/admin/realms/cmcnally/events?dateFrom=2024-08-30&user=5830af3f-2d58-4d15-885b-1fedbb1ff43f&type=LOGIN&type=LOGIN_ERROR&first=0&max=101' -H "Authorization: $TOK"
[
{
"time": 1725041408217,
"type": "LOGIN",
"realmId": "da448374-1c7f-44fc-be4d-3ff779c4b027",
"clientId": "cmcnally-application",
"userId": "5830af3f-2d58-4d15-885b-1fedbb1ff43f",
"sessionId": "b674af92-8b65-45f0-90b6-ce55f054da28",
"ipAddress": "##.##.##.##",
"details": {
"auth_method": "openid-connect",
"auth_type": "code",
"redirect_uri": "https://cmcnally.int.aws.folio.org/oidc-landing",
"consent": "no_consent_required",
"code_id": "b674af92-8b65-45f0-90b6-ce55f054da28",
"username": "cmcnally"
}
},
{
"time": 1725041406304,
"type": "LOGIN_ERROR",
"realmId": "da448374-1c7f-44fc-be4d-3ff779c4b027",
"clientId": "cmcnally-application",
"userId": "5830af3f-2d58-4d15-885b-1fedbb1ff43f",
"ipAddress": "##.##.##.##",
"error": "invalid_user_credentials",
"details": {
"auth_method": "openid-connect",
"auth_type": "code",
"redirect_uri": "https://cmcnally.int.aws.folio.org/oidc-landing",
"code_id": "b674af92-8b65-45f0-90b6-ce55f054da28",
"username": "cmcnally"
}
}
...elided...
]Admin Events
The following will return the first 100 admin events since August 30th w/ Resource Type = USER:
curl 'https://keycloak-evrk2.int.aws.folio.org/admin/realms/diku2/admin-events?resourceTypes=USER&dateFrom=2024-08-30&first=0&max=101' -H "Authorization: $TOK"
[
{
"time": 1725040740025,
"realmId": "da448374-1c7f-44fc-be4d-3ff779c4b027",
"authDetails": {
"realmId": "4b18a3bb-5737-4573-bb90-eb501564fc80",
"clientId": "a5bcffb0-0f42-4a05-b44f-ffe86d07aac7",
"userId": "c9fdad52-4d80-43f3-944f-4875b4d0da9a",
"ipAddress": "##.##.##.##"
},
"operationType": "UPDATE",
"resourceType": "USER",
"resourcePath": "users/1f3a5447-1d51-4498-a311-095b804729f4",
"representation": "{\"id\":\"1f3a5447-1d51-4498-a311-095b804729f4\",\"username\":\"22testuser33\",\"firstName\":\"Mary\",\"lastName\":\"Katarina\",\"email\":\"sampleemail@email.com\",\"emailVerified\":false,\"attributes\":{\"test\":[\"foo\"],\"user_id\":[\"76ef42f8-b7db-4a94-9997-1fa00aa2b57e\"]},\"createdTimestamp\":1721374579491,\"enabled\":true,\"totp\":false,\"disableableCredentialTypes\":[],\"requiredActions\":[],\"federatedIdentities\":[],\"notBefore\":0,\"access\":{\"manageGroupMembership\":true,\"view\":true,\"mapRoles\":true,\"impersonate\":true,\"manage\":true}}"
}
]Pros
Doesn’t require any additional configuration (compared to logging which does)
Flexible, you can get all events, or query for specific events or event types
Cons
Depending on the configuration, (e.g. types of events you want to save, how long you want to keep them, etc.) this could consume a large amount of storage.
Via Logging
I believe it’s possible to configure Keycloak so that these events are logged, even if not saved in the Keycloak database. Unfortunately I have been unable to verify this since we’re presently hardcoding the log level in our start.sh scripts built into the docker image. See KEYCLOAK-23: Allow log level to be specified via environment variableClosed
This blog post is quite old at this point, but may still be accurate. It suggests using the JBoss-CLI to configure this. However, I wonder if it could be as simple as specifying the log level of the org.keycloak.events category to DEBUG.
NOTE: additional investigation is needed to figure out the appropriate log category for the admin events.
Pros
Events are logged but aren’t required to be saved in the database. So database storage is unaffected.
Cons
Less control over which types of events are captured.
Could result in bloated logs
Post processing of logs is required to find the events you’re interested in, to generate reports, etc.
Via SQL
When configured to save events, this information is persisted in the Keycloak database, and can be directly queried via SQL.
Examples
Events
The following will return the LOGIN and LOGIN_ERROR events for this specific user since midnight August 30th:
keycloak=> SELECT * FROM event_entity WHERE (type = 'LOGIN' OR type = 'LOGIN_ERROR') AND realm_id = 'da448374-1c7f-44fc-be4d-3ff779c4b027' AND user_id = '5830af3f-2d58-4d15-885b-1fedbb1ff43f' AND event_time > 1724990400000;
[ RECORD 1 ]----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
id | fe221844-2f10-4175-9214-c2f2e3e653fa
client_id | cmcnally-application
details_json |
error | invalid_user_credentials
ip_address | ##.##.##.##
realm_id | da448374-1c7f-44fc-be4d-3ff779c4b027
session_id |
event_time | 1725041406304
type | LOGIN_ERROR
user_id | 5830af3f-2d58-4d15-885b-1fedbb1ff43f
details_json_long_value | {"auth_method":"openid-connect","auth_type":"code","redirect_uri":"https://cmcnally.int.aws.folio.org/oidc-landing","code_id":"b674af92-8b65-45f0-90b6-ce55f054da28","username":"cmcnally"}
-[ RECORD 2 ]----------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
id | b7c21882-ad5e-4cc9-8adc-e160d3351a9a
client_id | cmcnally-application
details_json |
error |
ip_address | ##.##.##.##
realm_id | da448374-1c7f-44fc-be4d-3ff779c4b027
session_id | b674af92-8b65-45f0-90b6-ce55f054da28
event_time | 1725041408217
type | LOGIN
user_id | 5830af3f-2d58-4d15-885b-1fedbb1ff43f
details_json_long_value | {"auth_method":"openid-connect","auth_type":"code","redirect_uri":"https://cmcnally.int.aws.folio.org/oidc-landing","consent":"no_consent_required","code_id":"b674af92-8b65-45f0-90b6-ce55
f054da28","username":"cmcnally"}Admin Events
The following will return the admin events since midnight August 30 w/ Resource Type = USER:
Pros
like using the APIs, very flexible
May be slightly easier to script in that you don’t have to worry about expiring access tokens like you do with the API approach.
Performing this doesn’t require authentication into Keycloak (thereby creating more events). Here you’re connecting directly to the database.
Cons
May be more cumbersome to work with these results since they’re not in JSON.
Open Issues/Questions
What exactly is required to setup the logging approach?
Does the logging approach still work w/o configuring Keycloak to save certain events?
Details of the specific event retention periods, event types, etc. are still TBD.