EUREKA-661: Review Permission Replaces for Upgrades

Purpose

https://folio-org.atlassian.net/browse/EUREKA-661

Permission Replacement is a critical mechanism in the FOLIO Eureka platform that manages the evolution of capabilities when applications are upgraded. When modules define new permissions that replace old ones, the system must ensure all users and roles maintain their access rights without disruption.

How Permission Replacement Works

Basic Flow

When an application is upgraded, the following sequence occurs:

Event Generation: mgr-tenant-entitlements publishes capability events to Kafka, which contain permission information that may include a 'replaces' field.
Event Processing: mod-roles-keycloak receives and processes these events.
Replacement Detection: mod-roles-keycloak identifies permissions with the ‘replace’ attribute.
Assignment Migration: New capabilities and sets are assigned to all users and roles that previously had the old capabilities, searching for them by permission name.
Cleanup: Old capabilities and sets are removed after successful migration.

Key Components

mgr-tenant-entitlements
- CapabilitiesEventPublisher: Starting the process by sending messages to the Kafka
mod-roles-keycloak
- CapabilityKafkaEventHandler: Main entry point for processing capability events.
- CapabilityEventProcessor: Processes capability events and creates/updates capabilities.
- CapabilityReplacementsService: Handles the replacement logic.

Implementation Details

Permission Replacement Detection

public Optional<CapabilityReplacements> deduceReplacements(CapabilityEvent newValue) {

    // Extract permissions with 'replaces' attribute

    var permissionReplacements = newValue.getResources().stream()
        .map(FolioResource::getPermission)
        .filter(Objects::nonNull)
        .flatMap(perm -> mapToReplacesByPermission(perm))
        .collect(groupingBy(Entry::getKey, mapping(Entry::getValue, toSet())));

    // Find affected users and roles
    var oldCapabilities = capabilityService.findByPermissionNames(permissionReplacements.keySet());
    var capabilityRoleAssignments = extractAssignments(oldCapabilities, ...);
    return Optional.of(new CapabilityReplacements(...));

}

Assignment Migration

public void processReplacements(CapabilityReplacements replacements) {

    // 1. Assign new capabilities to affected users/roles
    assignReplacementCapabilities(replacements);

    // 2. Update loadable permissions
    replaceLoadable(replacements);

    // 3. Remove old capabilities
    unassignReplacedCapabilities(replacements);
}

Upgrade Scenarios

Scenario 1: Simple Application Upgrade

When upgrading a single application (e.g., app-v1.0.0 to app-v2.0.0):

{
  "type": "UPDATE",
  "new": {
    "resources": [{
      "permission": {
        "permissionName": "new.permission.view",
        "replaces": ["old.permission.view"]
      }
    }]
  },
  "old": {
    "resources": [{
      "permission": {
        "permissionName": "old.permission.view"
      }
    }]
  }
}

Process:

New capability with permission new.permission.view is created
All roles/users with old.permission.view get new.permission.view assigned
Capability with permission old.permission.view is unassigned and removed

Scenario 2: One-to-Many Replacement

In this scenario, a single, coarse-grained permission is deprecated and replaced by multiple, more granular permissions. This is common when refactoring features to provide finer control over access rights.

An application app-v1.0.0 has a single permission items.manage. In app-v2.0.0, this is replaced by three distinct permissions: items.view, items.create, and items.delete.

{
  "type": "UPDATE",
  "new": {
    "resources": [
      {
        "permission": { "permissionName": "items.view", "replaces": ["items.manage"] }
      },
      {
        "permission": { "permissionName": "items.create", "replaces": ["items.manage"] }
      },
      {
        "permission": { "permissionName": "items.delete", "replaces": ["items.manage"] }
      }
    ]
  },
  "old": {
    "resources": [
      {
        "permission": { "permissionName": "items.manage" }
      }
    ]
  }
}

Process:

The system identifies that items.manage is being replaced.
It finds all users and roles that currently have items.manage.
After a successful assignment, the original items.manage permission is removed.

Scenario 3: Many-to-One Consolidation

This is the reverse of the previous scenario, where multiple old permissions are consolidated into a single, broader permission.

A module is simplifying its permission model. app-v1.0.0 had separate permissions for inventory.items.create and inventory.items.edit. In app-v2.0.0, these are both replaced by a single permission, inventory.items.manage.

{
  "type": "UPDATE",
  "new": {
    "resources": [
      {
        "permission": {
          "permissionName": "inventory.items.manage",
          "replaces": ["inventory.items.create", "inventory.items.edit"]
        }
      }
    ]
  },
  "old": {
    "resources": [
      { "permission": { "permissionName": "inventory.items.create" } },
      { "permission": { "permissionName": "inventory.items.edit" } }
    ]
  }
}

Process:

The system identifies that two old permissions are being replaced by one new one.
It finds all users/roles with either inventory.items.create OR inventory.items.edit.
The old permissions are removed after inventory.items.manage is assigned.

Scenario 4: Circular Dependency Replacement

Context

During a major refactoring across two different modules from app-A and app-B, permissions are being renamed and reassigned. An error has been made in the module descriptors.

Problematic Configuration

app-A-v2.0.0: Defines perm.A.new, which replaces: ["perm.B.old"].
app-B-v2.0.0: Defines perm.B.new, which replaces: ["perm.A.old"].
(In a more complex variant, perm.A.new replaces perm.A.old, which is then referenced by another module that replaces it with something else, eventually leading back to perm.A.new).

Challenges:

Logical Deadlock: The system cannot resolve this dependency chain. To process the replacement for app-A, it needs to know about users with perm.B.old. To process app-B, it requires users with perm.A.old. The replacement logic would likely fail.

Scenario 5: Multiple Interdependent Applications

A scenario where multiple applications, which have dependencies on each other, are upgraded in close succession or concurrently. Permission sets replacements.

app-ui-v2.0.0 (depends on app-orders-v2.0.0 and app-items-v2.0.0)
app-orders-v2.0.0 (being upgraded, replaces orders.old.view with orders.new.view)
app-items-v2.0.0 (being upgraded, replaces items.old.edit with items.new.edit)

This case can be executed in multiple combinations:

Only upgrade to the app-orders-v2.0.0
Only upgrade to the app-items-v2.0.0
Upgrade to the app-orders-v2.0.0 and app-items-v2.0.0
Only app-ui-v2.0.0 upgrade
Upgrade to the app-ui-v2.0.0 and app-orders-v2.0.0
Upgrade to the app-ui-v2.0.0 and app-items-v2.0.0
Upgrade to the app-ui-v2.0.0, app-items-v2.0.0 and app-orders-v2.0.0

Event 1 (from app-orders upgrade):

{
  "type": "UPDATE",
  "new": {
    "moduleId": "app-orders-2.0.0",
    "resources": [{
      "permission": {
        "permissionName": "orders.new.view",
        "replaces": ["orders.old.view"]
      }
    }]
  },
  "old": {
    "moduleId": "app-orders-1.0.0",
    "resources": [{
      "permission": {
        "permissionName": "orders.old.view"
      }
    }]
  }
}

Event 2 (from app-items upgrade):

{
  "type": "UPDATE",
  "new": {
    "moduleId": "app-items-2.0.0",
    "resources": [{
      "permission": {
        "permissionName": "items.new.edit",
        "replaces": ["items.old.edit"]
      }
    }]
  },
  "old": {
    "moduleId": "app-items-1.0.0",
    "resources": [{
      "permission": {
        "permissionName": "items.old.edit"
      }
    }]
  }
}

Event 3 (from app-ui upgrade):

{
  "type": "UPDATE",
  "new": {
    "moduleId": "app-ui-2.0.0",
    "resources": [{
      "permission": {
        "permissionName": "orders.new.manage",
        "subPermissions": ["items.new.edit", "orders.new.view"],
        "replaces": ["orders.old.edit"]
      }
    }]
  },
  "old": {
    "moduleId": "app-ui-1.0.0",
    "resources": [{
      "permission": {
        "permissionName": "orders.old.manage",
        "subPermissions": ["items.old.edit", "orders.old.view"],
      }
    }]
  }
}

Challenges:

Capability sets may include capabilities not yet created
- Dummy capabilities are not taking part in the replacements
A capability that is a part of the capability set from another application can be replaced
The order of event processing matters

Current Limitations

Replacing a capability that is a part of the capability set of another module or application
Replacing the capability that was a dummy capability(created during the CapabilitySet creation)
Replacement for “dummy technical” capabilities that represent a not-created capability set.
Replacement of the capability set with capabilities that have not yet been created.

Recommendations for FSE Teams

Upgrade Sequencing
- Process applications in dependency order or via one entitlement request
- Allow time for event processing between upgrades
Monitoring
- Watch for warnings and errors in mod-roles-keycloak during and after the upgrade.
- Monitor Kafka lags
- Verify permission assignments post-upgrade (smoke tests?)
Rollback Planning
- Have rollback scripts ready
- Test rollback procedures in non-production environments

Future Improvements

https://folio-org.atlassian.net/browse/UXPROD-5435

Conclusion

Permission replacement during upgrades is a complex but essential feature that ensures continuity of access rights. While the current implementation handles most scenarios effectively, careful planning and monitoring are necessary for successful upgrades, particularly in complex application dependency scenarios.

Part of the logic responsible for permission replacement did not account for the existence of dummy capabilities.

Permission sets inside permission sets are not handled properly(not only for replacements).