...
Institutions need to be able to schedule both the age to lost and aged to lost billing processes to be run during quieter periods for their libraries e.g. when the libraries are closed. This is to ensure these processes do not interfere with the day to day processes by shifting the load on the system to a quieter part of the day.
Previous Decisions
- This aspect was excluded from the initial aged to lost work
- It was decided that FOLIO would not have general job scheduling capability, preferring that each module will provide this capability separately
- A periodic invocation mechanism was implemented in Okapi which allows a task to be executed each time a period has elapsed
Current State
At the moment, these two processes run at 30 or 35 minute intervals respectively using the Okapi _timer interface for basic scheduling.
Previous Decisions
It was decided that FOLIO would not have general job scheduling capability.
Open Questions
It has been suggested that this need is also relevant for other background processes. What are those other processesThis approach means that the process runs throughout the working day.
The use of the timer interface means that
- only a single module instance is invoked for each process after the allotted period
- invocations of each process may overlap
- invocations of the two processes may overlap
Open Questions
- What are the needed scheduling options e.g. daily, weekly, specific days of the week?
- Can a process be executed on more than one schedule?
- What should happen if one of the processes fails?
- What are other background processes that need a similar capability (some are listed below)?
Possibly Related Areas
- Scheduled Sending of scheduled patron notices also needs scheduling at a specific time
- Bursar exporting of fees / fines
- Data Export of inventory records (as it defines the idea of a job)
- Data Import (as it defines the idea of a job)
Evaluation Criteria
- How are jobs configured?
- How is it decided which module instance executes a job?
- How are jobs job executions granted permissions?
- How does the approach address concurrent execution?
- What coupling does this design introduce?
Design Options
In-module Scheduling using _timer interface
Overview
Implement a scheduling capability within mod-circulation with the aid of the existing Okapi _timer interface
Considerations
Variations
- Entirely internal scheduling - how to get permissions needed for external requests? - log in as a user, similar to pub-sub
- publish job schedule directly to Kafka topic after each invocation - then read the topic to invoke the next instantiation of the job - means cannot change schedule until next invocation has occurred
Job Scheduling Module
Overview
Considerations
...
- Schedules would be set up using an API provided by mod-circulation
- Permissions would be provided as module permission for the process endpoints
- Choice of module instance is made by Okapi (assumed to be based upon version of module enabled for a tenant)
- Module instances would need to coordinate to avoid concurrent execution of the process
- Accuracy of when the process executes is dependent upon approach taken in mod-circulation
- Requires new interface dependency for storage of scheduling and job status (assuming business logic / storage constraint still in place, most likely
mod-circulation-storage)
Entirely In-module Scheduling
Overview
Implement a scheduling capability within mod-circulation that does not rely on the existing _timer interface. Processes would be executed with
Considerations
- Schedules would be set up using an API provided by mod-circulation
- Permissions would need to be granted to a user either assigned or created for the processes
- Choice of module instance would require agreement / coordination between instances of the same version
- Module instances would need to coordinate to avoid concurrent execution of the process
- Accuracy of when the process executes is dependent upon approach taken in mod-circulation
- Requires new interface dependency for storage of scheduling and job status (assuming business logic / storage constraint still in place, most likely
mod-circulation-storage)
Scheduler Module
Overview
Provide a scheduling
Considerations
- Schedules would be set up in an API provided by the scheduling module
- Permissions would need to be granted to a user either assigned or created for the processes
- Choice of module instance would likely be made by Okapi (assuming scheduler modules uses HTTP via Okapi to start processes)
- Concurrent execution would be managed by scheduler module
- Accuracy of when the process executes is dependent upon approach taken in scheduling module
- Storage for schedule and execution management could be internal to scheduling module or a separate storage module
Variations
- Extend the scheduling capabilities of Okapi beyond the current
_timerinterface
...
- Use Kafka instead of HTTP to manage process execution
External Scheduling Tool
Overview
Job scheduling Scheduling can be done done by external tools like cron
Considerations
- Statically configured by an administrator (new jobs cannot be created by the system)Schedules would be set up in the external tool (and cannot be configured from within FOLIO)
- Permissions would need to be granted to a user either assigned or created for the processes
- Choice of module instance would likely be made by Okapi (assuming scheduler modules uses HTTP via Okapi to start processes)
- Module instances would need to coordinate to avoid concurrent execution of the process
- Accuracy of when the process executes is dependent upon the external tool
- Could be left entirely up to the hosting provider or we would FOLIO may want to set expectations (e.g. like the use of AWS or min.io for file storage)
Related Information
- Design document for aged to lost processes
- JIRA issue for this design document
- JIRA issue where scheduling was first mentioned