Improve handling of COUNTER API Exceptions
Status
Status | In Progress |
Category | EPIC |
Start | June 2025 |
Description
The eUsage app should handle individual CoP error codes during harvesting. At the moment, the program only recognizes exceptions, but does not differentiate between them in terms of content. For this reason, the program must be expanded to respond to the various cases of exceptions.
Use case
Statistics show that CoP errors occur in the form of peaks. This means that as soon as errors occurs, it multiplies. Example: more than 1000 reports with CoP error codes during peak periods. This can happen, for example, if credentials changes or servers or reports are unavailable. This number of errors can only be handled by the user with great effort. These errors must be automatically intercepted and processed accordingly.
Workaround
Manual error analysis by users.
Deleting reports with errors by users.
Restarting harvesting.
Harvesting could be started manually and errors could be handled manually, but depending on the number of UDPs this could be a lot of work.
Concept
The following recommendation assumes that the exceptions are used as specified in the Code of Practice (CoP). If a usage data provider uses exceptions in a wrong way (which should be reported to COUNTER), a manual handling of the exceptions might be required.
Handling of exception codes
See also: COUNTER Code of Practice Release 5.1, Appendix D: Handling Errors and Exceptions
EPIC https://folio-org.atlassian.net/browse/UIEUS-429
Exception Code | UXPROD | Status | Exception Message | Comment | Implementation plan | Retry? | Retry after | Retry count |
|---|---|---|---|---|---|---|---|---|
0 |
| closed (Won’t Do) | {Info or Debug Message} | Exception with codes below 1000 are only for custom information, debugging, warning messages. They must not be used instead of the standard exceptions, so it should be possible to safely ignore them. | No individual handling. The report ends as a report with an error. | no |
|
|
1-999 |
| closed (Won’t Do) | {Warning Message} | No individual handling. The report ends as a report with an error. | no |
|
| |
1030 | closed (Won’t Do) | Insufficient Information to Process Request | Should not occur with FOLIO, unless the usage data host wronly uses this exception instead of the more specific ones below. | No individual handling. The report ends as a report with an error. | no |
| no | |
2011 | closed (Won’t Do) | Global Reports Not Supported |
| Global reports may be handled in more detail with eUsage at a later date. There is currently no use case. | Cannot be handled like CoP 3000 Consultation with PO from the FOLIO App Open Access: ‘A load-compatible variant should be available in the long term.’ | |||
3020 | closed (Won’t Do) | Invalid Date Arguments | Should not occur with FOLIO, unless wrongly used by the usage data host. | No individual handling. The report ends as a report with an error. | no |
| no | |
3040 | needs to be discussed (SME) | Partial Data Returned | These should be permanent conditions, unless wrongly used by the usage data host. |
|
|
|
| |
3050 | closed (Won’t Do) | Parameter Not Recognized in this Context | Should not occur with FOLIO unless a usage data host is not COUNTER compliant or not handling the request correctly. Exception 3063 should not occur with FOLIO unless component support would be added (very complex, only supported by a few usage data host). | No individual handling. The report ends as a report with an error. | no |
| no | |
3060 | closed (Won’t Do) | Invalid ReportFilter Value | ||||||
3061 | closed (Won’t Do) | Incongruous ReportFilter Value | ||||||
3062 | closed (Won’t Do) | Invalid ReportAttribute Value | ||||||
3063 | closed (Won’t Do) | Components Not Supported |
|
|
|
| ||
3070 | closed (Won’t Do)
| Required ReportFilter Missing | No individual handling. The report ends as a report with an error. | no |
| no | ||
3071 | Counter 4? | closed (Won’t Do) | Required ReportAttribute Missing | These exceptions should never occur, they were removed from the Code of Practice in Release 5.0.2. | The harvester for Counter 4 will be removed. | |||
3080 | Counter 4? | closed (Won’t Do) | Limit Requested Greater than Maximum Server Limit | |||||
The most error handling can be grouped into categories in eUsage. Most exceptions can be handled with these groups. The remaining cases will be added later.
Exception handling for CoP Error codes 1XXX
User story
https://folio-org.atlassian.net/browse/UXPROD-5746
Exception Code | UXPROD | Status | Exception Message | Comment | Implementation plan | Retry? | Retry after | Retry count |
1000 | needs to be discussed (SME) | Service Not Available | These should be a temporary conditions for the UDP (not report specific). If the retries would be counted, more failed attempts should be allowed than for 3031. | Solution proposed by the developer: Optional additional individual settings for the harvester per UDP. These optional settings would be created in the edit mode of the UDP record when configuring the harvester. This solution allows you to control behavior according to UDP. | yes | 10 min | no? | |
1010 | needs to be discussed (SME) | Service Busy | yes | 10 min | no? | |||
1011 | needs to be discussed (SME) | Report Queued for Processing | Not an error, try again after a short time. | yes | 10 min | no | ||
1020 | needs to be discussed (SME) | Client has made too many requests | Same as 1000/1010. Could be report specific, but sometimes is not. The handling of the exception will be changed. The harvester should stop and make no further attempts. Instead, an error message is displayed in the Detail View. This can also be filtered to search for it. | no | After correcting the configuration | no |
Exception handling for CoP Error codes 2XXX
Please note: 2011 is not included.
User story
https://folio-org.atlassian.net/browse/UXPROD-5754
A new feature is being developed that will enable credentials to be checked initially when creating a UDP.
The harvester is adjusted so that it responds to CoP error codes.
Exception Code | UXPROD | Status | Exception Message | Comment | Implementation plan | Retry? | Retry after | Retry count |
2000 | discussed - in preparation for Refinement | Requestor Not Authorized to Access Service | UDP configuration needs to be fixed. Automatically retry after fix? Exception 2011 could only occur if Customer_ID 0000000000000000 would be used to request global reports.
|
Exception 2011 Requires special handling.
Developer: There is a separation between UDP and Harvester in the backend. A solution is being sought that will allow the harvester to read the errors. Only then can the harvester respond accordingly. One suggestion would be to only allow the initial harvesting to start after prior verification of credentials. The credentials must therefore be checked. A solution is being sought during ongoing operations. A possible solution for ongoing harvesting would be that when something is reported, the harvester stops and the initial check must be repeated. The trigger for it must be checked from the technical standpoint. SME recommends checking the credentials by means of a retrieval, i.e., an attempt should be made to download a report during the initial check. | no | After correcting the configuration | no | |
2010 | discussed - in preparation for Refinement | Requestor is Not Authorized to Access Usage for Institution | no | After correcting the configuration | no | |||
2020 | discussed - in preparation for Refinement | APIKey Invalid | no | After correcting the configuration | no |
Exception handling for CoP Error codes 3XXX
User story
https://folio-org.atlassian.net/browse/UXPROD-5711
Exception Code | UXPROD | Status | Exception Message | Comment | Implementation plan | Retry? | Retry after | Retry count |
3000 | discussed - in preparation for Refinement | Report Not Supported | SME: R5 only, in R5.1 the HTTP status code 404 is used instead. | Possible redesign of report type selection. | no | After correcting the configuration | no | |
3010 | discussed - in preparation for Refinement | Report Version Not Supported | no | After correcting the configuration | no |
Exception handling for CoP Error codes 303X (UI only)
In these cases, only the icon used is changed. Therefore, nothing is changed in the harvester and only the UI controls what is displayed.
User stories
Exception Code | UXPROD | Status | Exception Message | Comment | Implementation plan | Retry? | Retry after | Retry count |
3030 | closed (Done) | No Usage Available for Requested Dates | Not an error, unless wrongly used by the usage data host. Maybe this exception could be shown with a different symbol or color (lighter or more yellowish green?), so that this case can easily be identified. | New Icon in the overview of the Counter Report statistics. Lighter green and a 0 as the symbol. Also called 0-reports. The reason for the symbol with a 0. | yes | User configuration in the settings | yes | |
3031 | discussed - in preparation for Refinement | Usage Not Ready for Requested Dates | Reports should be available within four weeks after the end of the month, but often are available much ealier, depending on the usage data host. How many retries should be permitted and how ofter a retry should be attempted depends on when the first attempt is started. | Only new Icon: calendar (yellow) | yes | 3 days? User configuration in the settings | yes | |
3032 | discussed - in preparation for Refinement | Usage No Longer Available for Requested Dates | These should be permanent conditions, unless wrongly used by the usage data host. | Only new Icon: calendar (red) | yes | User configuration in the settings | yes |
Harvester (Backend)
The backend of the harvester needs to be redesigned. The current design cannot meet the new requirements. See https://folio-org.atlassian.net/browse/UXPROD-5578