Sprint 182

Sprint 182

• Triage

  • previously created new Grails 6 project and incrementally added components
  • once pitfalls identified, each module took very little time
• Approach
  • will starting up a new grails project with the CLI even work?

• Components and Changes

  • Frontend

  • Backend

• Tests / Data / Dependencies

  • gradle upgrade is necessary

• Known Unknowns

  • will starting up a new grails project with the CLI even work?

• QA: snapshot | local | testing | other

• Release Target: 

• Development Estimate

• Current:
  • grails and gradle versions determined
  • web-toolkit passing tests
  • smoother process than 4 to 5, but integration is inevitably 
• TODO
  • service-interaction failing
  • need to build into okapi environment
    • may need to rebuild rancher, which could be risky at the moment
  • review migrations, but difficult to triage atm

Sprint 182

Sprint 178

• enforceVersionCheck is taken from grails default rest controller - not sure of impact to changing from false to true
• can add to to yaml without changing default to false
• would need to make changes to KIWT restful controller
• need to push version numbers
• likely not to be rolled in until Grails 6
• TODO: 
  • Jack to write up findings 
  • Owen to review with users

Sprint 178

Sprint 182

Sprint 178

Steve Osguthorpe

Sprint 180

Ethan Freestone 

Sprint 173

Ethan Freestone 

Sprint 179

Claudia Malzer 

• Triage

  • FOLIO session-level
• Approach
  • review implementation across other apps
    • eg, Orders, 
    • eg, stripes-smart-components useColumnManager 
  • can this be used as-is within ERM or does it need streamlining?

• Components and Changes

  • Frontend: TBC

  • Backend: none

• Tests / Data / Dependencies

  • gradle upgrade is necessary

  Known Unknowns

  will starting up a new grails project with the CLI even work

  : ??

• Known Unknowns: ??

• QA: snapshot | local | testing | other

• Release Target:  Quesnelia

• Development Estimate

  Current

  :

  • grails and gradle versions determined
  • web-toolkit passing tests
  • smoother process than 4 to 5, but integration is inevitably 
• TODO
  • service-interaction failing
  • need to build into okapi environment
    • may need to rebuild rancher, which could be risky at the moment
  • review migrations, but difficult to triage atm

Sprint 182

• enforceVersionCheck is taken from grails default rest controller - not sure of impact to changing from false to true
• can add to to yaml without changing default to false
• would need to make changes to KIWT restful controller
• need to push version numbers
• likely not to be rolled in until Grails 6
• TODO: 
  • Jack to write up findings 
  • Owen to review with users

Sprint 178

• ??

Jack Golding  

Triage

• • NamedThreadFactory is used in string templating service
  • out of end-of-life support, so should find a  way to upgrade/remove
• Approach
  • need to review any documented upgrade path - is there a straightforward path to go from 2 to 3
  • are we vulnerable to the jetty-io vulnerabilty?
    • influences backport requirements (esp Orchid which is on Grails 4)

• Components and Changes

  • Frontend: NA

  • Backend: 

• Tests / Data / Dependencies

• Known Unknowns

  • whether we are affected by virtue of using netty

• QA: snapshot | local | testing | other

• Release Target: 

• Development Estimate

Current: 

• upgrade path documentation doesn't help
• looks like (via grep) this is the only use of ehcache

Sprint 182

• need to decide whether to bail ehcache to use another NamedFactory provider
• potentially could do this internally, without another dependency
• run gradle clean  and check dependencies to see if ehcache/jetty-io are brought in transitively - CONFIRMED on v2

Sprint 182

Ethan Freestone 

Sprint 173

Ethan Freestone 

Sprint 179

Claudia Malzer 

Not vulnerable

• Triage

  • need to revisit what choice we made for Poppy
  • need to check whether Orchid/Poppy have the same vulnerability to determine backport requirement
  • otherwise to be dealt with (for Quesnalia) by Grails 6 upgrade
• Approach
  • Prefer not to patch grails, but should be able to bump data-binding 
  • Try 4.1.3, then 4.1.2, then 4.1.1
  • Otherwise will need to change databindings specifically - which will be messy
  • Owen: release plan

• Components and Changes

  • Frontend: NA

  • Backend: mod-agreements, mod-licenses, mod-service-interaction

    • shouldn't need grails-okapi or KIWT

• Tests / Data / Dependencies

• Known Unknowns

  • what's the minimum version that has the fix?
  • confirm that  grails-okapi and KIWT do not need bumping

• QA: snapshot | local | testing | other

  • dev: does it compile and run as expected?
  • regression testing
• Release Target: NA

• Development Estimate: NA

Sprint 182

• Triage

• • NamedThreadFactory is used in string templating service
  • out of end-of-life support, so should find a  way to upgrade/remove
• Approach
  • need to review any documented upgrade path - is there a straightforward path to go from 2 to 3
  • are we vulnerable to the jetty-io vulnerabilty?
    • influences backport requirements (esp Orchid which is on Grails 4)

• Components and Changes

  • Frontend: NA

  • Backend: fetch of loadMore is causing cache of what's opened to refresh
    • fetch is at rootLevel, so redraw pushes a different set of lines
  • which may be because of batch fetching PO lines 
  • redraw closes and remounts 
• Approach
  • needs investigation to understand chain of events leading to invalidation or refresh
    • what's the trigger?
    • is there a quick fix? 
    • if not will need a refactor of PO line fetch handling
  • Owen to revisit whether this is an issue in current versions

• Components and Changes

  • Frontend: ui-agreements

  • Backend: NA

• Tests / Data / Dependencies: 

  • none for now, potentially add to testrails catalog

• Known Unknowns

  • whether we are affected by virtue of using netty

• QA: snapshot | local | testing | other

• Release Target: 

• Development Estimate

Current: 

• upgrade path documentation doesn't help
• looks like (via grep) this is the only use of ehcache

Sprint 182

• need to decide whether to bail ehcache to use another NamedFactory provider
• potentially could do this internally, without another dependency
• run gradle clean  and check dependencies to see if ehcache/jetty-io are brought in transitively - CONFIRMED on v2

Sprint 182

• • then need to decide what's desirable behaviour - do we need to retain the user's view or reset it post-redraw?
    • if we don't then any data changes won't be reflected
    • if we do, we need to develop a reusable component for consistency across apps

• QA: snapshot 

• Release Target: Quesnelia (no backport)

• Development Estimate

Sprint 182

Owen Stephens 

21 Feb 2024  

• Not clear how to verify
• Need to follow up with Molly

Sprint 182

Ethan Freestone 

Not vulnerable

 Ethan Freestone 

• Approach

  • set up new open source project in K-Int gitlab
  • new micronaut service required (not using okapi or grails)
  • set up a test instance
  • will need a different dev workflow (probably using local copy of service)
  • working title: pushkb
  • avoid storing caches of transient data

• Components and Changes

  • Frontend: NAna

  • Backend: mod-agreements, mod-licenses, mod-service-interaction

    • shouldn't need grails-okapi or KIWT

• Tests / Data / Dependencies

• Known Unknowns

  • what's the minimum version that has the fix?
  • confirm that  grails-okapi and KIWT do not need bumping

    new

• Tests / Data / Dependencies

  • Align to when GBV start testing and using pushkb endpoint
  • ERM-3048 Possible package schema changes  with endpoint changes

• Known Unknowns

  • should be (but not clear yet) about TitleInstance Resolver
  • packageSchemaUpdater: potential conflicts when there are multiple copies of schemas in play where backwards compatibility is broken

• QA: snapshot | local | testing | other

  • dev: does it compile and run as expected?
  • regression testing
• Release Target: NA  Quesnalia

• Development Estimate: NA

Sprint 182

• Triage

  • fetch of loadMore is causing cache of what's opened to refresh
    • fetch is at rootLevel, so redraw pushes a different set of lines
  • which may be because of batch fetching PO lines 
  • redraw closes and remounts 
• Approach
  • needs investigation to understand chain of events leading to invalidation or refresh
    • what's the trigger?
    • is there a quick fix? 
    • if not will need a refactor of PO line fetch handling
  • Owen to revisit whether this is an issue in current versions

• Components and Changes

  • Frontend: ui-agreements

  • Backend: NA

• Tests / Data / Dependencies: 

  • none for now, potentially add to testrails catalog

• Known Unknowns

  • then need to decide what's desirable behaviour - do we need to retain the user's view or reset it post-redraw?
    • if we don't then any data changes won't be reflected
    • if we do, we need to develop a reusable component for consistency across apps

• QA: snapshot 

• Release Target: Quesnelia (no backport)

• Development Estimate

Sprint 182

Owen Stephens 

Not clear how to verify

• Triage

• Approach

• Components and Changes

  • Frontend

  • Backend

• Tests / Data / Dependencies

• Known Unknowns

• QA: snapshot | local | testing | other

• Release Target: 

• Development Estimate

Sprint 182

Ethan Freestone 

• Current State
  • pushKb can make a scheduled call (1s, 1h) to Gokb
  • will fetch roughly ~700k TIPPS 
  • Stores these in a local postgres db
  • proteus json transform implemented with proof of concept schema
• TODO
  • scaling
  • extensive transformation testing
  • write fetch chunks and push to FOLIO,
    • implementing algorithm to ensure no data gaps 
    • deciding what to do with the data volume (700k feels beyond FOLIO's performance capability)
    • at a minimum: need to move the FOLIO side to being not an open http request per chunk (would be killed by  okapi), by, eg, 
      • small batch sizes? would take full day for initial ingest
      • speeding up FOLIO? (not clear how this could be done)
      • move pushKb to background thread (like jobs, but on a much smaller scale)? 
• Constraints
  • want to avoid jobs being badly interrupted by a docker container being restarted
  • tradeoff is that we have a slower but more stable process 
• Development Estimate

 Ethan Freestone 

• Approach

  • set up new open source project in K-Int gitlab
  • new micronaut service required (not using okapi or grails)
  • set up a test instance
  • will need a different dev workflow (probably using local copy of service)
  • working title: pushkb
  • avoid storing caches of transient data

• Components and Changes

  • Frontend: na

  • Backend: new

• Tests / Data / Dependencies

  • Align to when GBV start testing and using pushkb endpoint
  • ERM-3048 Possible package schema changes  with endpoint changes

• Known Unknowns

  • should be (but not clear yet) about TitleInstance Resolver
  • packageSchemaUpdater: potential conflicts when there are multiple copies of schemas in play where backwards compatibility is broken

• QA: snapshot | local | testing | other

• Release Target:  Quesnalia

• Development Estimate

• Current State
  • pushKb can make a scheduled call (1s, 1h) to Gokb
  • will fetch roughly ~700k TIPPS 
  • Stores these in a local postgres db
  • proteus json transform implemented with proof of concept schema
• TODO
  • scaling
  • extensive transformation testing
  • write fetch chunks and push to FOLIO,
    • implementing algorithm to ensure no data gaps 
    • deciding what to do with the data volume (700k feels beyond FOLIO's performance capability)
    • at a minimum: need to move the FOLIO side to being not an open http request per chunk (would be killed by  okapi), by, eg, 
      • small batch sizes? would take full day for initial ingest
      • speeding up FOLIO? (not clear how this could be done)
      • move pushKb to background thread (like jobs, but on a much smaller scale)? 
• Constraints
  • want to avoid jobs being badly interrupted by a docker container being restarted
  • tradeoff is that we have a slower but more stable process 
• Development Estimate

Owen Stephens 

• Triage

  • needs some investigation work
  • priority is for GOKb
• Approach

• Components and Changes

• Known Unknowns

  • how do we want to track (count) the external (remote) counts? 
    • In the XML for the package rather than database
    • once the package XML is in hand, count all TIPPs with status=current 
  • where do we want to populate from in other cases?

• QA: snapshot | local | testing | other

• Release Target: 

• Development Estimate

Owen Stephens 

• Triage

  • needs some investigation work
  • priority is for GOKb
• Approach

• Components and Changes

• Known Unknowns

  • how do we want to track (count) the external (remote) counts? 
    • In the XML for the package rather than database
    • once the package XML is in hand, count all TIPPs with status=current 
  • where do we want to populate from in other cases?

• QA: snapshot | local | testing | other

• Release Target: 

• Development Estimate

• Triage

  • is this the kind of issue that pr707 is meant to have fixed?
  • review log file against pr707 zombie job
  • check that jobs are not stuck, just queued?
• Approach: subject to investigation findings

• Components and Changes

  • Frontend

  • Backend

• Tests / Data / Dependencies: NA

• Known Unknowns: check with GBV if restarting mod-agreements is viable workaround

• QA: NA

• Release Target: Quesnalia 
  • not expecting to backport, as restart may be viable workaround

• Development Estimate

21 Feb 2024 

• Issues tracking this back to federation service
• Need to revisit the pulse after zombie job changes

21 Feb 2024 

• Does title count need to be fetched also?
  • NO see ERM-3141
  • BUT test that it is fillable with a dummy number
• Does this need to be Nullable? YES 
• Update integration tests: glob this to related package schema tests
• Then ready for Code Review

• 21 Feb 2024 

• • Bug: clean up situation where log error causes downstream null pointer exception 
  • clean up for query optimisation 
  • title rename

• Triage

• Approach
  • revert locked changes and add fallback (ERM-3151) 
  • add query (ERM-3148)
  • investigate triplicate error message
  • remove the match key modelling (ERM-3153)

• Components and Changes

  • Frontend

  • Backend

• Tests / Data / Dependencies

• Known Unknowns

• QA:

•  

  • harvesting on snapshot and snapshot-2
  • initially all on live. then review 
• Release Target:

• Quesnalia

• Development Estimate

• Triage

• Approach
  • Extends the work on marking a job as interrupted
  • Closes loop on job interruption to mark data source as released ("idle")

• Components and Changes

  • Frontend: NA

  • Backend: mod-agreements

• Tests / Data / Dependencies: NA

• Known Unknowns

  QA: 

• harvesting on snapshot and snapshot-2
initially all on live. then review 

Unknowns: NA

• QA: dev

• Release Target: Quesnalia Quesnalia

• Development Estimate: Wed

21 Feb 2024 Closed as tested by dev. Not reproduceable for QA.

28 Feb 2024 

• confer with II about deprecation