[UICIRC-178] SPIKE - Tenant-selected timezone compliance for backend modules Created: 08/Jan/19  Updated: 19/Sep/19  Resolved: 15/Feb/19

Status: Closed
Project: ui-circulation
Components: None
Affects versions: None
Fix versions: None

Type: Story Priority: P2
Reporter: sthomas (Inactive) Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: Text File Timezones.txt    
Issue links:
Blocks
blocks FOLIO-1778 Add dev document for working with dat... Closed
Defines
defines MODCAL-29 Change GET /calculateopening API endp... Closed
defines MODCAL-30 Change GET /calendar/periods endpoint... Closed
defines MODCAL-32 Change GET, PUT and DELETE /calendar/... Closed
Relates
relates to UXPROD-1439 Tenant-selected timezone compliance f... Closed
relates to CIRC-434 Loan due date being pushed out 2 hour... Closed
Sprint: EPAM-Veg Sprint 7, EPAM-Veg Sprint 8, EPAM-Veg Sprint 9
Story Points: 8
Development Team: Vega

 Description   

All due date calculations need to respect the timezone chosen for the tenant. While UIORG-55 Closed established the ability to set a time zone for the tenant, dependent back end modules that are used in the calculation of specific date+time operations for circulation-related activities do not currently perform calculations and return accurately translated results in the tenant-selected form.

The SPIKE should be used to define general technical solution, how to make timezone setting effective throughout the system.

Investigation of scope of work across affected modules is out of scope of this particular SPIKE.

Original observations:

  • Dates and times are stored with the back-end modules as UTC.
  • Timesone is set in Settings - Organization - Language and localization - Time zone
  • Current timezone format: name of a timezone from Tz database (https://en.wikipedia.org/wiki/Tz_database) without offset values

Deliverables:
-Technical solution, approach how to apply timezone setting will work

  • Will create a document and present to Marc, Sean, and Vega team for review
  • Will post document to confluence

Questions:

  • What other kinds of date time calculations do we have (I've only described opening times and end of a specific day)?
  • What should happen when no timezone has been chosen (the UI currently internally defaults)?
  • How to support Daylight Saving Time?


 Comments   
Comment by sthomas (Inactive) [ 08/Jan/19 ]

Marc Johnson and Kostyantyn Khodarev - I currently have this spike set to Draft because I want to make sure that I'm accurately describing the issue and scoping this effort correctly. Can you let me know if there are any changes that need to be made? I also haven't set Vega to the assigned team as it wasn't clear during our discussion whether other teams should be a part of this effort. Can you let me know your thoughts? Thanks.

Comment by sthomas (Inactive) [ 08/Jan/19 ]

Marc Johnson and Kostyantyn Khodarev - Once we've established scope and investigator(s), we'll want to get an estimate in place for the level of effort required to perform the Spike and get that into our capacity planning. Thanks.

Comment by Marc Johnson [ 10/Jan/19 ]

sthomas Kostyantyn Khodarev Jakub Skoczen Zak Burke Here is a summary of my understanding. Thoughts welcome.

Situation

  • Backend modules are not aware of the chosen timezone and so do not respect it in calculations

Desired Outcome

  • All calculations involving date and time should respect the chosen timezone

Assumptions

  • Stripes provides mechanisms for creating date and time representations that respect the chosen time zone
  • Stripes provides mechanism for converting date and time representations between chosen time zone and UTC
  • UI modules use these mechanisms consistently
  • The UI defaults the timezone internally (to the UI) to UTC
  • The reference environments don't define a chosen timezone

Spike Scope

  • Focused on due date calculations involved in mod-circulation
  • Some general decisions / patterns may need to be established to achieve this
  • Might involve identification of other areas which need changing

Highlights of due date calculation (based upon current work in CIRC-159 Closed pull request)

  • Initial due date is calculated ignorant of the chosen timezone (by moving the loan date on by the duration stated in the policy)
  • Long term loan due dates are not the end of the day (as decided by the RA SIG and described in UICHKOUT-451 Closed )
  • The opening hours for the initial due date, the previous and next open days for a service point are calculated by mod-calendar based upon the opening hours (from recent discussions with Kostyantyn Khodarev )
  • Which of those days are chosen depends upon closed due date management

Decisions

  • What is the source of truth for the chosen timezone (I'm aware most people think that is already decided as a setting stored using the configuration API, however we don't really exercise that)
  • What should happen when no timezone has been chosen (the UI currently internally defaults)?
  • How to calculate the end of a specific day (being one second before midnight on that day in the chosen time zone)?
  • How to calculate a date and time from a time definition (e.g. turning 8:00 AM on a Friday, into a date time on a specific Friday in the chosen timezone)?
  • Where should those calculations be made (i.e. what is the interaction between modules are needed to do this)?
  • Do we need any additional tooling support to aid making these calculations?

Questions

  • What other kinds of date time calculations do we have (I've only described opening times and end of a specific day)?
  • I don't understand the FOLIO definition of a spike well enough to know what the deliverables typically are. Should this involve a demonstrate of the decisions in practice?
Comment by Khalilah Gambrell [ 10/Jan/19 ]

sthomas, Kostyantyn Khodarev, Marc Johnson, Vega will conduct sprint planning tomorrow. Do we want to consider this spike or some aspect of this spike in the next sprint? If so, can we confirm today?

Comment by Kostyantyn Khodarev [ 14/Jan/19 ]

sthomas Marc Johnson Jakub Skoczen Does FOLIO support Daylight Saving Time?
If no timezone selected - apply UTC?
If FOLIO currently stores all date-time values in UTC it makes sense to do all calculations in UTC.
Convert local date-time to UTC as soon as possible and convert back to local date-time as late as possible

Comment by Cate Boerema (Inactive) [ 15/Jan/19 ]

sthomas, Khalilah Gambrell, Marc Johnson it would be good to make some progress on this investigation and it looks like we want input from Core and Vega. Can we put a spike for each team into Sprint 55 to ensure it gets some attention?

Comment by Khalilah Gambrell [ 15/Jan/19 ]

Cate Boerema, Kostyantyn Khodarev will revise this spike so we can pull into Sprint 55. Once he completes revision, I believe the plan is to create additional spikes for Core and Vega.

Comment by Marc Johnson [ 16/Jan/19 ]

Kostyantyn Khodarev Here are my thoughts on your questions.

Does FOLIO support Daylight Saving Time?

I believe FOLIO uses the tz form of timezone which I believe means that daylight saving transitions are included in the definition. This is probably worth investigating as part of the spike.

If no timezone selected - apply UTC?

I believe this overlaps with the decision:

What should happen when no timezone has been chosen (the UI currently internally defaults)?

My personal preference is that the system should fail if not timezone has been chosen (though this also means that the system will fail if we choose to store this information in the configuration API and this is transitively unavailable).

If FOLIO currently stores all date-time values in UTC it makes sense to do all calculations in UTC.

I don't think it makes a difference which timezone calculations are performed in, as long as the date and times that are created by the system reflect the chosen timezone (e.g. end of day in ET, can be converted to UTC if desired, but the original decision needs to be ET).

It might be that I have a different understanding of calculations, what do you mean by calculations?

By calculations I refer to moving a date and time by an period of time, working out the difference or comparison. But not the decision of what a fragment (e.g. 08:00) should mean when turned into a date and time. Does that make sense?

Convert local date-time to UTC as soon as possible and convert back to local date-time as late as possible

It has been previously agreed that all date and time representations in requests or responses to API endpoints, and in storage, are intended to be UTC. I think we could review that decision.

By convert back to local date-time as late as possible, I interpret that as typically only done in presentation, e.g. UI. Is that a sensible interpretation?

Comment by Marc Johnson [ 18/Jan/19 ]

Kostyantyn Khodarev Khalilah Gambrell Given this has been pulled into the current sprint, is it possible to expand upon what

Technical solution, approach how to apply timezone setting

means in terms of scope for this work?

Comment by Kostyantyn Khodarev [ 18/Jan/19 ]

Marc Johnson Technical solution here means a document with a description of how to work with date-time (how to store, what java library to use, request-response parameters format etc.), some general, module ignorant approach

It has been previously agreed that all date and time representations in requests or responses to API endpoints, and in storage, are intended to be UTC. I think we could review that decision.

Could you provide a link to a doc or story where it was agreed that all date and time representations in requests or responses to API endpoints are intended to be UTC? The only document i found is about storing date and time in UTC.

Comment by Marc Johnson [ 18/Jan/19 ]

Kostyantyn Khodarev Ok, thank you. From your response, my understanding is that this work will provide all of the decisions I outlined in my post above, and likely more.

Could you provide a link to a doc or story where it was agreed that all date and time representations in requests or responses to API endpoints are intended to be UTC

I'm basing this on a comment Zak Burke made on UICHKOUT-451 Closed referring to the internationalization guidelines, that state that front-end code should assume the time-related values it exchanges with the backend are in UTC. When a date or time is formatted for display using react-intl, the tenant's timezone will be automatically incorporated.

There might be other documentation in JIRA issues, a cursory search didn't find the specific decision, [DMOD-256] and FOLIO-592 Closed includes some of the discussion. It is mostly based upon support for the date-time format defined in the validation standard leverages RFC3339

Comment by Kostyantyn Khodarev [ 18/Jan/19 ]

Marc Johnson ok, thanks. Probably it's good to have a single format for all date-time parameters that includes time zone information.

About DST support. Due to the fact FOLIO uses region-based IDs DST support is out of the box (https://docs.oracle.com/javase/8/docs/api/java/time/ZoneId.html, Time-zone IDs section)

Comment by Marc Johnson [ 22/Jan/19 ]

Given that this has been re-scoped to general guidance for all modules, I have renamed the issue.

Will create a document and present to Marc, Sean, and Vega team for review

Given that these decisions are owned by the core platform team, I think the review needs to include a broader audience, including Jakub Skoczen and other folks.

Comment by Jakub Skoczen [ 31/Jan/19 ]

Kostyantyn Khodarev Marc Johnson

IMHO, there are two main areas here:

  • date-time fields in various FOLIO APIs – those fields should be formatted with JSON Schema format: date-time which ALWAYS require a timezone designator (explicit indication of what is the timezone of the date-time). Those fields point at a specific point-in-time which is independent from the Tenant timezone setting changes. Those fields can be expressed using any timezone – it doesn't matter as long as the timezone IS specified. Internally FOLIO may store those fields by converting them to UTC – again this should be not a problem because all the conversion does is that it changes how the field is "rendered". The absolute point-in-time it represents does NOT change. Those fields are self-contained and there is no impact on them when the Tenant timezone setting changes. Example of such fields include loanDate, dueDate, returnDate, systemReturnDate, etc. There might cases in FOLIO where fields of this type are not properly defined as format: date-time, those would be bugs that need fixing.
  • time-of-day fields relative to the Tenant timezone – this would include things like opening hours or general time-of-day designations (end of day, begning of day). Those are timezone relative and usually take part in calculations (e.g due date) that as a result should provide a timezone-independent date-time field (so one of the things descibed in the previous bullet). When such calculation is performed the Tenant timezone setting should be read from mod-configuration.
Comment by Jakub Skoczen [ 04/Feb/19 ]

David Crossley Kostyantyn Khodarev we discussed that it makes sense to put up a short documentation on the dev website about working with dates and date calculations in FOLIO. Kostya will prepare the write-up, can you help him finding the right place to put it up?

Comment by David Crossley [ 04/Feb/19 ]

Please provide the content text and i will expand it into an actual document. See FOLIO-1778 Closed .

Comment by Kostyantyn Khodarev [ 04/Feb/19 ]

David Crossley updated attached document

Comment by Kostyantyn Khodarev [ 05/Feb/19 ]

Example of API endpoint with date-only query parameters:
/calendar/periods

  • startDate: 2018-05-01

Should this be converted to format: datetime (RFC3339) ?

Comment by Jakub Skoczen [ 08/Feb/19 ]

Kostyantyn Khodarev Marc Johnson Marc might have a different opinion but I don't think it should – if you do, changing the tenant timezone will not have any impact on them (as we have discussed above) so you would suddenly see "days" starting/ending in the middle of the day.

Comment by Marc Johnson [ 09/Feb/19 ]

Kostyantyn Khodarev I think there are likely cases where a date is a more appropriate construct, rather than a datetime.

My interpretation of RFC 3339 is that it defines the full-date representation for this purpose, and that RAML 1.0 defines the date-only type for this purpose.

Hence, I think each of the examples need considering in context, and we should use the most semantically appropriate type to convey the meaning and expectations to clients.

What does the endpoint GET /calendar/periods?startDate=X do?

When a client asks the following question, what should it expect as the answer?

GET /calendar/periods?startDate=2019-02-15
Comment by Kostyantyn Khodarev [ 11/Feb/19 ]

Marc Johnson
It returns opening hours starting from requested date, closed days are also included

{
    "openingPeriods": [
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "08:00",
                        "endTime": "17:00"
                    }
                ],
                "allDay": false,
                "open": true,
                "exceptional": false
            },
            "date": "2019-02-15T00:00:00.000+0000"
        },
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "00:00",
                        "endTime": "23:59"
                    }
                ],
                "allDay": true,
                "open": false,
                "exceptional": false
            },
            "date": "2019-02-16T00:00:00.000+0000"
        },
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "00:00",
                        "endTime": "23:59"
                    }
                ],
                "allDay": true,
                "open": false,
                "exceptional": false
            },
            "date": "2019-02-17T00:00:00.000+0000"
        },
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "08:00",
                        "endTime": "17:00"
                    }
                ],
                "allDay": false,
                "open": true,
                "exceptional": false
            },
            "date": "2019-02-18T00:00:00.000+0000"
        },
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "00:00",
                        "endTime": "23:59"
                    }
                ],
                "allDay": true,
                "open": false,
                "exceptional": false
            },
            "date": "2019-02-19T00:00:00.000+0000"
        },
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "00:00",
                        "endTime": "23:59"
                    }
                ],
                "allDay": true,
                "open": false,
                "exceptional": false
            },
            "date": "2019-02-20T00:00:00.000+0000"
        },
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "08:00",
                        "endTime": "17:00"
                    }
                ],
                "allDay": false,
                "open": true,
                "exceptional": false
            },
            "date": "2019-02-21T00:00:00.000+0000"
        },
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "08:00",
                        "endTime": "17:00"
                    }
                ],
                "allDay": false,
                "open": true,
                "exceptional": false
            },
            "date": "2019-02-22T00:00:00.000+0000"
        },
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "00:00",
                        "endTime": "23:59"
                    }
                ],
                "allDay": true,
                "open": false,
                "exceptional": false
            },
            "date": "2019-02-23T00:00:00.000+0000"
        },
        {
            "openingDay": {
                "openingHour": [
                    {
                        "startTime": "00:00",
                        "endTime": "23:59"
                    }
                ],
                "allDay": true,
                "open": false,
                "exceptional": false
            },
            "date": "2019-02-24T00:00:00.000+0000"
        }
    ],
    "totalRecords": 75
}
Comment by Kostyantyn Khodarev [ 11/Feb/19 ]

Marc Johnson Jakub Skoczen you are right, in some cases date-only option is more appropriate than full date
But date and time value is always stored as RFC3339 in DB, never seen different example

So, you propose to use formats like date-only for request parameters where time is not needed, correct?
For entity properties in json schemas i would go with full date-time values as it always stores in RFC3339

Comment by sthomas (Inactive) [ 15/Feb/19 ]

It's my understanding from a discussion with Kostyantyn Khodarev that there is general agreement to the approach that's described here. EPAM-Vega is moving forward with development related to mod-cal.

Comment by Jakub Skoczen [ 28/Feb/19 ]

Kostyantyn Khodarev should the attached timezone.txt document be updated with information about full-date and date-only query parameters before David puts the contents on dev.folio.org?

Marc Johnson?

Comment by Kostyantyn Khodarev [ 28/Feb/19 ]

Jakub Skoczen Yes, i'll update document with information about date-only query parameter
UPD: document updated

Comment by David Crossley [ 06/Mar/19 ]

The document is now added. See FOLIO-1778 Closed .

Generated at Fri Feb 09 00:15:23 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.