[FOLIO-1438] Upgrade to PostgreSQL 10 Created: 24/Aug/18  Updated: 07/Jan/20  Resolved: 09/Apr/19

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P2
Reporter: Marc Johnson Assignee: John Malconian
Resolution: Done Votes: 0
Labels: back-end, ci, core, devops, documentation, platform-backlog, sprint45, sprint48, sprint54
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
blocks FOLIO-1955 Create databases using und-x-icu coll... Open
blocks BF-264 Sorting of contributor types ignores ... Closed
blocks MODDICONV-34 Handle sorting issue for name field f... Closed
blocks UXPROD-1821 3rd party dependency upgrades (Q4 2019) Closed
blocks FOLIO-2411 Use SCRAM-SHA-256 for passwords on Po... Blocked
blocks CQLPG-26 ICU collations locale (independent of... Blocked
Duplicate
is duplicated by FOLIO-2406 SSL/TLS, SCRAM-SHA-256, migration to ... Closed
is duplicated by FOLIO-2410 EOL: PostgreSQL 9.x Closed
is duplicated by FOLIO-883 Postgresql v10 is now the current rel... Closed
Relates
relates to OKAPI-790 Upgrade PostgreSQL from 9.6 to 10 in ... Closed
Sprint: Core: Platform - Sprint 61
Story Points: 3
Development Team: Core: Platform

 Description   

The RAML module builder (and modules based upon it) has been using PostgreSQL 10 for testing since version 18.0.0 released in February 2018.

I believe the stated supported version is still 9.6 (I'm not aware of any use of 10 based features).

It would appear some behaviour is different. Julian Ladisch proposed on BF-264 Closed that we use PostgreSQL 10 as a potential short term resolution.

Based upon a quick scan of the 10.0 release notes, and the absence of mentions of ICU in the 9.6 documentation, it looks like ICU support was added in 10. Which I imagine means this is needed for CQLPG-26 Blocked .

Do we want to change our supported version to 10?

If we do, what needs to change in our modules and environments?



 Comments   
Comment by Heikki Levanto [ 24/Aug/18 ]

I am using Debian/stable (stretch) on my workstation, and it still has postgres 9.6.

On my home machine, I use Debian/testing (buster), and it has version 10 (actually, Debian version 10+192, what ever that means).

If we mandate using 10.0 for development, some people may need to install postgres manually, to get to version 10. Not impossible, but a bit of a hazzle, possibly for many people.

Comment by Julian Ladisch [ 24/Aug/18 ]

They can use Docker to run Postgres 10.x:
I use

docker run --name postgres5433 -e POSTGRES_PASSWORD=postgres -p 5433:5432 -d postgres:10-alpine
Comment by Julian Ladisch [ 24/Aug/18 ]

For the major linux distributions there is no need to install postgres 10 manually because repositories for automatic installation are available:
Follow the links at the top of the page of https://www.postgresql.org/download/linux/ for

  • Red Hat family Linux (including CentOS/Fedora/Scientific/Oracle variants)
  • Debian GNU/Linux and derivatives
  • Ubuntu Linux and derivatives
  • SuSE and OpenSuSE
Comment by Hongwei Ji [ 28/Aug/18 ]

Just a FYI: AWS Aurora does not support 10 yet. As of this writing, the latest is PostgreSQL 9.6.9.

Comment by Marc Johnson [ 29/Aug/18 ]

Hongwei Ji Is AWS Aurora planned for / is used for some of the FOLIO environments?

And if so, does that mean that upgrading to 10 is blocked on this support (or we may need to support both)?

Comment by Hongwei Ji [ 29/Aug/18 ]

Hi Marc Johnson yes, we use Aurora in our FOLIO environment. The hope is that AWS will support Aurora PostgreSQL 10 soon. If not, we can fall back to use AWS RDS. I commented it yesterday just a FYI. I was a little surprised to learn that FOLIO needs the latest feature from PostgreSQL to implement some functionalities.

Comment by Marc Johnson [ 29/Aug/18 ]

Hongwei Ji I believe a primary driver behind this upgrade is ICU collation support so that we can have localized searching and sorting. If there is an alternative mechanism for that, it might be worth exploring that.

At the moment, I believe PostgreSQL use OS provided libraries to perform this, and so can be different depending upon where it is deployed.

Comment by Julian Ladisch [ 29/Aug/18 ]

https://aws.amazon.com/rds/aurora/ says: "The Amazon Aurora database engine is fully compatible with existing MySQL and PostgreSQL open source databases, and adds compatibility for new releases regularly."
Amazon Aurora for PostgreSQL has been available for only 10 months: https://aws.amazon.com/blogs/aws/now-available-amazon-aurora-with-postgresql-compatibility/ I don't think that Folio should restrict it's PostgreSQL feature use to the limitations of such a young product.
Folio does not need the latest PostgreSQL features. PostgreSQL 10 has been released October 2017, PostgreSQL 10.5 and PostgreSQL 11 Beta 3 have been released August 2018.
I don't see that Amazon Aurora is a blocker for using PostgreSQL 10 features because there exists the AWS RDS fallback.

Comment by Julian Ladisch [ 01/Oct/18 ]

Aurora is no longer a blocker of this issue: "Amazon Aurora with PostgreSQL Compatibility now supports PostgreSQL major version 10.4, and is available in US East (N. Virginia, Ohio), US West (Oregon), and Europe (Ireland) Regions."
https://aws.amazon.com/about-aws/whats-new/2018/09/amazon-aurora-postgresql-compatibility-supports-postgresql104/

Comment by Julian Ladisch [ 07/Jan/19 ]

"Amazon Aurora with PostgreSQL Compatibility now supports PostgreSQL major version 10.5"
https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-aurora-postgresql-supports-postgresql-105/

Comment by Jakub Skoczen [ 04/Mar/19 ]

John Malconian Wayne Schneider should we consider Postgres 11?

Comment by Julian Ladisch [ 04/Mar/19 ]

The supported platforms for postgresql-embedded for postgresql 11 are Mac OS X and Windows x86-64.
Not supported: Linux x86-64, Linux x86-32, Windows x86-32.
References:
https://github.com/yandex-qatools/postgresql-embedded/commit/15685611972bacd8ba61dd7f11d4dbdcb3ba8dc1
https://www.enterprisedb.com/downloads/postgres-postgresql-downloads

Comment by Julian Ladisch [ 18/Mar/19 ]

"Amazon Aurora with PostgreSQL Compatibility Supports PostgreSQL 10.6" https://aws.amazon.com/about-aws/whats-new/2019/02/amazon-aurora-with-postgresql-compatibility-supports-postgresql-10-6/

Comment by Cate Boerema (Inactive) [ 09/Apr/19 ]

Hi Jakub Skoczen. It looks like this Core Platform issue is blocking a couple of bug fixes now. Is this on your radar to fix?

Comment by Ann-Marie Breaux (Inactive) [ 09/Apr/19 ]

Jakub Skoczen Cate Boerema And once this is fixed, it looks like there might also need to be a change to RMB before the bugs are fixed?

Comment by John Malconian [ 09/Apr/19 ]

I'm going to upgrade and test postgresql v10 in our snapshot/testing reference environments today.

Comment by John Malconian [ 09/Apr/19 ]

reference environments have been upgraded to postgresql10. Will take effect on tonight's build of folio-testing and folio-snapshot.

Comment by Julian Ladisch [ 09/Apr/19 ]

Ann-Marie Breaux: To fix the sorting bugs the default sorting (=collation) of the databases need to be changed. FOLIO-1955 Open has been created for this task, but we should first check if there are any issues with the change to PostgreSQL 10 before we change the default collation.
No change in RMB is needed to change the default collation of the databases, this happens automatically when the environments are recreated.

The changes needed for RMB ( RMB-369 Open ) are for postgresql-embedded only. postgresql-embedded is used for unit tests only, not on the reference environments.

Comment by Ann-Marie Breaux (Inactive) [ 09/Apr/19 ]

Julian Ladisch Thank you so much for jumping on this, and for FOLIO-1955 Open . Great to know this may soon get handled and the sorting fixed centrally.

Comment by Jakub Skoczen [ 10/Apr/19 ]

John Malconian Julian Ladisch the issue we discussed during the stand-up was FOLIO-1955 Open

Generated at Thu Feb 08 23:13:20 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.