[FOLIO-3094] SPIKE: Investigate Upgrade to Postgres 12 Created: 29/Mar/21 Updated: 20/May/21 Resolved: 20/May/21 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | P2 |
| Reporter: | Craig McNally | Assignee: | Steve Ellis |
| Resolution: | Done | Votes: | 0 |
| Labels: | R2 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue links: |
|
||||||||
| Sprint: | CP: sprint 113, CP: sprint 114 | ||||||||
| Story Points: | 3 | ||||||||
| Development Team: | Core: Platform | ||||||||
| Release: | R2 2021 | ||||||||
| Description |
| Comments |
| Comment by Steve Ellis [ 06/May/21 ] |
|
I created a wiki page documenting my process and progress. I'll update that as things progress. See https://folio-org.atlassian.net/wiki/display/FOLIJET/Upgrading+to+Postgres+12. |
| Comment by Steve Ellis [ 07/May/21 ] |
|
Julian Ladisch I've made some edits to the wiki page. Unfortunately I don't see a clear path to running tests against postgres 12 for modules that use embedded. I saw your edits to the wiki page suggesting that you could use 12 or 13 with embedded but I don't see a way to do that using their code. I do see that you made a contribution to the embedded pg repo back in 2018 to add version 11 support. But nobody seems to have added 12 or 13. I'm wondering if you're thinking that might be something we want to do here (fork embedded pg and add support for 12?). In a perfect world embedded pg would just let you point to the pg binary you wanted to use. But I don't see a way to do that. It's "embedded" too deep perhaps.
|
| Comment by Julian Ladisch [ 07/May/21 ] |
|
[https://github.com/yandex-qatools/postgresql-embedded There is a configuration option to load any version you want: However, enterprisedb.com offers PostgreSQL download of version >= 11 for Mac OS X and Windows x86-64 only. Embedded postgresql is deprecated. FOLIO switches to the more flexible use of Docker containers. Therefore we shouldn't waist time trying to make embedded postgres work with PostgreSQL 12. Removing startEmbeddedPostgres() in the unit tests and using DB_* is probably more easy. |
| Comment by Steve Ellis [ 08/May/21 ] |
|
For modules that haven't yet made the switch to RMB 32.2+ and are still using embedded, testing against version 12 still seems be possible by bypassing embedded postgres by using a postgres-conf.json file or environment variables. For more info: https://github.com/folio-org/raml-module-builder#postgresql-integration. Using the conf file I'm able to have the tests in mod-finance-storage run against my local postgres 9.6. However a lot of tests that don't fail with embedded are failing with local non-embedded, even though I appear to be connecting to the db and queries from the tests are hitting it. Will continue to investigate this. |
| Comment by Steve Ellis [ 09/May/21 ] |
|
I'm able to see all tests pass in mod-finance-storage when running against a local postgres version 10.16, so I'm thinking that embedded pg in RMB 32.1 isn't hitting 9.6, but rather 10.6, which is the default in embedded if no version is selected, and which appears to be the case in vertex-pg-client. |
| Comment by Steve Ellis [ 11/May/21 ] |
|
In our discussion on monday we decided that version 10 is what I should be testing against, not 9.6. I've now run most tests using 10.16. Attaching the script I'm using to run the tests for a given module. This approach works reasonably well, and I'm able to bypass embedded postgres with a local postgres. So far most modules where the approach succeeds (it doesn't work for everything) seem to indicate that upgrading to 12 will be no problem. Of course there are cases which this may not be the case, some of which may be due to the postgres version change, and some of which may just be because I'm not running the tests in the right way, because the module's configuration is slightly different than what my script expects. See the current state of things in the wiki: https://folio-org.atlassian.net/wiki/display/FOLIJET/Upgrading+to+Postgres+12
|
| Comment by Steve Ellis [ 14/May/21 ] |
|
Adding the testing-outcomes.zip which contains detailed logs (4 files) for each of the tests run. I changed my approach to logging part way through my testing as I tried to script more things. Mosts test produced 4 log files:
If your module has a warning sign in the wiki table these logs might provide meaningful information. |