[FOLIO-1335] document the differences between dev, CI environments Created: 11/Jul/18  Updated: 12/Nov/18  Resolved: 29/Aug/18

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Story Priority: P2
Reporter: Zak Burke Assignee: Matthew Jones
Resolution: Done Votes: 0
Labels: build-release, ci, ebsco, sprint42, sprint43, sprint44, sprint45, ui-core
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
blocks FOLIO-1293 Replace folio-testing-platform with p... Closed
Duplicate
is duplicated by STRIPES-544 Understand all the stripes-building s... Closed
Relates
relates to FOLIO-1266 Reorganizing CI and nightly with plat... Closed
relates to UITEST-38 remove all timed waits and experiment... Closed
relates to FOLIO-1450 Revise FOLIO release and deployment p... Open
relates to FOLIO-1266 Reorganizing CI and nightly with plat... Closed
relates to FOLIO-1344 use stripes-cli in the build and test... Closed
relates to STRIPES-543 Consider "stripes-framework" to wrap ... Closed
relates to STRIPES-544 Understand all the stripes-building s... Closed
Sprint:
Development Team: EBSCO - FSE

 Description   

Integration tests that pass locally frequently don't pass in CI. That's bad. Devs need to better understand the environment and workflow in CI so that we can make changes, either in CI or in our local environments, so we can close the gap between the dev environment and the CI environment. Some discrepancies are inevitable and we don't expect to eliminate them all (local code isn't minified or obfuscated, and React and other mods execute very differently when run with NODE_ENV=production) but we need the ability to replicate the CI environment and process locally so we can be more confident that our branches will be mergeable when we push them up to GitHub.

Document has been started to capture the content of the comments:

https://docs.google.com/document/d/1fzSU1xmtqdox3iYq82xrhwcurof89un2ybAKJGhgJIo/edit?usp=sharing

Once the document is complete Mike Taylor will transfer the relevant parts to Stripes developer documentation.

David Crossley



 Comments   
Comment by Jakub Skoczen [ 11/Jul/18 ]

John Malconian Matthew Jones guys can we list out the differences here?

Comment by Matthew Jones [ 11/Jul/18 ]

Adding some of my findings posted to the devops channel, that I think apply here. In response to Zak Burke's comment on devops about visual differences between a workspace build and folio-testing-platform build:

Building folio-testing-platform locally yields the same visual inconsistencies for me. Looking into it, I find three different versions of `stripes-components` in use including two separate major versions:

lpmc-mjones:folio-testing-platform mattjones$ yarn why @folio/stripes-components | grep Found
=> Found "@folio/stripes-components@3.0.1000426"
=> Found "@folio/eholdings#@folio/stripes-components@2.1.5000420"
=> Found "@folio/developer#@folio/stripes-components@2.1.8000425"
=> Found "@folio/calendar#@folio/stripes-components@2.1.8000425"
=> Found "@folio/plugin-find-user#@folio/stripes-components@2.1.8000425"
=> Found "@folio/checkout#@folio/stripes-components@2.1.8000425"
=> Found "@folio/search#@folio/stripes-components@2.1.8000425"
=> Found "@folio/stripes-core#@folio/stripes-components@2.1.8000425"
=> Found "@folio/finance#@folio/stripes-components@2.1.8000425"
=> Found "@folio/vendors#@folio/stripes-components@2.1.8000425"
=> Found "@folio/checkin#@folio/stripes-components@2.1.8000425"
=> Found "@folio/organization#@folio/stripes-components@2.1.8000425"
=> Found "@folio/requests#@folio/stripes-components@2.1.8000425"
=> Found "@folio/circulation#@folio/stripes-components@2.1.8000425"
=> Found "@folio/inventory#@folio/stripes-components@2.1.8000425"
=> Found "@folio/users#@folio/stripes-components@2.1.8000425"
=> Found "@folio/react-big-calendar#@folio/stripes-components@2.1.8000425"
=> Found "@folio/stripes-form#@folio/stripes-components@2.1.8000425"
=> Found "@folio/stripes-smart-components#@folio/stripes-components@2.1.8000425"

Using a workspace or aliasing in `stripes-components` from tip of master (at v3) would force all other v2 references to use v3 as well. When I alias in `stripes-components` and serve `folio-testing-platform` the visuals look okay.

It seems like we need a better way to manage the inclusion/exclusion of dependencies with major version differences. In this case, the latest major version resolves the issue, but I can imagine there are situations where the reverse is true.

I think the thing to note here is that a dev environment using workspaces/links/aliases to join multiple local clones of modules disregards any versions specified in package.json for those modules when used as dependencies. It may be obvious that we are often pointing to the tip of master here, but perhaps not so obvious that doing so hides instances where another module's package.json needs to be updated. Without such updates, stand-alone platform builds can go out of sync.

Comment by Matthew Jones [ 12/Jul/18 ]

Below is an outline of the different approaches we currently use for development and CI. I've highlighted the areas that are likely sources of discrepancies in dependencies between dev and CI.

Development Environments

  • Platform (standard, direct install)
    • All dependency versions are respected
    • No duplication for same module+version
    • Yarn.lock includes resolutions for all modules (source of discrepancy, when not using CI's yarn.lock)
    • Not for app/module editing
    • DevDependencies only for the platform itself are included (not for apps)
  • Platform with aliases or linking
    • For modules with an active alias/link to a local clone
      • Developer maintains updates w/origin (source of discrepancy)
      • Module's version, when used elsewhere as a dependency, is not respected (source of discrepancy)
      • Module maintains yarn.lock separate from platform (source of discrepancy)
      • Shared dependencies are duplicated
      • DevDependencies included
    • For all other modules
      • Same as standard platform
    • Differences between alias and linking
      • Aliases are applied to the Webpack config at build time (managed by CLI)
      • Links are applied as symlinks to the file system
  • Workspace
    • For modules with a local clone in the workspace
      • Developer maintains updates w/origin (source of discrepancy)
      • Module's version, when used elsewhere as a dependency, is not respected (source of discrepancy)
      • Module is omitted from top level resolutions in workspace's yarn.lock (source of discrepancy)
      • Shared dependencies are not duplicated
      • DevDependencies included
    • For all other modules
      • Same as standard platform

CI Environments

  • Platform (for snapshot build)
    • Package.json updated to use specific build versions of modules (or just latest?)
    • Otherwise the same as standard platform
  • Platform (for PR build)
    • Package.json updated to point to module under PR, via file reference in dependencies
    • Otherwise the same as standard platform

John Malconian Please let me know if I'm omitting something with regards to CI modifications

Comment by Matthew Jones [ 12/Jul/18 ]

Some thoughts on minimizing discrepancies:

  • Test (and possibly develop) using only the module(s) necessary for a feature (workspace or alias)
    • Brings things closer to CI's platform PR build
    • Fewer cloned modules in play reduces the surface for potential differences
    • Versions are respected for all non-cloned modules
      • Can help highlight specified dependency versions that may need updating, like above issue with stripes-components 2.x -> 3.x
  • Publish/use versioned platforms
    • Pull down a specific platform version (or yarn.lock from a stable snapshot)
    • Needs more thought, but possibly integrate with the CLI, something like "stripes test --snapshot=latest"
  • Perform module tests with virtual platform (both dev and PR)?
    • CLI could orchestrate
    • Minimizing differences to possibly the CLI version itself
  • Differences with back-end
    • Local vagrant box can be behind back-end used for CI tests
    • Run tests against same back-end used by CI

Testing considerations:

There are some potential differences associated with testing as well. I suspect these are less of an issue than the environment at this point, but still worth noting here.

  • Tests run from a separate ui-testing install:
    • Yarn.lock is independent of platform under test
  • Tests run from ui-testing within a workspace:
    • Similar points re: workspace above

Inverting the ui-tesing dependency, such that modules and platforms depend on ui-testing (directly or via CLI), rather than ui-tesitng depending on all modules, should avoid such discrepancies between the platform of tests and the explicit .

Comment by Mike Taylor [ 26/Jul/18 ]

(Side-question: can anyone explain to me what is the difference between a story such as this one, and a task such as FOLIO-1344 Closed ?)

Comment by Jakub Skoczen [ 08/Aug/18 ]

Matthew Jones Zak Burke can we consider this completed and the document created the result of this investigation?

Generated at Thu Feb 08 23:12:35 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.