[FOLIO-874] Update folio-testing-platform Created: 03/Oct/17  Updated: 12/Nov/18  Resolved: 06/Oct/17

Status: Closed
Project: FOLIO
Components: Continuous Integration
Affects versions: None
Fix versions: None

Type: Task Priority: P2
Reporter: Wayne Schneider Assignee: Wayne Schneider
Resolution: Done Votes: 0
Labels: ci, sprint23
Remaining Estimate: Not Specified
Time Spent: 6 hours, 45 minutes
Original estimate: Not Specified

Sprint:

 Description   

There may be a use for this project as a reference yarn platform to make it easier for stripes developers to catch cross-repo dependency mixups.



 Comments   
Comment by Matthew Jones [ 04/Oct/17 ]

I agree that maintaining a folio-testing-platform which CI uses, rather than one generated on the fly, will make it easier for developers to catch such errors. It will also help developers troubleshoot existing failures reported by CI more quickly.

There are a few other things we might consider, so I'm going to include my notes about the recent build failures in general. I'm hoping to identify areas for discussion and improvement, not point blame in any way. These can be move to another ticket if more appropriate.

Unintentional breaking changes

  • Configuration settings were shuffled around across core packages last week. This unintentionally created a breaking change at build time. While the latest version of stripes-core worked with the latest version of its dependencies, the latest version of stripes-core did not work when pointing to prior release of those dependencies as defined in the package.json.
  • Many developer's environments would appear to work, while others (and CI) did not. This was due to the next issue...

Locally linking packages in a development environment

  • Linking is often needed to develop, but can easily hide things. First, by developing with a locally linked dependencies, one is always pointing to the latest changes locally or on master. As described above, this may make one's cross-package changes appear to be non-breaking when in fact they are.
  • Second, locally linked packages are also independently installed. This means such a package will have both their NPM dependencies and devDependencies available. This is not the case when the same package is installed as standard, non-linked, dependency of another. At least two recent build failures were hidden in development environments by this.
  • To properly validate newly a added or modified devDependency, one would have to unlink the package (if they can - it may have code they need), point to a commit hash, or reinstall the linked packages's own dependencies with `--production` option to exclude the devDependencies. Not terribly friendly.

Use of interim snapshot or hash between releases

  • When relying on non-breaking changes between releases, one can point to the CI build's "snapshot". When breaking changes between releases are necessary, the current method is to target the commit hash of the dependency in question. The specific commit hash works well for those using to npm-folio or npm-folioci repos. However, in order to pull in latest available snapshot, one must be using the npm-folioci repo.
  • This alone may not have caused a build failure, but did create some confusion among developers. Switching locally between npm-folio and npm-folioci adds some complexity to the workflow. Should developers only be using npm-folioci?
  • For reference, the hash approach didn't work for stripes-loader last week as it relies on a prepublish script must have a release or interim snapshot with a bumped version. The loader has since been deprecated, but could

Webpack builds not reporting failures to CI

  • In perhaps the biggest cover-up, stripes.js build commands have been suppressing webpack compilation errors making it impossible for CI to detect any build failure reported by webpack. Recent changes helped to surface these errors to developers by adding console output, but a proper exit code was still missing for CI. This was probably a main source for most hidden build failures not found until runtime.

Difficulty in reproducing testing build

  • Finally, it is hard for a developer to reproduce the CI build locally, either when trying to troubleshoot an existing build failure, or checking that their new code won't cause a failure. Maintaining a folio-testing-platform will help with this, as one can easily attempt to build the CI configuration at any time.
  • This still requires using npm-folioci and remembering when to unlink when necessary. Perhaps we can include an `.npmrc` within the testing platform repo to always point to npm-folioci if developers aren't already doing so.
Comment by Wayne Schneider [ 04/Oct/17 ]

https://github.com/folio-org/folio-testing-platform has been updated to be current. I committed a yarn.lock to the repo, as well.

My thought is to commit a new yarn.lock every time one is generated by the regularly scheduled stripes-testing CI job. Does this make sense, or is it too aggressive? It will certainly lead to lots of commits, which is a lot of noise.

Comment by Wayne Schneider [ 04/Oct/17 ]

A couple of comments on Matt's useful summary:

  • I did include an .npmrc
  • I had some success with using yalc (https://www.npmjs.com/package/yalc) instead of yarn linking to better emulate the build environment locally. Any thoughts on this tool?
Comment by Matthew Jones [ 04/Oct/17 ]

Re: yarn.lock, I'm not too concerned about the added noise. However, it seems updating it every time is not much different than excluding it from the repo. Well, okay I do see having it there would help a developer to recreate locally exactly what is happening in CI at any given moment. This will be helpful in troubleshooting. The next CI build would then omit yarn.lock to get the latest available changes anyway, correct?

Comment by Matthew Jones [ 04/Oct/17 ]

I have not used yalc. The description sounds like it might be helpful though. Some other thoughts that come to mind for improvements:

  • Update CI to attempt to build folio-testing-platform before a PR is merged into selected repos (such as: core, connect, components, etc.) This will help catch many of the errors we've seen recently that are hard to surface locally. Even when running a local folio-testing-platform, there are some cases that could slip through the cracks.
  • Include commands and tests in stripes-core/CLI to simplify environment setup and validate that changes in core are non-breaking. This would be one part of a larger CLI effort to streamline development.
  • Further refine or better communicate process for handling breaking changes between releases.
Comment by Wayne Schneider [ 04/Oct/17 ]

I updated folio-ansible to use folio-testing-platform, but it caused unexpected side effects on folio-testing.aws.indexdata.com, so I rolled back part of the change.

Comment by Wayne Schneider [ 04/Oct/17 ]

OK, John Malconian found the problem, so folio-ansible (and folio-testing) are now being build from the folio-testing-platform repo.

Comment by Wayne Schneider [ 04/Oct/17 ]

John Malconian – I think if we are to update the yarn.lock file through CI, it will have to be part of the Ansible plays in the folio-infrastructure project.

Comment by Wayne Schneider [ 06/Oct/17 ]

The folio-testing-platform is working fine in CI. I'll open another issue about updating the yarn.lock file from a CI build.

Generated at Thu Feb 08 23:08:57 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.