[FOLIO-3418] clean up old Vagrant images from S3 Created: 17/Feb/22 Updated: 05/May/23 |
|
| Status: | Open |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Story | Priority: | P3 |
| Reporter: | Jakub Skoczen | Assignee: | Wayne Schneider |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Sprint: | DevOps Sprint 135, DevOps Sprint 136, Kitfox: sprint 165, DevOps Sprint 134 |
| Development Team: | FOLIO DevOps |
| RCA Group: | TBD |
| Description |
Review other existing S3 buckets to decide if they should be shutdown. |
| Comments |
| Comment by Marc Johnson [ 03/Mar/22 ] |
|
I'm not sure if this is the correct issue. Jakub Skoczen asked me to comment on the issue assigned to Wayne Schneider with feedback about the retirement of the testing vagrant images. In the Technical Council yesterday, it was mentioned that the snapshot-backend vagrant image is intended to be equivalent to the retired testing-backend image. I passed this on to folks who are using the testing-backend image. When a developer tried to use the snapshot-backend image, they got the following error:
Please can you help us find the snapshot-backend images? |
| Comment by Wayne Schneider [ 03/Mar/22 ] |
|
Sorry for the confusion, Marc Johnson. There is not a snapshot-backend image. The snapshot image backend should be the same as the testing-backend image (though they are constructed slightly differently), because interface dependencies for the backend were always enforced (the main difference between the snapshot and testing scripts is that the testing script used the most recent "snapshot" releases of front-end modules without enforcing interface dependencies). It's always been a rather confusing situation. This probably doesn't explain it completely, either |
| Comment by Wayne Schneider [ 03/Mar/22 ] |
|
To be clear, the only difference between the testing-backend and the testing images was that the testing image had a Docker container running nginx hosting a webpack bundle composed of the most recent snapshot releases of front-end modules. |
| Comment by Marc Johnson [ 03/Mar/22 ] |
|
Thank you for responding quickly.
That is good to know.
Thanks for explaining that. I don't think folks are concerned about the slightly different construction, except for the side-effect of that which means one of the images only has back end modules and the other has back end and front end.
The feedback I've been given is that this adds significantly to the resource usage of the image, especially when it is initially spun up, to the point where machines that can run the back end only one cannot run this one. |
| Comment by Jakub Skoczen [ 04/Mar/22 ] |
|
Wayne Schneider can you verify this (the res alloc for running with or without the nginx+bundle)? I am surprised that adding a nginx container makes a big difference so we I'd appreciate if you also checked if other factors could make snapshot image more heavy than the testing one (e.g individual container configurations). Thanks. |
| Comment by Wayne Schneider [ 07/Mar/22 ] |
|
I think the difference in resource allocation is not really about the nginx container, which is pretty small. The difference is that in the testing build, all backend modules were manually enumerated for deployment in a configuration file, rather than being pulled in based on dependency resolution. Some modules were deliberated excluded (e.g mod-ldp), and some modules were being inadvertently excluded. I think the best solution is to push forward the work for "platform-minimal" (
We could also look at building a backend-only box with a specified set of modules (for example, only the modules required for authn/z on the backend). If you feel like that would be useful for your team, please raise another ticket and we can look at what your requirements might be. |
| Comment by Marc Johnson [ 07/Mar/22 ] |
I don't understand how vagrant images are built. Are the steps run on the machine running the image, e.g. when a developer starts the image, does it spin up Okapi and deploy the modules and build the bundle for the front end?
Ok. I don't anticipate that work will progress quickly.
I don't think folks are going to do that. I think the conclusion of this conversation is that the belief that there were equivalent snapshot vagrant images to the removed testing ones turned out not to be the case and those folks using them will need to find other ways. |
| Comment by Jakub Skoczen [ 07/Mar/22 ] |
|
Marc Johnson The selection of modules in the "testing" backend Vagrant image did not follow any particular plan. It just so happened that some modules were held back from that image while others were simply forgotten. Either way the fact that the "testing" backend had a smaller footprint was a pure chance rather than "by design". It's certainly not something the developers should depend on. I think "platform-minimal" is something that we should priorize as a project as this would be a foundation for a small footprint Vagrant image that developers could use. If you bring this to the TC I will support any motions to get this work on track. |
| Comment by Marc Johnson [ 07/Mar/22 ] |
Ok. Given those images are now retired, it isn't something the folks who were using them are going to be concerned with now.
As far as I can tell, this initiative has stalled (since the tech leads meeting about it). I don't think I'm the person to represent or sponsor that initiative. |
| Comment by Marc Johnson [ 07/Mar/22 ] |
Though I do appreciate the offer to support this initiative. I do think it's an important area, especially given the concerns folks have around developer onboarding and community development. I think it would be great if FOLIO had a cohesive plan for how it wanted to structure both development and production systems (in terms of operations and how they are extended / enhanced). I don't think I'm the person to lead that initiative, given that I neither use the existing vagrant images, the scratch environments or operate hosted environments. |