[FOLIO-2008] Publish a minimal README at Docker Hub for each back-end module Created: 10/May/19  Updated: 16/Dec/20  Resolved: 07/Jun/19

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P3
Reporter: David Crossley Assignee: mark.stacy
Resolution: Done Votes: 0
Labels: devops, platform-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Relates
relates to FOLIO-2237 Modify CI script which publishes mini... Closed
relates to FOLIO-1696 SPIKE: provide operational "metadata"... Closed
relates to FOLIO-1698 determine additional module metadata ... Closed
relates to FOLIO-2003 platform-core backend modules metadata Closed
relates to FOLIO-2916 Modify CI job buildDocker to publish ... Closed
Sprint: CP: sprint 65, CP: sprint 64
Story Points: 5
Development Team: Core: Platform

 Description   

Gather the relevant pieces of information. Some can be extracted from its Dockerfile, some from its ModuleDescriptor and its launchDescriptor, some from its GitHub repository properties.

Keep this minimal, and refer back to the module's README. We want the module's git README to be the primary source of information.

Consider also automatically appending part of this, as a summary section of the module's README in git.

Add a CI task to do this as part of the build pipelines.



 Comments   
Comment by David Crossley [ 13/May/19 ]

Did some initial investigation. Do curl with the "/v2/repositories/" API to issue a "PATCH" with payload being the "registry" and the "full_description".

Comment by David Crossley [ 13/May/19 ]

https://stackoverflow.com/questions/34710513/how-can-i-update-full-description-on-docker-hub-automatically
See this answer: https://stackoverflow.com/a/55082686

That is described in a little more detail at https://www.sanisimov.com/2019/03/building-docker-images-for-multiple-architectures/
See the "Pushing README" section.

Comment by David Crossley [ 14/May/19 ]

I did an experiment today for my own test repo, following the abovementioned document
and https://success.docker.com/article/how-do-i-authenticate-with-the-v2-api

Hooray, success. Added "description" and "full_description".

Comment by mark.stacy [ 15/May/19 ]

David Crossley Checking in to see if you want to take this task or if you need any assistance? I see you where successful send readme to docker hub. I really like the script that you found, similar but better than my scripts that I have used in the past to accomplish. John Malconian how do you want to integrate this into your current workflow? bash script? Willing to work with both of you or keep looking for another task.

Comment by David Crossley [ 16/May/19 ]

Yes Mark, i have plenty else to do. If you and Jakub are happy, then please take this on.

Some considerations:

How to handle the credentials for Docker Hub.

Should the script do some verification for the presence of items in the ModuleDescriptor FOLIO-2003 Closed .

As part of the initial setup of a new GitHub repository for a back-end module, we currently manually create a "repository" space at Docker Hub (folioorg, folioci). This script could create that on its first run.

Comment by David Crossley [ 16/May/19 ]

The FOLIO-2003 Closed is adding a "metadata" section to the ModuleDescriptor of each platform-core backend module. See mod-notes for an actual example (still preparing the other 22 modules). Currently there are two items "containerMemory" and "databaseConnection". Later there may be more.

There will also be "port" coming from the launchDescriptor section of the ModuleDescriptor.

For some modules there may also be an "env" section in the ModuleDescriptor. Not yet done one in FOLIO-2003 Closed , but mod-login-saml will be one example. See OKAPI-732 Closed and its linked PR.

The "full_description" generated and uploaded to Docker Hub should also have a link back to the GitHub repository.

Another useful piece of information would be the short "description" from the top of the module's GitHub repository front page.

Comment by mark.stacy [ 29/May/19 ]

John Malconian I need to have ENV variables set within the jenkins environment. My code uses DOCKER_USERNAME and DOCKER_PASSWORD. I tested locally and the script is functioning correctly. https://github.com/folio-org/jenkins-pipeline-libs/blob/56e871b722f80a6a03030ebaf040d96b79ee5fc0/resources/org/folio/dockerHubPublishMetadata.sh#L16

Comment by John Malconian [ 29/May/19 ]

mark.stacy David Crossley Where and how will this script be invoked?

Comment by David Crossley [ 29/May/19 ]

I expect during the Jenkins "merge to master" after "Build" and before "Docker".

Comment by John Malconian [ 29/May/19 ]

You'll want to wrap the shell script with the necessary Docker Hub credentials using the following example when you insert it into the pipeline:

withCredentials([usernamePassword(credentialsId: 'DockerHubIDJenkins', passwordVariable: 'DOCKER_PASSWORD', usernameVariable: 'DOCKER_USERNAME')]) {
     writeFile file: 'dockerHubPublishMetadata.sh', text: libraryResource('org/folio/dockerHubPublishMetadata.sh')
     sh 'chmod +x 'dockerHubPublishMetadata.sh'
     sh "./dockerHubPublishMetadata.sh ${env.dockerRepo}/${env.name}"
}

The most appropriate pipeline script to add this to is 'jenkins-pipeline-libs/vars/buildJavaDocker.groovy' - probably within the following conditional on line 122:

 if ( (env.BRANCH_NAME == 'master' && publishMaster) || env.isRelease ) {

FWIW, I think this shell script could pretty easily be re-written as a pipeline code, but I'll leave that decision up to you.

Comment by mark.stacy [ 29/May/19 ]

John MalconianThanks for the information regarding the pipeline. I will take a look.

Comment by mark.stacy [ 29/May/19 ]

John MalconianDavid Crossley Creating the Repository within Docker Hub will have to be done by a user that is in the owners' team. No other way around this using Docker Hub. One of the drawbacks to Docker Hub. So, not sure if you want to keep it the same manual process or move the Jenkins user into the owners' team? I understand the security risks with the pipeline having the ability to create/delete repositories. Let me know what you want to do. Actually, there is no work to be done if you move into the owners' team.

Comment by John Malconian [ 29/May/19 ]

Hmm. Good to know! I added the FOLIO jenkins account to the 'owners' team for the folioci repos, but not the folioorg repos. Let's try it and see how it works out.

Comment by mark.stacy [ 07/Jun/19 ]

Added PR to Repo. Completed

Generated at Thu Feb 08 23:17:31 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.