[FOLIO-3020] use Okapi Docker container on reference environments Created: 16/Feb/21  Updated: 24/Oct/21  Resolved: 06/Jul/21

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P3
Reporter: Jakub Skoczen Assignee: Ankita Sen
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Relates
relates to FOLIO-3318 Update folio-ansible doc regarding Ok... Closed
Sprint: DevOps Sprint 116, DevOps Sprint 109, DevOps Sprint 114, DevOps Sprint 108, DevOps Sprint 115, DevOps Sprint 110, DevOps Sprint 117, DevOps Sprint 111, DevOps Sprint 112, DevOps Sprint 113
Development Team: FOLIO DevOps

 Description   

Switch from the Debian package to a container. The container should be built from the master branch of Okapi so that the version of Okapi used on folio-testing and folio-snapshot is the snapshot version.



 Comments   
Comment by Wayne Schneider [ 22/Feb/21 ]

Ankita Sen This issue refers to using the snapshot Docker image for Okapi as the Okapi runtime for the FOLIO reference builds (folio-testing, folio-snapshot, etc.) and Vagrant box builds. Currently, those builds use the released Okapi deb package instead.

Those systems are created with scheduled Jenkins jobs that use the folio-ansible roles to build a single-server development FOLIO environment, using Okapi for container orchestration.

The task will likely involved creating, testing, and documenting a new role in folio-ansible that deploys a properly configured Docker container for Okapi on system startup. A few things that will need to be worked out:

  • How should Docker networking be set up on the target system?
  • How should Okapi configuration be passed to the Okapi container? The container will need to be configured to use a database, as well as credentials for accessing the FOLIO Docker Hub mirror (docker.ci.folio.org)

I'm sure other issues will arise.

The easiest way to do development on a folio-ansible role is to check out the folio-ansible repository and use the included Vagrantfile to do test builds with VirtualBox VMs.

Comment by Ankita Sen [ 04/Mar/21 ]

Wayne Schneider it would be great if you could write down simply what steps the role should be comprised of? As of now I have a Pull Okapi image step which pulls the latest okapi image from the folioci docker repo. As you mentioned in the comment above I need to connect it to a database, so similarly, it would help me progress better and faster if I could just have a list of steps that need to be done, for example should there be a step that should be done immediately before or after pulling the Okapi image? 

This way I could breakdown the development into phases and that would fasten up my work speed.

Comment by Wayne Schneider [ 11/Mar/21 ]

Ankita Sen Here's what I think needs to happen:

  1. Set up the database for Okapi to store its data. See the okapi role for how this is done in the meta/main.yml file.
  2. Create a template for an Okapi JSON configuration file. See the Okapi guide for details. Again, look to the okapi role for guidance to see what needs to be set (though the conf file there is not exactly parallel to the JSON config file, so it may be more like rough guidance). This will be a Jinja2 template.
  3. Use the Ansible template module to copy the template onto the target system in a likely location, like /etc/folio/okapi/okapi.json, for example. You may need to make sure the target directory has been created. The file should be mode 0600 and owned by root, because it contains credentials.
  4. Use the Ansible docker_container module to start the container (you will not need a separate task to pull it). The container should expose port 9130. It will need to mount your configuration file from the filesystem. I've just done a (much simpler) Docker deployment, see https://github.com/folio-org/folio-ansible/tree/FOLIO-3053-minio/roles/minio-docker.

I may be missing some things, but that's how I would start it, anyway. If all goes well, you will have an Okapi running on port 9130 of the target system that will be able to be addressed by all other roles that require it just like the one installed from the Debian package.

A few notes:

  • The docker-engine role will be a dependency for this role
  • Any variables used in any of the files need to be defined in the defaults/main.yml file and documented in the README.md file of the role
  • One thing I'm not certain of is if the Docker container will be running under a user that has access to the Docker daemon running on the system, which is required for Okapi module deployment.
  • For development, make a branch of folio-ansible and use that as your working directory. Use the build_snapshot_backend_core target of the Vagrantfile, and hack on the folio.yml file to just run the part of the build you are interested in. You can add a fail play to the playbook to break out of the build.
Comment by Ankita Sen [ 18/Mar/21 ]

Wayne Schneider: I have worked on and added some of the points from above. 

  • I have added docker-engine as a dependency
  • I have also added create-database as a dependency
  • I have defined all the variables used for now in defaults/main.yml
  • Have defined the docker_container for building and launching the okapi docker container in tasks/main.yml

The branch is FOLIO-3020 Closed and linked to the issue. It would be great if you have a look at it and add further comments on whether the approach is right or wrong.

Current update: I am working on creating the template for Okapi JSON config file. 

 

EDIT 18.03.2021 16:30 CET:

  • Started writing the template for Okapi JSON configurations.
  • Have pushed the first version to folio-ansible repo branch FOLIO-3020 Closed .
  • Review required
Comment by Wayne Schneider [ 18/Mar/21 ]

Ankita Sen the approach looks right to me. Don't be afraid to make more commits than you might normally do in order to allow me to have a look at your work in progress, we can always rebase to clean up the git history.

Comment by Wayne Schneider [ 23/Mar/21 ]

Ankita Sen I've made some comments on the PR in Github. I think you have a good start. I would start testing the role to find out what pieces are missing and how it will work with the system as a whole.

To test, this is what I would do:

  1. Edit the folio.yml file to remove the okapi role
  2. Add a tasks dictionary to that same play with one task to fail the build. The play should look like this:
    - hosts: all
      roles:
        - common
        - postgresql
        - docker-engine
        - kafka-zk
        - okapi-docker-container
      tasks:
        - fail:
    
  3. Build a system with the command vagrant up build_snapshot_backend_core. This will run the playbook up to the fail task.

You'll then have a VM available that you can explore (connect with vagrant ssh build_snapshot_backend_core). If you need to rerun the playbook to reconfigure, you can use vagrant provision build_snapshot_backend_core.

Comment by Ankita Sen [ 25/May/21 ]

Latest update:

There seems to be networking issue. On building the

build_snapshot_backend_core

The docker logs gives the following error:

11:26:12 [] [] [] [] ERROR MainVerticle         finishConnect(..) failed: Connection refused: localhost/127.0.0.1:5432
11:26:12 [] [] [] [] ERROR MainCluster          finishConnect(..) failed: Connection refused: localhost/127.0.0.1:5432
io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/127.0.0.1:5432
Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused

This is the block I am facing currently and would need help.

Comment by Jakub Skoczen [ 07/Jun/21 ]

This requires work from Wayne Schneider to address the docker networking setup issue.

Comment by Wayne Schneider [ 25/Jun/21 ]

Tested PRs are ready for both folio-ansible and folio-infrastructure. Ankita Sen, let me know if you have any questions or concerns. Thanks!

Comment by Ankita Sen [ 06/Jul/21 ]

PR merged and tested before merging. Closing the issue

Generated at Thu Feb 08 23:25:00 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.