How to set up and use BigTest for Windows users (using Linux virtual environment)

Owned by Taras Tkachenko
Last updated: Oct 29, 2021 by Dmitry

Why do we Need This

BigTest testing framework cannot work on Windows OS due to several causes:

  • It uses deprecated and non-supported for now libraries and components like PhantomJS (which never worked on Windows from the time it was created);
  • It creates UNIX-specific symbolic links which are absent on Winows (FAT/NTFS)  and Windows shared (SMBFS/CIFS) file systems;
  • Despite it uses PhantomJS as a shadow renderer it cannot work in headless mode - it requires normal UI browser to run.

It makes us impossible to run BigTest not only on Windows but from the shared host machine folders on virtual environments as well.

At the top of this we cannot use it on headless server OS under any VM according to point 3 from the previous list above.

So the basic idea is to set up Linux virtual environment (using VirtualBox in this case) which contains UI-enabled OS and set all the Node development infrastructure there.

Codebase will be available in 2 ways:

  • via git operations (when clonning and pulling project codebase);
  • via bulk copy from CIFS shared dir of your projects home on Windows and vice versa (this will help us to avoid too many commits in the branch).

Step by Step Guide

So while the root causes are defined and approaches are pointed out, let's do it step by step.

  1. Install VirtualBox (if you'd not done it already):
    1. Download the latest version of Virtualbox from here:  https://www.virtualbox.org/wiki/Downloads;
    2. Download VirtualBox Extension Pack from here: https://download.virtualbox.org/virtualbox/6.0.8/VirtualBoxSDK-6.0.8-130520.zip (or newer depending on your VirtualBox version);
    3. Install VirtualBox and Extension Pack according to VirtualBox installation guide;
    4. Create VBox VM with the following settings:
      1. Architecture: x86_64 (see Figure 2);
      2. Chipset type: Intel ICH9 (see Figure 2);
      3. Memory: 8GB+ (16GB in my case, see Figure 2);
      4. Processors: 2, accelerations: ON (see Figure 3 and 4);
      5. Video settings: Memory: 128MB, 3D acceleration: ON (see Figure 5);
      6. Disk space: WDI, 40GB, disk type: Fixed (it is faster than dynamic while takes more space);
      7. Block devices config: SATA only (see Figure 6);
      8. Host only network (for ssh access): Internal router IP: 192.168.2.1, Netmask 255.255.255.0, DHCP: OFF (see Figure 10);
      9. Network adapters: 2 x Intel PRO/1000 MT Desktop: 1 - NAT (for external network bridging), 2 - Host-Only Adapter (for SSH access and the like) (see Figures 7 and 8);
  2. Install Ubuntu (in this guide, you can use your own distro):
    1. Default installation options could be used.
  3. Install Linux packages required:

    1. Update repos list and upgrade your distro: 

      Update your distro
      $ sudo apt update && apt upgrade -y
..............
$ sudo apt dist-upgrade -y

    2. Install prerequisite packages:

      Prerequisites
      $ sudo apt install -y aptitude mc ubuntu-restricted-extras build-essential git tig crl lynx links dkms inux-headers-generic linux-headers-$(uname -r)
  4. Install VirtualBox Guest Extensions (according to these guides: https://www.tecmint.com/install-virtualbox-guest-additions-in-ubuntu/, https://www.techrepublic.com/article/how-to-install-virtualbox-guest-additions-on-a-gui-less-ubuntu-server-host/):
    1. Insert VBox Guest Extensions ISO into VM CD drive;

    2. Become a root and mount Guest Additions ISO into your VM and change to its directory: 

      $ sudo su -
$ mount /dev/cdrom /media/cdrom
$ cd /media/cdrom

    3. Install the Guest Additions package with the command:

      $ ./VBoxLinuxAdditions.run
  5. Install Java:

    1. you can install OpenJDK and OpenJRE or Oracle Java (according to this guide: https://www.digitalocean.com/community/tutorials/how-to-install-java-with-apt-on-ubuntu-18-04):

      $ sudo apt install default-jdk default-jre
  6. Install Google Chrome browser (using this guide: https://askubuntu.com/questions/510056/how-to-install-google-chrome):

    1. Download and import APT key: 

      $ wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -

    2. Install Chrome repository:

      $ echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | sudo tee /etc/apt/sources.list.d/google-chrome.list

    3. install Chrome browser: 

      $ sudo apt update 
$ sudo apt install google-chrome-stable
    4. Personalize your new browser as usual by introducing your Google account and synchronising it (it will bring all your development extensions you usually have on host machine).
  7. Install NodeJS environment (according to these guides: https://itsfoss.com/install-nodejs-ubuntu/, https://websiteforstudents.com/install-the-latest-node-js-and-nmp-packages-on-ubuntu-16-04-18-04-lts/):

    1. Download APT repository installer for NodeJS version of your choice (let it be 11.x for example): 

      $ curl -sL https://deb.nodesource.com/setup_11.x | sudo bash -

    2. Set up NodeJS & NPM: 

      $ apt update && apt install nodejs
# checking installed versions:
$ node -v
# v11.6.0
$ npm -v
# 6.5.0-next.0

    3. Update your NPM installation and install all the Node global packages needed: 

      $ sudo npm i -g npm
$ sudo npm i -g yarn@latest babel-cli@latest babel-preset-env@latest webpack@latest webpack-dev-server@latest eslint@latest create-react-app@latest pm2@latest json-server@latest express@latest typescript@latest
  8. Install FOLIO development environment (using this guide: https://github.com/folio-org/stripes/blob/master/doc/quick-start.md):

    1. Change to your local user (not root!) home directory, generate your VM's SSH ID and list your public key file contents to the terminal:

      $ cd ~
$ ssh-keygen
$ cat ./id-rsa.pub
    2. Copy the key from terminal and register it in your FOLIO GitHub account.

    3. Set up FOLIO Yarn repository and install Stripes globally:

      $ yarn config set @folio:registry https://repository.folio.org/repository/npm-folio/
$ yarn global add @folio/stripes-cli

    4. Create your VM projects dir within your users home directory to run and debug your tests from it (/home/username/workspace in my case):

      $ mkdir ~/workspace
$ chmor -R 775 ./workspace
    5. Clone your app or Stripes platform from Github repo and follow the instructions on app or platform installation from the guide above. Example related with Stripes workspace and modules ui-eholdings, ui-notes and ui-quick-marc
      1. Go to folder without subfolder "stripes"
      2. Run "stripes workspace"
      3. Select by space items ui-eholdings, ui-inventory, ui-notes, ui-quick-marc, ui-users, stripes-sample-platform

      4. Press Enter. The result:

        stripes workspace
? Stripes modules to include ui-eholdings, ui-inventory, ui-notes, ui-quick-marc, ui-users, stripes-smart-components
Directory "/XXX/stripes" created.

Cloning modules...
Clone complete.

Installing dependencies...
Directory "/home/venn/workspace/stripes"
yarn install v1.22.17
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 133.84s.
Install complete.

Initializing Stripes configuration...
Configuration complete.

Done.

Edit "stripes/.stripesclirc.json" to modify CLI configuration including aliases for this workspace.

UI modules available: "ui-eholdings", "ui-inventory", "ui-notes", "ui-quick-marc", "ui-users"
"cd" into the above dir(s) and run "stripes serve" to start a module in isolation.
        ls -la ./stripes/
total 800
drwxrwxr-x 9 venn venn 4096 окт 29 12:22 .
drwxrwxr-x 4 venn venn 4096 окт 29 12:18 ..
drwxrwxr-x 1452 venn venn 45056 окт 29 12:22 node_modules
-rw-rw-r-- 1 venn venn 143 окт 29 12:18 package.json
-rw-rw-r-- 1 venn venn 76 окт 29 12:18 .stripesclirc.json
drwxrwxr-x 9 venn venn 4096 окт 29 12:21 stripes-smart-components
drwxrwxr-x 11 venn venn 4096 окт 29 12:21 ui-eholdings
drwxrwxr-x 9 venn venn 4096 окт 29 12:21 ui-inventory
drwxrwxr-x 8 venn venn 4096 окт 29 12:21 ui-notes
drwxrwxr-x 6 venn venn 4096 окт 29 12:21 ui-quick-marc
drwxrwxr-x 9 venn venn 4096 окт 29 12:21 ui-users
-rw-rw-r-- 1 venn venn 727968 окт 29 12:22 yarn.lock
        ls -la ./stripes/stripes-smart-components/
total 144
drwxrwxr-x 9 venn venn 4096 окт 29 12:21 .
drwxrwxr-x 9 venn venn 4096 окт 29 12:22 ..
-rw-rw-r-- 1 venn venn 42147 окт 29 12:19 CHANGELOG.md
drwxrwxr-x 2 venn venn 4096 окт 29 12:19 .circleci
-rw-rw-r-- 1 venn venn 148 окт 29 12:19 CONTRIBUTING.md
-rw-rw-r-- 1 venn venn 216 окт 29 12:19 .editorconfig
-rw-rw-r-- 1 venn venn 25 окт 29 12:19 .eslintignore
-rw-rw-r-- 1 venn venn 658 окт 29 12:19 .eslintrc
drwxrwxr-x 8 venn venn 4096 окт 29 12:19 .git
drwxrwxr-x 3 venn venn 4096 окт 29 12:19 .github
-rw-rw-r-- 1 venn venn 59 окт 29 12:19 .gitignore
-rw-rw-r-- 1 venn venn 4292 окт 29 12:19 index.js
-rw-rw-r-- 1 venn venn 239 окт 29 12:19 Jenkinsfile.deprecated
-rw-rw-r-- 1 venn venn 881 окт 29 12:19 karma.conf.js
drwxrwxr-x 23 venn venn 4096 окт 29 12:19 lib
-rw-rw-r-- 1 venn venn 11357 окт 29 12:19 LICENSE
drwxrwxr-x 3 venn venn 4096 окт 29 12:21 node_modules
-rw-rw-r-- 1 venn venn 3402 окт 29 12:19 package.json
-rw-rw-r-- 1 venn venn 782 окт 29 12:19 README.md
-rw-rw-r-- 1 venn venn 1027 окт 29 12:19 .stripesclirc.js
-rw-rw-r-- 1 venn venn 354 окт 29 12:19 .stylelintrc
drwxrwxr-x 3 venn venn 4096 окт 29 12:19 tests
drwxrwxr-x 3 venn venn 4096 окт 29 12:19 translations

      5. Create file stripes-sample-platform/.stripesclirc

        {
"configFile": "stripes.config.js",
"port": 8080,
"aliases": {
"@folio/users": "../ui-users",
"@folio/inventory": "../ui-inventory",
"@folio/eholdings": "../ui-eholdings",
"@folio/notes": "../ui-notes",
"@folio/quick-marc": "../ui-quick-marc"
}
}
      6. Update stripes-sample-platform/stripes.config.js:
        module.exports = {
okapi: { 'url':'http://localhost:9130', 'tenant':'diku' },
config: {
// autoLogin: { username: 'diku_admin', password: 'admin' }
// logCategories: 'core,redux,connect,connect-fetch,substitute,path,mpath,mquery,action,event,perm,interface,xhr'
// logPrefix: 'stripes'
// logTimestamp: false
// showPerms: false
// showHomeLink: false
// listInvisiblePerms: false
// disableAuth: false
// hasAllPerms: false
},
modules: {
'@folio/users': {},
'@folio/inventory': {},
'@folio/eholdings': {},
'@folio/quick-marc': {},
},
branding: {
logo: {
src: './tenant-assets/opentown-libraries-logo.png',
alt: 'Opentown Libraries',
},
favicon: {
src: './tenant-assets/opentown-libraries-favicon.png',
},
},
};
      7.  Update stripes-sample-platform/package.json
        "dependencies": {
"@folio/stripes": "^7.0.0",
"@folio/users": "^6.0.0",
"moment": "~2.25.3",
"react": "^17.0.2",
"react-dom": "^17.0.2",
"react-redux": "^7.2.2",
"react-router": "^5.2.0",
"react-router-dom": "^5.2.0",
"react-query": "^3.13.0",
"redux": "^4.0.5"
},
"devDependencies": {
"@folio/eslint-config-stripes": "^5.0.0",
"@folio/stripes-cli": "^2.0.0",
"eslint": "^6.2.1",
"lodash": "^4.17.5"
},
"resolutions": {
"@folio/stripes-cli": "^2.0.0",
"rxjs": "^6.6.7",
"minimist": "^1.2.3",
"moment": "~2.29.0",
"redux-form": "^8.0.0",
"final-form": "4.20.1"
}
      8. Run "yarn stripes platform clean --removeLock"
      9. Run "yarn install"
      10. Run "yarn stripes serve --okapi https://folio-snapshot-okapi.dev.folio.org". The result:
        yarn stripes serve --okapi https://folio-snapshot-okapi.dev.folio.org
yarn run v1.22.17
$ stripes serve --okapi https://folio-snapshot-okapi.dev.folio.org
Waiting for webpack to build...
Listening at http://localhost:8080
webpack built 696609be5483229a9645 in 56285ms
  9. Set up second network interface that will be used as internal one for ssh and the other similar stuff (it uses static IP address so you don't need to check out what next IP it has via DHCP):
    1. IP: 192.168.2.2/24, Netmask 255.255.255.0, DNS: 192.168.2.1/24;
    2. Put VM's IP into host machine's hosts file and vice versa.
  10. Set up Projects shared directory:
    1. Point VirtualBox sharing facilities to your projects folder (D:/Users/username/workspace in this example) which we are going to share with our VM (see Figure 9);

    2. Create empty directory for the mount point and give it access rights of your VM user (not root!):  

      $ sudo mkdir /usr/www
$ sudo chown -R username:usergroup /usr/www
$ sudo chmod -R 775 /usr/www

    3. Mount shared folder into your directory of choice (/usr/www in my case) using fstab (/etc/fstab) through vboxsf virtual file system (as root):

      fstab example
      UUID=9a4b3832-6258-11e9-859a-080027de5c74	/		ext4	defaults			0	0
/swap.img					none		swap	sw				0	0
workspace					/usr/www	vboxsf	auto,uid=1000,gid=1000		0	0
# //192.168.2.1/workspace			/usr/www	cifs	rw,users,credentials=/etc/samba/credentials/sith.local,gid=1002,dir_mode=0775,file_mode=0775		0	0

    4. Check newly created mount point via the following command:

      $ sudo mount -a

       this will re-mount all the existing mount points figured out in fstab and in the case of success you will be able to change to your new shared directory and check its contents and access rights using the following commands: 

      $ cd /usr/www
$ ls -las

So this way now you are capable of BigTest tests development in the following way:

  1. Write tests on Windows host machine as usual.
  2. Switch your VM project to your current git branch and pull the latest changes.
  3. Commit and push tests to git repo from host machine or copy tests directory to your VM projects folder (/home/username/workspace/project_name in my case) to avoid using git for now.
  4. Run and debug your tests on VM.
  5. Copy ready to use tests back to your host machine project folder for commit or commit them from VM directly.

Illustrations

1 - VM Settings: Summary:

2 - VM System Settings - Motherboard:

 

3 - VM System Settings - Processors:

4 - VM System Settings - Accelerations:

5 - VM Settings - Display:

6 - VM Settings - Storage:

7 - VM Settings - Network 1:

8 - VM Settings - Network 2:

9 - VM Settings - Shares:

10 - VM Settings - Host only networks: