[FOLIO-3184] execute formatjs-compile in CI for translation PRs Created: 02/Jun/21  Updated: 13/Dec/21  Resolved: 13/Dec/21

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: TBD
Reporter: Zak Burke Assignee: John Malconian
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Relates
relates to FOLIO-3340 Errors in Korean translation files ca... Closed
relates to FOLIO-3067 Replace Jenkins UI and Stripes CI pip... Closed
Sprint: DevOps Sprint 127, DevOps Sprint 128, DevOps Sprint 129
Development Team: FOLIO DevOps

 Description   

Summary: Execute the "Run yarn formatjs-compile" build step during translation PRs to prevent corrupt translation files from being merged.

Details: The CI build step "Run yarn formatjs-compile" will fail if an input file contains a syntax error, e.g. if a keyword in a substitution block is inadvertently translated (e.g. 1, e.g. 2). Because this step is part of the regular-PR UI build pipeline but not part of the translation-commit build pipeline, corrupt translation files may be merged, causing future PRs to fail. It would be preferable for the translation-commit to fail.



 Comments   
Comment by Jakub Skoczen [ 03/Jun/21 ]

Peter Murray how important is this?

Comment by Peter Murray [ 03/Jun/21 ]

If these corrupt translation files are caught earlier in the process, it does prevent failed deployments later down the line.  I don't have a way of checking for this corruption in an automated fashion during the pull request process from Lokalise to GitHub.  I think Zak Burke has a better idea of how often this occurs, but I would guess this is a medium to medium-high priority.

Comment by Zak Burke [ 03/Jun/21 ]

I'd put this at P2: there's a workaround (manually fix the syntax errors in a separate PR, merge it to master, merge master into the original branch), but it's kinda painful. You could argue for P3 and I wouldn't fight you. Resolving the problem isn't hard once you understand it, though it is some scutwork.

This ticket amounts to "do a syntax check before committing translation-file changes in order to avoid committing corrupt files".

WRT "how often" this occurs, I dunno. There were errors in every single repository when we compiled translations for the first time (i.e., catching existing errors) back in April ( FOLIO-3111 Closed ). Since then, I bumped into it two PRs this week but there may have been more; I alerted Peter Murray to the problem and he took over, because he is awesome like that.

Comment by Zak Burke [ 17/Jun/21 ]

Fuel for this fire: Attempting to add automated compilation to ui-users ( UIU-2115 Closed ) fails because of existing corrupt translations:

$ yarn formatjs-compile
yarn run v1.22.10
$ formatjs compile-folder --ast --format simple ./translations/ui-users ./translations/users/compiled
Error validating message "{total, number} 单件已选。 {items, number} {items, plural, one {item} 其他 {items}} 无法 {action} 因为已关闭。" with ID "accounts.warning.itemSelected" in file "./translations/ui-users/zh_TW.json"
Error validating message "{total, number} 单件已选。 {items, number} {items, plural, one {item} 其他 {items}} 无法 {action} 因为已关闭。" with ID "accounts.warning.itemSelected" in file "./translations/ui-users/zh_CN.json"
SyntaxError: MISSING_OTHER_CLAUSE
    at Object.parse (/Users/zburke/temp/ui-users/node_modules/@formatjs/cli/node_modules/@formatjs/icu-messageformat-parser/index.js:34:21)
    at Object.<anonymous> (/Users/zburke/temp/ui-users/node_modules/@formatjs/cli/src/compile.js:44:69)
    at step (/Users/zburke/temp/ui-users/node_modules/tslib/tslib.js:143:27)
    at Object.next (/Users/zburke/temp/ui-users/node_modules/tslib/tslib.js:124:57)
    at fulfilled (/Users/zburke/temp/ui-users/node_modules/tslib/tslib.js:114:62)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

As we complete the "automate translation compilation" tickets linked to STCLI-158 Closed , blocked PRs due to corrupt translations will become a more and more frequent problem.

Comment by Peter Murray [ 17/Jun/21 ]

Zak Burke: I looked at these and others, and I'm speculating that there is some sort of bug in the Lokalise user interface that is not surfacing these placeholder issues when I use the "Multilanguage" view.  If I go into the "bilingual" view, I see them.  So I've gone through each of the languages defined in Lokalise with the "missing/incorrect placeholder" filter: fixing what I could figure out (most of them) or deleting the translation strings for those that I couldn't (just a few).  Fortunately, none involved a R-t-L language, although I had to puzzle out a few CJK strings.

These changes are being pulled into the GitHub repos now.  I'll let you know when that task is done.

Comment by Peter Murray [ 17/Jun/21 ]

Now done.

Comment by Zak Burke [ 21/Jun/21 ]

Yay! Thanks Peter Murray! The PR for UIU-2115 Closed looks clean now.

Comment by John Malconian [ 10/Nov/21 ]

Run 'yarn formatjs-compile' step to Jenkins and Github Actions CI pipelines when translation-only updates are made.

Comment by John Malconian [ 13/Dec/21 ]

I've made a change to Jenkins pipelines such that the 'runScripts' option is no longer bypassed if translations are made via the FOLIO translations bot. Therefore, as long as 'yarn formatjs-compile' is configured as a runScripts option, it will be run on for all builds. e.g.

runScripts = [
['formatjs-compile': ''],
]

Comment by John Malconian [ 13/Dec/21 ]

For repos that use Github Actions, no further action is needed except to ensure the environment variable COMPILE_TRANSLATION_FILES is set to 'true' in the workflows.

Generated at Thu Feb 08 23:26:14 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.