/
Technik (Deploy)

Technik (Deploy)

Translation deployment

Die aktuelle Fassung steht auf How to Translate FOLIO#Translationdeployment.

Eine veraltete Kopie, die hier nicht aktualisiert wird:

  • Translations are moved from Lokalise to the /translations/ directories of GitHub repositories by a script that Peter Murray runs every few days. They show up as a pull request on each stripes-* and ui-* repository, are run through the CI tests, and automatically merged. CI tests don't always pass and may require manual restart.

  • This is a one-way process. Any manual change to locale files on GitHub will be overwritten and are lost when new locale files from Lokalise come in.

  • https://folio-testing.aws.indexdata.com/ is the first reference environment where new or changed translations appear because it uses the master branch. Other reference environments take longer because they only use released front-end module versions, see Reference Environments for details.

  • Any FOLIO release like Honeysuckle, Iris, etc. ships with the translations that come with each front-end module. Each front-end module has a different release date (Honeysuckle example). This is the time when translations are taken from module's GitHub translations directory and is different for each front-end module. A new front-end module release is needed to get more recent translation included into a FOLIO release.

  • However, the sysOp of a FOLIO installation may manually update the translations of their installation by exchanging the files in the /translations/ directory from where the static files are shipped. One option is to take the latest translations from https://folio-testing.aws.indexdata.com/ , use the browser developer tools to find out the current number folio-testing uses, for example https://folio-testing.dev.folio.org/translations/de-1605513539114.json, take the .json file, and rename the .json filename to match the number of the own installation. If the own installation has front-end modules that not exist on folio-testing then merge the folio-testing into the own translation file: jq -s '.[0] * .[1]' de-1601343197440.json de-1605513539114.json > de-1601343197440.json.new

(Planung): Lokalise-Zuständigkeiten und unzugewiesene aus Github aktualisieren

Bei Lokalise ist es sehr aufwendig, herauszufinden, welche neuen Übersetzungsdateien hinzugekommen sind und bei welchen alten Dateien neue Begriffe hinzugekommen oder alte weggefallen sind. Eine Möglichkeit dies zu verbessern, wäre, dass bei "Translation deployment" oben beschriebene Verfahren zu kopieren. Man holt also gelegentlich aus Github die Json-Dateien aus den Translation-Ordnern und guck, was sich seit dem letzten Abholen geändert hat und erzeugt darauf die Liste für Zuständigkeiten., statt sie ziemlich aufwendig von Hand zu pflegen.

Die Scripts unten sind erstmals 2024-07-10 eingestellt und zur Anschauung. Da ist sicher Optimierungspotential.

Schritt 1: Json-Dateien holen

fetch-folio-reps.sh
#!/bin/bash # Set the GitHub organization name ORG="folio-org" # Set the API endpoint API_URL="https://api.github.com/orgs/$ORG/repos" # Set the number of items per page (max 100) PER_PAGE=100 # Initialize page number PAGE=1 # Initialize an empty array to store filtered repository names declare -a FILTERED_REPOS while true; do # Make the API request with pagination RESPONSE=$(curl -s "${API_URL}?page=${PAGE}&per_page=${PER_PAGE}") # Extract repository names from the response, filter, and add to array while read -r repo; do if [[ $repo == mod-* || $repo == stripes-* || $repo == ui-* ]]; then FILTERED_REPOS+=("$repo") fi done < <(echo "$RESPONSE" | jq -r '.[].name') # If less than PER_PAGE repositories are returned, we've reached the end if [ $(echo "$RESPONSE" | jq '. | length') -lt $PER_PAGE ]; then break fi # Move to the next page ((PAGE++)) done # Sort the filtered repositories alphabetically IFS=$'\n' sorted=($(sort <<< "${FILTERED_REPOS[*]}")) unset IFS # Print all filtered and sorted repository names printf '%s\n' "${sorted[@]}" # Print the total number of filtered repositories echo "Total filtered repositories: ${#sorted[@]}" # Create a directory to store the downloaded files mkdir -p translations # Download en.json and de.json for each repository for repo in "${sorted[@]}"; do echo "Downloading translations for $repo" # Download en.json en_url=https://raw.githubusercontent.com/folio-org/$repo/master/translations/$repo/en.json de_url=https://raw.githubusercontent.com/folio-org/$repo/master/translations/$repo/de.json if curl --output /dev/null --silent --head --fail "$en_url"; then # Download en.json curl -s "$en_url" -o "translations/${repo}_en.json" else echo "File does not exist or cannot be accessed." fi # Download de.json if curl --output /dev/null --silent --head --fail "$en_url"; then curl -s "$de_url" -o "translations/${repo}_de.json" echo "File downloaded successfully." else echo "File does not exist or cannot be accessed." fi done echo "Download complete. Files are stored in the 'translations' directory."

Schritt 2: Welche Schlüssel in de.json sind nicht übersetzt (also nur in en.json vorhanden): oder: welche ganzen Zeilen (key+value) sind identisch, d.h. nicht übersetzt

compare-keys.sh
#!/bin/bash # Directory containing the translation files TRANSLATIONS_DIR="translations" # Function to extract keys from a JSON file extract_keys() { jq -r 'keys[]' "$1" | sort } # Function to count keys in a JSON file count_keys() { jq 'keys | length' "$1" } # Function to compare two sets of keys compare_keys() { local en_file="$1" local de_file="$2" local repo_name="$3" local out_de_missing="" local out_de_extra="" local out_all_match="" local out_untranslated="" # Extract keys en_keys=$(extract_keys "$en_file") de_keys=$(extract_keys "$de_file") # Count total entries in English file en_total=$(count_keys "$en_file") de_total=$(count_keys "$de_file") # Find keys in en.json but not in de.json missing_in_de=$(comm -23 <(echo "$en_keys") <(echo "$de_keys")) missing_count=$(echo "$missing_in_de" | grep -c '^') # Find keys in de.json but not in en.json extra_in_de=$(comm -13 <(echo "$en_keys") <(echo "$de_keys")) extra_count=$(echo "$extra_in_de" | grep -c '^') # Find whole matching lines in the de and en files (= not translated) untranslated_de=$(comm -12 <(cat "$en_file" | sort | awk '{gsub(/^ +| +$/,"")} {print $0}') <(cat "$de_file" | sort | awk '{gsub(/^ +| +$/,"")} {print $0}') | grep -v '^[\{\}]$' ) untranslated_count=$(echo "$untranslated_de" | grep -c '^') # Report findings out_head=$(echo -e "Comparing keys for $repo_name: Total entries in file EN: $en_total / DE: $de_total\n================================== \n") if [ -n "$missing_in_de" ]; then out_de_missing=$(echo -e "\n--- Keys in en.json but missing in de.json ($missing_count): ---\n $missing_in_de\n") fi if [ -n "$extra_in_de" ]; then out_de_extra=$(echo -e "\n--- Keys in de.json but not in en.json ($extra_count): ---\n $extra_in_de\n") fi if [ -z "$missing_in_de" ] && [ -z "$extra_in_de" ]; then out_all_match=$(echo -e "\n--- All keys (!) match between en.json and de.json ---\n") fi if [ "$untranslated_count" -gt "1" ]; then out_untranslated=$(echo -e "\n\n--- These strings are not translated to de yet ($untranslated_count): ---\n $untranslated_de \n\n") fi # output the final result (modify it to your likings) echo -e "$out_head $out_de_missing $out_de_extra $out_all_match $out_untranslated\n" } # Main script echo "My Compares" > ./compares.txt for en_file in "$TRANSLATIONS_DIR"/*_en.json; do # Extract repository name from filename repo_name=$(basename "$en_file" _en.json) de_file="${en_file%_en.json}_de.json" # Check if both files exist if [ -f "$en_file" ] && [ -f "$de_file" ]; then compare_keys "$en_file" "$de_file" "$repo_name" >> ./compares.txt else echo "Missing translation file for $repo_name" >> ./compares.txt echo >> ./compares.txt fi done

Schritt 3: Zuständigkeiten formatiert ausgeben

Die Liste unter Zuständigkeiten nun in 3 Spalten mit den Mods und UIs ausgeben (Übersetzungsdatei, Name Zuständigkeit, Fehlende Übersetzung). Eine CSV mit halb-manuell gepflegter "Übersetzungsdatei, Name Zuständigkeit" muss es natürlich geben. Das fehlt hier noch.

Related content