Translate Content Externally

With Course Builder, you can export course content to be translated by an external translation service (also called a translation bureau). Here is an overview of the steps:

  1. You produce and download a set of files that contain the content of your course and send them to a translation service.
  2. The service adds translated content to the files and returns them.
  3. You upload the translated files to your course.
  4. Course Builder processes and stores the translated content.
  5. You go to Settings > Translations and, under Other Languages, update the language(s) to Available.

Note that if you get a 403 authentication error that says, “You do not have permission to modify this app” (for any of the operations on this page), then you may need to install the gcloud command line tool so the app knows that it’s really you.

Translation files

The files that you send to the translation service are .po files. PO/POT is the standard format for translating software and websites.

By default, Course Builder produces one .po file (also called a translation file) for each language/locale that you have added to the course’s translation settings. A translation file consists of a set of entries. Each entry is a translatable item from the course, such as a question prompt or some lesson text.

ETL

You manage translation files from the command line using ETL (Extract, Transform, Load). This page describes the ETL commands used in managing external translation:

You can run these commands from your coursebuilder directory.

You can view the code executed by these commands and more documentation at https://github.com/google/coursebuilder-core/blob/master/coursebuilder/modules/i18n_dashboard/jobs.py.

Produce and Download Translation Files

Here is the basic ETL command for producing and downloading a set of translation files:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DownloadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/download.zip'

In this command, <app_server_name> is your project ID and hostname, and <course_url_prefix> is the name of the specific course. For example, if your project ID is underwater-basket-weaving, you might have an intermediate-level course available at

https://underwater-basket-weaving.appspot.com/intermediate

In this case, the command to download translation files would be

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DownloadTranslations \
  /intermediate underwater-basket-weaving.appspot.com \
  --job_args='/tmp/download.zip'

If you want files for the course at the root URL, you provide just the / with no <course_url_prefix>:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DownloadTranslations \
  / underwater-basket-weaving.appspot.com \
  --job_args='/tmp/download.zip'

By default, this command produces one translation (.po) file for each language/locale. These files are then stuffed in a .zip file named in the --job_args argument. Note that to prevent accidental overwrites, the specified file must not already exist. If it does, ETL will treat it as an error.

Set Download Options

Depending on your situation and on the translation service’s requirements, you may need to adjust what translation files get produced. The download command provides the following file options:

Downloading translation files for specific locales

To download one or more specific locales instead of every locale listed for the course, use the --locales argument. Its value is a comma-delimited list of the desired locales.

For example, the following command prepares and downloads files for the en_US and fr locales:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DownloadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/download.zip --locales=en_US,fr

Reduce translation file size

By default, a translation file contains all the entries for a course. This can lead to large files, which are difficult to manage. A translation service may request smaller files.

There are three ways to produce smaller translation files:

You can supply one or more of these arguments to ensure that your files are small enough to meet the translation service’s requirements.

--export=new

By default, if a course has been previously translated, the existing translations are included when a new translation files are produced. Including existing translations helps translators provide consistent translations for a course.

If you are updating an existing translation and need to send smaller files, you can choose to ignore the existing translations and only include items that need new translations (because the items are new or because they have been changed since the last translation was uploaded).

To get files containing only entries for items that need new translations, you supply the --export=new flag to the download command:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DownloadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/download.zip' --export=new

Note that setting --export=new has no effect on courses or locales that do not have existing translations. For those files, everything is new, so everything is exported.

--separate_files_by_type

Setting the --separate_files_by_type flag divides entries into separate files according to content type.

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DownloadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/download.zip' --separate_files_by_type

For each locale, you get a set of smaller files rather than one large one. The size of the set and its divisions will vary depending on the size and makeup of the course: all course settings go in one file, all questions go in another file, and if individual units, assessments, and lessons are large enough, separate files are created for each.

Note that separating entries by type can also make translation easier and more accurate because the translator can easily see the context of each entry.

--max_entries_per_file

You can control file size by specifying a maximum number of entries per file using the --max_entries_per_file argument:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DownloadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/download.zip' --max_entries_per_file=500

If the number of entries for a course did not exceed the maximum, you would get the default single file for each locale. Otherwise, the entries would be split across multiple files.

When using --max_entries_per_file, the entries in a course are not divided logically; they are simply split when the maximum number of entries is reached.

Reduce total number of translation files

Typically, reducing the size of translation files leads to an increase in the number of files that are sent out for translation. If you have many locales and many files per locale, the number of files can get out of hand, and the translation service may ask you to reduce the number of files sent.

--locale_agnostic

You can reduce the number of files without increasing their size by setting the --locale_agnostic flag to produce a single set of files not tied to any particular locale.

To download a single set of files for all locales, you supply the --locale_agnostic flag to the download command:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DownloadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/download.zip' --locale_agnostic

Note that the locale-agnostic set of files will not contain any existing translations. Thus, when using this flag, you want to be sure that either your course has no existing translations to be overwritten or that the translation service has the existing translations and understands the need to retain and match them when creating the new files.

The translation service will make a copy of the set of files for each locale that it will be translating content into and then return all of the copies with the appropriate translated content.

When uploading the translated files, you upload the files for each locale separately and supply the --force_locale argument with the value of the intended locale to tell Course Builder which locale the translated content is for.

Format translation files without HTML markup

Some translation services may not be able to cope with embedded HTML markup in course content. Angle brackets in the content itself can interfere with their processing of the translation files.

--encoded_angle_brackets

You can replace the angle brackets in your course content with square brackets escaped with backslashes by supplying the --encoded_angle_brackets to the download command:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DownloadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/download.zip' --encoded_angle_brackets

Note that you must supply the --encoded_angle_brackets flag when uploading the translated files to reverse the change and get the angle brackets back where they belong.

Don’t set this flag until you know that your translation service requires it. The backslashes that precede the square brackets are particularly susceptible to accidental deletion. A deleted backslash will break the conversion back to angle brackets when uploading, so this flag is not worth setting unless you are sure that it is necessary.

Check For Non-Textual Material

The entries in the translation files capture the textual material of your course, but they can’t capture non-textual material, such as words within an image.

To identify course material that will not be included in any translation files, you can run the following command:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.TranslateToReversedCase \
  /<course_url_prefix> <app_server_name>

This command reverses the case of the textual material in the course. After running the command, you go through the course and look for items that still appear in their normal case. These items are non-textual material that will not be translated by the normal process. Once you have identified these items, you can decide if they require translating and make separate arrangements to do so.

For example, consider the following screenshot from a lesson that presents a graph.

reversecaseexample.png

The heading and introductory text will be entries in the translations files; the graph’s legend and labels will not. You would check with your translation service to discuss translating images like this graph.

Upload Translated Files

When the translation service returns the files with translated content, you upload them to your course with the following command:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.UploadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/file.zip'

The file to be uploaded is named in the --job_args argument. You can upload a .po file or a .zip file. A .zip file does not need a particular structure. The script will find and process all of the .po files included in the .zip file.

Set Upload Options

When uploading translated files, you can supply arguments that correspond to certain download options and that turn upload warnings on or off.

Counterparts to download options

If you set the --locale_agnostic flag or the --encoded_angle_brackets flag when downloading translation files, you must supply the corresponding upload flag to ensure that Course Builder handles the translated files correctly.

--force_locale

If you used --locale_agnostic when downloading files, you must upload the returned files in separate batches and supply the intended locale for each batch using the --force_locale argument.

For example, to upload a set of files translated into Brazilian Portuguese (pt_BR), you would run the following command:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.UploadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/file.zip' --force_locale=pt_BR

The value for this argument is a single locale code.

--encoded_angle_brackets

If you used --encoded_angle_brackets when downloading files, you must also supply it to the upload command to get the angle brackets back:

sh scripts/etl.sh run modules.i18n_dashboard.jobs.UploadTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='/tmp/file.zip' --encoded_angle_brackets

Turn upload warnings on and off

Ideally, the entries in the uploaded files will perfectly and completely map onto items in your course. There are two flags that turn on warnings to tell you when this ideal is not satisfied:

These flags only turn on warnings: the upload will proceed, and you will need to follow up on any findings afterward.

--warn_not_used

The --warn_not_used setting is on by default. Whent this setting is on, you get a warning when an entry in the uploaded translation files does not match any item in the course. This means that the translated content for that entry was not incorporated into the course and will not appear to students accessing the course in that language.

This warning usually appears when course content has been changed since the translation files were produced and downloaded. The action you take to correct the problem depends on the type and the extent of the changes to the course content. Consider the following options:

  • Use Course Builder’s translation console to fix the out-dated or missing translation.
  • Leave the content untranslated and let the base-language version appear to students.
  • Download a new set of files to send to translators.

In general, you want to keep this setting on, but there are rare cases where it causes too much noise. You can turn this setting off by supplying --warn_not_used=False to the upload command.

--warn_not_found

Supply the --warn_not_found argument to see a warning when an item in the course does not have a matching item in the uploaded translation files. This flag is not on by default and must be set manually.

Do not set this flag when uploading files that were downloaded with --separate_files_by_type, --max_entries_per_file, or --export=new. These arguments intentionally produce files containing entries from only part of the course, and you will get warnings for the supposedly missing entries in each file.

Delete Translations

The following command deletes all of the translations for the course specified by <course_url_prefix>.

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DeleteTranslations \
  /<course_url_prefix> <app_server_name>

Delete translations for specific locales

You can also delete translations for particular locales. For example, the following command deletes the translations for the en_US and fr locales for the specified course.

sh scripts/etl.sh run modules.i18n_dashboard.jobs.DeleteTranslations \
  /<course_url_prefix> <app_server_name> \
  --job_args='--locales=en_US,fr'

Make Translations Available

When the translations are uploaded and you are ready for students to access the course in that language, go to Settings > Translations. In the Other Languages setting, update the language to Available.