Download App Data

Course Builder has two types of data that is uses: course and datastore. Course data is data that represents the course itself, like HTML and CSS files, while datastore data revolves around the student interactions with your course.

You can back up both course and datastore data by downloading them using the ETL command or through the Developers Console. We suggest using the Developers Console because it is easier but both ways are illustrated in this page.

Course Data

When you download course data, you are saving the structure and content of the course on your site. Examples of course data include:

  • Units — course unit information like ID, type, title, release date, and availability
  • Lessons — information about each lesson, such as the video’s URL and the lesson’s title
  • Assets — course CSS files, images, activities, assessments, and libraries

Datastore Data

When you download datastore data, you get data about students. By default, you also get data about student events. Examples include:

  • EventEntity — system level events and historical data about each activity or assessment completed.
  • Student — information about students, their enrollment status and assessment scores
  • StudentAnswersEntity — the most recent detailed answers given by a student for each assessment

Recording student event data is enabled by default, but it can be disabled to save time and potential costs. Learn more about time and costs.

Use the Developers Console

Back up your datastore data using the Developers Console.

Before you can back up to the datastore, you need to set up a Cloud Storage Bucket in which to back up your data. Learn how to set up a Cloud Storage Bucket.

  1. Go to the Developers Console
  2. Click Datastore from the main menu.
  3. Click Admin from the menu.
  4. Click Open Datastore Admin.
  5. Select Entity Kind to back up all datastore data.
  6. Type a name for your backup.
  7. Choose whether you want to back up datastore data for the entire Google App Engine instance on which Course Builder is on or one particular course. You will also need to choose a Cloud Storage Bucket for your backed-up data.

After a wait (duration depends on amount of data), you’ll see your backup at the bottom of the page.

Download App Data with ETL

The most common reason to download app data is to be able restore it another Google App Engine instance running Course Builder. You download course data and datastore data separately.

Download Course Data

The following command downloads your course data.

sh scripts/etl.sh download course <course_url_prefix> <app_server_name> --archive_path <archive_path>

It is best to use this command from the root of your Course Builder installation.

cd /Users/sammich/coursebuilder
sh scripts/etl.sh download course /physics zabaione.appspot.com --archive_path ./physics-course.zip

In the above example, you’d download course data only for the “physics” course. You’d run this command for each course for which you want to download course data. If you want to download a course on the root URL, you would just specify the slash / instead of slash with name (/physics).

Do Not Edit Downloaded Data

You can view and make copies of data exported from Course Builder, but you must upload the unmodified file back into Course Builder when you restore the data to the upgraded version. Modifying the data may cause an immediate upload error or (even worse) pass uploading and create downstream errors that are difficult to troubleshoot. If you modify it, however, don’t upload it back to your Course Builder app.

Download Datastore Information

Downloading datastore data uses a similar command:

sh scripts/etl.sh download datastore <course_url_prefix> <app_server_name> --archive_path <archive_path> --datastore_types <model1,model2>

Again, it is best to use this command from the root of your Course Builder installation. Here is an example of downloading datastore data using ETL:

cd /Users/sammich/coursebuilder
sh scripts/etl.sh download datastore /poetry bbq.appspot.com --archive_path ./poetry.zip --datastore_types Student,StudentAnswersEntity

The first line makes the current directory the Course Builder root directory (in this example, /Users/sammich/coursebuilder). The second line finds the course data for the course and downloads it as a file named physics.zip in the current directory. It also uses the --datastore_types option to specify which models to download. If you do not include this option, the command will default to downloading everything:

cd /Users/sammich/coursebuilder
sh scripts/etl.sh download datastore /poetry bbq.appspot.com --archive_path ./poetry-datastore.zip

Consider time and costs

Downloading data (whether to upgrade or for other reasons) will count against your site’s quota and may incur costs. Keep this in mind, especially when downloading datastore data. The amount of datastore data depends on the number of students enrolled, and for courses with high enrollment (tens of thousands), the file can be quite large and the download command can take hours to run.

Learn how to disable student analytics to reduce your resource usage.