GAE/J datastore backup

Question

What is the easiest way to do a GAE/J datastore backup?

It looks like there is python bulkloader.py tool to do backup for Python apps, but what should I do to backup Java app? Is there any way to use python tool?

score 16 · Accepted Answer · answered Mar 02 '10 at 17:24

It is possible to use python tool bulkloader.py to create datastore backup of GAE Java app. You just have to set up remote_api by adding following lines to web.xml:

<?xml version="1.0" encoding="utf-8"?>
<web-app>
  <!-- Add this to your web.xml to enable remote API on Java. -->
  <servlet>
    <servlet-name>remoteapi</servlet-name>
    <servlet-class>com.google.apphosting.utils.remoteapi.RemoteApiServlet</servlet-class>
  </servlet>
  <servlet-mapping>
    <servlet-name>remoteapi</servlet-name>
    <url-pattern>/remote_api</url-pattern>
  </servlet-mapping>
  <security-constraint>
    <web-resource-collection>
      <web-resource-name>remoteapi</web-resource-name>
      <url-pattern>/remote_api</url-pattern>
    </web-resource-collection>
    <auth-constraint>
      <role-name>admin</role-name>
    </auth-constraint>
  </security-constraint>
</web-app>

After that you can use bulkloader.py with --dump to download backup and with --restore to upload backup to datastore.

The security-constraint part can now be safely removed (AppEngine 1.4.3), as the Servlet itself now also performs this check — Axel Fontaine, Apr 13 '11 at 12:50
This is not a valid solution with HR datastore: "If you attempt to download data, you'll see a high_replication_warning error in the Admin Console, and the downloaded data might not include recently saved entities." — Luca Matteis, Aug 31 '11 at 12:53
Please see [Here](http://stackoverflow.com/a/12458521/1328880) for steps required to use RemoteApiServlet — Maithilish, Sep 17 '12 at 12:04

score 2 · Answer 2 · answered Dec 16 '12 at 05:21

2

I know this question is quite old, but this came out as a feature of the Datastore Administration in the app-engine dashboard.

answered Dec 16 '12 at 05:21

JohnIdol

48,899
61
158
242

Gopi · Answer 3 · 2010-03-06T14:49:27.953

2

Or if you can, you may wait for the datastore backup-restore feature in GAE upcoming versions as seen in the roadmap. http://code.google.com/appengine/docs/roadmap.html

edited Mar 06 '10 at 14:49

answered Mar 06 '10 at 14:43

Gopi

10,073
4
31
45

score 1 · Answer 4 · answered Mar 02 '10 at 16:29

1

Just set up remote_api for your app using the directions here - notably the tip:

Tip: If you have a Java app, you can use the Python bulkloader.py tool by installing the Java version of the remote_api handler, which is included with the Java runtime environment. The handler servlet class is com.google.apphosting.utils.remoteapi.RemoteApiServlet.

Then, use the Python bulkloader with --dump or --restore.

answered Mar 02 '10 at 16:29

Nick Johnson

100,655
16
128
198

can bulkloader --dump do incremental downloads? – Thilo May 09 '10 at 08:58
1

No. There's currently no generic way to do incremental downloads, since App Engine doesn't expose a total order over entities. – Nick Johnson May 10 '10 at 17:51

score 0 · Answer 5 · answered Mar 02 '18 at 00:49

0

You can now use the managed export and import feature, which can be accessed through gcloud or the Datastore Admin API:

Exporting and Importing Entities

Scheduling an Export

answered Mar 02 '18 at 00:49

Juan Lara

6,454
1
22
31

GAE/J datastore backup

5 Answers5

Linked