3

I have a postgres database running locally that I'd like to migrate to AWS Aurora (or AWS postgres).

I've pg_dump'd the database that I want, and it's ~30gb compressed.

How should I upload this file and get the AWS RDS instance to pg_restore from it?

Requirements:

There's no one else using the DB so we're ok with a lot of downtime and an exclusive lock on the db. We want it to be as cheap as possible to migrate

What I've tried/looked at so far:

  • Running pg_restore on the local file with the remote target - unknown pricing total

I'd also like to do this as cheaply as possible, and I'm not sure I understand their pricing strategy.

Their pricing says:

Storage Rate            $0.10 per GB-month
I/O Rate                $0.20 per 1 million requests
Replicated Write I/Os   $0.20 per million replicated write I/Os

Would pg_restore count as one request? The database has about 2.2 billion entries, and if each one is 1 request does that come out to $440 to just recreate the database?

  • AWS Database Migration Service - it looks like this would be the cheapest (as it's free?) but it only works by connecting to the local database. Uncompressed the data is about 200gb, and I'm not sure it makes sense to do a one for one copy using DMS

  • I've read this article but I'm still not clear on the best way of doing the migration.

We're ok with this taking a while, we'd just like to do it as cheap as possible.

Thanks in advance!

JonLuca
  • 850
  • 6
  • 25
  • AFAIK, incoming data is free of charge anyway, so a simple `pg_restore` (if possible allowing multi-threaded using `--format=directory --jobs=`) will do the job just fine. In any case, you will be charged for the I/O's and storage. For reducing the number of I/O's you can e.g. turn off logging before creating the dump (`ALTER TABLE xyz SET UNLOGGED;`) and turn it back on after the restore. – Ancoron Apr 02 '19 at 23:21

1 Answers1

1

There are some points you should note when migrating

AWS Database Migration Service - it looks like this would be the cheapest (as it's free?)

The service they provide for free is a Virtual machine ( with softwares included ) that provide the computing power and functionality to move Databases to some of their RDS service.

Even when that service is free, you would be charged normal fee for any RDS usage

The number they provided is roughly related to EBS (the underlying disks ) they use to serve your data. A very big and complex query can take some I/O, the two are not equal to eachother. The estimation for EBS usage can be seen here

As an example, a medium sized website database might be 100 GB in size and expect to average 100 I/Os per second over the course of a month. This would translate to $10 per month in storage costs (100 GB x $0.10/month), and approximately $26 per month in request costs (~2.6 million seconds/month x 100 I/O per second * $0.10 per million I/O).

My personal advice: Make a clone of your DB with only part of the set ( 5% maybe). Use DMS on that piece. You can see how the bills work out for you in a few minutes. Then you can estimate the price on a full DB migration

qkhanhpro
  • 4,371
  • 2
  • 33
  • 45