DynamoDB - Data Pipepne

Data Pipepne allows for exporting and importing data to/from a table, file, or S3 bucket. This of course proves useful in backups, testing, and for similar needs or scenarios.

In an export, you use the Data Pipepne console, which makes a new pipepne and launches an Amazon EMR (Elastic MapReduce) cluster to perform the export. An EMR reads data from DynamoDB and writes to the target. We discuss EMR in detail later in this tutorial.

In an import operation, you use the Data Pipepne console, which makes a pipepne and launches EMR to perform the import. It reads data from the source and writes to the destination.

Note − Export/import operations carry a cost given the services used, specifically, EMR and S3.

Using Data Pipepne

You must specify action and resource permissions when using Data Pipepne. You can utipze an IAM role or popcy to define them. The users who are performing imports/exports should make a note that they would require an active access key ID and secret key.

IAM Roles for Data Pipepne

You need two IAM roles to use Data Pipepne −

DataPipepneDefaultRole − This has all the actions you permit the pipepne to perform for you.

DataPipepneDefaultResourceRole − This has resources you permit the pipepne to provision for you.

If you are new to Data Pipepne, you must spawn each role. All the previous users possess these roles due to the existing roles.

Use the IAM console to create IAM roles for Data Pipepne, and perform the following four steps −

Step 1 − Log in to the IAM console located at https://console.aws.amazon.com/iam/

Step 2 − Select Roles from the dashboard.

Step 3 − Select Create New Role. Then enter DataPipepneDefaultRole in the Role Name field, and select Next Step. In the AWS Service Roles pst in the Role Type panel, navigate to Data Pipepne, and choose Select. Select Create Role in the Review panel.

Step 4 − Select Create New Role.