Real-time migration from on-premise to cloud

Enable real-time data migration from on-premise data centre to AWS (cloud environment data lake)

Build datamart over data lake leveraging complex transformation logic to make the resultant data instantly available for the end user

Data load scenarios covering:

• Source – RDBMS Data load through transformation
• Web-service – Data load through transformation with Lambda

Set up AWS Services (DMS, RDS, Glue Lambda) and verified data load

Configured S3 Bucket and Redshift

Facilitated creation of target tables & views

Leveraged transformation logic using Glue

Achieved real-time data migration from on-premise to cloud environment (data lake)

Instant end user access to data

Amazon S3, Amazon DMS, Amazon RDS, AWS Glue, AWS Lambda, Amazon Redshift, Python/Scala/Java, SparkSQL, AWS API Gateway