Amazon recently announced that AWS DataSync now supports Google Cloud Storage and azure Files as storage locations. The two new data service options allow data to be moved both in and out of AWS, but data transfer charges can still be a limitation.
DataSync can copy and sync data across different storage locations and move it to AWS, supporting multi-cloud workflows or data retention requirements. Based on a proprietary network protocol, DataSync performs and verifies one-time and periodic data transfers and scales them according to the size of the job. The service provides include/exclude filters, bandwidth throttling controls, and supports automatic rollbacks from temporary network issues. Danilo PocciaEMEA Chief Evangelist at AWS, Explains:
This way, you can simplify your data processing or storage consolidation tasks. It’s also useful if you need to import, share, and exchange data with customers, vendors, or partners who use Google Cloud Storage or Microsoft Azure Files. DataSync provides end-to-end security, including encryption and integrity validation.
Google Cloud Storage and Azure Files Storage are the primary sources targeted at concurrent services, but they are not the only supported storage locations: AWS DataSync can sync data with NFS, SMB, Hadoop Distributed File Systems (HDFS), self-managed object storage and various AWS services such as Amazon S3 or Amazon FSx.
While Poccia explains how to move data from Google Cloud Storage to Amazon S3, Rodney Underkoffler and Aidan Keane, Senior Specialist Solutions Architects at AWS, show how to move data from Azure Files SMB shares. In the past, the cloud provider explained how to transfer data between AWS accounts using DataSync. Corey Quinn in his newsletter Written:
This is perhaps one of the most impressive AWS features I’ve seen in some time. It recognizes both the existence of other clouds and the fact that customers may sometimes want data to live there. Good game.
Larry Hau, Director of Products at Rackspace Technology, Accepted:
It seems like a huge deal (…) AWS has always locked in customers and it reverses that. Egress is still a hurdle, but it makes data migration much easier, especially for BCP/DR.
AWS DataSync can help migrate data between major cloud providers, but data transfer fees can be a significant barrier. To mitigate the impact and reduce egress charges, AWS recommends deploying the DataSync agent in the source environment as it compresses data in-flight while copying.
There are no specific DataSync costs for the new storage options but moving data to AWS, customers are subject to Google Cloud and Microsoft Azure data transfer charges. When moving data from AWS, customers are charged for transferring data from EC2 to the Internet.
The speed at which AWS DataSync copies depends on the amount of data and network conditions. For example, for data transfer between on-premises and AWS, a task can fully utilize a 10 Gbps network link.