Data Migration 101 — Storage Transfer Service.

Whether you want to migrate your on-prem data to cloud or between cloud providers, teams within an Organization should come together, choose a service that best fits, estimate costs, plan and execute.

Storage Transfer Service or simply Transfer Service is one of GCP’s offering that help you move data either from On-Prem, or from a different cloud or even from one GCS bucket to other. You can seamlessly create and process transfer operations to GCP Storage in a fully managed and serverless way.

No code — No infrastructure

User may choose either to perform one-time transfer or schedule transfers.

One can access Storage Transfer Service from Navigation -> Storage -> Data Transfer.

Transfer Service | Cloud when you want to transfer from other cloud storage service such as Amazon S3 or Azure Blob storage (Available in BETA).

Transfer Service | on-premises when you want to transfer On-Prem data.

Transfer Appliance A hardware setup that is governed by Google when you wanted to do large data transfers at low bandwidths.

Imagine you want to transfer data via object URLs. Here’s the brief of what needs to be done:

• Make these source files available via internet and generate a TSV (tab- separated values) file that contains list of files to be transferred.
• Once we have the URL, navigate to Transfer Service and create New Transfer Job.

Create new transfer job from the storage console

• Select List of Object URLs as source and pass the URL of TSV file.

Select source and enter the URL of the TSV file.

• Select the destination storage bucket where you would like to save this data. You can also configure your transfer to run at scheduled timings as per teams requirements.

Provide source and enter the schedule to run your transfer job

Once you click create, setting up the transfer job comes to an end.

Once the transfer is completed, you can navigate to storage bucket browser and can view and verify the transferred files under the bucket details.

Points to remember:

• Since the service do not require custom code and infrastructure, there is a significant reduction in operational costs.
• Transfer speed depends on the network bandwidth. Please use transfer calculator to estimate time and costs of transfers.
• Check for data size restrictions.
• IAM Permissions required would depend on type of data you are transferring.
• Always check costs (Both Ingress and Egress) before choosing a service.