Agentless transfer permissions

Agentless transfers include transfers within Cloud Storage, or from Amazon S3, Azure Blob Storage or Data Lake Storage Gen2, or URL lists of public objects. They do not require transfer agents or agent pools.

When transferring data between cloud providers in an agentless transfer, you must grant permissions to two entities:

• The user or user-managed service account that's creating or managing the transfer.
• A Google-managed service account that's automatically created the first time you interact with Storage Transfer Service.

In addition, you must configure access to the source files on AWS S3, Microsoft Azure, or in Cloud Storage.

To learn how to grant IAM roles, see Granting, changing, and revoking access to resources.

Simplest method of granting permissions

The gcloud CLI can be used to grant the required permissions to the user / user-managed service account and to the Google-managed service account. These permissions allow the user to create, edit, and delete transfer jobs, and set or modify bandwidth limits.

If these permissions are too broad for your organization's policies, refer to the later sections in this document for the minimum permissions required by Storage Transfer Service.

To inspect existing permissions and print out any missing roles, run the following command:

gcloud transfer authorize

To automatically apply those roles, use the --add-missing flag:

gcloud transfer authorize --add-missing

To grant permissions to a user-managed service account, pass the service account key file:

gcloud transfer authorize --add-missing --creds-file=path/to/key.json

For instructions on creating a service account, see Creating and managing service accounts.

The command grants the following permissions.

• To the user / user-managed service account:

  • roles/owner
  • roles/storagetransfer.admin
  • roles/storagetransfer.transferAgent
  • roles/storage.objectAdmin
  • roles/pubsub.editor

• To the Google-managed service account:

  • roles/storage.admin
  • roles/storagetransfer.serviceAgent

For instructions on installing the gcloud CLI, see the gcloud quickstart.

User or user-managed service account permissions

This section covers the roles required for the accounts managing and executing transfers. Your organization's requirements will dictate the exact roles for each persona; this section assumes that you will create an administrator and a user.

Administrator accounts

Storage Transfer Service administrator accounts set bandwidth usage limits and delete transfer jobs.

To set up an administrator account, assign the following IAM permissions and roles:

User accounts

Storage Transfer Service user accounts are used to create and execute transfers. These accounts typically don't have access to delete transfer jobs.

A user account can be a Google Cloud console user, or a service account. If you're using a service account, the method you use to pass the credentials to Storage Transfer Service varies depending on the interface you use.

To set up a user account, assign the following permissions and roles to the account:

Google-managed service account permissions

Storage Transfer Service uses a Google-managed service account to move your data. This service account is automatically created the first time you create a transfer job, call googleServiceAccounts.get, or visit the job creation page in the Google Cloud console.

The service account's format is typically project-PROJECT_NUMBER@storage-transfer-service.iam.gserviceaccount.com. To retrieve your service account identifier, use the googleServiceAccounts.get API call.

Automatically assigning roles

There are two ways to automatically assign the correct roles to the Google-managed service account:

• Create your transfer using the Google Cloud console. The console automatically applies the required permissions.

• Use gcloud transfer authorize --add-missing. See Simplest method of granting permissions.

Manually assigning roles

To allow the Google-managed service account access to resources needed to complete transfers, assign the following roles, or equivalent permissions, to the service account.

See Grant the required permissions for instructions.

Cross-project Cloud Storage transfers

To transfer objects between Cloud Storage buckets in different projects, you must grant permission to a single Google-managed service account on both the source bucket and the destination bucket. The Google-managed service account belongs to the project from which you are creating the transfer.

The user or user-managed service account only needs permissions on the project that is creating the transfer.

To obtain the Google-managed service account's email, refer to Find the service agent's email.

On the source bucket, grant the following roles at the bucket level to the Google-managed service account:

• Storage Legacy Bucket Reader
• Storage Object Viewer

On the destination bucket, grant the following role at the bucket level to the same Google-managed service account:

• Storage Legacy Bucket Writer

For instructions on granting roles at the bucket level, refer to the following pages:

• Configure access to a source: Cloud Storage
• Configure access to a sink: Cloud Storage

Organizational restrictions

If your organization has applied organization policy constraints such as domain restrictions (constraints/iam.allowedPolicyMemberDomains) or disabling attachment of service accounts to resources in other projects (constraints/iam.disableCrossProjectServiceAccountUsage), these constraints must be updated before cross-project transfers can succeed.

You may also need to update any VPC service control settings to support cross-project transfers. See Configure VPC Service Controls for cloud object storage transfers.

AWS and Microsoft permissions

You must also configure access to your source files if they're in AWS S3 or Microsoft Azure Storage. The following documents detail the required steps:

• Configure access to AWS S3
• Configure access to Microsoft Azure Storage