AWS DataSync Enhanced mode now supports OpenID Connect (OIDC) authentication for Google Cloud Storage and Microsoft Azure Blob Storage locations. DataSync uses an AWS Identity and Access Management (IAM) role to get short-lived credentials from Google Cloud or Microsoft Entra ID and refreshes them automatically so your transfers run without interruption. You can now use DataSync to access your Google Cloud Storage and Azure Blob Storage without long-term credentials such as hash-based message authentication code (HMAC) keys or shared access signature (SAS) tokens. This lets you move data across clouds while complying with security policies that might prohibit long-term credentials. You can get started by adding an IAM role to your DataSync location with the AWS CLI, API, or AWS CloudFormation. This capability is available in all AWS Regions where DataSync Enhanced mode is supported. To learn more, visit the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.

AWS DataSync Enhanced mode now supports OpenID Connect (OIDC) authentication for Google Cloud Storage and Microsoft Azure Blob Storage locations. DataSync uses an AWS Identity and Access Management (IAM) role to get short-lived credentials from Google Cloud or Microsoft Entra ID and refreshes them automatically so your transfers run without interruption. You can now use DataSync to access your Google Cloud Storage and Azure Blob Storage without long-term credentials such as hash-based message authentication code (HMAC) keys or shared access signature (SAS) tokens. This lets you move data across clouds while complying with security policies that might prohibit long-term credentials. You can get started by adding an IAM role to your DataSync location with the AWS CLI, API, or AWS CloudFormation. This capability is available in all AWS Regions where DataSync Enhanced mode is supported. To learn more, visit the AWS DataSync User Guide.

AWS DataSync Enhanced mode now supports Amazon FSx for Windows File Server and Amazon FSx for NetApp ONTAP as source or destination locations. You can now run Windows to Windows transfers, such as from an on-premises file server to FSx for Windows File Server. Additionally, Enhanced mode agents can now run on Red Hat OpenShift Virtualization version 4.20 or later. With this launch, you can use the same location options in Enhanced mode as in Basic mode. Enhanced mode processes data in parallel for higher performance, removes file count limitations, and provides detailed data transfer metrics. OpenShift Virtualization expands your agent deployment options alongside VMware ESXi, KVM, Microsoft Hyper-V, and Amazon EC2. You can get started by choosing Enhanced mode when you create a task in the AWS DataSync console. These capabilities are available in all AWS Regions where DataSync Enhanced mode is supported. To learn more, visit the AWS DataSync documentation.

AWS DataSync Enhanced mode now supports Amazon FSx for Windows File Server and Amazon FSx for NetApp ONTAP as source or destination locations. You can now run Windows to Windows transfers, such as from an on-premises file server to FSx for Windows File Server. Additionally, Enhanced mode agents can now run on Red Hat OpenShift Virtualization version 4.20 or later. With this launch, you can use the same location options in Enhanced mode as in Basic mode. Enhanced mode processes data in parallel for higher performance, removes file count limitations, and provides detailed data transfer metrics. OpenShift Virtualization expands your agent deployment options alongside VMware ESXi, KVM, Microsoft Hyper-V, and Amazon EC2. You can get started by choosing Enhanced mode when you create a task in the https://console.aws.amazon.com/datasync/home. These capabilities are available in all AWS Regions where DataSync Enhanced mode is supported. To learn more, visit the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.

AWS DataSync Enhanced mode now supports Amazon FSx for OpenZFS as a source or destination location, NFS version 4.2 for on-premises NFS servers and FSx for OpenZFS file systems, and a new ObjectMetadata option for object storage transfers. Previously, customers transferring data to or from FSx for OpenZFS were limited to Basic mode. NFS 4.2 lets you transfer data from NFS servers on the latest version without reconfiguring them to an older protocol version. The ObjectMetadata option gives you control over whether an object's user metadata is copied to the destination, which helps when destination metadata must stay intact, such as buckets served by AWS Storage Gateway, or when source metadata keys are not compatible with the destination, such as Azure Blob Storage. Existing tasks remain unaffected and continue to run as scheduled. These capabilities are available for Enhanced mode tasks in all AWS Regions where AWS DataSync Enhanced mode is supported. To learn more, visit the AWS DataSync documentation.

AWS DataSync Enhanced mode now supports Amazon FSx for OpenZFS as a source or destination location, NFS version 4.2 for on-premises NFS servers and FSx for OpenZFS file systems, and a new ObjectMetadata option for object storage transfers. Previously, customers transferring data to or from FSx for OpenZFS were limited to Basic mode. NFS 4.2 lets you transfer data from NFS servers on the latest version without reconfiguring them to an older protocol version. The ObjectMetadata option gives you control over whether an object's user metadata is copied to the destination, which helps when destination metadata must stay intact, such as buckets served by AWS Storage Gateway, or when source metadata keys are not compatible with the destination, such as Azure Blob Storage. Existing tasks remain unaffected and continue to run as scheduled. These capabilities are available for Enhanced mode tasks in all AWS Regions where AWS DataSync Enhanced mode is supported. To learn more, visit the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.

AWS DataSync now supports shared Virtual Private Clouds (VPCs). You can create DataSync agents and run transfer tasks using subnets shared across AWS accounts with AWS Resource Access Manager (RAM). It allows you to transfer data privately over a shared subnet and VPC endpoint managed in a central account, rather than one per account. Customers that centralize their networking previously had to create a separate DataSync VPC endpoint in every account that connected privately through AWS PrivateLink. Each endpoint consumed IP addresses and added operational overhead to maintain across accounts. With shared VPC support, a single endpoint in the account that owns the VPC serves every account the subnet is shared with, conserving IP address space and removing the need for a per-account endpoint. This launch also helps within a single account setup. You can now use one VPC endpoint across multiple subnets, removing the earlier requirement for a matching VPC endpoint in each subnet. Shared VPC is supported for both Enhanced mode and Basic mode agent-based tasks. To get started, create a DataSync agent using the console or the CreateAgent API and specify a subnet shared with your account via AWS RAM. AWS DataSync support for Shared VPCs is available in all AWS Regions, except AWS Secret Regions, where AWS DataSync is offered. To learn more, visit the AWS DataSync feature documentation.

AWS DataSync now supports shared Virtual Private Clouds (VPCs). You can create DataSync agents and run transfer tasks using subnets shared across AWS accounts with AWS Resource Access Manager (RAM). It allows you to transfer data privately over a shared subnet and VPC endpoint managed in a central account, rather than one per account. Customers that centralize their networking previously had to create a separate DataSync VPC endpoint in every account that connected privately through AWS PrivateLink. Each endpoint consumed IP addresses and added operational overhead to maintain across accounts. With shared VPC support, a single endpoint in the account that owns the VPC serves every account the subnet is shared with, conserving IP address space and removing the need for a per-account endpoint. This launch also helps within a single account setup. You can now use one VPC endpoint across multiple subnets, removing the earlier requirement for a matching VPC endpoint in each subnet. Shared VPC is supported for both Enhanced mode and Basic mode agent-based tasks. To get started, create a DataSync agent using the console or the CreateAgent API and specify a subnet shared with your account via AWS RAM. AWS DataSync support for Shared VPCs is available in all AWS Regions, except AWS Secret Regions, where AWS DataSync is offered. To learn more, visit the https://docs.aws.amazon.com/datasync/latest/userguide/choose-service-endpoint.html#datasync-in-vpc.

Move large amounts of data into AWS quickly, securely, and without writing your own copy scripts....

AWSDataSyncのビルダーズセッションに参加しましたのでご紹介します概要Jointhisbuilderssessiontoimmerseyourselfintheworldofmul

AWS DataSync Enhanced mode now supports agent-based data transfers with Hadoop Distributed File Systems (HDFS), Microsoft Azure Blob Storage, and self-managed object storage. Additionally, Enhanced mode agents can now be deployed on Microsoft Hyper-V. Using a DataSync agent, customers can transfer data to and from these locations with the parallelism, unlimited file counts, and detailed metrics that Enhanced mode provides. Enhanced mode HDFS support includes multiple NameNode configurations for high availability and Transparent Data Encryption (TDE) with Kerberos authentication, enabling organizations in regulated industries to securely migrate petabyte-scale encrypted Hadoop data without sacrificing availability. This capability is available in all AWS Regions where AWS DataSync is offered. To get started, visit the https://console.aws.amazon.com/datasync/home. For more information, see the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.

AWS DataSync Enhanced mode now supports Amazon EFS and Amazon FSx for Lustre as source or destination locations. AWS DataSync is a secure, high-speed data transfer service that simplifies moving data over a network. Enhanced mode processes data in parallel, removes file count limitations, and provides detailed transfer metrics. Previously, customers who wanted to move data to or from EFS or FSx for Lustre were limited to Basic mode. Now customers can use Enhanced mode for these locations, simplifying workflows for large-scale migrations, agentic AI and machine learning training, high-performance computing, genomics processing, and media rendering. This capability is available in all https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ where AWS DataSync is offered. To get started, visit the https://console.aws.amazon.com/datasync/home. For more information, see the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.

AWS DataSync now supports AWS Secrets Manager for credential management across all location types, including Hadoop Distributed File System (HDFS), Amazon FSx for Windows File Server, and Amazon FSx for NetApp ONTAP. Previously, Secrets Manager integration was limited to a subset of location types, requiring you to provide credentials directly through the DataSync API or console. You can centralize credential management for all DataSync locations in Secrets Manager, providing a single, consistent approach across all your data transfers. You can also encrypt credentials with your own AWS KMS key instead of the default AWS-owned key, helping you meet your organization's security requirements and governance policies. All secrets are stored in your account, allowing you to update credentials as needed, independent of the DataSync service. DataSync supports two approaches for credential management. You can provide a secret ARN referencing credentials you manage in Secrets Manager for full control over rotation, auditing, and access policies. Alternatively, DataSync can automatically create and manage secrets on your behalf. This capability is available is available in the majority of AWS regions where AWS DataSync is offered. For the full list of supported regions, visit the https://builder.aws.com/build/capabilities in Builder Center. To get started, visit the https://console.aws.amazon.com/datasync/. For more information, see https://docs.aws.amazon.com/datasync/latest/userguide/location-credentials.html in the AWS DataSync documentation.

The https://registry.terraform.io/modules/aws-ia/datasync/aws/latest now supports Enhanced mode for transfers between S3 locations, making it easier for you to set up high-performance data transfers at scale. https://aws.amazon.com/datasync/ is a secure, high-speed file transfer service that optimizes data movement over a network. Enhanced mode uses parallel processing to deliver higher performance and scalability for datasets of any size, while removing file count limitations and providing detailed transfer metrics for better monitoring and management. You can now use Terraform to automatically provision DataSync tasks configured for Enhanced mode. This eliminates manual configuration steps that can be time-consuming and error-prone, while giving you a consistent, repeatable, version-controlled deployment process that can scale across your organization. You can access the AWS DataSync Terraform module on https://github.com/aws-ia/terraform-aws-datasync or through the https://registry.terraform.io/modules/aws-ia/datasync/aws/latest. To learn more about DataSync, see the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html. To see all Regions where DataSync is available, visit the https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/.

https://aws.amazon.com/datasync/ Enhanced mode now supports data transfers between on-premises file servers and https://aws.amazon.com/s3/, enabling customers to transfer datasets that scale to virtually unlimited numbers of files at higher levels of performance than DataSync Basic mode. AWS DataSync is a secure, high-speed file transfer service that optimizes data movement over a network. https://aws.amazon.com/about-aws/whats-new/2024/10/aws-datasync-performance-scalability-data-transfers/ uses parallel processing to deliver higher performance and scalability for datasets of any size, while removing file count limitations and providing detailed transfer metrics for better monitoring and management. Previously, Enhanced mode was available for data transfers between Amazon S3 locations and for multicloud transfers. This launch extends the capabilities of Enhanced mode to support transfers between on-premises NFS or SMB file servers, and Amazon S3. Using Enhanced mode, customers can accelerate generative AI workloads by rapidly moving training datasets to AWS, power data lake analytics by synchronizing on-premises data with cloud-based pipelines, and drive large-scale migrations for archival and cloud modernization. This new capability is available in all https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ where AWS DataSync is offered. To get started, visit the https://console.aws.amazon.com/datasync/home. For more information, see the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.

https://aws.amazon.com/datasync/ now supports virtual private cloud (VPC) endpoint policies, allowing you to control access to DataSync API operations through DataSync VPC service endpoints and Federal Information Processing Standard (FIPS) 140-3 enabled VPC service endpoints. This new feature helps organizations strengthen their security posture and meet compliance requirements when accessing DataSync API operations through VPC endpoints. VPC endpoint policies allow you to restrict specific DataSync API actions accessed through your VPC endpoints. For example, you can control which AWS principals can access DataSync operations such as CreateTask, StartTaskExecution, or ListAgents. These policies work in conjunction with identity-based policies and resource-based policies to secure access in your AWS environment. This feature is available in all https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ where AWS DataSync is available. For more information about FIPS 140-3 at AWS, visit https://aws.amazon.com/compliance/fips/. To learn more about VPC endpoint policies for AWS DataSync, see the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.

https://aws.amazon.com/datasync/ customers can now transfer data directly between storage in other clouds and https://aws.amazon.com/s3/, without needing to deploy DataSync agents. This new capability uses DataSync https://aws.amazon.com/about-aws/whats-new/2024/10/aws-datasync-performance-scalability-data-transfers/ to deliver increased performance and scalability, helping customers streamline their data pipelines and accelerate migrations from other clouds to AWS. AWS DataSync is a secure, reliable, high-speed data transfer service that simplifies moving data over networks. This launch enables direct transfers to and from storage services in other clouds including Google Cloud Storage, Microsoft Azure Blob Storage, and Oracle Cloud Object Storage. Using Enhanced mode, this new capability provides faster transfer speeds through parallel processing of data preparation, transfer, and verification, while supporting virtually unlimited object counts. Customers can monitor transfers using detailed metrics and reporting capabilities, all with a simplified setup that requires no agent deployment. This new capability is available in all https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ where AWS DataSync is offered. To get started, visit the https://console.aws.amazon.com/datasync/home. For more information, see the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.  

https://aws.amazon.com/datasync/ now supports Kerberos authentication for self-managed file servers that use the Server Message Block (SMB) network protocol. This update provides enhanced security options for connecting to SMB file servers commonly found in Microsoft Windows environments. DataSync is a secure, high-speed data transfer service that simplifies and accelerates moving data over a network. It automates copying files and objects between AWS Storage services, on-premises storage, and other clouds. DataSync uses protocols like SMB to transfer data to and from network storage systems. With this launch, you can configure your DataSync SMB locations to authenticate access to your storage using Kerberos, in addition to existing support for NT LAN Manager (NTLM) authentication. DataSync supports any Kerberos server, such as Microsoft Active Directory, that implements Kerberos protocol version 5. Kerberos authentication is available in all https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ where AWS DataSync is offered. To get started, visit the https://console.aws.amazon.com/datasync/home. To learn more, refer to the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.  

https://aws.amazon.com/datasync/ now supports configuration updates for all supported storage locations, including Amazon S3, Amazon EFS, Amazon FSx for Lustre, Amazon FSx for NetApp ONTAP, Amazon FSx for OpenZFS, and Amazon FSx for Windows File Server. With this enhancement, you can update storage locations directly, removing the need to delete and create new locations in order to update key parameters. AWS DataSync is a high-speed data transfer service that securely and reliably copies data over a network. With this new capability, you gain greater flexibility in managing data transfer configurations across supported AWS storage services. With configuration update support extended to Amazon S3, Amazon EFS, and Amazon FSx storage locations, you can now adjust properties, such as access roles, directory paths, and authentication credentials, without recreating tasks and locations, making it easier than ever to adapt DataSync tasks to meet your evolving business needs. This new capability is available in all https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/ where AWS DataSync is available, except the China (Beijing) and the China (Ningxia) Regions. To get started, visit the https://console.aws.amazon.com/datasync/home. To learn more, view the https://docs.aws.amazon.com/datasync/latest/userguide/what-is-datasync.html.  

AWS DataSync Enhanced mode now supports Amazon EFS and Amazon FSx for Lustre as source or destination locations. AWS DataSync is a secure, high-speed data transfer service that simplifies moving data over a network. Enhanced mode processes data in parallel, removes file count limitations, and provides detailed transfer metrics. Previously, customers who wanted to move data to or from EFS or FSx for Lustre were limited to Basic mode. Now customers can use Enhanced mode for these locations, simplifying workflows for large-scale migrations, agentic AI and machine learning training, high-performance computing, genomics processing, and media rendering. This capability is available in all AWS Regions where AWS DataSync is offered. To get started, visit the AWS DataSync console. For more information, see the AWS DataSync documentation.

AWS DataSync Enhanced mode now supports agent-based data transfers with Hadoop Distributed File Systems (HDFS), Microsoft Azure Blob Storage, and self-managed object storage. Additionally, Enhanced mode agents can now be deployed on Microsoft Hyper-V. Using a DataSync agent, customers can transfer data to and from these locations with the parallelism, unlimited file counts, and detailed metrics that Enhanced mode provides. Enhanced mode HDFS support includes multiple NameNode configurations for high availability and Transparent Data Encryption (TDE) with Kerberos authentication, enabling organizations in regulated industries to securely migrate petabyte-scale encrypted Hadoop data without sacrificing availability. This capability is available in all AWS Regions where AWS DataSync is offered. To get started, visit the AWS DataSync console. For more information, see the AWS DataSync documentation.

AWS DataSync now supports AWS Secrets Manager for credential management across all location types, including Hadoop Distributed File System (HDFS), Amazon FSx for Windows File Server, and Amazon FSx for NetApp ONTAP. Previously, Secrets Manager integration was limited to a subset of location types, requiring you to provide credentials directly through the DataSync API or console. You can centralize credential management for all DataSync locations in Secrets Manager, providing a single, consistent approach across all your data transfers. You can also encrypt credentials with your own AWS KMS key instead of the default AWS-owned key, helping you meet your organization's security requirements and governance policies. All secrets are stored in your account, allowing you to update credentials as needed, independent of the DataSync service. DataSync supports two approaches for credential management. You can provide a secret ARN referencing credentials you manage in Secrets Manager for full control over rotation, auditing, and access policies. Alternatively, DataSync can automatically create and manage secrets on your behalf. This capability is available is available in the majority of AWS regions where AWS DataSync is offered. For the full list of supported regions, visit the AWS Capabilities tool in Builder Center. To get started, visit the AWS DataSync console. For more information, see Managing credentials with AWS Secrets Manager in the AWS DataSync documentation.

The AWS DataSync Terraform module now supports Enhanced mode for transfers between S3 locations, making it easier for you to set up high-performance data transfers at scale. AWS DataSync is a secure, high-speed file transfer service that optimizes data movement over a network. Enhanced mode uses parallel processing to deliver higher performance and scalability for datasets of any size, while removing file count limitations and providing detailed transfer metrics for better monitoring and management. You can now use Terraform to automatically provision DataSync tasks configured for Enhanced mode. This eliminates manual configuration steps that can be time-consuming and error-prone, while giving you a consistent, repeatable, version-controlled deployment process that can scale across your organization. You can access the AWS DataSync Terraform module on GitHub or through the Terraform Registry. To learn more about DataSync, see the AWS DataSync documentation. To see all Regions where DataSync is available, visit the AWS Region table.

AWS DataSync Enhanced mode now supports data transfers between on-premises file servers and Amazon S3, enabling customers to transfer datasets that scale to virtually unlimited numbers of files at higher levels of performance than DataSync Basic mode. AWS DataSync is a secure, high-speed file transfer service that optimizes data movement over a network. Enhanced mode uses parallel processing to deliver higher performance and scalability for datasets of any size, while removing file count limitations and providing detailed transfer metrics for better monitoring and management. Previously, Enhanced mode was available for data transfers between Amazon S3 locations and for multicloud transfers. This launch extends the capabilities of Enhanced mode to support transfers between on-premises NFS or SMB file servers, and Amazon S3. Using Enhanced mode, customers can accelerate generative AI workloads by rapidly moving training datasets to AWS, power data lake analytics by synchronizing on-premises data with cloud-based pipelines, and drive large-scale migrations for archival and cloud modernization. This new capability is available in all AWS Regions where AWS DataSync is offered. To get started, visit the AWS DataSync console. For more information, see the AWS DataSync documentation.

AWS DataSync now supports virtual private cloud (VPC) endpoint policies, allowing you to control access to DataSync API operations through DataSync VPC service endpoints and Federal Information Processing Standard (FIPS) 140-3 enabled VPC service endpoints. This new feature helps organizations strengthen their security posture and meet compliance requirements when accessing DataSync API operations through VPC endpoints. VPC endpoint policies allow you to restrict specific DataSync API actions accessed through your VPC endpoints. For example, you can control which AWS principals can access DataSync operations such as CreateTask, StartTaskExecution, or ListAgents. These policies work in conjunction with identity-based policies and resource-based policies to secure access in your AWS environment. This feature is available in all AWS Regions where AWS DataSync is available. For more information about FIPS 140-3 at AWS, visit FIPS 140-3 Compliance. To learn more about VPC endpoint policies for AWS DataSync, see the AWS DataSync User Guide.