Data Transfer and Bandwidth Cost | Pure CBS on Azure
Introduction
As businesses increasingly rely on cloud services like Pure Cloud Block Store (CBS) instances in Azure to store and manage their data, understanding the nuances of data transfer costs becomes crucial.
This article aims to demystify Azure bandwidth costs when transferring data to and from Pure Cloud Block Store instances, outlining different scenarios and their associated costs. Following are the type of scenarios in which CBS data transfers takes place:
- Replication traffic between a CBS instance and an on-prem FlashArray.
- Replication traffic across CBS instances.
- Data backup/restore between a CBS instance and an Azure Blob via Purity CloudSnap.
The prices listed in this article are examples based on Azure prices at the time of writing. Refer to Azure Bandwidth Pricing for updated pricing.
Understanding CBS Data Transfer Amount
In order to understand CBS data transfer costs, we need to first understand the amount of data that is transferred during the different types of CBS data transfers, which can be divided in the following 2 types:
- Replication across CBS instances, or between CBS instances and on-prem FlashArrays
- CBS backups and restores to an Azure Blob Storage using Purity CloudSnap.
The Amount of Data Transferred During Replication
CBS replication includes replication across CBS instances, and between CBS instances and on-prem FlashArrays.
Similar to the FlashArray, CBS employs a method of storing data in a compressed and deduplicated manner to optimize space usage and minimize storage expenses. When data is replicated asynchronously, this compression and deduplication are preserved during data transfers. This strategy not only decreases data transfer durations but also curtails Azure data transfer cost.
Furthermore, when a volume is replicated from one CBS instance or FlashArray to another, the initial copy contains the entire content, while subsequent copies exclusively include incremental changes. This process is explained in the following manner:
- Upon initiating volume replication on CBS (or a FlashArray), since the destination array does not have the volume, the entirety of the volume's contents is transmitted from the source array to the destination array in a compressed and deduplicated format. This version is stored completely on the destination array, setting up a baseline.
- Once the baseline is established, during subsequent replications of the same volume, only the delta changes in data since the previous snapshot are dispatched to the destination array. To avoid unnecessary network traffic and Azure transfer costs, the latest snapshot's local copy is retained on the source array. This copy is used to compute the differences for the subsequent snapshot. Upon receiving the delta changes on the destination array, the existing data from prior replications is seamlessly combined with the delta to create a complete snapshot.
- In the same manner, if the volume would be fail back from destination source, only the new data changes occurred on the destination will be sent back. In other words, no byte will be sent twice.
Replication Example
The following is a simplified example, serves as a simplified example that will be employed within the article to calculate the costs associated with bandwidth for various replication scenarios. Let’s assume we have a volume called Vol_A located on a Pure CBS instance in the East US region with the following properties:
- Data size = 1TB
- Compression = 2:1
- Deduplication = 2:1
- Daily change rate = 5%
After data reduction (compression & deduplication), the amount of data stored on the CBS instance for Vol_A is 250GB.
If we start replicating this volume to an on-prem FlashArray (or another CBS in a different region) at a “once per day” frequency, the initial baseline snapshot will transfer 250GB of data, and each subsequent snapshot will transfer 1.25GB of data (250GB X 5% = 12.5GB).
This highly efficient replication process has the following benefits:
- Reduction in storage footprint, leading to decreased Azure costs.
- Minimized data transfer cost and network utilization.
- Shorten replication durations.
The Amount of Data Transferred During CloudSnap Offload
For information about what is Purity CloudSnap and how to configure it, refer to this article.
CloudSnap preserves the data compression used in CBS for storage efficiency. However, it does not preserve the deduplication used by CBS. Let's dive into the life of protecting a volume using CloudSnap:
- During CBS backups to Azure Blob Storage, after the initial baseline snapshot of a volume is transferred to an Azure Blob, CloudSnap sends only the delta changes for subsequent snapshots of the same volume. A local copy of the latest snapshot is always kept on CBS to compute the delta changes. This prevents back and forth network traffic for calculating deltas between snapshots.
- During restores from Azure Blob, CloudSnap pulls back only the missing data blocks from the Azure Blob. This is combined with the data blocks that are already present on the CBS instance in order to rebuild the complete snapshot. This further reduces network traffic, as well as restore times.
In summary, the amount of data transferred by CloudSnap is equal to the size of the customer’s data in a compressed (but not deduped) format. After the initial snapshot of a volume has been backed up to the Azure Blob, only the delta changes are transferred for subsequent snapshots of the same volume.
Offload Example
The following is a simplified example, serves as a simplified example that will be employed within the article to calculate the costs associated with bandwidth for offload scenarios. Let’s assume we have a volume called Vol_A located on a Pure CBS instance in the East US region with the following properties:
- Data size = 1TB
- Compression = 2:1
- Deduplication = 2:1
- Daily change rate = 5%
After data reduction (compression & deduplication), the amount of data stored on the CBS instance for Vol_A is 250GB.
If we start offloading this volume via CloudSnap to an Azure Blob (location will be a variable for each scenario) “once per day” frequency, the initial baseline snapshot will transfer 500GB of data (the data is compressed, but not deduped), and each subsequent snapshot will transfer 25GB of data (500GB X 5% = 25GB).
CloudSnap backups are highly efficient, and provide the following benefits:
- The storage footprint in the S3 bucket is reduced
- Network utilization is minimized
- Backup & restore windows are shorter
- Data retrieval costs from the Azure Blob are minimized.
Pure Cloud Block Store on Azure is not only limited to Azure Blob as an offload target, you can offload to any supported object storage by Purity CloudSnap.
Data Transfer Scenarios – Replication
Following are the different CBS replication scenarios, along with the Azure data transfer costs associated with each of them.
Inbound Data Transfer
Ingress traffic directed towards Azure data centers is not subject to charges. Whether it originates from an on-premises environment, another Azure region, or any other public cloud provider, there are no associated costs.
| Data Transfer | Price |
|---|---|
|
All data transfer in |
$0.00 per GB |
Data Transfer within Availability Zones
An Availability Zone represents an isolated area within an Azure Region. These zones are designed to facilitate the deployment of resources within a highly available architecture, offering protection against Azure datacenter-level disruptions.
Resources deployed within the same Virtual Network (VNet) are exempt from data transfer charges when communicating within the confines of an Availability Zone. Conversely, if resources situated within the same Availability Zone communicate across peered VNet, data transfer costs will be applicable according to the rates stipulated for VNet peering. Further details can be found in the Virtual Network Pricing documentation.
| Data Transfer | Price |
|---|---|
|
Within the same AZ, same VNet |
$0.00 per GB |
| Within the same AZ, different VNet, using VNet Peering (Egress and Ingress) |
$0.01 per GB |
Same AZ - VNet Peering Example
To illustrate, consider the replication example above where 250GB of data is transferred between two VNets — both are situated in the same region, and in the same Availability Zones.
In this instance, charges apply in both directions. The calculation for the initial baseline would be: 2 (Egress and Ingress) x 250 GB x $0.01/GB = $5 ; after that, the daily cost of replicating a subsequent snapshot will be: 2 x 12.5GB x $0.01/GB = $0.25.
| Estimated Upfront (Baseline) Cost | Estimated Monthly Cost |
|---|---|
| 2 (Egress and Ingress) x 250 GB x $0.01/GB = $5 | 2 x 12.5GB x $0.01/GB x 730 hours = $7.60 |
Data Transfer Between Availability Zones - Within the Same Region
Currently, there are no charges for data transmission between Availability Zones within a shared Virtual Network (VNet). However, Microsoft has indicated that charges for this kind of transfer may be introduced in the future.
The data transfer billing between virtual machines across Availability Zones will start in the future. We will announce the launch date three months in advance of billing.
In the case of data transfer between VNet resources residing in peered VNets across distinct Availability Zones, the associated data transfer costs will be in line with the rates prescribed for VNet peering. For more details, please check the Virtual Network Pricing documentation.
| Data Transfer | Price |
|---|---|
|
Cross AZ, same VNet |
$0.00 per GB* |
| Cross AZ, different VNet, using VNet Peering (Egress and Ingress) |
$0.01 per GB |
* Refer back to the Microsoft statement above, the price is only for the current time, and will change in the future.
Cross AZ - VNet Peering Example
To illustrate, consider the replication example above where 250GB of data is transferred between two VNets — both are situated in the same region, but in different Availability Zones.
In this instance, charges apply in both directions. The calculation for the initial baseline would be: 2 (Egress and Ingress) x 250 GB x $0.01/GB = $5 ; after that, the daily cost of replicating a subsequent snapshot will be: 2 x 12.5GB x $0.01/GB = $0.25.
| Estimated Upfront (Baseline) Cost | Estimated Monthly Cost |
|---|---|
|
2 directions [1] x 250 GB x $0.01/GB = $5 [1] Ingress and Egress
|
(30.41 days [1] x 2 directions [2] x 12.5 GB x $0.01/GB) [3] = $7.60 / mo. |
Data Transfer Between Azure Regions
Azure imposes charges for data transmission that occurs beyond the confines of a single Azure region. This encompasses data transfers between VNet peers (also known as Global Peering), and transfers facilitated through a VPN, ExpressRoute Gateway.
Each of the aforementioned scenarios adheres to its own distinct pricing model. Setting up Global VNET Peering is free of charge, however you have to pay for ingress and egress pricing which is based on a zonal structure. Every set of regions geographical grouped and correspond to Zone1 Zone2, and Zone3, which can be found at this documentation.
| Data Transfer | Bandwidth Price | Gateway Hours |
|---|---|---|
|
Cross regions, using VNet Global Peering(Egress and Ingress) |
$0.035 per GB for Zone 1 $0.09 per GB for Zone 2 $0.16 per GB for Zone 3 |
N/A |
| Cross regions, using VPN Gateway (Egress only) |
$0.035 per GB for Zone 1 $0.09 per GB for Zone 2 $0.16 per GB for Zone 3 |
Depends on the GW type. Check the prices and SKUs here. Ex: VpnGw2AZ $0.564/hour for 1 Gbps |
| Cross regions, using ExpressRoute (Depends on the Plan) | Circuits type (Metered / Unlimited). Refer here. |
Depends on the GW type. Check the plans and SKUs here. |
Global Peering Example
To illustrate, consider the replication example above where 250GB of data is transferred between two VNETs using Global Peering —one situated in the Central US and the other in the East US region.
In this instance, charges apply in both directions, and given that both VNETs are within Zone 1. The calculation for the initial baseline would be: 2 (Egress and Ingress) x 250 GB x $0.0350/GB = $17.5 ; after that, the daily cost of replicating a subsequent snapshot will be: 2 x 12.5GB x $0.0350/GB = $0.875.
| Estimated Upfront (Baseline) Cost | Estimated Monthly Cost |
|---|---|
|
2 directions [1] x 250 GB x $0.0350/GB = $17.5 [1] Ingress and Egress for the sender VPN Gateway
|
(30.41 days [1] x 2 directions [2] x 12.5 GB x $0.0350/GB) [3] = $26.60 / mo. |
VPN Gateway Example
This example assumes a cross region, VNet-to-VNet connection configuration, it's similar to creating a Site-to-Site IPsec connection to an on-premises location. Both connection types use a VPN gateway to provide a secure tunnel with IPsec/IKE and function the same way when communicating. However, they differ in the way the local network gateway is configured, adding a bi-directional VPN gateways cost to the calculation.
Let's try to do the same cost analysis, this time using VPN Gateway between the two regions (or to on-prem, or another cloud provider). The 250GB of data is transferred between two VNETs using Global Peering —one situated in the Central US and the other in the East US region.
In this instance, charges apply in one direction plus the gateway hours (assuming VpnGw2AZ) $0.564/hour, and given that both VNETs are within Zone 1. The calculation for the initial baseline would be: 1 (Egress for the sender VPN Gateway) x 250 GB x $0.0350/GB = $8.75; after that, the daily cost of replicating a subsequent snapshot will be: 1 x 12.5GB x $0.0350/GB = $0.4375. Plus $0.564/hour x 730hour = $411.72 Monthly.
| Estimated Baseline Cost | Estimated Monthly Cost |
|---|---|
|
1 direction [1] x 250 GB x $0.0350/GB = $8.75 [1] Egress for the sender VPN Gateway only
|
(30.41 days [1] x 1 direction [2] x 12.5 GB x $0.0350/GB) [3] + ($0.564/hour x 730 hours) [4] = $425.02 / mo. |
Monitoring and Cost Management
Proactive monitoring and cost management are essential to keep Azure bandwidth costs in check and under predefined thresholds. Here's how to effectively manage costs:
Using Azure Cost Management
Utilize Azure Cost Management to monitor, analyze, and optimize spending on Azure resources, including bandwidth costs.
Set up budgets and spending alerts to receive notifications when bandwidth costs approach predefined limits.
Using Azure Monitor
Leverage Azure Monitor to gain insights into the performance and usage of Pure Cloud Block Store instances.
Create custom monitoring dashboards to track data transfer metrics and identify potential cost optimization opportunities.