Skip to main content
Contact Us
Register
Pure Technical Services

ActiveDR Guide | Azure VMware Solution and Pure CBS

  1. Last updated
  2. Save as PDF

Currently viewing public documentation. Please login to access the full scope of documentation.

Introduction

Disaster recovery planning is a critical piece of any infrastructure – any location. Azure VMware Solution is a VMware environment runs on Microsoft Azure and can extend its storage capacity from the integration with Pure Cloud Block Store. This integration comes with a set of built-in replication features, with an addition of offloading the storage and replication operation to the storage array-level. 

One of the replication feature with Pure Arrays is ActiveDR. ActiveDR is a near-sync replication that provides an array-based failover and fallback mechanism to protect your important data in a simple and robust way over great distances with extremely low Recovery Point Objectives (RPO). The simplicity of management makes disaster recovery (and importantly, testing your disaster recovery plans) straight-forward and repeatable.

This document describes how to manage ActiveDR in an Azure VMware Solution using Pure Storage Cloud Manager for AVS. 

Scenarios

There are four scenarios to be covered in this guide: 

The high level process for AVS ActiveDR is identical to scenarios between on-premises VMware environment. The difference is that AVS storage management is simplified and limited to using Run Commands via PureStorage.CBS.AVS. 

The scenario and the test environment for this guide was based on:

  • Source: Azure VMware Solution (Version: 7.0.3) and On-Premises VMware environment (Version: 7.0.3) 
  • Target: Azure VMware Solution (Version: 7.0.3).

Configuration

Prior to following the scenarios below, make sure you have satisfied the prerequisites configuration:

  1. Connectivity between both environment
    1. For On-Prem to AVS configured using Site-2-Site VPN, Alternatively Express Route can be used, and connected to AVS using Global Reach. 
    2. For two AVS private cloud in the same region, you can configure AVS Interconnect.
    3. For two AVS private cloud in different regions, you can configure ExpressRoute Global Reach.
  2. Deploy and setup Run Commands for use with AVS and CBS, you can follow this guide to setup the integration.
  3. Initializing AVS and CBS iSCSI Connectivity, you can follow this guide to build and initialize the connectivity.

Test Failover: Promoting and Restoring a Target Pod

ActiveDR was specifically designed to make the test of a recovery (or recovery drill) the same operation as an actual recovery.

The overall process of testing is:

  1. The target pod gets promoted while source remains promoted and online (No disruptive to source workload). Refer to Promoting an ActiveDR Pod with AVS.
  2. The Restore-PCSBVmfsDatastore gets executed by passing the name of the target pod or volume names. Refer to Restoring a Promoted ActiveDR Pod with AVS.
  3. The VMFS datastore(s) gets mounted on the specified AVS cluster. The next step would be registering and powering on the VMs.

Note: In a testing workflow, the source pod stays online and in promotion state. The source pod continues to replicate writes and object changes to the target array--these changes are not reflected in the target pod while it is promoted.

clipboard_effa4429055bd63e1929e3c7d39cca4f9.png

Cleanup Test Failover: Demoting and Removing a Target Pod

Once testing is complete, cleaning up the recovery would be a target only process, source is not impacted. Once the cleanup process is over, any changes to objects in the target pod will be discarded (new volumes, writes, snapshots, etc).

The process of cleanup the test is:

  1. Shutdown VMs running on restored datastores, and unregister them.
  2. Unmount the datastores by executing Remove-PCSBVmfsDatastoresee Removing a Promoted ActiveDR Pod with AVS.
  3. Demote the target pod. See Demoting an ActiveDR Pod with AVS. clipboard_eda8ab3f6acf80a7e7849c63f78688327.png

Failover: Promoting a Target Pod after Source Pod Demotion

If the source pod has been demoted, and you promote the target pod, this process is generally called a "recovery" or a "failover". The purpose here is to bring down the workload on the original “source” site and is being brought up in a new site. The recovery process is:

On the source environment:

  1. Shutdown VMs running on restored datastores, and unregister them.
  2. Unmount the datastore.
    1. If the source is AVS/CBS, execute the remove command. See Removing a Promoted ActiveDR Pod with AVS.
    2. If the targat is VMware on-premises/FlashArray. See Demoting an ActiveDR Pod in a VMware Environment.
  3. Demote the source pod. See Demoting an ActiveDR Pod with AVS.

On the target environment:

  1. Promote the target pod. See Promoting an ActiveDR Pod with AVS.
  2. Mount the datastore.
    1. If the target is AVS/CBS, execute the restore command. See Restoring a Promoted ActiveDR Pod with AVS.
    2. If the targat is VMware on-premises/FlashArray. See Promoting an ActiveDR Pod in a VMware Environment
  3. Register and power on the virtual machines. 

clipboard_e439d18abde84f3221d0f91111991caf7.png

Re-Protect: Re-Promoting a Source Pod after Target Pod Demotion

The last scenario is to re-protect or fallback to the original “source” environment. To archive that, you would need to bring down the target -- demote the target pod, and then bring up the source environment by promoting the pod. 

Note:  Once you demote the target and promote the source, the replication direction will flip, and the source will be ready to brought up and VMs can be registered ed and powered up. 

clipboard_e62ab21cf9e03214ec778616e0548c16c.png

Appendix 

Promote an ActiveDR Pod with AVS

In this environment, the target pod is located on Pure CBS. The promotion can be performed on the CBS GUI or from within the Cloud Manager OVA or any machine has PureStoragePowerShellSDK2 installed on it and can connect and authenticate to CBS. 

► CBS GUI

This section covers how to promote an ActiveDR Pod on Pure Cloud Block Store, using the User Interface (UI). 

  1. Navigate to the CBS UI > Protection > ActiveDR, where the replica link is configured and in replicating status.

    clipboard_e4f5e6b91becf6dcfbbfa26dcd4f7cff0.png 
  2.  Promoting the pod can be done from the Replica link or from Pod view. Click on the vertical ellipsis and choose Promote Local Pod.
    clipboard_ea893220546700719d6f9c70c009b6bc5.png
  3. This will promote the pod using the latest available recovery point, which is 2023-10-04 04:21. Confirm the promotion.
►  PowerShell SDK2 

This section covers how to promote an ActiveDR Pod on Pure Cloud Block Store, using Pure Storage PowerShell SDK2.

  1. From a PowerShell session console, get a list of the pods and replica links.  
    # Get the list of the pods on the CBS array
Get-Pfa2Pod -Array $CBSConnection
  2. Promote the pod using the name of the pod. 
    # Pass the pod name
$TargetPodName = "avs-adr-pod-cbs"
# Promote the pod
Update-Pfa2Pod -Array $CBSConnection -Name $TargetPodName -RequestedPromotionState "promoted"

Restore a Promoted ActiveDR Pod with AVS

Executing the Restore run command can be done via Cloud Manager OVA or any machine has PureStoragePowerShellSDK2 installed on it and can connect and authenticate to CBS. 

►  Restore with -PodName flag

Running this command with the -PodName flag will enumerate over the list of volumes within a promoted pod and cause a volume or volumes to be mounted to AVS cluster. 

Sample Command with minimum required arguments:

Restore-PCBSVmfsDatastore -ClusterName "myCluster" -PodName "avs-adr-pod-cbs" -AVSCloudName "myAVSCloudName" -AVSResourceGroup "myAVSResourceGroup"

Additional Optional Arguments:

  -DatastoreName "ActiveDRvMFSName"
  -PureCloudBlockStoreConnection $CBSConnection

If you have multiple CBS connections in your PowerShell session, you may pass this variable to select the CBS connection you want to use.

  -AzureSubscriptionID "myAzureSubscriptionID"

If left out, this command will use the default/active Azure Subscription ID.  If you wish to specify a specific Azure subscription, use this parameter.

►  Restore with -VolumeName flag

Running this command with the -VolumeNameflag will cause a member volume of the promoted pod to be mounted to the AVS cluster.

Restore-PCBSVmfsDatastore -ClusterName "myCluster" -VolumeName "avs-adr-pod-cbs::AVS-ActiveDR-Test-AM" -AVSCloudName "myAVSCloudName" -AVSResourceGroup "myAVSResourceGroup"

Additional Optional Arguments:

  -DatastoreName "ActiveDRvMFSName"
  -PureCloudBlockStoreConnection $CBSConnection

If you have multiple CBS connections in your PowerShell session, you may pass this variable to select the CBS connection you want to use.

  -AzureSubscriptionID "myAzureSubscriptionID"

If left out, this command will use the default/active Azure Subscription ID.  If you wish to specify a specific Azure subscription, use this parameter.

Example output: 

PS /home/azureuser/AVS-RunCommand> Restore-PCBSVmfsDatastore -ClusterName $cluster2 -VolumeName "CBS-avs-adr-pod::AVS-ActiveDR-Test-AM" -AVSCloudName $avsPrivateCloudName  -AVSResourceGroup $avsResourceGroupName
Creating datastore from volume...                                                                                       
Connecting volume CBS-avs-adr-pod::AVS-ActiveDR-Test-AM to host group Cluster-2...                                      
Rescanning HBA...                                                                                                       
Using Azure default subscription: xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx                                                  
Invoking RunCommand /subscriptions/xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx/resourceGroups/fa-soln-vmware-uswest2/providers/Microsoft.AVS/privateClouds/avs-cbs/scriptPackages/Microsoft.AVS.VMFS@1.0.77-dev/scriptCmdlets/Sync-VMHostStorage ...
Restoring VMFS Datastore  from volume CBS-avs-adr-pod::AVS-ActiveDR-Test-AM...                                          
Using Azure default subscription: xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx                                                
Invoking RunCommand /subscriptions/xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx/resourceGroups/fa-soln-vmware-uswest2/providers/Microsoft.AVS/privateClouds/avs-cbs/scriptPackages/Microsoft.AVS.VMFS@1.0.77-dev/scriptCmdlets/Restore-VmfsVolume ...
Datastore 'snap-68b3cfbe-AVS-ActiveDR-AM' is successfully restored    

Name                               FreeSpaceGB      CapacityGB
----                               -----------      ----------
snap-68b3cfbe-AVS-ActiveDR-AM          994.128       1,023.750

vCenter UI view:

clipboard_e3997c76b8660bfe02f441f90ba019ef9.png

Remove a Restored ActiveDR Pod with AVS

Before demoting a target pod (part of the test cleanup scenarios), or demoting a source pod mounted to AVS cluster, you have to remove the datastore. Therefore, the process to prepare a VMware environment involves shutting down virtual machines running off of the ActiveDR-protected storage and informing the ESXi hosts that this storage will go away. 

Refer to Demoting an ActiveDR Pod in a VMware Environment, for more in-depth information. 

Executing the Remove run command can be done via Cloud Manager OVA or any machine has PureStoragePowerShellSDK2 installed on it and can connect and authenticate to CBS. 

The Remove-PCBSVmfsDatastore command will perform two actions: 

  1. Dismounts and removes an existing CBS VMFS datastore from AVS.
  2. Removes the volume from the CBS host group and deletes the volume in CBS.
Remove-PCBSVmfsDatastore  -ClusterName "mycluster" -DatastoreName "myDatastore -AVSCloudName "myAVSCloudName" -AVSResourceGroup "myAVSResourceGroup"

Demote an ActiveDR Pod with AVS

Demoting an ActiveDR pod with AVS can be part of two scenarios: 

  • Cleanup test recovery: demoting a target pod.
  • Recovery or failover: demoting a source pod. 

Both scenarios require preparing the VMware environment involves shutting down and unregistering virtual machines, then removing the datastores. For removing steps, check the previous section.

Another requirement prior to demotion is to verify if there is still active I/O going to the pod. If so, examine the workloads and stop them. If they should/cannot be stopped, move the workload to a volume not in the target pod. 

The simplest way to ensure the pod is not actively serving I/O is to check the performance statistics.

Navigate to Analysis -> Performance -> Pods, then select the pod.
clipboard_ea1145335af2cb080ba936af59efdeae6.png

The promotion can be performed on the CBS UI or from within the Cloud Manager OVA or any machine has PureStoragePowerShellSDK2 installed on it and can connect and authenticate to CBS. 

► CBS GUI

This section covers how to demote an ActiveDR Pod on Pure Cloud Block Store, using the User Interface (UI). 

  1. Navigate to the CBS UI > Protection > ActiveDR, where the replica link is configured and in replicating status.
    clipboard_e268843f44a99abf40576fa58c460e8c1.png
     
  2.  Demoting the pod can be done from the Replica link or from Pod view. Click on the vertical ellipsis and choose Demoted Local Pod.
    clipboard_e76a43b332a60ade874ddfd671c85f023.png
  3. If it is determined that after the pod is demoted that some data from the point-in-time immediately prior to demotion is needed, the contents of the pod at that time is protected temporarily in an "undo" pod. This undo pod is kept in the "destroyed pods" area for a period designated by the eradication timer which defaults to 24 hours, but can be increased upon request to Pure Storage support.
    clipboard_ef6de8c3195a65c55f7e20b45619ebda0.png

    This undo pod can be copied to a new pod so the contents can be used/manipulated.

►  PowerShell SDK2 

This section covers how to promote an ActiveDR Pod on Pure Cloud Block Store, using Pure Storage PowerShell SDK2.

  1. From a PowerShell session console, get a list of the pods and replica links.  
    # Get the list of promoted pods on the CBS array
Get-Pfa2Pod -Array $CBSConnection | Where-Object { $_.PromotionStatus -eq "promoted" }
  2. Promote the pod using the name of the pod. 
    # Pass the pod name
$TargetPodName = "avs-adr-pod-cbs"
# Promote the pod
Update-Pfa2Pod -Array $CBSConnection -Name $TargetPodName -RequestedPromotionState "demoted"

Common Issues and Fixes 

Restore Command Failed 

Symptom:

Executing the Restore-PCBSVmfsDatastore returns:

'Failed to re-signature VMFS volume.'

Applies to: 

When restoring VMFS datastore located in ActiveDR pods using -PodName "podName" or -VolumeName "podname::volumename".

Cause: 

The pod was not promoted.

Resolution: 

For restoring a Pod or Pod::Volume, you have to promote it first. Once pod is promoted it will be available for I/O, and the Restore command could re-signature it.