Link

High Available Backup Repository using Windows Storage Replica

Build the environment

The following steps will guide you through the configuration of volumes with Windows Storage Spaces Direct (referred as S2D), the creation of a replication group and the integration with Veeam Backup & Replication.

Install prerequisites

Windows Storage Replica is a feature that must be installed as it is not part of the standard installation of Windows Server 2016. What follows must be done on both nodes:

  • Open the server manager and click “Add Roles and Features”

server manager

  • In “Features” menu select Storage Replica

storage replica

  • Complete the wizard and reboot both the servers

confirm

Prepare log volumes

What follows must be done on both nodes:

  • Bring the disk online, initialize it as GPT disk and create a volume

prepare log disk create volume

  • Format it as ReFS (no 64KB required here)

format volume

Configure S2D

Tips: the entities that will be created are three on each node

  • Storage Pool
  • Virtual Disk
  • Volume

Use names that can easily identify the entity and it’s location (what node). Here are the names that will be used:

Object Node WSRSOURCE01 Node WSRTARGET01
Storage Pool sp_wsrsource01 sp_wsrtarget01
Virtua Disk vdisk_repo_src00 vdisk_repo_tgt00
Volume vol_repo_src00 vol_repo_tgt00

What follows must be done on both nodes:

  • Create the Storage Pool

create storage pool

create storage pool

  • Specify name and description

name and description

  • select disks

select disks

  • confirm to complete the process

confirm

Create virtual disk

What follows must be done on both nodes:

  • Create a new virtual disk

sp

  • Name it and specify enclosure awareness

vdisk name

  • Select a storage layout based on the protection level desired (Simple, Mirror, Parity)
  • Select thin provisioning

vdisk provisioning type

  • Set a size and complete the wizard leaving the option to create the volume checked

vdisk created

Create volumes

What follows must be done on both nodes:

  • Go through the wizard, set a size, letter and format it with ReFS (use 64 KB cluster size)

volume fs

Integration with Veeam Backup & Replication

Before setting up the replication partnership, it is necessary to configure both the repositories (source and destination) in Veeam Backup & Replication as you would with any standard ReFS repository.

vbr configuration

Once the replication partnership is set, the destination volume will remain not accessible thus VBR will not let you configure it.

Set on both the repositories to use per-VM backup files:

per-vm chain

Create the replication partnership

From now on, if not different noted, all the commands must be issued on the source server.

  • The first step is to test if the partnership can be created. The cmdlet will test for 10 minutes (it can be changed) both log and data disk performances and it calculates latency between source and destination.

As result an html report will be generated under C:\tmp. Check it out for any warning before moving forward.

$sourcesrv="wsrsource01.democenter.int"
$sourcevolume="r:"
$sourcelogvolume="e:"
$destsrv="wsrtarget01.democenter.int"
$destvolume="r:"
$destlogvolume="e:"


Test-SRTopology -SourceComputerName $sourcesrv -SourceVolumeName $sourcevolume -SourceLogVolumeName $sourcelogvolume -DestinationComputerName $destsrv -DestinationVolumeName $destvolume -DestinationLogVolumeName $destlogvolume -DurationInMinutes 10 -ResultPath c:\tmp
  • Create the partnership and set log size
$sourcesrv="wsrsource01.democenter.int"
$sourcevolume="r:"
$sourcelogvolume="e:"
$destsrv="wsrtarget01.democenter.int"
$destvolume="r:"
$destlogvolume="e:"


New-SRPartnership -SourceComputerName $sourcesrv -SourceRGName replica-rg01-source -SourceVolumeName $sourcevolume -SourceLogVolumeName $sourcelogvolume -DestinationComputerName $destsrv -DestinationRGName replica-rg01-dest -DestinationVolumeName $destvolume -DestinationLogVolumeName $destlogvolume -ReplicationMode Asynchronous
Set-SRGroup -LogSizeInBytes 97710505984 -Name replica-rg01-source
  • Set log size on destination node (use either Enter-PSSession -ComputerName xxxx or RDP)
Set-SRGroup -LogSizeInBytes 97710505984 -Name replica-rg01-source
  • Set the restriction on the interface that must be used for replication traffic (optional)
#Use Get-NetIPConfiguration to find interface names

Get-SRPartnership | Set-SRNetworkConstraint -SourceNWInterface 7 -DestinationNWInterface 3

Update-SmbMultichannelConnection

Monitoring

There are many methods to keep track of replication progress and errors. However, it is not possible to use just one tool and, in some cases, commands must be executed on the target or on the source node.

Replica status and remaining GBs

To get replication status use (on source server):

PS C:\Windows\system32> (Get-SRGroup).Replicas

CurrentLsn          : 70366
DataVolume          : R:\
LastInSyncTime      :
LastKnownPrimaryLsn : 70366
LastOutOfSyncTime   :
NumOfBytesRecovered : 1410928
NumOfBytesRemaining : 0
PartitionId         : f1b8ba33-c631-43f6-88b6-494a2c5d9f12
PartitionSize       : 805170053120
ReplicationMode     : Asynchronous
ReplicationStatus   : ContinuouslyReplicating_InRPO
PSComputerName      :

This command gives also useful information about the replication groups: it tells the replication mode (synchronous or asynchronous) and either if the replica is aligned or not.

NumOfBytesRemaining here will always be 0.

To get remaining data to be replicated use the same command on the destination server, or use this script to constantly update the Powershell window with the remaining amount.

Change the -Name “xxxxx” accordingly. To get SRGroup names use Get-SRGroup.

#must be used on destination

do{
    $r=(Get-SRGroup -Name "replica-rg01-dest").replicas
    [System.Console]::Write("Number of remaining GB {0}`n", [math]::Round($r.NumOfBytesRemaining/1GB,2))
    Start-Sleep 2
}until($r.NumOfBytesRemaining -eq 0)
Write-Output "Replica Status: synced"

Replication events

Windows Storage Replica events can be easily retrieved with Powershell. Run:

Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | Select-Object -First 20

Relevant events

When monitoring replication events, the most useful ones that can give you a complete outlook on the replication status can be found on the target node.

Here is some example:

events

  • Event 5010 is triggered every time the replication state return within the RPO, that means all the modified blocks on the source have been copied and the replication is completely aligned
  • Events 1200, 1203, 1201, 1237 are triggered when new blocks are written on the source and the replication process starts

Regarding events on the source, one thing worth noticing is an error that should be considered as warning instead:

event1239

Events ID 1239 is triggered when new data is written on the source and, at that moment in time, the replication is not longer in RPO. As soon as the data has been copied to the destination, event 1240 is logged (usually along with event 5010 on the target server).

Windows Performance Monitor

Performance monitor can be used to build charts, with different metrics, on replication statistics. Run “perfmon.msc”, click on the “+” (plus) sign to build a new dashboard, in the counter list select counters from these two containers.

perfmon

Failover

In case of failure of the source node, replication direction can be easily switched allowing VBR to continue backup chains (no need to create new full) on the target repository. Use the following command on the target server to switch the direction:

$sourcesrv="wsrsource01"
$sourcerg="replica-rg01-source"
$destsrv="wsrtarget01"
$destrg="replica-rg01-dest"

Set-SRPartnership -NewSourceComputerName $destsrv -SourceRGName $destrg -DestinationComputerName $sourcesrv -DestinationRGName $sourcerg

As soon as the command has been executed, the volume that contains replicated data is mounted in writeable state and from now on all new written data will be replicated on the former source server (that is now the new target). If the source server is offline, the replica state will not be in RPO, but it will be “Waiting for the destination”. That is expected as the destination is offline. Use, on the target server, Get-SRPartnership and Get-SRGroup to check the state:

inversed replica

Note how source and destination are changed.

Edit job configuration in VBR

As soon as the data is accessible on the destination server, job in VBR can be pointed to secondary repository:

vbr job

Jobs will continue as usual (in this case the next run was an incremental):

job run

All the job metadata has successfully been updated:

metadata update

Failback

Same destination

To failback to the original state, just switch the replica repeating all the above to inverse target with source:

$sourcesrv="wsrsource01"
$sourcerg="replica-rg01-source"
$destsrv="wsrtarget01"
$destrg="replica-rg01-dest"

Set-SRPartnership -NewSourceComputerName $sourcesrv -SourceRGName $sourcerg -DestinationComputerName $destsrv -DestinationRGName $destrg

This scenario is intended to be used when the source server is returned online, and the replication baseline still exists.

A consistency check will be performed and only needed blocks will be transferred.

New destination

In case of a disaster the original location could not be available anymore. It is possible to recreate the replication partnership with a new server, that will become the new source but in this case the replication partnership must be first destroyed.

Use the following commands on the target server to break the partnership:

Get-SRPartnership | Remove-SRPartnership
Get-SRGroup | Remove-SRGroup

Remove the old primary repository from VBR configuration, set up the new one pointing the new server and create the partnership using the secondary server as source. As soon as the replica is synced, inverse it and configure VBR accordingly.

Upgrade to Windows Server 2019

The in-place upgrade process to Windows Server 2019 will be supported and will introduce a new feature that gives the ability to mount a snapshot of the destination volume while replication keeps happening.

Thinking about this in Veeam’s world, it would open different scenario where data at destination can be used as source for DataLabs as well for SureBackup jobs without impact on the primary infrastructure or even use data at destination as source for replication jobs using backup files.

A short demo of this capability can be found on this Microsoft blog.

Upgrade Procedure

As in-place upgrade is supported, it’s possible to upgrade both the servers without impact on the backup process. Lets assume that Storage Replica happens between node A and node B, where A is the source. The recommended process is:

  • Upgrade B
  • After upgrade get A and B in sync
  • Switch direction making B as primary and A as secondary
  • Upgrade A
  • After upgrade get B and A in sync
  • Switch direction to get back to the normal state (A to B)

Mount destination volume

Windows Server 2019 allows target disks to be mounted. A snapshot of the data will be created and saved into a temporary position. Depending on the size of the expected changes (while the snapshot is open) more free space might be required.

Use the following command to mount the disk:

Mount-SRDestination -Name replica-rg01-desk -Computername wsrtarget01 -TemporaryPath C:\wsrmount

The target volume will be mounted with its own letter and it will be writeable. Any changes made to the filesystem will be saved on the temporary path given with the command and they will be discarded as soon as the snapshot is removed with the following command:

Dismount-SRDestination -Name replica-rg01-dest -Computername wsrtarget01

References