Providing the business with a disaster recovery capability in Microsoft Azure is a great way for organisations to start their Cloud journey.  For many, this starts by backing up on-premises databases to Azure blob storage.  In this article, I'll explain how you might (unknowingly) be spending more than you think on this hybrid disaster recovery method, along with possible solutions to mitigate that extra spend.

Scenario

We recently came across a scenario whereby on-premises database backups were being written to Blob Storage via the BACKUP TO URL feature in SQL Server.  As per recommended practice, the backups were being verified via the RESTORE VERIFYONLY command.  Whilst this doesn't restore the backup, it ensures the file is readable and free of additional errors.

Problem

Within two weeks of backups being written to Blog Storage, we observed a significant upward trend in cost associated with a Storage Account.  When compared to the previous month, there was an increase of c. £270.  After some further analysis, we were able to associate this increase with "bandwidth" charges.  This didn't feel right - you don't pay anything to upload data to Azure (ingress), you only pay when downloading data from Azure (egress).

Using Azure Monitor, we profiled the ingress and egress rates for the affected Storage Account and noticed the following pattern:

Each day, c. 150GB of backups were being written to blob storage (in blue), but shortly after, the same amount was being downloaded (in red).  Over this period, we calculated 4TB of data had been uploaded and then downloaded again.  Based on Microsoft's latest Bandwidth pricing, whilst the first 5GB of egress per month is free, the next 5GB - 10TB is charged at £0.065 per month.  Some simple maths confirms it to be the additional £270 we observed.

Why was there data egress?  Well, verifying database backups requires the backup to be read on the machine from where the RESTORE VERIFYONLY command is being run.  When that virtual machine is on-premises and the backup is on blob storage, that means you pay for the egress!  In this scenario, with some additional growth over a full month, verifying the backups could cost upwards of £1,000 per month!  Depending on the value you place on verifying backups, that may be acceptable, but for others it may be unacceptable.  Let's explore some ways of working around this.

Solutions

There are several alternative solutions that could be used to reduce this spend:

• Turn off backup verification: By turning off verification the backup doesn't need to be read locally, resulting in no egress charge.  However, the integrity of the backups are not known until you need to recover from a disaster, by which point it's too late!

• Verify the backup on-premises, then upload to Azure: Verifying your backups on-premises will result in no egress charge, but an alternative method for uploading the backups to blob storage will be required.  This could be achieved via a PowerShell script that backs up and verifies the database locally, then uploads it to Azure.

• Turn off backup verification, upload and verify in Azure: This option uses the BACKUP TO URL feature but without verification.  Instead, a relatively cheap virtual machine could be provisioned in Azure to restore the backups from blob storage using the RESTORE VERIFYONLY command.  However, this option only makes sense when the cost of the virtual machine is less than the egress cost.  Time also needs investing in setting up the necessary infrastructure, which may not be justifiable.

Verifying backups is an important part of your disaster recovery solution.  If you're verifying backups and operating in a hybrid model, there may be a cost you didn’t anticipate.  If you feel the cost justifies the means, there's no need to do anything!  However, if you're looking for new ways to save on your Cloud spend, this might offer a good saving.