backup azure storage

I always keep a backup of my azure storage so that in case code deletes something by mistake in azure storage, i have the backup ready to get the file from. Even though azure replicates storage, but it is not fail proof in case of manual deletion and the replica will remove the blob too.

I use AZCOPY to move my data from storage to storage or storage to azure file. This is then run as a 6 hourly job to sync data with storage, giving me enough time to get copy from manual replica in case I delete something.

to download azcopy, go to this link – http://aka.ms/downloadazcopy

then use this powershell script to run AZcopy. (replace with your source and destination storage)

$theSource = @{path=”; accessKey=”; recursion=”; pattern=”}
$theDestination = @{path=”; accessKey=”}

$theSource.path = ‘/Source:https://STORAGENAME.blob.core.windows.net/CONTAINERNAME’
$theSource.AccessKey = ‘/SourceKey:KEY’
$theDestination.path = ‘/Dest:https://STORAGENAME.file.core.windows.net/FILESTORAGENAME’
$theDestination.AccessKey = ‘/DestKey:KEY’

$theSource.recursion = ‘/S /V /XO’
$supressConfirmationPrompt = ‘/Y’
$listingOnlyOption = ” # or /L – use this option if you just want to list.

$arguments = $theSource.path + ” ” + $theDestination.path + ” ” + $theSource.AccessKey + ” ” + $theDestination.AccessKey + ” ” + $theSource.recursion + ” ” + $supressConfirmationPrompt + ” ” + $listingOnlyOption

$pinfo = New-Object System.Diagnostics.ProcessStartInfo
$pinfo.FileName = “C:\AzCopy\AzCopy.exe ”
$pinfo.Arguments = $arguments

$pinfo.RedirectStandardError = $true
$pinfo.RedirectStandardOutput = $true
$pinfo.UseShellExecute = $false

$p = New-Object System.Diagnostics.Process
$p.StartInfo = $pinfo
$p.Start() | Out-Null
$p.WaitForExit()
$stdout = $p.StandardOutput.ReadToEnd()
$stderr = $p.StandardError.ReadToEnd()
Write-Host “stdout: $stdout”
Write-Host “stderr: $stderr”
Write-Host “exit code: ” + $p.ExitCode

if you dont want to use storage key but SAS keys, replace /SourceKey and /DestKey with /SourceSAS and /DestSAS

Then to schedule this job in jenkins, use the following to run it as batch command -Powershell.exe  -NonInteractive -ExecutionPolicy Bypass -File C:\JenkinsJobs\LLProdDatabaseBackups.ps1

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s