backup azure storage

I always keep a backup of my azure storage so that in case code deletes something by mistake in azure storage, i have the backup ready to get the file from. Even though azure replicates storage, but it is not fail proof in case of manual deletion and the replica will remove the blob too.

I use AZCOPY to move my data from storage to storage or storage to azure file. This is then run as a 6 hourly job to sync data with storage, giving me enough time to get copy from manual replica in case I delete something.

to download azcopy, go to this link – http://aka.ms/downloadazcopy

then use this powershell script to run AZcopy. (replace with your source and destination storage)

$theSource = @{path=”; accessKey=”; recursion=”; pattern=”}
$theDestination = @{path=”; accessKey=”}

$theSource.path = ‘/Source:https://STORAGENAME.blob.core.windows.net/CONTAINERNAME’
$theSource.AccessKey = ‘/SourceKey:KEY’
$theDestination.path = ‘/Dest:https://STORAGENAME.file.core.windows.net/FILESTORAGENAME’
$theDestination.AccessKey = ‘/DestKey:KEY’

$theSource.recursion = ‘/S /V /XO’
$supressConfirmationPrompt = ‘/Y’
$listingOnlyOption = ” # or /L – use this option if you just want to list.

$arguments = $theSource.path + ” ” + $theDestination.path + ” ” + $theSource.AccessKey + ” ” + $theDestination.AccessKey + ” ” + $theSource.recursion + ” ” + $supressConfirmationPrompt + ” ” + $listingOnlyOption

$pinfo = New-Object System.Diagnostics.ProcessStartInfo
$pinfo.FileName = “C:\AzCopy\AzCopy.exe ”
$pinfo.Arguments = $arguments

$pinfo.RedirectStandardError = $true
$pinfo.RedirectStandardOutput = $true
$pinfo.UseShellExecute = $false

$p = New-Object System.Diagnostics.Process
$p.StartInfo = $pinfo
$p.Start() | Out-Null
$p.WaitForExit()
$stdout = $p.StandardOutput.ReadToEnd()
$stderr = $p.StandardError.ReadToEnd()
Write-Host “stdout: $stdout”
Write-Host “stderr: $stderr”
Write-Host “exit code: ” + $p.ExitCode

if you dont want to use storage key but SAS keys, replace /SourceKey and /DestKey with /SourceSAS and /DestSAS

Then to schedule this job in jenkins, use the following to run it as batch command -Powershell.exe  -NonInteractive -ExecutionPolicy Bypass -File C:\JenkinsJobs\LLProdDatabaseBackups.ps1

Advertisements

Missing Nuget Packages in jenkins

In Jenkins, during build, we encountered missing dlls, lot of them.

the following was checked and seemed ok

  • “Enable Nuget Package Restore” was selected.
  • nuget.exe had been added to source control.
  • the version of the nuget packages was stable

to fix this, you don’t have to check in the packages folder to your repository that jenkins is pulling. Nuget should be able to tell which packages it needs and download any missing ones. Make sure the package file is up to date. The error (below) was a little different but the root issue was the same: missing packages.

The solution ended up being remarkably simple: repeat the steps we perform locally in source control.

  1. In Visual Studio, open the Source Control Explorer.
  2. Navigate to the packages folder (it should be on the same level as the projects in the solution).
  3. Right click and select “Delete”. A small red x should appear next to the name.
  4. Check in the pending change (deleting the packages folder) and wait.

If all goes well, that automatic build should kick off with the new check in and build successfully.

Note: the first build should take longer than usual as the server has to recreate the packages folder and download all packages.

configure MSBuild to use a saved publishProfile for deployment

this can be done from VS 11 and above only.

msbuild.exe MyProject.csproj /p:DeployOnBuild=true;PublishProfile=MyProfile

 

more details on MSBuild command parameters looked as follows:

/P:DeployOnBuild=true
/P:PublishProfile=[Name of My Profile (without a file extension)]
/P:PublishProfileRootFolder=”%teamcity.build.checkoutDir%\Source\Web Application\My Project\PublishProfiles” (where your publish profiles are stored)
/P:AllowUntrustedCertificate=true
/P:Password=%env.DomainPassword%

FTP Update using WinSCP

you need to make the below script and then call it using below syntax

“c:\Program Files (x86)\WinSCP\WinSCP.com” -script=”Script Path”

——————————————————————-SCRIPT————————————————————————————

# Automatically abort script on errors
option batch abort

# Disable overwrite confirmations that conflict with the previous
option confirm off

# Force binary mode transfer
option transfer binary

#if you want to contiue on error. does not abort even the failing command.

option batch continue

 

#open connection to ftp

open ftp://user:password@ftp_address

# Publish local theme up to server, while deleting obsolete files from server.

#you can ignore this if you just to syncronize.

rm /site/wwwroot/file_or_folder_1
rm /site/wwwroot/file_or_folder_2

synchronize remote -delete local_path_where_new_code_is /site/wwwroot    ##/site/wwwroot is the destination path

echo Job is complete.

exit