Let's begin by addressing the issue we aim to resolve. My employer operates a significant number of websites on Azure, primarily utilizing Azure Web Apps. However, the Azure web app backup service has a 10GB limit. While it is possible to use the _backup.filter to exclude certain files, we wish to back up everything.

But we wanted to back up all the things :)

Solving the problem

First thing first I needed a way to get the web app files. I leverage the KUDU API to download the files I needed.

If you want to download large files with the HttpClient, it is important to specify the HttpCompletionOptions, for example var response = await httpClient.SendAsync(httpRequest, HttpCompletionOption.ResponseHeadersRead).

Otherwise, the HttpClient would buffer the entire response in memory. You can then process the response file as a stream via var stream = response.Content.ReadAsStreamAsync().

Otherwise, you'll get a timeout exception (TaskCanceledException).

Or you can use GetStreamAsync() as it has HttpCompletionOptions built in.

   public static async Task<Stream> DownloadZipFile(string TARGETURL)
        {
            var byteArray = Encoding.ASCII.GetBytes(MainInformation.SiteUser + ":" + MainInformation.SitePassword);
            client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", Convert.ToBase64String(byteArray));

            return await client.GetStreamAsync(TARGETURL);
        }

And if you are uploading the file, pass the stream straight to the upload method.