Save-AzureWebsiteLog The maximum message size quota for incoming messages (65536) has been exceeded

[andybatescc]

I'm trying to use the Azure powershell Save-AzureWebsiteLog command to get the logs for an azure site. Get the message below

Save-AzureWebsiteLog -Name <*******> -Output .\logs.zip
Save-AzureWebsiteLog : The maximum message size quota for incoming messages (65536) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding
element.
At line:1 char:1

• Save-AzureWebsiteLog -Name <name removed> -Output .\logs.zip

• - CategoryInfo          : CloseError: (:) [Save-AzureWebsiteLog], CommunicationException
  - FullyQualifiedErrorId : Microsoft.WindowsAzure.Commands.Websites.SaveAzureWebsiteLogCommand
  
  

Is this a known problem, and is there any resolution?

[rickrain]

Would like to see this issue corrected since it's referenced in the Azure documentation.

[NaveenGoli]

I have looked into this, seems like this is happening when the log size exceeds 65536. we will look at a solution for this. in the meantime if possible you can delete the old logs after downloading so the command continues to work if the size of the file is below the threshold..

[parth270592]

I made this script as a workaround:-

$username = "<enter git deployment username>"
$password = "<enter git deployment password>"
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
$apiUrl = "https://<website name>.scm.azurewebsites.net/api/zip/LogFiles/"
$response = Invoke-WebRequest -Uri $apiUrl -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method GET 
try
{
$filename = [System.IO.Path]::GetFileName($response.BaseResponse.ResponseUri.OriginalString)
$filepath = [System.IO.Path]::Combine("c:\asdf\", "http1.zip")
$filestream = [System.IO.File]::Create($filepath)
$response.RawContentStream.WriteTo($filestream)
}
finally
{
$filestream.Close()
}

[devigned]

@NaveenGoli is there an update for this?

[NaveenGoli]

I haven't heard back from the customer on this, I think the limits are there for security reason, I am going to close this issue. Please let me know if the proposed solution doesn't work

[sayedihashimi]

I ran into this problem last night. I created a script which runs over several sites and makes 50 web request and other operations. During development of the script I was partially running the script on those same sites. After getting the script in good shape it ran for several hours and when I woke up in the morning I received the error message above. I had to harden my script and then re-run it :(.

I wanted to find out what the breaking point of this method was so I created a brand new azure website and executed the following PS snippet to determine when the command would fail.

$maxnum = 1000
$url = 'http://sayedreqtest.azurewebsites.net/'

$tempfile = 'C:\temp\logs.zip'
for($num = 0; $num -le $maxnum; $num++){    
    try{
        Invoke-WebRequest -Uri $url
        if(Test-Path $tempfile){
            Remove-Item $tempfile
        }
        Save-AzureWebsiteLog -Name 'sayedreqtest' -Output $tempfile
    }
    catch{
        throw ('$num: [{0}]. Exception: [{1}]' -f $_.Exception)
    }
}

The script failed after just 81 requests!

IMO this limitation is sending users down a _pit of failure_. What I mean is that you work on developing a script and typically use a test site which doesn't receive any traffic. Then your team signs off on it and you run it in production and then you get this error. Considering the fact that I ran into this error after just 81 requests I don't understand how this can be considered useful for anything besides new sites. To add to this the script will work for new sites but then quickly stop working and the team will be left scrambling to figure out what happened.

If we cannot increase the default message size then it would be good to have a parameter where the max message size is passed in.

Another thing to consider, there are other entries going into the log file than just direct requests to your website. For example, browsers will request favicon.ico and if you use the scm site, you'll notice log entries for most actions there.

If you are not able to increase the message size then minimally there should be a warning when invoking the cmdlet so that users are aware of this severe limitation.

The workaround is not suitable because it requires that my script be able to manage secrets. That's what Add-AzureAccount is for. I don't want to be in the business of managing Azure secrets, that's for the Azure PowerShell team.

If you do close this issue I'd like more details on the "security threat" here. From the error message it sounds like a WCF error where the default message size is used. If the user were able to specify the max message size what's the threat?

[andybatescc]

I don't agree with the closure of this. While we don't require this at the moment, it remains a fault with this function and requires a fix.

[SherifTalaat]

"I haven't heard back from the customer on this" is not the response we were expecting. Isn't github the right place to report Azure PowerShell issue? Please Let me know about the channels on which you hear for customer so I can report the problem there.
I am using Azure PowerShell because I've an Azure subscription I want to manage. Isn't it enough to be considered as a customer?!. I am still facing this issue, I've a website and from day zero I cannot use Save-AzureWebsiteLog cmdlet not even after 7-10 days of using the website.

By the way, people use PowerShell because it makes their life easier. And now you are telling them to replace this very simply cmdlet with this script because you THINK that the limit is there for a reason although that you said "we will look at a solution for this".

If there is not plans to fix cmdlet sooner or later, then it's better to remove in the future releases because it's not useful in production use with this message limit.

Regards,
Sherif Talaat
PowerShell MVP

[djkazoo2002]

@parth270592 I am having this error. if you knwo the solution please let me know, Thank you so much for sharing the code.

Invoke-WebRequest : Server Error
401 - Unauthorized: Access is denied due to invalid credentials.
You do not have permission to view this directory or page using the credentials that you supplied.
At line:5 char:13

• $response = Invoke-WebRequest -Uri $apiUrl -Headers @{Authorization=("Basic {0}" ...

• - CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
  - FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
    You cannot call a method on a null-valued expression.
    At line:11 char:1
  

• $response.RawContentStream.WriteTo($filestream)

• - CategoryInfo          : InvalidOperation: (:) [], RuntimeException
  - FullyQualifiedErrorId : InvokeMethodOnNull
  

[markcowl]

@akurmi Can you make sure someone on the websites team looks at this - this appears to be the default WCF message size quota - the quota should definitely be increased for log download.

[cormacpayne]

@twitchax: "Azure PowerShell continues to maintain RDFE support on a legacy basis: new functionality and bug fixes are addressed sparingly. This issue does not meet the current threshold for consideration."