Deploying Flask app using VSCode App Service fails. The zip file is 2.2 gb - too large.
What procedure can I use to deploy large projects?
Can I upload the project in 2 or more steps ?
https://learn.microsoft.com/en-us/azure/app-service/deploy-zip
Created new web app "mysite": https://mysite.azurewebsites.net
mysite: Starting deployment...
mysite: Creating zip package...
mysite: Zip package size: 2.22 GB
mysite: Fetching changes.
mysite: Cleaning up temp folders from previous zip deployments and extracting pushed zip file /tmp/zipdeploy/b02a0e95-4e86-4bcc-a71f-9997355a55fb.zip (54.56 MB) to /tmp/zipdeploy/extracted
mysite: Offset to Central Directory cannot be held in an Int64.
mysite: Deployment failed.
As I mentioned in your post here "The zip file limit of 2048MB (2GB) can't be increased in App service. This is by the design and is not a limitation of Kudu but of IIS since the request goes through the Frontends. When the integrated pipeline is used, we go through webengine code (webengine4!MgdGetRequestBasics) which doesn't support more than 2 GB content-length."
Related
I have a Node.js Express project hosted via the serverless framework on AWS.
Prior to this I had no problems deploying my code, but after changing a couple of typo's and trying to redeploy I get the error "Resource handler returned message: "Unzipped size must be smaller than 262144000 bytes".
Note: I am excluding node_modules
When I inspect my project folder via finder, I can see that my project is 280.2 Mb (!!) - This is after I delete my node_modules folder in the project.
When I lookup the current uploaded website, I can see that the size of the project is 3.78 Mb.
When I inspect all the files in the in the project via the finder app, I cannot find any large files (images, lottie-files etc).
In other words: nothing stands out.
Have anyone experiences something similar and could have an explanation of this?
I read a bunch of websites (included stack overflow posts and Microsoft website) about the file size limitation of 2GB
Microsoft Docs: Compress-Archive
"The Compress-Archive cmdlet uses the Microsoft .NET API
System.IO.Compression.ZipArchive to compress files. The maximum file
size is 2 GB because there's a limitation of the underlying API."
I have a PowerShell script to backup my Azure DevOPS projects using the Azure DevOPS RestAPI 5.0.
I download all my projects directly in zip format with the RestAPI then i use Compress-Archive to consolidate all the zipped projects into 1 "big" zip file
My total zipped projects files all together is equal to 5.19GB
After compressing to 1 big zip file with Compress-Archive, i got a zip file size of 5.14GB
I don't have any issue to uncompress it and I don't get any error although the 2GB limitation on the documentation.
I wonder if it's because i'm using Windows Server 2016 (so 64bits) so I don't have the 2GB file size limitation?
Anyone can confirm that? As the Microsoft documentation doesn't specify anything about it.
Indeed, they stipulate that the issue is due to the API limitation of System.IO.Compression
Microsoft Docs: system.io.compression.ziparchive
As my zip will continue to grow, i would like to be sure that the zip won't be corrupted due to a file size limitation.
I can indeed use a 3rd-party library like Zip64, 7Zip, ... but i would like to only use the built-in compress method from PS/.Net
The 2GB limitation is for single Files inside the zip.
For example, if you try to Expand a .zip with an 2.1GB file inside it, it will trow an error.
I have a backend Nodejs application and I am fetching and streaming files in the background when a certain event happens in the client.
I have deployed the backend to Google App Engine.
The file downloading is working fine but I am a bit confused where the files are downloaded and stored ? In the app I am creating a folder relative to the deployed app folder and storing them there with createWriteStream. I also init a git repository where the files are (using simple-git npm module)
It seems the files are not accessible via the cloud shell since I can not find them there
Can I for example create a storage bucket and use "normal" file operations command there (and init the repo there)
-Jani
To store data downloaded you want to store it in Cloud Storage, you can find a complete guide in this Using Cloud Storage documentation.
Under almost any circumstances you want to download files into the App Engine Deployment since the instances doesn't have much memory to store data, and also when the deployment scales up and down you are prone to lost data
I'm deploying files to an Azure Web App via Octopus Deploy, and want to clean out the Azure Web App directories before deploying new versions. This way I can be sure I'm deploying each app version onto a clean slate. I don't want to entirely delete and re-create the app, because there are some app settings that need to carry over from previous deployments.
Kudu documentation lists the web app file structure here (all under D:\home), but I'm wondering if there's any possibility of other files outside of the D:\home directory that could affect app performance.
I tried running get-childItem D:\ -recursive in the kudu powershell console before and after deployment to compare results and found 268 new files (not counting those in wwwroot) after deployment, all within these directories:
D:\Windows\Temp
D:\Windows\Logs
D:\Windows\security\logs
D:\Users\\AppData\Roaming\Microsoft\SystemCertificates
D:\home\LogFiles
D:\home\Microsoft\Windows\PowerShell
D:\home\data\aspnet\CompilationSnapshots
D:\local\VirtualDirectory0\LogFiles
D:\local\VirtualDirectory0\data
D:\local\VirtualDirectory0\site\wwwroot
D:\local\VirtualDirectory0\Microsoft\Windows\PowerShell
D:\local\Config\
D:\local\Temporary ASP.NET Files\msdeploy
So which files do I need to clear or reset in order to ensure that new versions of the web app run as intended? Is it sufficient to clear out the wwwroot directory?
The only writable folders are d:\home and d:\local. But d:\local is temporary, and gets wiped clean on app restart. So effectively, you should only be concerned about d:\home when it comes to deployment.
Within that, wwwroot is typically the most important, though if you set up virtual directories and applications, you can end up with other folders as part of your app.
See also https://github.com/projectkudu/kudu/wiki/Understanding-the-Azure-App-Service-file-system which has related info.
I have a standard azure webrole that requires some 3rd party software to be installed on the cloud service. The webrole itself is very basic. The 3rd party software has a few prerequisites, so everything is being included in the role's content (a total of 5 MSI files) and is installed via a an elevated startup task.
The software is installing successfully and everything's working, but including these MSIs as content for the webrole results in an 80MB *.cspkg file.
Excluding the installers yields a package size of 10MB. The total size of all the 5 MSI files is 20.5MB. I don't understand why including 20MB of installers results in an 80MB cloud service package.
I think the packaging process might be attempting to compress the files (unintentionally increasing the payload). Is what I'm seeing normal? Or is there any way of reducing the resulting package when installers are included?
The package for WebRoles is usually bigger then one for worker roles because your role content is included twice! Once is approot folder and once in sites\0\ folder.
As a second thought of yours - compressing. This is true. Azure package is just a regular zip file. You can safely rename yourpackage.cspkg to yourpackage.zip and browse the content.
However your strategy for keeping installation files in package is not effective in its roots. What I will suggest is that you put your 3rd party installer in a blob storage. Then first download it and then install it. For downloading the content you can use various techniques. From PowerShell to Azure Bootstrapper and probably many more.
Having your (all and any of them) 3rd party installers in a Blob storage is a recommended best practice for designing applications for Windows Azure.
As to the question why your Web Application is copied twice in the Azure Deployment package - it has its roots in the times where Windows Azure Web Role was using IIS Hostable Web Core and not full IIS. And is probably still there for backward compatibility.
I found if you add the msi to the \bin folder of the Azure project it doesn't copy multiple times, in this style:
https://learn.microsoft.com/en-us/azure/cloud-services/cloud-services-dotnet-install-dotnet
Using Visual Studio 2017, Azure SDK 2.9, Build Engine version 14.0.27530.0