Failure to unzip application packages in Azure Batch - azure

I followed the guidance in this wiki page to create an application package for my Azure Batch pool, but now my nodes are stuck in an unusable state because it fails to unzip. I can't find anything in the documentation that talks about what kind of compressed file is acceptable here, other than "a zip file".
I have a collection of database files used for some genomic sequencing tools that I have stored in a folder structure, which I created a compressed archive with using tar -zvcf and gave a .zip extension to. That did not work, so I tried uploading the same file with a .tar.gz extension and it also failed.
The Batch Node is running the CentOS image Azure Batch recommends for container applications, and my startup task is not running in the context of the container.
Can anyone point me to documentation or personal experience that helps clarify what kind of files can be used for this? Thank you in advance!

Yes, you are correct, but let me emphasise on the confusion, tar is the different compress archive file format then zip i.e. more detail here: What is the difference between tar and zip? it is mentioned many times in the documentation you mentioned along with
Batch App Package feature only support *.zip format and hence changing file extension from *.tar to *.zip is not the right way as they are 2 different way they get compressed et. al.
Extra docs:
https://azure.microsoft.com/en-au/blog/application-packages-and-task-dependencies-now-available-on-azure-batch/
https://kb.winzip.com/help/winzip/AboutZIPsAndOtherArchives_4.htm
Thanks and hope it helps.

Related

What is the best way to retrieve a single folder from a github repository in a python script?

I need to download a single folder from a github repository in a Python 3 script.
Listing all raw file URLs to download is tedious and eventual new files would need to be added manually
Downloading the whole repository as zip takes rather long; there are lots of unneeded files. [This how I do at the moment]
I have read about web services that do what I would need, such as downgit.github.io, but the problem is that generating the relevant URL and fetching it via urllib.request.urlretrive() downloads the website rather than the actual file.
What can I do? Is there a web service that provides raw file links that I can download as described above?
I just found a solution: The ffspec library may be used to download a single folder from a GitHub repository using Python. See https://sebastianwallkoetter.wordpress.com/2022/01/30/copy-github-folders-using-python/

Transfer zip file to web server

I would like to develop an app that targets everything from Gingerbread(version 2.3 API 9) to JellyBean(version 4.3 API 18).
The problem:
I need to transfer large images(40 to 50 at a time) either independently or in a zip file without the user having to click on each file being transferred. As far as I can tell I need to use the HttpClient(org.apache) that was deprecated after JellyBean.
Right now the application takes the images and zips them to a zip file prior to uploading. I can create additional zip files, for example if I have 50MB to transfer I can make each zip file about 10MB and have 5 files to be transferred if I have to. I need to transfer these files to a web server. I cant seem to find anything about transferring files after Jellybean. All the searching I've done uses the deprecated commands and the posts are 2-5 years old. I have installed andftp and transferred a 16MB zip file last night that was created by my app, but I really don't want to use that as it will require additional steps from the user. I will try andftp today and setup an intent to transfer the files to see how that works out. Supposedly andftp works until Lollipop(5.0). If there is an easier way please let me know, hopefully I've missed something about transferring files. Is there another way to do this after JellyBean?

SmartDL for ftp

I need a python code which can download files from a ftp server. I need a built in multi-part download managing package which can help me to retrieve files faster. I tried SmartDL but the problem is I don't know how to retrieve files in a ftp server. Also I used the add_basic_authentication to ensure that, I am passing the right credentials. Please help me with a solution.
I have no problem using any other solution/package which uses Multipart download.
P.S:- I need to save the Downloaded files on to an Object storage on Cloud. The size of each file may be 300MB and I need to download 20TB of data.
Thanks in anticipation.
Take a look at ftplib, it's a simple FTP library which will permit you to download files from a FTP server.

About updating a node-webkit app

I want to set auto-updates up for my apps before I release. I'm a budding programmer, so when I looked into node-webkit-updater I was pretty confused. It seems under-documented to me. Can someone explain the overall update mechanism that it helps implement?
As an alternative to node-webkit-updater, I was thinking of creating my own update system. I kinda like how Apple handles extension updates and I was thinking about replicating it. This would involve putting a JSON/XML manifest file on Amazon S3 along with the latest versions of the app for all platforms. The app checks the file at startup and replaces itself with the new version.
Is the latter sound plausible? Am I better off going with node-webkit-updater? If so, can someone explain it to me please? My app is a Mac + Windows project.
This is what we did:
The first script of the page checks a custom "manifest" (.txt file) on the server, which contains some arbitrary text, e.g. version number.
If this value differs from a local version of the manifest, then download a .zip file from server. (The zip contains the latest nwjs website. You could have a separate one for each platform).
Unzip into a local directory (we use 7za command line util).
Set window.location.href to above local directory (index.html).
I know this is a old question, but here is the answer :)
https://www.npmjs.org/package/node-webkit-updater

Creating a CAB file on WinCE

I want to make a CAB file on a WinCE 5/6 device.
The idea is that someone would run a backup script that creates the CAB file, which can then be loaded back onto the device at any point restoring settings to a previously known point.
The settings are xml and custom files in various folders on the device, I am not talking specifically about registry settings.
If this cannot be done is there (commercially) free ZIP file creation software that could be used?
Thankyou in advance
Chris
DotNetZip is a free zip library that will let you do this.
See also:
zip and unzip files and folders on Mobile Device
Is doing this from the desktop side any good?
You can use this registry tool using ActiveSync. The tool has a nice feature of Reg->CAB. Once you want to restore settings, you can clean the registry using OALIoCtlHalGetHiveCleanFlag and then run the CAB to restore the settings.
I know this does not include the requirement:
run a backup script
But it is a quick solution that might give you an idea.

Resources