I need to download a single folder from a github repository in a Python 3 script.
Listing all raw file URLs to download is tedious and eventual new files would need to be added manually
Downloading the whole repository as zip takes rather long; there are lots of unneeded files. [This how I do at the moment]
I have read about web services that do what I would need, such as downgit.github.io, but the problem is that generating the relevant URL and fetching it via urllib.request.urlretrive() downloads the website rather than the actual file.
What can I do? Is there a web service that provides raw file links that I can download as described above?
I just found a solution: The ffspec library may be used to download a single folder from a GitHub repository using Python. See https://sebastianwallkoetter.wordpress.com/2022/01/30/copy-github-folders-using-python/
Related
I followed the guidance in this wiki page to create an application package for my Azure Batch pool, but now my nodes are stuck in an unusable state because it fails to unzip. I can't find anything in the documentation that talks about what kind of compressed file is acceptable here, other than "a zip file".
I have a collection of database files used for some genomic sequencing tools that I have stored in a folder structure, which I created a compressed archive with using tar -zvcf and gave a .zip extension to. That did not work, so I tried uploading the same file with a .tar.gz extension and it also failed.
The Batch Node is running the CentOS image Azure Batch recommends for container applications, and my startup task is not running in the context of the container.
Can anyone point me to documentation or personal experience that helps clarify what kind of files can be used for this? Thank you in advance!
Yes, you are correct, but let me emphasise on the confusion, tar is the different compress archive file format then zip i.e. more detail here: What is the difference between tar and zip? it is mentioned many times in the documentation you mentioned along with
Batch App Package feature only support *.zip format and hence changing file extension from *.tar to *.zip is not the right way as they are 2 different way they get compressed et. al.
Extra docs:
https://azure.microsoft.com/en-au/blog/application-packages-and-task-dependencies-now-available-on-azure-batch/
https://kb.winzip.com/help/winzip/AboutZIPsAndOtherArchives_4.htm
Thanks and hope it helps.
Is there any programmatic way to access and download the already-stored video recordings from MS Teams? I have been reading MS Graph API documentation, but could not find anything related to this.
There is no specific API or chrome extension for this but it is possible using destreamer. Prerequisite: Node 8.0 or higher
Steps for downloading the video:
Download zip files of these repositories Destreamer Youtube-dl
Now in a new folder copy all the contents of these repositories. The folder structure is like Folder Structure. Do not worry about files with same names as they are not important. So when windows asks you to decide what to do just skip it.
Download ffmpeg as described in this link ffmpeg download. Also add it to the environment variables as described in this link.
Download chromium from this link chromium and just paste the folder present in zip file anywhere.
In the folder described in step 2 open src/destreamer.ts and edit code as code edit on line 55 in the pic and replace the path with your own chromium path.
Open cmd and navigate to the main folder that we created in step 1 and run npm install. After successful installation of node_modules run npm run build. After successful build run
destreamer.cmd -i "https://web.microsoftstream.com/video/VIDEO-1". (You need double quotations for link). This will launch chromium and ask you to login. Enter your MS account credentials. On successful login your download will start.
Note: In powershell you need to write ./destreamer.cmd instead of destreamer.cmd. In linux you need to write ./destreamer.sh.
You can also refer to destreamer github repository described in step 1 for other download options such as custom path download, re-encode, different format, etc.
Video recordings in Teams are actually stored in Microsoft Stream (another one of the services in Office 365) but unfortunately there's currently no API for accessing Stream itself (it's on the roadmap for release later this calendar year, see here (page 2 at the moment).
I couldn't open download link from provided FFMPEG instruction, found release for windows here:! instead. The rests have no issues.
Decided to it add like an answer, because it can be a temporary issue from my side.
Please consider provided link as an alternative/additional.
I am trying to transfer files using the GitHub API by curl. The files may have any extension (eg: .java, .jar, .jpg, etc). I have found no solutions in the net. Is there a way I can do this? Or is it even possible?
What do you mean by transfer file?
there is an option of uploading a blob files through the github api /repos/git/blobs.
you should add a parameter of the content of the file in base64 format.
if you want to use them later on you can create new tree with the returned SHA.
base64 online converter: https://www.base64decode.org/
I'm trying to place some config files under version control using Github. The ONLY way to get these particular config files is via a GET request to the hosting server. Server responds with a .zip file.
.zip file structure looks similar to:
|-Folder1
--|-File1.json
--|-File2.json
--|-Pic1.jpg
|-Folder2
--|-File3.json
--|Folder3
----|Pic2.jpg
|-File4.json
|-File5.json
As you can see, its a mix of text and binary files. Also has a nested folder structure.
I need to:
Commit this .zip file to Github using the Github REST API (no problem there)
But BEFORE committing, how can I programmatically extract the folders and files (LEAVING THE FOLDER STRUCTURE IN PLACE)
Commit the results to Github
I've checked the question/solutions mentioned here, but they don't quite match/are a few years old.
Anyone have a workflow for doing this? I have to imagine SOMEONE has come across similar requirements.
I ended up using Power Automate to handle this.
Basically,
GET .zip file
Add it to OneDrive for Business
Extract it there (folder structure is "flattened", but luckily OneDrive "keeps" it by renaming each file)
Example: Folder1/File1.json in the .zip becomes Folder1_File1.json once extracted
When committing to Github via api, I just did a replace for the filename; INCLUDING the / as part of the path.
Example URL: https://api.github.com/repos/SeaDude/replace(items('FOR_EACH_app')?['properties']?['DisplayName'], ' ', '-')/contents/replace(items('FOR_EACH_source_file_2')?['Name'],'_','/')
Apparently, you can specify an empty (or non-empty) path as part of an API commit. If the directory is empty, Github will create it!
I was thinking to use dropbox to upload my source code of a web-application. For this folder i would create a shared link. This link i like to use to download all the latest source files on my test server (instead of using s/FTP).
Now i know you can use dropbox with linux by installing their version, but it requires to create account. I don't want to use a account, and for sure don't want to use my own account.
Is there anyway to use a shared (folder) link, and download all the files in that folder command-line, without a account (maybe something like wget) ? There is no need for live-syncing, it would be fine to trigger the download with some bash script.
Thanks.
If you're ok with your links being public (which i think is not a good idea) , then you can just create a file with a list of links to your files and then create a bash script to loop over each line of the file get the link with wget
If you want to use authentication, you'll have to register for a Dropbox API key and then create a script (in python,ruby or java etc) to authenticate and get the files.
If you don't have a specific need for dropbox, i'll recommend you use git (or similar). With git you'll just have to create the repository on your server and clone it on your desktop. Then you can just edit your files and push it to the server.... it's so much easier.
Rogier, github has become the norm for hosting code. There are other options (Sourceforge, Google Code, Beanstalk) or you can set up a private git repository on your own computer.
Somewhere deep in my browser history there's an article about how to do that.
However a little googling turned up http://news.ycombinator.com/item?id=1652414. Let me know if you can't find some satisfactory instructions on your own of how to set up a git repo on your computer.