I am trying to deploy different groups of files (of many different types) to different environments using Bitbucket Pipelines and AWS CodeDeploy. In Pipelines, I use the zip command (from apt-get) to package up all the files for the specific environment and upload them to CodeDeploy (using the CodeDeploy pipe, which seems to expect a zip file).
A recent change has made it unwieldy to use a single line command to feed the all the necessary files to zip, so I instead would like to use a file list. The problem is that I have several sub-folders where I need to recursively grab all files, while in others I need to grab specific files. I also need to preserve the folder structure.
I also don't want to have to add every single path by hand if possible, as there are a lot of files in these sub-directories, and I also want to reduce the amount of cases where we forget to add new files to these lists. I also don't want to require the developer running a script locally, but I am okay with creating a script for use in Pipelines.
I tried to use the -R# option with zip, but it gives the error zip error: Invalid command arguments (nothing to select from).
Example folder structure:
file1.txt
ziplist.txt
folder1/file2.js
folder1/file3.txt
folder1/folder2/file4.png
folder1/folder2/file5.jpg
folder1/folder3/file6.tsx
folder1/folder3/file7.mp3
The contents of ziplist.txt:
file1.txt
folder1/file2.js
folder1/folder2/file5.jpg
folder1/folder3/*
Using the command cat ziplist.txt | zip -R# application.zip, I'd expect to have a zip with the following files inside:
file1.txt
folder1/file2.js
folder1/folder2/file5.jpg
folder1/folder3/file6.tsx
folder1/folder3/file7.mp3
Appreciate any help.
I was able to make it work by removing the /* for the folder path in the zip list, and then use xargs to convert the list to command line parameters, like so:
cat ziplist.txt | xargs zip -r application.zip
Related
I find myself in a situation similar to this question:
Linux: Overwrite all files in folder with specified data?
The answers there work nicely, however, they are for typed-out text. Allow me to provide context.
I have a Linux terminal which the following file structure: (with files & folders irrelevant to the question removed)
root/
empty.svg
svg/
257238.svg
297522.svg
a7yf872.svg
236y27fh.svg
38277.svg
... (~200 other .svg files with arbitrary names)
2903852.svg
The framework I am working with requires those .svg files to exist with those specific filenames, but obviously, it does not care about SVG image they contain. I do not plan on using such files and they take up a hefty amount of space on disk, so I wish to convert them all into empty SVGs, aka the empty.svg file on my root directory, which is a 12x12 transparent SVG file (124 bytes). This way the framework shouldn't error out like it did when I tried simply overwriting the raw data of those SVGs with plaintext using the answer of the question linked at the top of this question. I've tried many methods by trying to be creative with my basic Linux command-line knowledge but no success. How do I accomplish this?
TL;DR: How to recursively overwrite all files in a folder with the raw data of another file from Linux CLI?
Similar to the link, you can use tee command, but instead of echo use cat to copy file contents, where cat is the command to read the contents of the file.
cat empty.svg | tee svg/257238.svg svg/297522.svg <etc>
But if there are a lot of files in svg directory it will be useful to use loop to automate the previous command:
for f in svg/*; do
if [[ "$f" == *.svg ]]; then
cat empty.svg > "$f"
fi
done
Here we use pipes and redirections to connect commands and redirect previous command output.
I'm trying to zip all the files within a directory which contains .py files individually. But after zipping the files the output that I'm seeing is .py.zip vs just .zip
Here's the one liner command that I'm trying to execute.
cd scripts/python/
for i in *; do zip $i.zip $i; done
This is what you are looking for:
for i in *py; do
zip "${i%.*}".zip "$i";
done
Explanation
${i%.*}: This makes use of Bash's built in parameter expansion. Here it tries to match everything after %. If it does find a match, it uses everything before the match. https://www.gnu.org/software/bash/manual/bash.html#Shell-Parameter-Expansion for more information.
I currently have about 10 zip files I would like to extract into one folder. Each zip file contains around 1000 images. As a result, lots of the names of the images are duplicated. For example, in the first zip file, we have things like Img.jpg, Img(1).jpg, Img(2).jpg. I know that to extract multiple zip files into a single folder, I would do something like:
unzip '*.zip'
However, when it tries to put a file from the first zip file that has the same name as a file in the second zip file, it starts to ask:
replace duplicatefile.mp4? [y]es, [n]o, [A]ll, [N]one, [r]ename:
At this point, what do I do if I want to keep ALL files, including the duplicates, and possibly have them named to image(1).jpg instead?
In short, is there a way to call the unzip command on all the zip files, have them extracted into a single folder, without losing any files due to same names?
Thanks.
Invoke unzip --help for details.
But it appears unzip *.zip -n should do the trick?
(Make sure it does what you expect before going ahead!)
I used command zip in linux (RedHat), this is my command:
zip -r /home/username/folder/compress/zip.zip /home/username/folder/compressed/*
Then, i open file zip.zip, i see architecture as path folder compress.
I want to in folder zip only consist list file *.txt
Because i used this command in script crontab hence i can't use command cd to path folder before run command zip
Please help me
I skimmed the zip man page and this is what I have found. There is not an option archive files relative to a different directory. The closest I have found is zip -j which removes the entire path and stores the files directly in the zip rather than sub directories. I do not know what happens in the case of file name conflicts such as if /home/username/folder/compressed/a.txt and /home/username/folder/compressed/subdir/a.txt both exist. If this is not a problem for you, you can use this option, but I am concerned because you did specify the -r option indicating that you expect zip to traverse sub folders.
I also thought of the possibility that your script could somehow call zip with a different working directory, but I took a look at this unix stack exchange page and it looks like their options use cd.
I have to admit I do not understand why you cannot use cd and I am very curious about it. You said something about using crontab, but I have never heard of anything wrong with changing directories in a crontab script.
I used option -j in command zip
zip -jr /home/username/folder/compress/zip.zip /home/username/folder/compressed/*
and i was yet settled this problem, thanks
In my workplace, there's one Perl script that runs on a Unix machine every time someone tries to check-in a file to the SVN repo for any of the 10-20 projects.
The way it works is that each project has its own "Hooks" folder with a file called "pre-commit" which SVN automatically executes when someone check-in something. Except the "pre-commit" file is actually a symbolic link to the one central Perl script common to all projects just so that if a change needs to be made to the Perl script it doesn't need to be done for every project.
So my problem is this: I need to put a text file in each of these projects' "hooks" directory, each one containing some settings specific to that project. So there will be 10-20 settings files (one per project) each in their respective "hooks" directory.
The problem is that I need to open these text files in the Perl script and read from them but I'm having issues letting Perl know where to find it. I tried using the $0 parameter which is supposed to tell me where the script is being executed from but because it's a symbolic link it just says "Not a directory" and the script terminates. I need to get the path of the "hooks" directory so that I can find the text file.
The SVN pre-commit script is supposed to be invoked with the path to the repository as its first argument. Inside a Perl script, that argument should be available as $ARGV[0]. You should be able to build the path to the corresponding hooks directory or to a file inside that directory by simply appending to the repository path, like this:
$repopath = $ARGV[0];
$hookspath = $repopath . "/hooks";
$myfilepath = $hookspath . "/myfile";
although for maximum portability it would be cleaner to use the pathname-manipulation functions in the File::Spec module to do this.
If this approach doesn't work then you'll have to explain more about how your Perl script gets invoked. For instance, if your pre-commit script is really a shell script wrapper that eventually invokes perl then perhaps it's not passing the pre-commit arguments along properly.
Showing us your current code that's failing would be a good thing too.