I'm trying to zip all the files within a directory which contains .py files individually. But after zipping the files the output that I'm seeing is .py.zip vs just .zip
Here's the one liner command that I'm trying to execute.
cd scripts/python/
for i in *; do zip $i.zip $i; done
This is what you are looking for:
for i in *py; do
zip "${i%.*}".zip "$i";
done
Explanation
${i%.*}: This makes use of Bash's built in parameter expansion. Here it tries to match everything after %. If it does find a match, it uses everything before the match. https://www.gnu.org/software/bash/manual/bash.html#Shell-Parameter-Expansion for more information.
Related
Here, (on linux)
there is an existing archive named A.zip, which include File1 and File2:
A.zip:
File1
File2
and I run this command: zip A.zip File1 File3, then the archive A.zip becomes like:
A.zip:
File1
File2
File3
however, what I really want to get is a brand new archive A.zip! like:
A.zip:
File1
File3
I know it can be done by run rm A.zip and then run zip A.zip File1 File3, but it is not elegant and if I write these commands into a shell script so A.zip may not exist while the action to remove a non-existent file is not elegant as well.
Is there any options for me to get this done?
Use these option to works:
zip -FSr A.zip File1 File3
OPTIONS
-FS
Synchronize the contents of an archive with the files on the OS. Normally when an archive is updated, new files are added and changed files are updated but files that no longer exist on the
OS are not deleted from the archive. This option enables a new mode that checks entries in the archive against the file system. If the file time and file size of the entry matches that of
the OS file, the entry is copied from the old archive instead of being read from the file system and compressed. If the OS file has changed, the entry is read and compressed as usual. If
the entry in the archive does not match a file on the OS, the entry is deleted. Enabling this option should create archives that are the same as new archives, but since existing entries are
copied instead of compressed, updating an existing archive with -FS can be much faster than creating a new archive. Also consider using -u for updating an archive.
For this option to work, the archive should be updated from the same directory it was created in so the relative paths match. If few files are being copied from the old archive, it may be
faster to create a new archive instead.
Note that the timezone environment variable TZ should be set according to the local timezone in order for this option to work correctly. A change in timezone since the original archive was
created could result in no times matching and recompression of all files.
This option deletes files from the archive. If you need to preserve the original archive, make a copy of the archive first or use the --out option to output the updated archive to a new
file. Even though it may be slower, creating a new archive with a new archive name is safer, avoids mismatches between archive and OS paths, and is preferred.
-r
Travel the directory structure recursively; for example:
zip -r foo.zip foo
or more concisely
zip -r foo foo
In this case, all the files and directories in foo are saved in a zip archive named foo.zip, including files with names starting with ".", since the recursion does not use the shell's file-
name substitution mechanism. If you wish to include only a specific subset of the files in directory foo and its subdirectories, use the -i option to specify the pattern of files to be in‐
cluded. You should not use -r with the name ".*", since that matches ".." which will attempt to zip up the parent directory (probably not what was intended).
Multiple source directories are allowed as in
zip -r foo foo1 foo2
which first zips up foo1 and then foo2, going down each directory.
Note that while wildcards to -r are typically resolved while recursing down directories in the file system, any -R, -x, and -i wildcards are applied to internal archive pathnames once the di‐
rectories are scanned. To have wildcards apply to files in subdirectories when recursing on Unix and similar systems where the shell does wildcard substitution, either escape all wildcards
or put all arguments with wildcards in quotes. This lets zip see the wildcards and match files in subdirectories using them as it recurses.
I find myself in a situation similar to this question:
Linux: Overwrite all files in folder with specified data?
The answers there work nicely, however, they are for typed-out text. Allow me to provide context.
I have a Linux terminal which the following file structure: (with files & folders irrelevant to the question removed)
root/
empty.svg
svg/
257238.svg
297522.svg
a7yf872.svg
236y27fh.svg
38277.svg
... (~200 other .svg files with arbitrary names)
2903852.svg
The framework I am working with requires those .svg files to exist with those specific filenames, but obviously, it does not care about SVG image they contain. I do not plan on using such files and they take up a hefty amount of space on disk, so I wish to convert them all into empty SVGs, aka the empty.svg file on my root directory, which is a 12x12 transparent SVG file (124 bytes). This way the framework shouldn't error out like it did when I tried simply overwriting the raw data of those SVGs with plaintext using the answer of the question linked at the top of this question. I've tried many methods by trying to be creative with my basic Linux command-line knowledge but no success. How do I accomplish this?
TL;DR: How to recursively overwrite all files in a folder with the raw data of another file from Linux CLI?
Similar to the link, you can use tee command, but instead of echo use cat to copy file contents, where cat is the command to read the contents of the file.
cat empty.svg | tee svg/257238.svg svg/297522.svg <etc>
But if there are a lot of files in svg directory it will be useful to use loop to automate the previous command:
for f in svg/*; do
if [[ "$f" == *.svg ]]; then
cat empty.svg > "$f"
fi
done
Here we use pipes and redirections to connect commands and redirect previous command output.
There are multiple files in /opt/dir/ABC/ named allfile_123-abc allfile_123-def allfile_123-ghi allfile_123-xxx.
I need the files to be named new_name-abc.pgp new_name-def.pgp new_name-ghi.pgp new_name-xxx.pgp and then moved to /usr/tst/output
for file in /opt/dir/ABC/allfile_123* ;
do mv $file /usr/tst/output/"$file.pgp";
rename allfile_123 new_name /usr/tst/output/*.pgp ; done
I know the above doesn't work because $file = /opt/dir/ABC/allfile_123*. Is it possible to make this work, or is it a different command instead of 'for loop'?
This is for the Autosys application in which the jil contains a command to pass to the command line of a linux server running bash.
I could only find versions of each part of my question but not altogether and I was hoping to keep it on the command line of this jil. Unless a script is absolutely necessary.
No need for the loop, you can do this with just rename and mv:
rename -v 's/$/.pgp/' /opt/dir/ABC/allfile_123*
rename -v s/allfile_123/new_name/ /opt/dir/ABC/allfile_123*
mv /opt/dir/ABC/new_name* /usr/tst/output/
But I'm not sure the rename you are using is the same as mine.
However,
since the replacement you want to perform is fairly simple,
it's easy to do in pure Bash:
for file in /opt/dir/ABC/allfile_123*; do
newname=new_name${file##*allfile_123}.gpg
mv "$file" /usr/tst/output/"$newname"
done
If you want to write it on a single line:
for file in /opt/dir/ABC/allfile_123*; do newname=new_name${file##*allfile_123}.gpg; mv "$file" /usr/tst/output/"$newname"; done
The requirement is to make copies of n(n>10000) files in the same linux directory.
The extension of the files has to be intact and can probably add numbers to distinguish among files.
For e.g. if one file is text1.txt the other could be text2.txt
But I have to create multiple copies from multiple files and not from a single file.
Please help.
Bash pattern substitution might help you here. If you e.g. want to copy all .txt files, you can do it like this:
for file in *.txt # add any other name wildcards
do
filename=${file%.*} # removes everything after the last dot
extension=${file##*.} # removes everything before the last dot
cp "$file" "${filename}-copy.${extension}" # adds the -copy suffix to every copy
done
You might want to look into tools like logrotate which can take for example a glob and rotate each of the files on a regular basis.
I am looking to write a script to move files from a directory:
/home/mydir/
To another directory based on tokens in the file name. I have a bunch of files named as such:
red_office_mike_2015_montreal_546968.ext
or
$color_$location_$name_$year_$city_$numbers.extension (files will be various movie files: mov, mp4, mkv, etc.)
I would like the script to move the files to the following location:
/dir/work/$color/$name
Then verify the file has successfully copied, and delete the original file once it has.
I would also love it if the script would create the to directory if it does not already exist.
So in summary, I need a script to move files based on underscore separated tokens, create the to directory if it doesn't already exist, verify the successful copy (maybe with a size check), then delete the original file.
I am working on linux, and would prefer a bash script. The variables I have given are generic, and I will incorporate some other things to the script, I'm just looking for help on building the skeleton.
Thanks in advance for any help!
It's not a bash script, but perl is much better at this kind of thing and is installed on all Linux systems
while(<>) {
chomp;
$file = $_;
($colour, $location, $name, $year, $city, $numbers) = split(/_/,$file);
$dest0 = "/dir/work/$colour";
$dest1 = "$dest0/$name";
mkdir ($dest0) unless (-d $dest0);
mkdir ($dest1) unless (-d $dest1);
rename ($file, "$dest1/$file");
}
The script splits your input file on the underscore character, creates all the directories to the destination and then renames the file to the new filename. Rename takes care of all the copying and deleting for you. In fact it just changes the directory entries without any copying at all.
UPDATE
The above version takes its input from a file containing a list of filenames to process. For an alternative version which processes all files in the current directory, replace the while line with
while(glob("*")) {
I was able to fumble around online and come up with a for loop to do this task. I used cut and it made things simple. Here is what worked for me:
#!/bin/sh
cd "${1:-.}"
for f in *.*; do
color=`echo "$f" | cut -d'_' -f1`
name=`echo "$f" | cut -d'_' -f3`
todir="/dir/work/$color/$name"
mkdir -p "$todir"
mv "$f" "$todir"
done
This worked perfectly and I hope it can help others who might need to create directories based on portions of filenames.
The first line under the shebang made it so that it will either look at the current working directory or a directory you pass it as an argument.
Thanks to those who chimed in on the original post. I'm new with scripting so it take me a while to figure this stuff out. I love this site though, it is super helpful!