If I choose a zip file and right click "extract here" a folder with the zip filename is created and the entire content of the zip file is extracted into it.
However, I would like to convert several zip files via shell.
But when I do
unzip filename.zip
the folder "filename" is not created but all the files are extracted into the current directory.
I have looked at the parameters but there is no such parameter.
I also tried
for zipfile in \*.zip; do mkdir $zipfile; unzip $zipfile -d $zipfile/; done
but the .zip extension of the 2. $zipfile and 4. $zipfile have to be removed with sed.
If I do
for zipfile in \*.zip; do mkdir sed 's/\.zip//i' $zipfile; unzip $zipfile -d sed 's/\.zip//i' $zipfile/; done
it is not working.
How do I replace the .zip extension of $zipfile properly?
Is there an easier way than a shell script?
unzip file.zip -d xxx will extract files to directory xxx, and xxx will be created if it is not there. You can check the man page for details.
The awk line below should do the job:
ls *.zip|awk -F'.zip' '{print "unzip "$0" -d "$1}'|sh
See the test below,
note that I removed |sh at the end, since my zips are fake archives; I just want to show the generated command line here.
kent$ ls -l
total 0
-rw-r--r-- 1 kent kent 0 Nov 12 23:10 001.zip
-rw-r--r-- 1 kent kent 0 Nov 12 23:10 002.zip
-rw-r--r-- 1 kent kent 0 Nov 12 23:10 003.zip
-rw-r--r-- 1 kent kent 0 Nov 12 23:10 004.zip
-rw-r--r-- 1 kent kent 0 Nov 12 23:10 005.zip
-rw-r--r-- 1 kent kent 0 Nov 12 23:10 006.zip
-rw-r--r-- 1 kent kent 0 Nov 12 23:10 007.zip
kent$ ls *.zip|awk -F'.zip' '{print "unzip "$0" -d "$1}'
unzip 001.zip -d 001
unzip 002.zip -d 002
unzip 003.zip -d 003
unzip 004.zip -d 004
unzip 005.zip -d 005
unzip 006.zip -d 006
unzip 007.zip -d 007
"extract here" is merely a feature of whatever unzip wrapper you are using. unzip will only extract what actually is in the archive. There is probably no simpler way than a shell script. But sed, awk etc. are not needed for this if you have a POSIX-compliant shell:
for f in *.zip; do unzip -d "${f%*.zip}" "$f"; done
(You MUST NOT escape the * or pathname expansion will not take place.) Be aware that if the ZIP archive contains a directory, such as with Eclipse archives (which always contain eclipse/), you would end up with ./eclipse*/eclipse/eclipse.ini in any case. Add echo before unzip for a dry run.
p7zip, the command line version of 7zip does the job
7z x '*.zip' -o'*'
On Debian and Ubuntu, you have to install p7zip-full.
Add the folder name and slash after the switch, example:
unzip -d newfolder/ zipfile.zip
zip will create the folder 'newfolder' and extract the archive into it.
Note that the trailing slash is not required but I guess it's just from old habit (I use Debian and Ubuntu).
aunpack from atool will do this by default for all sorts of archives.
In Termux bash I ended up with this, tested for spaces and dots in filenames. It only removes the last file extension and dot for the folder name. This assumes the zip file has a .zip extension otherwise it won't work. Went nuts trying to \ escape the quotes or use single quotes before realizing they just needed to be there as plain quote marks. Drop an echo in front of the zip command, or add a -l to audit the command. For some reason the position of the -d seems important for this implementation.
for f in *.zip; do unzip "$f" \-d "./${f%.*}/"; done; echo -end-
Open the terminal and locate the 00* files
cat filename.zip.00* > filename.zip
wait until the process ends, it depends on the file size.
The file joining process finished, now you can run the output file
unzip filename.zip
Related
There are 3 txt files called
1.txt 2.txt 3.txt
I want to batch copy with the name
1.txt.cp 2.txt.cp 3.txt.cp
using the wildcard *
I entered the command cp *.txt *.txt.cp
but it wasn't working...
cp : target *.txt.cp : is not a directory
what was the problem???
Use: for i in *.txt; do cp "$i" "$i.cp"; done
Example:
$ ls -l *.txt
-rw-r--r-- 1 halley halley 20 out 27 08:14 1.txt
-rw-r--r-- 1 halley halley 25 out 27 08:14 2.txt
-rw-r--r-- 1 halley halley 33 out 27 08:15 3.txt
$ ls -l *.cp
ls: could not access '*.cp': File or directory does not exist
$ for i in *.txt; do cp "$i" "$i.cp"; done
$ ls -l *.cp
-rw-r--r-- 1 halley halley 20 out 27 08:32 1.txt.cp
-rw-r--r-- 1 halley halley 25 out 27 08:32 2.txt.cp
-rw-r--r-- 1 halley halley 33 out 27 08:32 3.txt.cp
$ for i in *.txt; do diff "$i" "$i.cp"; done
$
If you are used to MS/Windown CMD shell, it is important to note that Unix system handle very differently the wild cards. MS/Windows has kept the MS/DOS rule that said that wild cards were not interpreted but were passed to the command. The command sees the wildcard characters and can handle the second * in the command as noting where the match from the first should go, making copy ab.* cd.* sensible.
In Unix (and derivatives like Linux) the shell is in charge of handling the wildcards and it replaces any word containing one with all the possible matches. The good news is that the command has not to care about that. But the downside is that if the current folder contains ab.txt ab.md5 cd.jpg, a command copy ab.* cd.* will be translated into copy ab.txt ab.md5 cd.jpg which is probably not want you would expect...
The underlying reason is Unix shells are much more versatile than the good old MS/DOS inherited CMD.EXE and do have simple to use for and if compound commands. Just look at #Halley Oliveira's answer for the syntax for your use case.
I have a range of directories from 2010 to 2017 and a sub-directory in them from 1 to 12. There is a file in each sub-directory, I need to add a line to each of these files.
This is part of my script:
#!/bin/bash
mkdir -p test/201{0..7}/{1..12}/
touch test/201{0..7}/{1..12}/file_{0..9}.txt
echo "42" | tee test/201{0..7}/{1..12}/file_{0..9}.txt
Append to files:
echo "42" | tee -a test/201{0..7}/{1..12}/file_{0..9}.txt
In your script, it was created a range of directories from 2010 to 2017 and a range of sub-directories that goes from 1 to 12 in each directory. You also created nine files in each sub-directory at the end.
So, to append a new line in each of these files you should use the echo command to create the contents of your line and redirect it to the tee command to add it to all your files at once, as follows:
echo "new line" | tee -a test/201{0..7}/{1..12}/file_{0..9}.txt
That should be enough.
With sed I try to replace the value 0.1.233... On the command line there is no problem; however, when putting this command in a shell script, I get an error:
sed: couldn't open temporary file ../project/cas-dp-ap/sedwi3jVw: Permission denied
I don't understand where this temporary sedwi file comes from.
Do you have any idea why I have this temporary file and how I can pass it?
$(sed -i "s/$current_version/$version/" $PATHPROJET$CREATE_PACKAGE/Chart.yaml)
++ sed -i s/0.1.233/0.1.234/ ../project/cas-dp-ap/Chart.yaml
sed: couldn't open temporary file ../project/cas-dp-ap/sedwi3jVw: Permission denied
+ printf 'The version has been updated to : 0.1.234 \n\n \n\n'
The version has been updated to : 0.1.234
+ printf '***********************************'
sed -i is "in-place editing". However "in-place" isn't really. What happens is more like:
create a temporary file
run sed on original file and put changes into temporary file
delete original file
rename temporary file as original
For example, if we look at the inode of an edited file we can see that it is changed after sed has run:
$ echo hello > a
$ ln a b
$ ls -lai a b
19005916 -rw-rw-r-- 2 jhnc jhnc 6 Jan 31 12:25 a
19005916 -rw-rw-r-- 2 jhnc jhnc 6 Jan 31 12:25 b
$ sed -i 's/hello/goodbye/' a
$ ls -lai a b
19005942 -rw-rw-r-- 1 jhnc jhnc 8 Jan 31 12:25 a
19005916 -rw-rw-r-- 1 jhnc jhnc 6 Jan 31 12:25 b
$
This means that your script has to be able to create files in the folder where it is doing the "in-place" edit.
The proper syntax is identical on the command line and in a script. If you used $(...) at the prompt then you would have received the same error.
sed -i "s/$current_version/$version/" "$PATHPROJET$CREATE_PACKAGE/Chart.yaml"
(Notice also the quoting around the file name. Probably your private variables should use lower case.)
The syntax
$(command)
takes the output from command and tries to execute it as a command. Usually you would use this construct -- called a command substitution -- to interpolate the output of a command into a string, like
echo "Today is $(date)"
(though date +"Today is %c" is probably a better way to do that particular thing).
There are multiple directories which contain a file with the same name:
direct_afaap/file.txt
direct_fgrdw/file.txt
direct_sardf/file.txt
...
Now I want to extract them to another directory, direct_new and with a different file name such as:
[mylinux~ ]$ ls direct_new/
file_1.txt file_2.txt file_3.txt
How can I do this?
BTW, if I want to put part of the name in original directory into the file name such as:
[mylinux~ ]$ ls direct_new/
file_afaap.txt file_fgrdw.txt file_sardf.txt
What can I do?
This little BaSH script will do it both ways:
#!/bin/sh
#
# counter
i=0
# put your new directory here
# can't be similar to dir_*, otherwise bash will
# expand it too
mkdir newdir
for file in `ls dir_*/*`; do
# gets only the name of the file, without directory
fname=`basename $file`
# gets just the file name, without extension
name=${fname%.*}
# gets just the extention
ext=${fname#*.}
# get the directory name
dir=`dirname $file`
# get the directory suffix
suffix=${dir#*_}
# rename the file using counter
fname_counter="${name}_$((i=$i+1)).$ext"
# rename the file using dir suffic
fname_suffix="${name}_$suffix.$ext"
# copy files using both methods, you pick yours
cp $file "newdir/$fname_counter"
cp $file "newdir/$fname_suffix"
done
And the output:
$ ls -R
cp.sh*
dir_asdf/
dir_ljklj/
dir_qwvas/
newdir/
out
./dir_asdf:
file.txt
./dir_ljklj:
file.txt
./dir_qwvas:
file.txt
./newdir:
file_1.txt
file_2.txt
file_3.txt
file_asdf.txt
file_ljklj.txt
file_qwvas.txt
while read -r line; do
suffix=$(sed 's/^.*_\(.*\)\/.*$/\1/' <<<$line)
newfile=$(sed 's/\.txt/$suffix\.txt/' <<<$line)
cp "$line" "~/direct_new/$newfile"
done <file_list.txt
where file_list is a list of your files.
You can achieve this with Bash parameter expansion:
dest_dir=direct_new
# dir based naming
for file in direct_*/file.txt; do
[[ -f "$file" ]] || continue # skip if not a regular file
dir="${file%/*}" # get the dir name from path
cp "$file" "$dest_dir/file_${dir#*direct_}.txt"
done
# count based naming
counter=0
for file in direct_*/file.txt; do
[[ -f "$file" ]] || continue # skip if not a regular file
cp "$file" "$dest_dir/file_$((++counter)).txt"
done
dir="${file%/*}" removes all characters starting from /, basically, giving us the dirname
${dir#*direct_} removes the direct_ prefix from dirname
((++counter)) uses Bash arithmetic expression to pre-increment the counter
See also:
Why you shouldn't parse the output of ls(1)
Get file directory path from file path
How to use double or single brackets, parentheses, curly braces
It may not be quite what you want, but it will do the job. Use cp --backup=numbered <source_file> <destination_directory:
$ find . -name test.sh
./ansible/test/integration/roles/test_command_shell/files/test.sh
./ansible/test/integration/roles/test_script/files/test.sh
./Documents/CGI/Code/ec-scripts/work/bin/test.sh
./Documents/CGI/Code/ec-scripts/trunk/bin/test.sh
./Test/test.sh
./bin/test.sh
./test.sh
$ mkdir BACKUPS
$ find . -name test.sh -exec cp --backup=numbered {} BACKUPS \;
cp: './BACKUPS/test.sh' and 'BACKUPS/test.sh' are the same file
$ ls -l BACKUPS
total 28
-rwxrwxr-x. 1 jack jack 121 Jun 9 10:29 test.sh
-rwxrwxr-x. 1 jack jack 34 Jun 9 10:29 test.sh.~1~
-rwxrwxr-x. 1 jack jack 34 Jun 9 10:29 test.sh.~2~
-rwxrwxr-x. 1 jack jack 388 Jun 9 10:29 test.sh.~3~
-rwxrwxr-x. 1 jack jack 388 Jun 9 10:29 test.sh.~4~
-rwxrwxr-x. 1 jack jack 20 Jun 9 10:29 test.sh.~5~
-rwxrwxr-x. 1 jack jack 157 Jun 9 10:29 test.sh.~6~
If you really want to put part of the folder name in, you have to decide exactly what part you want. You could, of course, just replace the directory separator character with some other character, and put the whole path into the filename.
I'm trying to create a zip file from file contents which are being piped in, e.g.
mysql [params and query] | zip -q output.zip -
This writes the zip correctly, but when you open the zip, the file within it is called "-". Is there any way of specifying what the filename of the piped in data should be within the zip?
You can do this.
ls | zip test.zip -#
this is done from the notion that i have 3 files in the dir.
-rw-rw-r-- 1 xxx domain users 6 Jan 7 11:41 test1.txt
-rw-rw-r-- 1 xxx domain users 6 Jan 7 11:41 test2.txt
-rw-rw-r-- 1 xxx domain users 6 Jan 7 11:41 test3.txt
and the file itself, the result is then
Archive: test.zip
Length Date Time Name
-------- ---- ---- ----
6 01-07-10 11:41 test1.txt
6 01-07-10 11:41 test2.txt
6 01-07-10 11:41 test3.txt
-------- -------
18 3 files
From the Linux Zip Man page
If the file list is specified as -#,
zip takes the list of input files
from standard input.
You can use a named pipe, and send the request output to it, while zipping from it.
mkfifo output.txt ; mysql [params and query] > output.txt & zip output.zip -FI output.txt ; rm output.txt
I couldn't manage with the PHP answer (out of memory on bigger mysql dumps), and the FIFO was not working as I wanted, so my solution is to rename the file inside the ZIP archive after running the dump, using zipnote (which is included with the zip package on Debian).
mysql [params and query] | zip -q output.zip -
echo -e "# -\n#=newname.sql" | zipnote -w output.zip
From what i can gather you cannot do both with the zip command, i mean you cannot both specify the filenames and pipe the content. You can either pipe the contents and the resulting file is - or you can pipe the filenames with -#.
That does not mean that doing so is impossible using other techniques. I outline one of those below. It does mean that you have to have PHP installed and the zip extension loaded.
There could be a whole bunch of other ways to do it. But this is the easiest that I know of. Oh and it's a one-liner too.
This is a working example using PHP
echo contents | php -r '$z = new ZipArchive();$z->open($argv[1],ZipArchive::CREATE);$z->addFromString($argv[2],file_get_contents("php://stdin"));$z->close();' test.zip testfile
To run on windows just swap single and double quotes. Or just place the script in a file.
"test.zip" is the resulting Zip file, "testfile" is the name of the contents that are being piped into the command.
Result from unzip -l test.zip
Archive: test.zip
Length Date Time Name
--------- ---------- ----- ----
6 01-07-2010 12:56 testfile
--------- -------
6 1 file
And here is a working example using python
echo contents | python -c "import sys
import zipfile
z = zipfile.ZipFile(sys.argv[1],'w')
z.writestr(sys.argv[2],sys.stdin.read())
z.close()
" test5.zip testfile2
Result of unzip -l
Archive: test5.zip
Length Date Time Name
-------- ---- ---- ----
9 01-07-10 13:43 testfile2
-------- -------
9 1 file
Result of unzip -p
contents
mysql [params and query] | python3 -c "import sys as s,os,zipfile as m;z=m.ZipFile(os.fdopen(s.stdout.fileno(),'wb'),'w');z.writestr(s.argv[1],s.stdin.read());z.close()" 'filename.sql' > output.zip
Python 3.5 added support for writing to unseekable streams.
It's not pretty but works.
Just stick with the - as filename, as zip --help suggests:
The default action is to add or replace zipfile entries from list,
which can include the special name - to compress standard input.
but rename it after. You can use 7zip rn for this purpose or any other compression tool.
Here an example reading from curl a plain text, compressing it with best compression and then renaming the minus file after archive is ready.
curl -q 'https://ws-export.wmcloud.org/?lang=en&page=In+Event+of+Moon+Disaster&format=txt&fonts=' | zip -9 moon.zip -
7z rn moon.zip -- - moondisaster.txt
Why double --? To pass - as filename instead of command option.
Result is a zip named moon.zip with a single moondisaster.txt file inside.
You can try:
mysql [params and query] | zip -q output.zip -#