I´m using zip to create a backup.
In the directory I´m processing, there are some files not intended to be included. My issue is that the filenames are named in this style:
abc-1 (excluded)
abc-2 (excluded)
abc-3.ini (included)
I don´t know how to specify the -x option in zip command line so that the first two files that have no extension are left out, and the third one is included.
I´ve tried
zip -r mybackup.zip mydir -x mydir/abc-*.
but it´s not working.
Thanks!
Well, this worked:
zip -r mybackup.zip mydir -x mydir/abc-!(*.*)
...but only from the command line. When trying to include the command in a bash script, it didn´t work and I had to include the following line before:
shopt -s extglob
which enables the extended globbing.
Related
in a script this
(cd "$amdir/archive" && zip -rm "$amdir/archive/a.zip" "$amdir/archive/*")
causes zip warning: name not matched
when I echo that and copy paste it to a command line it works fine
any idea why that doesn't work in a bash script on linux ?
You are already in $amdir/archive after your cd.
So your zip tries to find another directory $amdir/archive, when already being there.
I can reproduce the error message, when I try to do a zip -rm on a nonexisting directory.
In addition you should consider the remark from Inian: by quoting the * you escape it and therefore have no bash pattern matching - see for example here
So the second part should simply read:
zip -rm a.zip *
I'm new to Bash scripting. My script intended role is to access a provided path and then apply some software (RTG - Real time Genomics) commands on the data provided in that path. However, when i try to execute the bash from CLI, it gives me following error
ERROR:There were invalid input file paths
The path I have provided in the script is accurate. That is, In the original directory, where the program 'RTG' resides, I have made folders accordingly like /data/reads/NA19240 and placed both *_1.fastq and *_2.fastq files inside NA19240.
Here is the script:
#!/bin/bash
for left_fastq in /data/reads/NA19240/*_1.fastq; do
right_fastq=${left_fastq/_1.fastq/_2.fastq}
lane_id=$(basename ${left_fastq/_1.fastq})
rtg format -f fastq -q sanger -o ${lane_id} -l ${left_fastq} -r ${right_fastq} --sam-rg "#RG\tID:${lane_id}\tSM:NA19240\tPL:ILLUMINA"
done
I have tried many workarounds but still not being able to bypass this error. I will be really grateful if you guys can help me fixing this problem. Thanks
After adding set -aux in bash script for debugging purpose, I'm getting following output now
adnan#adnan-VirtualBox[Linux] ./format.sh
+ for left_fastq in '/data/reads/NA19240/*_1.fastq'
+ right_fastq='/data/reads/NA19240/*_2.fastq'
++ basename '/data/reads/NA19240/*'
+ lane_id='*'
+ ./rtg format -f fastq -q sanger -o '*' -l '/data/reads/NA19240/*_1.fastq' -r '/data/reads/NA19240/*_2.fastq' --sam-rg '#RG\tID:*\tSM:NA19240\tPL:ILLUMINA'
Error: File not found: "/data/reads/NA19240/*_1.fastq"
Error: File not found: "/data/reads/NA19240/*_2.fastq"
Error: There were 2 invalid input file paths
You need to set the nullglob option in the script, like so:
shopt -s nullglob
By default, non-matching globs are expanded to themselves. The output you got by setting set -aux indicates that the file glob /data/reads/NA19240/*_1.fastq is getting interpreted literally. The only way this would happen is if there were no files found, and nullglob was disabled.
In the original directory, where the program 'RTG' resides, I have
made folders accordingly like /data/reads/NA19240 and placed both
*_1.fastq and *_2.fastq files inside NA19240.
So you say, your data folders are in the original directory (whatever that may be), but in the script you wrongly specify them to be in the root directory (by the leading /).
Since you start the script in the original directory, just drop the leading / and use a relative path:
for left_fastq in data/reads/NA19240/*_1.fastq
I want to copy a file from a directory using shell script
Suppose I save the directory and file name seperately as
dir=/home/user/directory/
file=file_1
to copy the file Im using this command in my script
cp $dir$file .
But I get this error
/bin/cp omitting directory '/home/user/directory'
I have tried all combination eg. omitted the trail backslah from variable dir, etc but nothings working. I cant understand what is wrong with this code. Pleas help
Maybe the command $dir$file is not getting unpacked in the shell (ie only the directory variable is getting unpacked, not the file variable)!!!!!
It looks like you are having problem with expansion in cp $dir$file . In order to prevent possible problems, it is better to protect your variable with braces and double quote the full path/file to make sure you don't get caught by spaces in either the filename or heaven forbid the user's dirname:
cp "${dir}${file}" .
This will prevent the possibility the second $ is missed. Also make sure you have read access to other users /home (if you are root or using sudo you should be fine)
If you see this, when you somehow assign an empty string to file somewhere. Search your script for file= and unset file.
You can also debug this by adding
echo ".${file}."
in the line before the cp command. I'm pretty sure it prints .., i.e. the variable is empty or doesn't exist.
Right now to create an extension with Google Chrome Extensions page we select a directory that contains created extension and it generates .crx file.
The problem is that it contains all files from this directory - for example, all docs, asset drafts, etc.
Is it possible to create some blacklist to ignore specified files like *.psd, *.pdf, docs/* ... ?
The Chromium team decided not to implement a manifest (or similar mechanism) for including only the desired files in a .CRX.
The recommended workflow is to have a build step that outputs only the needed files in a dist directory, and to create the CRX from that directory. This is common practice for JavaScript libraries.
My solution
I have created a .crxignore custom ignore file, like this:
.*
Makefile
*.md
bin
It is made for zip command! It is different than .gitignore. You can't add comments eg! See documentation: https://linux.die.net/man/1/zip , look for --exclude word.
Now you can create a zip without ignred files:
$ zip -qr -9 -X .out/out.zip . -x#.crxignore
# ^^^^^^^^^^^^^ using ignore file
After that I can convert zip to crx file with this go script: https://github.com/mmadfox/go-crx3 You have to build it with go build -o out/crx3 crx3/main.go command (you may get missing mod errors, but there will be the commands what you have to run). --> you can move your out/crx3 to where you want. Eg: [project]/bin/crx3.
I haven't tried, maybe chrome command also can convert zip to crx.
You have to generate a private key:
$ bin/crx3 keygen .out/key.pem
Final Makefile
build:
rm -f .out/out.zip
zip -qr -9 -X .out/out.zip . -x#.crxignore
bin/crx3 pack .out/out.zip -p .out/key.pem -o .out/out.crx
Build process:
$ make build
# --> build the .out/out.crx file
Try the command line program "zip", which could be found in cygwin if you're on windows, or is likely present on OSX, and easy to install if you're using linux.
zip package.zip -r * -x package.sh -x *.git* -x "*.*~" -x "*.pdf" docs/* "*.psd"
Well, I'm a linux newbie, and I'm having an issue with a simple bash script.
I've got a program that adds to a log file while it's running. Over time that log file gets huge. I'd like to create a startup script which will rename and move the log file before each run, effectively creating separate log files for each run of the program. Here's what I've got so far:
pastebin
DATE=$(date +"%Y%m%d%H%M")
mv server.log logs/$DATE.log
echo program
When run, I see this:
: command not found
program
When I cd to the logs directory and run dir, I see this:
201111211437\r.log\r
What's going on? I'm assuming there's some syntax issue I'm missing, but I can't seem to figure it out.
UPDATE: Thanks to shellter's comment below, I've found the problem to be due to the fact that I'm editing the .sh file in Notepad++ in windows, and then sending via ftp to the server, where I run the file via ssh. After running dos2unix on the file, it works.
New question: How can I save the file correctly in the first place, to avoid having to perform this fix every time I resend the file?
mv server.log logs/$(date -d "today" +"%Y%m%d%H%M").log
The few lines you posted from your script look okay to me. It's probably something a bit deeper.
You need to find which line is giving you this error. Add set -xv to the top of your script. This will print out the line number and the command that's being executed to STDERR. This will help you identify where in your script you're getting this particular error.
BTW, do you have a shebang at the top of your script? When I see something like this, I normally expect its an issue with the Shebang. For example, if you had #! /bin/bash on top, but your bash interpreter is located in /usr/bin/bash, you'll see this error.
EDIT
New question: How can I save the file correctly in the first place, to avoid having to perform this fix every time I resend the file?
Two ways:
Select the Edit->EOL Conversion->Unix Format menu item when you edit a file. Once it has the correct line endings, Notepad++ will keep them.
To make sure all new files have the correct line endings, go to the Settings->Preferences menu item, and pull up the Preferences dialog box. Select the New Document/Default Directory tab. Under New Document and Format, select the Unix radio button. Click the Close button.
A single line method within bash works like this.
[some out put] >$(date "+%Y.%m.%d-%H.%M.%S").ver
will create a file with a timestamp name with ver extension.
A working file listing snap shot to a date stamp file name as follows can show it working.
find . -type f -exec ls -la {} \; | cut -d ' ' -f 6- >$(date "+%Y.%m.%d-%H.%M.%S").ver
Of course
cat somefile.log > $(date "+%Y.%m.%d-%H.%M.%S").ver
or even simpler
ls > $(date "+%Y.%m.%d-%H.%M.%S").ver
I use this command for simple rotate a file:
mv output.log `date +%F`-output.log
In local folder I have 2019-09-25-output.log
Well, it's not a direct answer to your question, but there's a tool in GNU/Linux whose job is to rotate log files on regular basis, keeping old ones zipped up to a certain limit. It's logrotate
You can write your scripts in notepad but just make sure you convert them
using this ->
$ sed -i 's/\r$//' yourscripthere
I use it all they time when I'm working in cygwin and it works. Hope this helps
First, thanks for the answers above! They lead to my solution.
I added this alias to my .bashrc file:
alias now='date +%Y-%m-%d-%H.%M.%S'
Now when I want to put a time stamp on a file such as a build log I can do this:
mvn clean install | tee build-$(now).log
and I get a file name like:
build-2021-02-04-03.12.12.log