Piping into a part of bash command stored in variable [duplicate] - linux

This question already has answers here:
Conditional step in a pipeline
(2 answers)
Can I make a shell function in as a pipeline conditionally "disappear", without using cat?
(1 answer)
Closed 4 months ago.
EMPTY_VAR=''
MMDDYYYY='6.18.1997'
PIPE_VAR=' | xargs echo "1+" | bc'
echo "$MMDDYYYY" | cut -d "." -f 2${EMPTY_VAR}
>> 18
Command above would give me correct output, which is 18, but if I try to use PIPE_VAR instead it would give me bunch of errors:
echo "$MMDDYYYY" | cut -d "." -f 2${PIPE_VAR}
cut: '|': No such file or directory
cut: xargs: No such file or directory
cut: echo: No such file or directory
cut: '"1+"': No such file or directory
cut: '|': No such file or directory
cut: bc: No such file or directory
OR:
echo "$MMDDYYYY" | cut -d "." -f 2"$PIPE_VAR"
cut: invalid field value ‘| xargs echo "1+" | bc’
Try 'cut --help' for more information.
What I'm really trying to find out is that even possible to combine commands like this?

You can't put control operators like | in a variable, at least not without resorting to something like eval. Syntax parsing comes before parameter expansion when evaluating the command line, so Bash is only ever going to see that | as a literal character and not pipeline syntax. See BashParsing for more details.
Conditionally adding a pipeline is hard to do well, but having a part of the pipeline conditionally execute one command or another is more straightforward. It might look something like this:
#!/bin/bash
MMDDYYYY='6.18.1997'
echo "$MMDDYYYY" | cut -d "." -f 2 |
if some_conditional_command ; then
xargs echo "1+" | bc
else
cat
fi

It looks like you're trying to calculate the next day. That's hard to do with plain arithmetic, particularly with month/year ends.
Let date do the work. This is GNU date. It can't parse 6.18.1997 but it can parse 6/18/1997
for MMDDYYYY in '2.28.1996' '2.28.1997'; do
date_with_slashes=${MMDDYYYY//./\/}
next_day=$(date -d "$date_with_slashes + 1 day" '+%-m.%-d.%Y')
echo "$next_day"
done
2.29.1996
3.1.1997

Related

Bash how to loop through the output of a command and assign that output to variable [duplicate]

This question already has answers here:
Looping through the content of a file in Bash
(16 answers)
Closed 11 months ago.
I have a for loop in my bash script.
for dir in $directories
do
dockerfile=$(find ./repo/$dir -name "Dockerfile");
docker build -f $dockerfile . -t $dir:version;
new_image=$(echo "$dir:version" | cut -d '/' -f2);
done
How can I pass $new_image output to another variable?
If I do something like this
new_image=$(echo "$image:version" | cut -d '/' -f2 >> list_images.txt)
I get the list of docker images in my .txt file but is there a way to pass the output to a variable? And update that variable value on each iteration?
The expected output after one iteration is something like
testing1:version
So when the for loop stops executing I get something like this in my .txt file.
testing1:version
kubernetes:version
node:version
And this is what I need, I can't figure out a better way to do this.
Using an answer, because the comments make it unreadable.
Do I understand correctly that you want:
for dir in $directories
do
dockerfile=$(find ./repo/$dir -name "Dockerfile");
docker build -f $dockerfile . -t $dir:version;
echo "$dir:version" | cut -d '/' -f2
done > list_images.txt
or do you actually have a need for the variable(s) further-on in your script?
And
new_image=$(echo "$dir:version")
trimmed=${new_image#*/}
will do your cut in bash, but that's just for bonuspoints :-)

Grep a word out of a file and save the file as that word

I am using Ubuntu Linux and grepping info out of a file (lets say filename.log) and want to save the file using some of the info inside of (filename.log).
example:
The info in the (filename.log) has version_name and date.
When displaying this info on screen using cat it will display:
version_name=NAME
date=TODAY
I then want to save the file as NAME-TODAY.log and have no idea how to do this.
Any help will be appreciated
You can chain a bunch of basic linux commands with the pipe character |. Combined with a thing called command substitution (taking the output of a complex command, to use in another command. syntax: $(your command)) you can achieve what you want to do.
This is what I came up with, based on your question:
cp filename.log $(grep -E "(version_name=)|(date=)" filename.log | cut -f 2 -d = | tr '\n' '-' | rev | cut -c 2- | rev).log
So here I used cp, $(), grep, cut, tr and finally rev.
Since you said you had no idea where to start, let me walk you trough this oneliner:
cp - it is used to copy the filename.log file to a new file,
with the name based on the values of version_name and date (step 2 and up)
command substitution $() the entire command between the round brackets is 'resolved' before finishing the cp command in step 1. e.g. in your example it would be NAME-TODAY. notice the .log at the end outside of the round brackets to give it a proper file extension. The output of this command in your example will be NAME-TODAY.log
grep -E "(version_name=)|(date=)" grep with regexp flag -E to be able to do what we are doing. Matches any lines that contain version_name= OR date=. The expected output is:
version_name=NAME
date=TODAY
cut -f 2 -d = because I am not interested in version_name
, but instead in the value associated with that field, I use cut to split the line at the equals character = with the flag -d =. I then select the value behind the equals character (the second field) with the flag -f 2. The expected output is:
NAME
TODAY
tr '\n' '-' because grep outputs on multiple lines, I want to remove all new lines and replace them with a dash. Expected output:
NAME-TODAY-
rev | cut -c 2- | rev I am grouping these. rev reverses the word I have created. with cut -c 2- I cut away all characters starting from the second character of the reversed word. This is required because I replaced new lines with dashes and this means I now have NAME-TODAY-. Basicly this is just an extra step to remove the last dash. See expected outputs of each step:
-YADOT-EMAN
YADOT-EMAN
NAME-TODAY
remember this value is in the command substituion of step 2, so the end result will be:
cp filename.log NAME-TODAY.log
I manged to solve this by doing the following: grep filename.log > /tmp/file.info && filename=$(echo $(grep "version_name" /tmp/filename.info | cut -d " " -f 3)-$(grep "date" /tmp/filename.info | cut -d " " -f 3)-$filename.log

Looping though a file in bash and filtering for a directory name beginning

I'm really new to bash scripting and I am trying to write a bash script to use with subversion hooks.
The goal is to get a list of the java projects that were comitted, then write that list to a file (so I can create a change log from it and display it inside an application) along some other information gathered using svnlook that I've already worked out.
Since subversion itself doesn't really care for projects that I commit in eclipse and instead works with directories, I have to use the "svnlook changed" command which spits out each and every file with its full path that was included in the commit.
Here's an example of what "svnlook changed" returns:
U Project/branches/11.4.11.001/LIB1/com/some/other/directory/some_java_class.java
U Project/branches/11.4.11.001/LIB2/com/some/other/directory/another_thingy.java
U Project/branches/11.4.11.001/new/directories/LIB1/com/something/some_java_class.java
U Project/branches/11.4.11.001/PRJ1/com/directory/some_java_class.java
Now I can't guarantee that this directory structure will always be the same, so what I want to do is to find the java project names by filtering for "PRJ" and "LIB", since they will always begin with those letters and I can be sure this will not change.
So what I am trying to do in the script:
Step 1: Put the output of "svnlook changed" into a file:
tempfile=/data/svn/scratch.txt
input=/data/svn/scratch2.txt
touch $tempfile
touch $input
$SVNLOOK changed -r "$REVISION" "$REPOS" >> $input
This works.
Step 2: iterate over every line of the file, get the substring that represents the project by looking for "LIB" or "PRJ" in the path. Write these into a file. For now I am just assuming there are no more than 9 child directories and I am looping over the path 10 times to look at each:
for i in {1..10}
do
cat $input | cut -d "/" -f $i | grep -i -s -e PRJ -e LIB | tr [:lower:]äöü [:upper:]ÄÖÜ >> $tempfile
done
This doesn't work in the script. The file at /data/svn/scratch.txt is always empty. When I run the command from the command line and replace $input and $i, it works and all of the following commands work, too.
Step 3: Set all the other variables. I iterate over the temporary file, filtering out the duplicates and putting commas to seperate them:
DATE=$(date '+%d-%m-%Y')
REVISION=$($SVNLOOK youngest "$REPOS")
CHANGEDPRJ=$(cat $tempfile | sort -u | xargs | sed -e 's/ /, /g')
COMMENT=$($SVNLOOK log -t "$TXN" "$REPOS")
Step 4:Output this variable along other stuff into the file in which I'd like to store it all:
echo "$DATE" , "$REVISION" , "$CHANGEDPRJ" , "$COMMENT" >> /data/svn/commit.log
So right now I am getting:
17-02-2020 , 571 , , Cleanup of outdated comments
Instead of:
17-02-2020 , 571 , LIB1,LIB2,PRJ1 , Cleanup of outdated comments
I'm pretty sure there is a very obvious and easy solution to get this working, but I'm bash scripting for the very first time today and I can't seem to google for the right thing to find out what I'm doing wrong.... If someone could point me into the right direction that'd be amazing!
The following:
cat <<EOF |
U Project/branches/11.4.11.001/LIB1/com/some/other/directory/some_java_class.java
U Project/branches/11.4.11.001/LIB2/com/some/other/directory/another_thingy.java
U Project/branches/11.4.11.001/new/directories/LIB1/com/something/some_java_class.java
U Project/branches/11.4.11.001/PRJ1/com/directory/some_java_class.java
EOF
# remove the U<space><space>
sed 's/^U //' |
# replace `/` with a newline
tr '/' '\n' |
# grep only lines with PRJ and LIB
grep -e 'PRJ\|LIB' |
# sort unique
sort -u |
# join elements with a comma
paste -sd,
outputs:
LIB1,LIB2,PRJ1
So you want:
"$SVNLOOK" changed -r "$REVISION" "$REPOS" |
sed 's/^U //' | tr '/' '\n' | grep -e 'PRJ\|LIB' | sort -u | paste -sd,
Note: remember to quote variable expansions. Don't touch $tempfile do touch "$tempfile".

Read first characters of a variable in bash script? [duplicate]

This question already has answers here:
In Bash, how can I check if a string begins with some value?
(13 answers)
Closed 6 years ago.
I have a file where i get some informations through a bash script to put data in a DB table and i'd like to know how to read the first characters of a variable because if it starts with "CE-" that line's data will go into a table if not they must be inserted in an other one, how can i do this?
Like this-
var=CE-xxxxx
echo "$var"
output- CE-xxxxx
var2=$(echo "$var" | cut -c 1-3)
echo "$var2"
output- CE-
Then you can check if $var2 matches your criteria and use it further.
You can use cut to get the bytes that you need:
V="CE-IMPORTANT"
I=$(echo $V | cut -b 4-)
If you want to use the - as separator:
I=$(echo $V | cut -d '-' -f 2)
In both cases you get "IMPORTANT" in I var

Modification of file names

I have a list of more than 1000 files on the following format.
0521865417_roman_pottery_in_the_archaeological_record_2007.pdf
0521865476_power_politics_and_religion_in_timurid_iran_2007.pdf
0521865514_toward_a_theory_of_human_rights_religion_law_courts_2006.pdf
0521865522_i_was_wrong_the_meanings_of_apologies_2008.pdf
I am on Linux and want to change them as follows
2007_roman_pottery_in_the_archaeological_record.pdf
2007_power_politics_and_religion_in_timurid_iran.pdf
2006_toward_a_theory_of_human_rights_religion_law_courts.pdf
2008_i_was_wrong_the_meanings_of_apologies.pdf
Using rename and awk I managed to get
2007_roman_pottery_in_the_archaeological_record_2007.pdf
2007_power_politics_and_religion_in_timurid_iran_2007.pdf
2006_toward_a_theory_of_human_rights_religion_law_courts_2006.pdf
2008_i_was_wrong_the_meanings_of_apologies_2008.pdf
The remaining task is now to remove the last field that holds the year.
A solution that uses sed to generate the new names and the rename commands then pipes them to bash:
ls -1 | sed -r 's/[0-9]*_([A-Za-z_]*)_[a-z]{3}_([0-9]{4})\.pdf$/mv & \2_\1.pdf/g' | bash
A work around from where you left of...
echo 2007_roman_pottery_in_the_archaeological_record_2007.pdf | awk -F '_' '{$NF=""; OFS="_"; print substr($0, 0, length($0)-1)".pdf";}'

Resources