I'm creating a PATH environment variable that begins with /foo and ends with /bar; and in between I "sandwich" in a collection of related bin paths that I programmatically concatenate together, as shown:
export PATH=/foo:$(ls -1d /path/to/*/bin | xargs | sed -e 's/ /:/'g):/bar
The issue I'm having is that the in-between collection of paths are interpreted as one giant string, rather than as tokenized paths separated by a colon (":").
I tried variations in a attempt to get a tokenized result, such as:
export PATH=/foo:$(echo $(ls -1d /path/to/*/bin | xargs | sed -e 's/ /:/'g)):/bar
but no luck. I also tried variants that include an eval(1), and so on. I suppose I can next try using find(1) and friends.
What does work is pasting in the literal long string, but that's a bummer because it's not programmatic.
Incidentally, this issue isn't specific to PATH because substituting in export BLAH=... results in the same issue.
Any ideas for modifying the above so that the result is tokenized? You can try this yourself by creating, say, user$ mkdir -p /tmp/{1,2,3,4}/bin and playing around.
Whatever the solution is, it's just not coming to me today. =:)
Related
i have a variable of filenames that end with a vowel. I need to delete all of these files at once. I have tried using
rm "$vowels"
but that only seems to return the files within the variable and state that there is "No such file or Directory"
Its your use of quotes: they tell rm that your variables contents are to be interpreted as a single argument (filename). Without quotes the contents will be broken into multiple arguments using the shell rules in effect.
Be aware that this can be risky if your filenames contain spaces - as theres no way to tell the difference between spaces between filenames, and spaces IN filenames.
You can get around this by using an array instead and using quoted array expansion (which I cant remember the syntax of, but might look something like rm "${array[#]}" - where each element in the array will be output as a quoted string).
SOLUTION
assigning the variable
vowel=$(find . -type f | grep "[aeiou]$")
removing all files within variable
echo $vowel | xargs rm -v
I have several documents hosted on a cloud instance. I want to extract all words conforming to a specific pattern into a .txt file. This is the pattern:
ABC123A
ABC123B
ABC765A
and so one. Essentially the words start with a specific character string 'ABC', have a fixed number of numerals, and end with a letter. This is my code:
grep -oh ABC[0-9].*[a-zA-Z]$ > /home/user/abcLetterMatches.txt
When I execute the query, it runs for several hours without generating any output. I have over 1100 documents. However, when I run this query:
grep -r ABC[0-9].*[a-zA-Z]$ > /home/user/abcLetterMatches.txt
the list of files with the strings is generated in a matter for seconds.
What do I need to correct in my query? Also, what is causing the delay?
UPDATE 1
Based on the answers, it's evident that the command is missing the file name on which it needs to be executed. I want to run the code on multiple document files (>1000)
The documents I want searched are in multiple sub-directories within a directory. What is a good way to search through them? Doing
grep -roh ABC[0-9].*[a-zA-Z]$ > /home/user/abcLetterMatches.txt
only returns the file names.
UPDATE 2
If I use the updated code from the answer below:
find . -exec grep -oh "ABC[0-9].*[a-zA-Z]$" >> ~/abcLetterMatches.txt {} \;
I get a no file or directory error
UPDATE 3
The pattern can be anywhere in the line.
You can use this regexp :
~/ grep -E "^ABC[0-9]{3}[A-Z]$" docs > filename
ABC123A
ABC123B
ABC765A
There is no delay, grep is just waiting for the input you didn't give it (and therefore it waits on standard input, by default). You can correct your command by supplying argument with filename:
grep -oh "ABC[0-9].*[a-zA-Z]$" file.txt > /home/user/abcLetterMatches.txt
Source (man grep):
SYNOPSIS
grep [OPTIONS] PATTERN [FILE...]
To perform the same grepping on several files recursively, combine it with find command:
find . -exec grep -oh "ABC[0-9].*[a-zA-Z]$" >> ~/abcLetterMatches.txt {} \;
This does what you ask for:
grep -hr '^ABC[0-9]\{3\}[A-Za-z]$'
-h to not get the filenames.
-r to search recursively. If no directory is given (as above) the current one is used. Otherwise just specify one as the last argument.
Quotes around the pattern to avoid accidental globbing, etc.
^ at the beginning of the pattern to — together with $ at the end — only match whole lines. (Not sure if this was a requirement, but the sample data suggests it.)
\{3\} to specify that there should be three digits.
No .* as that would match a whole lot of other things.
I need to look for and replace certain strings for all files in a directory, including sub-directories. I think I'm nearly there using the following method which illustrates my general approach. I do much more inside the -exec than just this replace, but have removed this for clarity.
#!/bin/bash
#call with params: in_directory out_directory
in_directory=$1
out_directory=$2
export in_directory
export out_directory
#Duplicate the in_directory folder structure in out_directory
cd "$in_directory" &&
find . -type d -exec mkdir -p -- "$out_directory"/{} \;
find $in_directory -type f -name '*' -exec sh -c '
for file do
#Quite a lot of other stuff, including some fiddling with $file to
#get rel_file, the part of the path to a file from
#in_directory. E.g if in_directory is ./ then file ./ABC/123.txt
#will have rel_file ABC/123.txt
cat $file|tr -d '|' |sed -e 's/,/|/g' > $out_directory/$rel_file
done
' sh {} +
One issue is likely how I've tried to write the file to pipe the output to. However, this isn't the main/only issue as when I replace it with an explicit test path I still get the error
|sed -e 's/,/|/g' |No such file or directory
which makes me think the cat $file part is the problem?
Any help is massively appreciated as always - this is only the second BASH script I've ever had to write so I expect I've made a fairly basic mistake!
Your "inner" single quotes are being seen as "outer" single quotes and causing you problems. You think you are quoting the | in the tr command but what you are actually doing is ending the initial single-quoted string having an unquoted | and then starting a new single-quoted string. That second single-quoted string then ends at the single-quote that you believe is starting the sed script but is instead ending the previous single-quoted string, etc.
Use double quotes for those embedded single quotes if you can. Where you can't do that you have to use the '\'' sequence to get a literal single-quote in the single-quoted string.
Log file name: "/home/msubra/WORK/tmo/LOG/BCH1043.9987.log"
From the above string i need to extract the content BCH1043.
The directory structure may differ so the solution should check for the string with BCH until the dot
No need to call basename, you can use parameter substitution that is built-in to the shell for the whole thing:
$ cat x.sh
filepath="/home/msubra/WORK/tmo/LOG/BCH1043.9987.log"
# Strip off the path. Everything between and including the slashes.
filename=${filepath##/*/}
# Then strip off everything after and including the first dot.
part1=${filename%%.*}
echo $part1
$ ./x.sh
BCH1043
$
A dot in the filepath will not cause trouble either.
See section 4.5.4 here for more info: http://docstore.mik.ua/orelly/unix3/korn/ch04_05.htm
Oh and resist the temptation to get tricky and do it all in one line. Breaking into separate components is much easier to debug and maintain down the road, and who knows you may need to use those components too (the path and the rest of the file name).
basename will reduce /home/msubra/WORK/tmo/LOG/BCH1043.9987.log to BCH1043.9987.log
echo /home/msubra/WORK/tmo/LOG/BCH1043.9987.log | basename
You can use regular expressions, awk, perl, sed etc to extract "BCH1043" from "BCH1043.9987.log". First I need to know what the range of possible filenames is before I can suggest a regular expression for you.
Use basename to extract only the filename and then use parameter expansion to strip off the data you don't want.
log=/home/msubra/WORK/tmo/LOG/BCH1043.9987.log
log=$(basename "$log")
echo "${log%%.*}"
The following is almost equivalent but doesn't use the external basename process. However there are cases where it will give different results (though whether those cases are relevant here is up to you and your usage/input). See this answer for examples/details.
log=/home/msubra/WORK/tmo/LOG/BCH1043.9987.log
log=${log#*/}
echo "${log%%.*}"
try like this:
a="/home/msubra/WORK/tmo/LOG/BCH1043.9987.log"
echo ${a##*/} | cut -d "." -f 1
or
basename $a | cut -d "." -f 1
or
var=${a##*/}; echo ${var%%.*}
output:
BCH1043
It dosent include dot. Your question is not clear, but you can extract like that
${a##*/} will extract after last / like same as basename
Currently I have a file that has a unique line with the pattern
alphanumeric_ChangeMe_moreAlphaNumeric
Actual looks like this:
127.0.0.1 local.com localhost HostType_test_HostNumber2
I'm trying to replace the string test with a variable determined by another command, run as another user using the following code.
site=$(su admin -c get_local_site | less | sed 's/Local Site Name: //')
sed -i -e "s/RecoverPoint_[[:alnum:]]*_RPA/RecoverPoint_$site_RPA/" fakehostfile
I've tested the individual codes and they echo the correct values, but when I try to use the $site variable in the second it fails to replace the section.
I can't seem to find the correct syntax to replace just what's between the underscores with a string (containing only alphanumerics) that's stored in a variable
I've already been looking on here, and found some similar problems, however the solutions don't seem to work, since part of the replacement string is a variable. I've tried to concatenate the string as three separate variables, but it replaces things strange (Maybe due to the underscores?)
What am I missing here??
Questions with similar problems that didn't work:
sed variable replacement does not seem to work
Sed replacement not working when using variables
As others recommended, no less command required, and if you need show the line of Local Site Name only, use -n option in Sed.
Second, put varies in braces, it should fix your problem.
site=$(su admin -c get_local_site | sed -n 's/Local Site Name: //p')
sed -i "s/RecoverPoint_[[:alnum:]]*_RPA/RecoverPoint_${site}_RPA/" fakehostfile