How do I extract specific strings from multiple files and write them to .txt in bash? - linux

I have a lot of files in the folder filesToCheck, some examples given below. I need the output result.txt as also shown below. I can use linux bash with any commands that do not require extra installations.
The try-out below (with help from stackoverflow) has two problems when I execute it. It only looks for one instance of CAKE_FROSTING despite the global flag g and the file result.txt remains empty despite > result.txt.
sed -Enz 's/.*CAKE_FROSTING\(\n?"([^"]*).*/\1\n/gp' filesToCheck/* > result.txt
What do I need to change?
file1.cpp
something
CAKE_FROSTING("is.simply.the.best", "[no][matter][what]") { DO(something(0) == 1); }
file2.h
something else
CAKE_FROSTING(
"is.kinda.neat",
"[i][agree]") something else
something more
file3.cpp
random_text CAKE_FROSTING("Can be nice") "more random text"
CAKE_CREAM("totally.sucks", "[trust][me]")
random_text CAKE_FROSTING("Can be very nice") "indeed"
desiredResult.txt
is.simply.the.best
is.kinda.neat
Can be nice
Can be very nice
currentResult command line output:
is.simply.the.best
is.kinda.neat
Can be nice

Assuming the string CAKE_FROSTING occurs once per line, you can try this sed
$ sed -En ':a;N;s/.*CAKE_FROSTING\(\n?"([^"]*).*/\1/p;ba' filesToCheck/*
is.simply.the.best
is.kinda.neat
Can be nice
Can be very nice

Related

bash: How to replace an entire line in a text file by a part of its content

I have a text file, called texto.txt in Documentos folder, with some values like the ones below:
cat ~/Documentos/texto.txt
65f8: Testado
a4a1: Testado 2
So I want to change a whole line by using a customized function which gets as parameters the new value.
The new value will always keep the first 6 characters, changing only what comes after them. Although I am testing only the first four.
Then I edited my .bashrc including my function like shown below.
muda()
{
export BUSCA="$(echo $* | cut -c 1-4)";
sed -i "/^$BUSCA/s/.*/$*/" ~/Documentos/texto.txt ;}
When I run the command below it works like a charm, but I feel it could be improved.
muda a4a1: Testado 3
Result:
cat ~/Documentos/texto.txt
65f8: Testado
a4a1: Testado 3
Is there a smarter way to do this? Maybe by getting rid of BUSCA variable?
I'd write:
muda() {
local new_line="$*"
local key=${newline:0:4}
sed -i "s/^${key//\//\\/}.*/${new_line//\//\\/}/" ~/Documentos/texto.txt
}
Notes:
using local variables, not exported environment variables
does not call out to cut, bash can extract a substring
escaping any slashes in the variable values so the sed code is not broken.

"Cat" into multiple files using brace expansion

I am quite new to bash and trying to type some text into multiple files with a single command using brace expansion.
I tried: cat > file_{1..100} to write into 100 files some text that I will type in the terminal. I get the following error:
bash: file_{1..100}: ambiguous redirect
I also tried: cat > "file_{1..100}" but that creates a singe file named: file_{1..100}.
I tried: cat > `file_{1..100}` but that gives the error:
file_1: command not found
How can I achieve this using brace expansion? Maybe there are other ways using other utilities and/or pipelines. But I want to know if that is possible using only simple brace expansion or not.
You can't do this with cat alone. It only writes its output to its standard output, and that single file descriptor can only be associated with a single file.
You can however do it with tee file_{1..100}.
You may wish to consider using tee file_{01..100} instead, so that the filenames are zero-padded to all have the same width: file_001, file_002, ... This has the advantage that lexicographic order will agree with numerical order, and so ls, *, etc, will process them in numerical order. Without this, you have the situation that file_2 comes after file_10 in lexicographic order.
target could be only a pipe, not a multiple files.
If you want redirect output to multiple files, use tee
cat | tee file_{1..100}
Don't forget to check man tee, for example if you want to append to the files, you should add -a option (tee -a file_{1..100})
This types the string or text into file{1..4}
echo "hello you just knew me by kruz" > file{1..4}
Use to remove them
rm file*

What does this bash script command mean (sed - e)?

I'm totally new to bash scripting but i want to solve this problem..
the command is:
objfil=`echo ${srcfil} | sed -e "s,c$,o,"`
the idea about the bash script program is to check for the source files, and check if there is an adjacent object file in the OBJ directory, if so, the rest of the program runs smoothly, if not, the iteration terminates and skips the current source file, and moves on to the next one.. it works with .c files but not on the headers, since the object filenames depend on .c files.. i want to write this command so it checks the object files not just the .c but the .h files too.. but without skipping them. i know i have to do something else too, but i need to understand what this line of command does exactly to move on. Thanks. (Sorry for my english)
UPDATE:
if test -r ${curOBJdir}/${objfil}
then
cp -v ${srcfil} ./SAVEDSRC/${srcfil}
fdone="NO"
linenums=ALL
else
fdone="YES"
err="${curOBJdir}/${objfil} is missing - ${srcfil} skipped)"
echo ${err}
echo ${err} >>${log}
fi
while test ${fdone} == "NO"
do
#rest of code ...
here is the rest of the program.. i tried to comment out the "test" part to ignore the comparison just because i only want my script to work on .h files, but without checking the e.g abc.h files has an abc.o file.. (the object file generation is needed because the end of the script there's a comparison between the hexdump of the original and modified object files). The whole script is for changing the basic types with typedefs like int to sint32_t for example.
This concrete command will substitute all c's right before line-end to o:
srcfill=abcd.c
objfil=`echo ${srcfil} | sed -e "s,c$,o,"`
echo $objfil
Output:
abcd.o
P.S. It uses a different match/replace separator: default is / but it uses ,.

replace a line in linux file containing special characters

Here is an extract from a script showing the variables for the script
PathToPiconPNG="/var/OscamSrvidPicon/picon/19.2E/"
PathToOscamSrvid="/var/OscamSrvidPicon/picon/19.2E/oscam.srvid"
PathToPiconTPL="/var/OscamSrvidPicon/oscam_picons/"
PathToTmp="/tmp/"
I want to run this script numerous times replacing (for example) this line:
PathToPiconPNG="/var/OscamSrvidPicon/picon/19.2E/"
with this lines
PathToPiconPNG="/var/OscamSrvidPicon/picon/28.2E/"
I have tried using sed (I know this example is wrong but you might get what im trying to achieve)
sed 's/{PathToPiconPNG="/var/OscamSrvidPicon/picon/19.2E/"}/{PathToPiconPNG="/var/OscamSrvidPicon/picon/28.2E/"}/g' filename.txt > newfilenam.txt
If that is not possible, is there any way that I can set the variable externally from another script
sed -E 's/picon\/.+\//picon\/28.2E\//' filename.txt > newfilenam.txt

How can I replace a specific line by line number in a text file?

I have a 2GB text file on my linux box that I'm trying to import into my database.
The problem I'm having is that the script that is processing this rdf file is choking on one line:
mismatched tag at line 25462599, column 2, byte 1455502679:
<link r:resource="http://www.epuron.de/"/>
<link r:resource="http://www.oekoworld.com/"/>
</Topic>
=^
I want to replace the </Topic> with </Line>. I can't do a search/replace on all lines but I do have the line number so I'm hoping theres some easy way to just replace that one line with the new text.
Any ideas/suggestions?
sed -i yourfile.xml -e '25462599s!</Topic>!</Line>!'
sed -i '25462599 s|</Topic>|</Line>|' nameoffile.txt
The tool for editing text files in Unix, is called ed (as opposed to sed, which as the name implies is a stream editor).
ed was once intended as an interactive editor, but it can also easily scripted. The way ed works, is that all commands take an address parameter. The way to address a specific line is just the line number, and the way to change the addressed line(s) is the s command, which takes the same regexp that sed would. So, to change the 42nd line, you would write something like 42s/old/new/.
Here's the entire command:
FILENAME=/path/to/whereever
LINENUMBER=25462599
ed -- "${FILENAME}" <<-HERE
${LINENUMBER}s!</Topic>!</Line>!
w
q
HERE
The advantage of this is that ed is standardized, while the -i flag to sed is a proprietary GNU extension that is not available on a lot of systems.
Use "head" to get the first 25462598 lines and use "tail" to get the remaining lines (starting at 25462601). Though... for a 2GB file this will likely take a while.
Also are you sure the problem is just with that line and not somewhere previous (ie. the error looks like an XML parse error which might mean the actual problem is someplace else).
My shell script:
#!/bin/bash
awk -v line=$1 -v new_content="$2" '{
if (NR == line) {
print new_content;
} else {
print $0;
}
}' $3
Arguments:
first: line number you want change
second: text you want instead original line contents
third: file name
This script prints output to stdout then you need to redirect. Example:
./script.sh 5 "New fifth line text!" file.txt
You can improve it, for example, by taking care that all your arguments has expected values.

Resources