I am having problems in execution a sepefic sed command.
I have a bunch of files which will be target to several "find and replace" via sed.
In the file_test there is the following text example:
*FILENAME TESTE "DIR1.PGM1";*
My desired output is:
*FILENAME TESTE "/NEW_ROOT/DIR1.PGM1";*
The following command works:
*sed -i -e '/FILENAME/s!"!"/NEW_ROOT/!' file_test*
However having several alterations to make I opted to list the on a file a loop through it.
list_changes
FILENAME /NEW_ROOT/
Below its the logic I've implemented:
*#!/bin/bash
diretoria=pasta_b
ficheiro=teste
arquivo_2="$2" # serve para guardar o nome do txt
while read -r linha; do
grep_input=$(echo "$linha" | cut -d " " -f 1) #get input for sed
grep_output=$(echo "$linha" | cut -d " " -f 2) #get output for sed
cd ~/Desktop/FUENTES/"$diretoria"
novo_fich="novo_$ficheiro" #rename file
cat "$ficheiro" >> "$novo_fich".txt #rename file
comand_sed="sed -i -e '/"$grep_input'/s!"!"/'$grep_output"/!' "$novo_fich".txt"
echo "Comando : $comand_sed"
call_comand=$(${comand_sed})
cd ~/Desktop/FUENTES/
done < "$arquivo_2"*
The error it gives is the following:
sed: -e expression #1, char 1: unknown command: `''
However the command_sed variable outputs to a well constructed sed command:
sed -i -e '/LIBNAME/s!"!"/2sasdata2/!' novo_base08.txt
To sum up, my issue is giving a command based on a previously assigned variable (comand_sed)
Related
I have a data in file and need to extract particular element based on input command.
Example: Data in file look like:
NM1*IL*1*JESSI*PINKMAN*XM*SIRUS~
NM1*IL**WALTER*WHITE*XM*PANDORA*EM*ARIZONA~
Input Command:
NM1*IL*XM*
Expected Output:
SIRUS
PANDORA
Input Command:
NM1*IL*EM*
Expected Output:
ARIZONA
I have code which extract the segment.
grep -P -o '(?<=NM1[* ]IL).*?(?=~)'
this code gives the data between NM1*IL and ~
*1*JESSI*PINKMAN*XM*SIRUS
**WALTER*WHITE*XM*PANDORA*EM*ARIZONA
Now i have to search again for string XM in this output and get data next to XM*
so i appended code as below
grep -P -o '(?<=NM1[*]IL).*?(?=~)' | grep -P -o '(?<=XM[*]).*?(?=[*]ORNULL)'
Not sure how i can give end string to be as * or ENDOFLINE
I am using sed command to solve this
grep -P -o '(?<=NM1[* ]IL).*?(?=~)' FILE.txt | sed -ne 's/$/EOLINE&/p' | grep -P -o '(?<=XM[ * ]).*?(?=[ * ]|~|EOLINE)'
Explanation command by command
grep -P -o '(?<=NM1[* ]IL).*?(?=~)' FILE.txt
extracts data between NM1* IL and ~
sed -ne 's/$/EOLINE&/p'
sed suffix EOLINE for the output of grep
grep -P -o '(?<=XM[* ]).*?(?=[ * ]|~|EOLINE)'
again taking the data between XM* and (* or ~ or EOLINE)
I have the following code:
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail
set -u
se () { sed -n 's/\s*r('"$1"').*|r| =\s\+//p'}
### gets the number of cell opt steps
exec 0<"DEMLIR.out"
while read -r line
do
gawk 'BEGIN{FS="OPTIMIZATION STEP:"} {print $2}' | tr -s " "
done>results
sed -i '/^$/d' results
#sed -e 's/^[ \t]*//' results
step=$(tail -n 1 results)
echo "${step}"
### gets the number of steps in each geo_opt output
for i in $(seq 1 $step)
do
exec 0<"DEMLIR-GEO_OPT-$i.out"
while read -r line
do
gawk 'BEGIN{FS="OPTIMIZATION STEP:"} {print $2}' | tr -s " "
done>results_geo_$i
sed -i '/^$/d' results_geo_$i
step_geo=$(tail -n 1 results_geo_$i)
echo "${step_geo}"
### goes through each line in distance.out and prints distances to array
exec 0<"DEMLIR-GEO_OPT-$i-distance-1.coordLog"
while read -r line
do
for j in $(seq 0 $step_geo)
do
"$line" | se
paste -d' ' <(printf '%s\n' $j) <(se 1,5) <(se 2,5) <(se 2,8)
done
done
done>DEMLIR_task.txt
I try to run this program but I keep getting the unexpected end of line erroron line 53. I saw that there are already answers for this problem but when I try dos2unixit says
dos2unix: converting file script_step.sh to Unix format...
but that's it. And when I run the code again it won't work.
I am expecting some errors at the last for loop, so if you see some you can also point them out. But if they are not related to the original problem then you can just ignore them.
The problem is here:
se () { sed -n 's/\s*r('"$1"').*|r| =\s\+//p'}
You need a semicolon or newline before the }. Otherwise, it's treated as part of the sed argument.
I want to write an xml tag in a file at some line number.I will be requesting the user to fill out few information which i will use to generate the xml tag.
After echoing it , i got the desired out. But when i write it to file im getting few exceptions.
Script file : test.sh
cd /opt/shibboleth-ds/conf
chmod 777 wayfconfig.xml
echo "Please Enter MetadataProvider displayName : "
read -r -p "" mdp_displayname
echo "Please Enter MetadataProvider identifier : "
read -r -p "" mdp_identifier
echo "Please Enter Federation Metadata URL : "
read -r -p "" fmd_url
configure_metadata_xml='<MetadataProvider
displayName="'$mdp_displayname'"
identifier="'$mdp_identifier'"
backingFile="'$ds_home'/metadata/fed-metadata-backup.xml"
url="'$fmd_url'"/>'
echo $configure_metadata_xml
#sed -i '70i\test string' wayfconfig.xml
sed -i '70i\'$configure_metadata_xml'' wayfconfig.xml
Im trying to write the xml tag in wayfconfig.xml at line number 70.
Output:
./test.sh
Please Enter MetadataProvider displayName :
MyDisplayName
Please Enter MetadataProvider identifier :
MyIdentifier
Please Enter Federation Metadata URL :
http://plamakerandroid.com/file.xml
<MetadataProvider displayName="MyDisplayName" identifier="MyIdentifier" backingFile="/metadata/fed-metadata-backup.xml" url="http://plamakerandroid.com/file.xml"/>
sed: can't read displayName="MyDisplayName": No such file or directory
sed: can't read identifier="MyIdentifier": No such file or directory
sed: can't read backingFile="/metadata/fed-metadata-backup.xml": No such file or directory
sed: can't read url="http://plamakerandroid.com/file.xml"/>: No such file or directory
What am i missing here
Your variable will contain spaces, so you need to quote it. try this:
sed -i '70i'"$configure_metadata_xml" wayfconfig.xml
Let me show you how it works:
No space in the test variable no quote needed:
test="abc"; sed -re '1i'$test <<< $'a\nb\nc'
abc
a
b
c
With space in the test variable but without quoting:
test="a bc"; sed -re '1i'$test <<< $'a\nb\nc'
sed: can't read bc: No such file or directory
With space in the test variable and quoting:
test="a bc"; sed -re '1i'"$test" <<< $'a\nb\nc'
a bc
a
b
c
With awk command,
awk 'NR==7{print "This is the new line"}7' filename
With sed command,
sed -i .bak '7i\
This is the new line\
' filename
these commands with insert This is the new line at line number 7.
In my script I am taking a text file and splitting into sections. Before doing any splitting, I am reformatting the name of the text file. PROBLEM: Creating a folder/directory and naming it the formatted file name. This is where segments are placed. However the script breaks when the text file has spaces in it. But that is the reason I am trying to reformat the name first and then do the rest of the operations. How could I do so in that sequence?
execute script: text_split.sh -s "my File .txt" -c 2
text_split.sh
# remove whitespace and format file name
FILE_PATH="/archive/"
find $FILE_PATH -type f -exec bash -c 'mv "$1" "$(echo "$1" \
| sed -re '\''s/^([^-]*)-\s*([^\.]*)/\L\1\E-\2/'\'' -e '\''s/ /_/g'\'' -e '\''s/_-/-/g'\'')"' - {} \;
sleep 1
# arg1: path to input file / source
# create directory
function fallback_out_file_format() {
__FILE_NAME=`rev <<< "$1" | cut -d"." -f2- | rev`
__FILE_EXT=`rev <<< "$1" | cut -d"." -f1 | rev`
mkdir -p $FILE_PATH${__FILE_NAME};
__OUT_FILE_FORMAT="$FILE_PATH${__FILE_NAME}"/"${__FILE_NAME}-part-%03d.${__FILE_EXT}"
echo $__OUT_FILE_FORMAT
exit 1
}
# Set variables and default values
OUT_FILE_FORMAT=''
# Grab input arguments
while getopts “s:c” OPTION
do
case $OPTION in
s) SOURCE=$(echo "$OPTARG" | sed 's/ /\\ /g' ) ;;
c) CHUNK_LEN="$OPTARG" ;;
?) usage
exit 1
;;
esac
done
if [ -z "$OUT_FILE_FORMAT" ] ; then
OUT_FILE_FORMAT=$(fallback_out_file_format $SOURCE)
fi
Your script takes a filename argument, specified with -s, then modifies a hard-coded directory by renaming the files it contains, then uses the initial filename to generate an output directory and filename. It definitely sounds like the workflow should be adjusted. For instance, instead of trying to correct all the bad filenames in /archive/, just fix the name of the file specified with -s.
To get filename and extension, use bash's string manipulation ability, as shown in this question:
filename="${fullfile##*/}"
extension="${filename##*.}"
name="${filename%.*}"
You can trim whitespace from the input string using tr -d ' '.
You can then join this to your FILE_PATH variable with something like this:
FILE_NAME=$(echo $1 | tr -d ' ')
FILE_PATH="/archive/"
FILE_PATH=$FILE_PATH$FILE_NAME
You can escape the space using a back slash \
Now the user may not always provide with the back slash, so the script can use sed to convert all (space) to \
sed 's/ /\ /g'
you can obtain the new directory name as
dir_name=`echo $1 | sed 's/ /\ /g'
I need to pipe a list of two delimeter separated variables to a command in BASH. I deleted my girlfriend's files from her SD card accidentally. I cloned an image of it using dd and used Sleuth Kit to recover the inode number and names of the deleted files.
fls -d -r bckup_irmasSD1.img | awk 'gsub(/\t|.*\*/,"")' | less
This gives me an example output:
6689308:DCIM/Camera/2014-02-05 20.51.30.jpg
6689560:DCIM/Camera/2013-08-10 16.37.44.jpg
6689563:DCIM/Camera/2013-08-10 16.37.52.jpg
6689566:DCIM/Camera/2013-09-14 19.00.06.jpg
6689567:DCIM/Camera/_I966F~2.MP4
29211:Android/data/com.google.android.apps.maps/cache/_ACHE_~8.M
29298:Android/data/com.google.android.apps.maps/cache/_ACHE_~2.6
29301:Android/data/com.google.android.apps.maps/cache/cache_vts_GMM.7
29304:Android/data/com.google.android.apps.maps/cache/cache_vts_GMM.8
73224:bluetooth/DSC00360.jpg
73227:bluetooth/DSC00360_2.jpg
14728713:.downloadTemp/1616021_716182491801349_1111393555_n.mp4
14728718:.downloadTemp/1616117_10151911525912011_1690760246_n.mp4
18898441:download/1595926_47757
18898445:download/1614824_234800313358133_914357470_n.mp4
18898449:download/_24316~1.MP4
To recover a deleted file by inode number, you can use the command line tool icat:
icat -d /tmp/disk.img 18898449 > /recover/download/_24316~1.MP4
How can I pipe this cleanly to a command to recover all files?
fls -d -r bckup_irmasSD1.img |
awk 'gsub(/\t|.*\*/,"")' |
while IFS=: read -r inode filename; do
mkdir -p /recover/"${filename%/*}"
icat -d /tmp/disk.img $inode > /recover/"$filename"
done
You could use awk again to split your lines and then call your command:
fls -d -r bckup_irmasSD1.img | awk 'gsub(/\t|.*\*/,"")' > indoes.txt
awk -F: '{system("icat -d " $1 " > " #2}' inodes.txt
Make sure none of your filenames contain a : and buy your girlfriend some flowers!