I need to display the text in the utf-8 encoding.
echo "sometext1 ${DATA[1]} \
sometext2 ${DATA[2]} \
sometext3 ${DATA[3]} \
sometext4 ${DATA[4]} \
sometext5 ${DATA[5]}"
sometext* text with the encoding of utf-8
${DATA[1]} Contains digits
${DATA[2]} Contains digits
${DATA[3]} Contains text with the encoding of windows-1251
${DATA[4]} Contains digits
${DATA[5]} Contains text with the encoding of windows-1251
I tried to do the following:
DATA_A=$(${DATA[3]}|iconv -f "windows-1251" -t "UTF-8")
DATA_B=$(${DATA[5]}|iconv -f "windows-1251" -t "UTF-8")
echo "sometext1 ${DATA[1]} \
sometext2 ${DATA[2]} \
sometext3 $DATA_A \
sometext4 ${DATA[4]} \
sometext5 $DATA_B"
And I do not get the conversion of a part of line text.
If i just convert these variables, then everything good:
echo ${DATA[3]}|iconv -f "windows-1251" -t "UTF-8"
echo ${DATA[5]}|iconv -f "windows-1251" -t "UTF-8"
echo was missing in following commands
DATA_A=$(echo "${DATA[3]}"|iconv -f "windows-1251" -t "UTF-8")
DATA_B=$(echo "${DATA[5]}"|iconv -f "windows-1251" -t "UTF-8")
or to avoid a pipe
DATA_A=$(iconv -f "windows-1251" -t "UTF-8" <<< "${DATA[3]}")
DATA_B=$(iconv -f "windows-1251" -t "UTF-8" <<< "${DATA[5]}")
Related
first of all my code:
#!/bin/tcsh
#linebroadening, zero filling, Fouriertransform, Phase Correction, automatic baseline corr.,output
xyz2pipe -in fid/test%03d.fid -x -verb \
| nmrPipe -fn EM -lb 20.0 \
| nmrPipe -fn ZF -auto \
| nmrPipe -fn FT -auto \
| nmrPipe -fn PS -p0 90.0 -p1 0.0 -di \
| nmrPipe -fn POLY -auto \
| pipe2xyz -out ft2_01/test%03d.ft2 -ov -verb
\
#ROIs:
xyz2pipe -in ft2_01/test%03d.ft2 -x -verb \
| nmrPipe -fn EXT -x1 -109ppm -xn -120ppm -sw \
| pipe2xyz -out ft2_roi/test%03d.ft2 -ov -verb
\
mkdir -p ft2_sum
mkdir -p ft2_sum_txt
proj2D.tcl -in ft2_roi/test001.ft2 -sum -out ft2_sum/test001
pipe2txt.tcl ft2_sum/test001/1H.dat > ft2_sum_txt/test001.txt
Short description of code until code line no. 17:
processing of raw 3D data which is saved into planes of 2D data as .ft2 files. Their file names are like that:
test001.ft2
test002.ft2
.
.
.
now I wanted to iterate with two different scripts over those files. Above is an working example how it works for a single file. Unfortunately I dont understand how this notation %03d works inside xyz2pipe. Is there a way to access this iteration method directly with the scripts proj2D.tcl and pipe2txt.tcl (maybe via shell pipes)?
So alternatively I tried to manipulate the filenames myself to iterate over all relevant files:
foreach file (ft2_roi/*)
echo $file
echo $file/ft2_roi
echo ${file%.ft2}
echo ${file#ft2_roi/}
end
which gives following output:
ft2_roi/test001.ft2
ft2_roi/test001.ft2/ft2_roi
Missing '}'.
So the echo ${file%.ft2} and echo ${file#ft2_roi/} string manipulation don't work correctly. What am I doing wrong?
Thanks for any advice or help!
best wishes,
Hannes
managed to do it with another program "sed":
foreach dir (ft2_roi/*)
echo 'dir:' $dir
set file = `echo $dir | sed 's/.*\///'`
set filename = `echo $file | sed -E 's/(.*?)\..*/\1/'`
echo 'filename:' $filename
end
output:
dir: ft2_roi/test001.ft2
filename: test001
question about an elegant way via %03d remains.
I've found this bash script to check status of URLs from text file and print the destination URL when having redirections :
#!/bin/bash
while read url
do
dt=$(date '+%H:%M:%S');
urlstatus=$(curl -kH 'Cache-Control: no-cache' -o /dev/null --silent --head --write-out '%{http_code} %{redirect_url}' "$url" )
echo "$url $urlstatus $dt" >> urlstatus.txt
done < $1
I'm not that good in bash : I'd like to add - for each url - the value of its Robots meta tag (if is exists)
Actually I'd really suggest a DOM parser (e.g. Nokogiri, hxselect, etc.),
but you can do this for instance (Handles lines starting with <meta and "extracts" the value of the robots' attribute content):
curl -s "$url" | sed -n '/\<meta/s/\<meta[[:space:]][[:space:]]*name="*robots"*[[:space:]][[:space:]]*content="*\([^"]*\)"*\>/\1/p'
This will print the value of the attribute or the empty string if not available.
Do you need a pure Bash solution? Or do you have sed?
You can add a line to extract the meta header for robots from the source code of the page and modify the line with echo to show its value:
#!/bin/bash
while read url
do
dt=$(date '+%H:%M:%S');
urlstatus=$(curl -kH 'Cache-Control: no-cache' -o /dev/null --silent --head --write-out '%{http_code} %{redirect_url}' "$url" )
metarobotsheader=$(curl -kH 'Cache-Control: no-cache' --silent "$url" | grep -P -i "<meta.+robots" )
echo "$url $urlstatus $dt $metarobotsheader" >> urlstatus.txt
done < $1
This example records the original line with the meta header for robots.
If you want to put a mark "-" when the page has no meta header for robots, you can change the metarobotsheader line, and put this one:
metarobotsheader=$(curl -kH 'Cache-Control: no-cache' --silent "$url" | grep -P -i "<meta.+robots" || echo "-")
If you want to extract the exact value of the attribute, you can change that line:
metarobotsheader="$(curl -kH 'Cache-Control: no-cache' --silent "$url" | grep -P -i "<meta.+robots" | perl -e '$line = <STDIN>; if ( $line =~ m#content=[\x27"]?(\w+)[\x27"]?#i) { print "$1"; } else {print "no_meta_robots";}')"
When the URL doesn't contain any meta header for robots, it will show no_meta_robots.
I've created a cURL bash script in which I want to save the response body into a file called output.log, but when I open the file output.log it looks like this:
Here is my bash script:
#!/bin/bash
SECRET_KEY='helloWorld'
FILE_NAME='sma.txt'
function save_log()
{
printf '%s\n' \
"Header Code : $1" \
"Executed at : $(date)" \
"Response Body : $2" \
'==========================================================\n' > output.log
}
while IFS= read -r line;
do
HTTP_RESPONSE=$(curl -I -L -s -w "HTTPSTATUS:%{http_code}\\n" -H "X-Gitlab-Event: Push Hook" -H 'X-Gitlab-Token: '$SECRET_KEY --insecure $line 2>&1)
HTTP_STATUS=$(echo $HTTP_RESPONSE | tr -d '\n' | sed -e 's/.*HTTPSTATUS://')
save_log $HTTP_STATUS $HTTP_RESPONSE
done < $FILE_NAME
Can anyone help me get my desired output in my output.log?
From the Curl documentation:
-I, --head Show document info only
Removing the -I or replace it with -i should solve your problem
I have script import.sh
. properties
sql=$(cat sql.sql)
${psql_path}/psql \
-X \
--set ON_ERROR_STOP=on \
--set AUTOCOMMIT=on \
--echo-all \
${sql}
${dbname} > ${log_file} 2>> ${log_file}
and file properties
psql_path="/usr/pgsql-9.6/bin"
dbname="postgres"
path="/opt/files"
log_file="ok.log"
and file sql.sql
-c "truncate table SCHEMA.TABLE1;" \
-c "\\copy SCHEMA.TABLE1 FROM '${path}/TABLE1.tsv' DELIMITER ';' CSV HEADER ENCODING 'UTF8' QUOTE '\"'" \
-c "truncate table SCHEMA.TABLE2;" \
-c "\\copy SCHEMA.TABLE1 FROM '${path}/TABLE2.tsv' DELIMITER ';' CSV HEADER ENCODING 'UTF8' QUOTE '\"'" \
i need to run psql command with importing lines from file sql.sql on the line ${sql} in script import.sh but including text /opt/files instead of the variable itself ${path}, eg:
${psql_path}/psql \
-X \
--set ON_ERROR_STOP=on \
--set AUTOCOMMIT=on \
--echo-all \
-c "truncate table SCHEMA.TABLE1;" \
-c "\\copy SCHEMA.TABLE1 FROM '/opt/files/TABLE1.tsv' DELIMITER ';' CSV HEADER ENCODING 'UTF8' QUOTE '\"'" \
-c "truncate table SCHEMA.TABLE2;" \
-c "\\copy SCHEMA.TABLE2 FROM '/opt/files/TABLE2.tsv' DELIMITER ';' CSV HEADER ENCODING 'UTF8' QUOTE '\"'" \
${dbname} > ${log_file} 2>> ${log_file}
edit: all im getting right now is example below. How do i insert the text of variable ${path} ?
${psql_path}/psql \
-X \
--set ON_ERROR_STOP=on \
--set AUTOCOMMIT=on \
--echo-all \
-c "truncate table SCHEMA.TABLE1;" \
-c "\\copy SCHEMA.TABLE1 FROM '${path}/TABLE1.tsv' DELIMITER ';' CSV HEADER ENCODING 'UTF8' QUOTE '\"'" \
-c "truncate table SCHEMA.TABLE2;" \
-c "\\copy SCHEMA.TABLE2 FROM '${path}/TABLE2.tsv' DELIMITER ';' CSV HEADER ENCODING 'UTF8' QUOTE '\"'" \
${dbname} > ${log_file} 2>> ${log_file}
This line:
sql=$(cat sql.sql)
Will put contents of "sql.sql" into this variable $sql but won't evaluate any variables inside it. The right way to do this is to use a heredoc, instead. But since I figure out you want to keep your script, a simple solution would be:
sql=$(cat sql.sql | sed -e 's#${path}#'"${path}"'#')
This is simple running sed to substitute the contents of "${path}" into the contents of shell variable of the same name. But anyway consider using heredocs, because your "sql" file is obviously not SQL at all.
As advised, the HEREDOC is the best option and then simple parameter substitution can be done:
. properties
sql=$(cat sql.sql)
${psql_path}/psql <<-EOF
-X
--set ON_ERROR_STOP=on
--set AUTOCOMMIT=on
--echo-all
${sql//\$\{path\}/$path}
${dbname} > ${log_file} 2>> ${log_file}
EOF
Notice too that the line extensions are no longer required either. You should be able to remove them from sql.sql file as well
Script to insert line in file
> cat text
BBLAYERS ?= " \
/home/neeraj/yocto/poky/meta \
/home/neeraj/yocto/poky/meta-yocto \
/home/neeraj/yocto/poky/meta-yocto-bsp \
"
In that I have to insert a line below a pattern as
> cat text
BBLAYERS ?= " \
/home/neeraj/yocto/poky/meta \
/home/neeraj/yocto/poky/meta-yocto \
/home/neeraj/yocto/poky/meta-yocto-bsp \
/home/neeraj/yocto/poky/meta-ti \
"
(Not tested)
Probably what you are looking for:
#!/bin/bash
awk -v search="$1" -v add="$2" '$0 ~ search{$0=$0"\n"add}1' "$3" >$4
exit
Where you would execute with the following format:
user#system:$ ./path-to-script.sh "Line you want to find" "Line you want to add" Input_File Output_File
Let me know if this needs more explanation. :-)