I am creating an Excel file through shell script. The data should be in 4 columns,
but the output data came in one column.
Create a comma-separated values file.
Perhaps, depending on your delimiters:
some process creates colon-separated data | while IFS=: read v1 v2 v3 v4; do
printf "\"%s\",\"%s\",\"%s\",\"%s\"\n" \
"$(sed 's/"/""/g' <<< "$v1")" \
"$(sed 's/"/""/g' <<< "$v2")" \
"$(sed 's/"/""/g' <<< "$v3")" \
"$(sed 's/"/""/g' <<< "$v4")" \
>> my_data.csv
done
Try adding new line at the end of each statement.
Related
i have this very simple script:
data=$(<data.txt)
counter=10
#just a dbquery using $data
result=(`psql -X -t -AF $'\t' -h $POSTGRES_HOST -d $POSTGRES_DATABASE -U $POSTGRES_USERNAME -w -c "select COUNT(field_value) from table where field_value in ($data)"`)
if [ $result == counter ]; then
echo -e "\e[92m Success \e[39m"
else
while read -r dataLine;
do
result=(`psql -X -t -AF $'\t' -h $POSTGRES_HOST -d $POSTGRES_DATABASE -U $POSTGRES_USERNAME -w -c "select field_value from fields where field_value = $dataLine" `)
if [ -z "$result" ]; then
echo "$dataLine failed"
fi
done < $data
fi
I am getting line 17: $data: ambiguous redirect (where fi is) i imagine the issue is something with the first line reading data.txt then later referencing $data on the read -r but im not exactly sure what is wrong.
Any ideas?
edit: error in line 17, not 21.
edit2: fixed "results" typo inside the loop.
edit3: data.txt contains a list of UUIDs:
'5dce6dcc-5368-4dc2-b26e-01b92c3dd3aa',
'6dab9b13-1734-4766-93f5-a96d0e0afd38',
'c365e709-296b-4e8e-acf9-1d9e252325f6'
The < redirection takes a filename, but you are passing it string data.
To redirect from a string, use a <<< here-string instead:
data=$(< data.txt)
cat <<< "$data"
Or if you are using sh instead of bash, use a short here-doc:
cat << end
$data
end
Though it would be better to just read directly from the file, since this allows you to stream arbitrary amounts of data without reading it into memory first:
data="data.txt"
cat < "$data"
While not applicable to this question, but for the benefit of future readers, you also get this error if you try to redirect from a file with spaces in the name:
$ file="my file.txt"
$ cat < $file
bash: $file: ambiguous redirect
In this case you simply have to quote the variable:
$ cat < "$file"
My file contents
RECORDS=$(aws route53 list-resource-record-sets --hosted-zone-id $HOSTED_ZONE_ID \
| $JQ -r '.ResourceRecordSets[] | select (.Name == "ap.com.") | .Name')
The Output value from $RECORDS prints in the format shown below in separated lines.
>> echo "recordType: $RECORDS"
recordType: ap.com.
ap1.com.
ap2.com.
How to print output in the format as (in inverted commas separating by comma)
>> echo "recordType: $RECORDS"
recordType: "ap.com, ap1.com, ap2.com"
I would read the jq output into an array, and use bash's own ability to join things:
# redirect from a process substitution into mapfile
mapfile -t records < <(aws ...| jq ...)
printf 'recordType: "%s"\n' "$(IFS=,; echo "${records[*]}")"
That joins with just a comma, not comma+space:
recordType: "ap.com.,ap1.com.,ap2.com."
Get out of the habit of using ALLCAPS variable names, leave those as
reserved by the shell. One day you'll write PATH=something and then
wonder why
your script is broken.
Try this line :
echo "recordType: $RECORDS"|sed -e "s/\.$/\", /"|sed -e "s/ap/\"ap/"|tr -d '\r\n'| rev | cut -c 3- | rev
Not probably the most short and efficient but it works ;) (and it could be easily customize)
You can use:
echo recordType:\"$RECORDS\"|sed 's/ /,/g'|sed 's/:/: /'
Where the first sed to replace the commas, the second one to add the one space after the colon.
You can just change slightly your jq command :
records=$(aws route53 list-resource-record-sets --hosted-zone-id $HOSTED_ZONE_ID \
| $JQ -r '[.ResourceRecordSets[] | select (.Name == "ap.com.") | .Name]|join(", ")')
This one is kinda related to my previous question. Now I'm dealing with a long string:
sed -b -i.old $inifile \
-e "s/a_very_long_pattern_i_should_problably_improve/terribly_long_replace_string"
Is there a way to get it well formatted (ie indented and not larger than 80 characters)? Use of backslashes adds extra-spaces:
$ echo 'first_part'\
> 'second_part'
first_part second_part
$ echo 'first_part'\
> 'second_part'
first_part second_part
Bash concatenates string literals that are adjacent.
So for example if you had echo "first_part" "second_part" you would get
first_part second_part
But, if you had echo "first_part""second_part" you would get first_partsecond_part
so to take advantage of that for your
-e "s/a_very_long_pattern_i_should_problably_improve/terribly_long_replace_string"
You could try doing something like
sed -b -i.old $inifile \
-e "s/a_very_long_pattern_"
"i_should_problably_improve/"
"terribly_long_replace_string"
You can use printf to paste together a bunch of arguments:
sed -b -i.old $inifile \
-e "$(printf %s
's/long long pattern'
'/long long replacement/;'
's/another/command/'
)"
I have an xmlstarlet command that looks like this:
xml sel -T-t -m /xml/path -v "concat(name,','value,',')" -n filename.xml > output.csv
It outputs like so
#output.csv
name,value,
name,value,
name,value,
I want it to look like
name,name,name,
value,value,value,
I have been focused on trying different combinations within concat:
"concat(name,'<p>'value,',')"
"concat(name,'<br />'value,',')"
"concat(name,'"<p>"'value,',')"
"concat(name,'\n'value,',')"
Am I looking at the completely wrong area?
The route I ended up taking was using a macro that transposed 8 rows into columns, within excel.
Well, your concat statement explicitly concatenates names and values. It seems what you want to do is loop over the elements twice, selecting first the names and then the values.
If you couldn't format the output using xml, then below script :
awk 'BEGIN{FS=","}
NR==FNR{if(NR!=1){array[$1]=$2;next}}
END{
for (key in array) {printf "%s,",key;valuelist=valuelist""array[key]","}
{printf "\n";print valuelist}
}' your_file_name > temp.txt && mv temp.txt your_file_name
will do the work for you.
You can process your data after the xml command:
unset allnames allvalues
while IFS=, read -r name value; do
allnames+="${name},"
allvalues+="${value},"
done < <(echo "name1,value1
name2,value2,
name3,value3" )
echo "${allnames}"
echo "${allvalues}"
You can not pipe it through the while loop (vars set inside the loop will got lost), so you can use it with your command like this:
unset allnames allvalues
while IFS=, read -r name value; do
allnames+="${name},"
allvalues+="${value},"
done < <(xml sel -T-t -m /xml/path -v "concat(name,','value,',')" -n filename.xml )
echo "${allnames}" > output.csv
echo "${allvalues}">> output.csv
In my script I am taking a text file and splitting into sections. Before doing any splitting, I am reformatting the name of the text file. PROBLEM: Creating a folder/directory and naming it the formatted file name. This is where segments are placed. However the script breaks when the text file has spaces in it. But that is the reason I am trying to reformat the name first and then do the rest of the operations. How could I do so in that sequence?
execute script: text_split.sh -s "my File .txt" -c 2
text_split.sh
# remove whitespace and format file name
FILE_PATH="/archive/"
find $FILE_PATH -type f -exec bash -c 'mv "$1" "$(echo "$1" \
| sed -re '\''s/^([^-]*)-\s*([^\.]*)/\L\1\E-\2/'\'' -e '\''s/ /_/g'\'' -e '\''s/_-/-/g'\'')"' - {} \;
sleep 1
# arg1: path to input file / source
# create directory
function fallback_out_file_format() {
__FILE_NAME=`rev <<< "$1" | cut -d"." -f2- | rev`
__FILE_EXT=`rev <<< "$1" | cut -d"." -f1 | rev`
mkdir -p $FILE_PATH${__FILE_NAME};
__OUT_FILE_FORMAT="$FILE_PATH${__FILE_NAME}"/"${__FILE_NAME}-part-%03d.${__FILE_EXT}"
echo $__OUT_FILE_FORMAT
exit 1
}
# Set variables and default values
OUT_FILE_FORMAT=''
# Grab input arguments
while getopts “s:c” OPTION
do
case $OPTION in
s) SOURCE=$(echo "$OPTARG" | sed 's/ /\\ /g' ) ;;
c) CHUNK_LEN="$OPTARG" ;;
?) usage
exit 1
;;
esac
done
if [ -z "$OUT_FILE_FORMAT" ] ; then
OUT_FILE_FORMAT=$(fallback_out_file_format $SOURCE)
fi
Your script takes a filename argument, specified with -s, then modifies a hard-coded directory by renaming the files it contains, then uses the initial filename to generate an output directory and filename. It definitely sounds like the workflow should be adjusted. For instance, instead of trying to correct all the bad filenames in /archive/, just fix the name of the file specified with -s.
To get filename and extension, use bash's string manipulation ability, as shown in this question:
filename="${fullfile##*/}"
extension="${filename##*.}"
name="${filename%.*}"
You can trim whitespace from the input string using tr -d ' '.
You can then join this to your FILE_PATH variable with something like this:
FILE_NAME=$(echo $1 | tr -d ' ')
FILE_PATH="/archive/"
FILE_PATH=$FILE_PATH$FILE_NAME
You can escape the space using a back slash \
Now the user may not always provide with the back slash, so the script can use sed to convert all (space) to \
sed 's/ /\ /g'
you can obtain the new directory name as
dir_name=`echo $1 | sed 's/ /\ /g'