AWK argument prints unwanted newline - excel

Question

Disclaimer: I used an extremely simple example thinking each argument had some hidden encoding I wasn't aware of. Turns out my
formatting was entirely wrong. As #miken32 said, I should be using
commas. I changed my format and it works perfectly. Valuable lesson
learned.
I've exported a csvfile from an xlsl with Excel 2013 (on Windows). I emailed myself the new csv file and am running these tests on Unix (MacOS Sierra).
Consider the following CSV file:
John
Adam
Cameron
Jordan
I'm trying to format each line to look like this:
{'operator':'EQ', 'property':'first_name', 'value':'John'},
{'operator':'EQ', 'property':'first_name', 'value':'Adam'},
{'operator':'EQ', 'property':'first_name', 'value':'Cameron'},
{'operator':'EQ', 'property':'first_name', 'value':'Jordan'}
So value is the only argument changing between each line.
Here is the awk file I wrote:
BEGIN { }
{
print "{'operator':'EQ', 'property':'first_name', 'value':'"$0"'},";
}
END { }
But after executing this is the output I get:
{'operator':'EQ', 'property':'first_name', 'value':'John
'},
{'operator':'EQ', 'property':'first_name', 'value':'Adam
'},
Notice how right after the argument ($0) is printed out, a newline is printed? This is messing with my JSON format. I have a feeling this has to do with the excel exporting (which was done by Save as .csv).
Any suggestions?

Answer 1

In awk, $0 represents the entire line, whereas $1, $2, $n represent the delimited fields in the line.
The sample provided isn't a CSV file, since there aren't any values separated by commas. If it were, you could do this:
awk -F, '{print "{'"'"'operator'"'"':'"'"'EQ'"'"', '"'"'property'"'"':'"'"'first_name'"'"', '"'"'value'"'"':'"'"'"$1"'"'"'},"}' foo.txt
Which gets a bit crazy with the shell-friendly quoting!
You should be aware that there are tools such as jq, which are designed to create and work with JSON data. If this is more than a one-off task you might be better served looking at those.
Edit using a suggestion by Ed Morton from a comment:
awk -F, '{print "{\047operator\047:\047EQ\047, \047property\047:\047first_name\047, \047value\047:\047"$1"\047},"}' foo.txt
(But from your original question it looks like you're using a separate script file anyway, so you won't have to worry about escaping quotes.)

Answer 2

As has been noted, your sample output with '-based quoting isn't valid JSON, where only " may be used.
Ensuring valid JSON output is a good reason to
use the jq CLI, which not only makes the task more robust, but also simplifies it:
jq -Rnc 'inputs | { operator: "EQ", property: "first_name", value: . }' <<EOF
John
Adam
Cameron
Jordan
EOF
yields:
{"operator":"EQ","property":"first_name","value":"John"}
{"operator":"EQ","property":"first_name","value":"Adam"}
{"operator":"EQ","property":"first_name","value":"Cameron"}
{"operator":"EQ","property":"first_name","value":"Jordan"}
Explanation:
-R reads Raw input (input that isn't JSON)
-n suppresses automatic reading of the input, so that special variables input and inputs can be used instead.
-c produces compact output (not pretty-printed)
inputs represents all input lines, and the expression after | sees each line as ., iteratively.
The output object can be specified using JavaScript syntax, which simplifies matters because the property names don't require quoting; the expanded value of { ... } is converted to JSON on output.

Answer 3

Perl:
perl -MJSON -nlE 'push #p,{operator=>"EQ",property=>"first_name",value=>$_}}{say JSON->new->pretty->encode(\#p)' file
output is valid, pretty-printed JSON:
[
{
"operator" : "EQ",
"property" : "first_name",
"value" : "John"
},
{
"operator" : "EQ",
"value" : "Adam",
"property" : "first_name"
},
{
"operator" : "EQ",
"property" : "first_name",
"value" : "Cameron"
},
{
"property" : "first_name",
"value" : "Jordan",
"operator" : "EQ"
}
]
more readble:
perl -MJSON -nlE '
push #p, { operator=>"EQ", property=>"first_name", value=>$_}
END {
say JSON->new->pretty->encode(\#p)
}' file
If you generating JSON, a final note: in the JSON the single quotes aren't allowed.