I have a column in string format like below:
["name": "XXX","active": true,"locale": "EN","Channel":["1","2"]]
I would like to explode them like below in spark sql(preserving the quotes in string values).
This is code I used:
SELECT EXPLODE(from_json(col, 'map<string, string>>'))
FROM XXX;
I am not able to preserve the quotes in "XXX" and "EN" after exploding.
This is what I want:
key
value
name
"XXX"
active
true
locale
"EN"
Channel
[1,2]
The quotes are part of the JSON representation of the data and not the data itself. If there were embedded quotes in the data it would look like:
"\"SOME DATA\""
If you need to add quotes on strings, you can always concatenate them to the specific columns. You can use the concat operator to accomplish this, https://spark.apache.org/docs/latest/api/sql/index.html#concat
Alternatively, you can use get_json_object, which allows you to extract specific parts of a JSON object. https://spark.apache.org/docs/3.1.2/api/python/reference/api/pyspark.sql.functions.get_json_object.html
Related
I have varibale which contains raw data as shown below.
I want to replace the comma inside double quotes to nothing/blank.
I used replaceAll(',',''), but all other commas are also getting replaced.
So need regex function to identify pattern like "123,456,775" and then replace here comma into blank.
var = '
2/5/2023,25,"717,990","18,132,406"
2/4/2023,27,"725,674","19,403,116"
2/3/2023,35,"728,501","25,578,008"
1/31/2023,37,"716,580","26,358,186"
2/1/2023,37,"720,466","26,494,010"
1/30/2023,37,"715,685","26,517,878"
2/2/2023,37,"723,545","26,603,765" '
Tried replaceAll, but did not work
If you just want to replace "," with "", you have to escape the quotes this will do:
var.replaceAll(/\",\"/, /\"\"/)
If you want to replace commas inside the number strings, "725,674" with "725674" you will have to use a regex and capture groups, like this:
var.replaceAll(/(\"\d+),(\d+\")/, /$1$2/)
It will change for three groupings, like "18,132,406", you will have to use three capture groups.
How to split json data in PowerApps
I have a json format text
{"ID":"1","name":"yashpal"}
and I need to split the data and assign 1 to textbox1 and Yashpal to textbox2
Power Apps currently does not have a general JSON parsing mechanism, but if you know that the text you have will always have the same format, and the 'name' property cannot have double quotes ("), then you can use a regular expression to extract the values, something along the lines of
With(
Match(
<<the json text>>,
"\""ID\"":\""(?<id>[^\""]+)\"",\""name\"":\""(?<name>[^\""]+)\"""),
UpdateContext({defaultId: id, defaultName: name}))
And you can use the variables defaultId and defaultName as the Default property of 'textbox1' and 'textbox2', respectively.
join works BUT i want to keep the double quotes join gives me this
[ben,linda,john]
BUT i want this
["ben", "linda", "john"]
this is getting crazy, spent over 2 hours trying to fix this
i want to pass in a list as a string variable
why can't terraform just take in my list as a string? why is this so difficult?
so i have
name = ["ben", "linda", "john"]
and i want to pass this to variable used in terrform
var.name
why can't terrform take this as is?
i get the error saying epxtected a string and i can not find a solution online after sarching everywhere
i have been able to get
[ ben,linda,john ] using join(",", var.name) but i want ["ben", "linda", "john"]
$ terraform --version
Terraform v0.12.18
+ provider.aws v2.42.0
+ provider.template v2.1.2
Conversion from list to string always requires an explicit decision about how the result will be formatted: which character (if any) will delimit the individual items, which delimiters (if any) will mark each item, which markers will be included at the start and end (if any) to explicitly mark the result as a list.
The syntax example you showed looks like JSON. If that is your goal then the easiest answer is to use jsonencode to convert the list directly to JSON syntax:
jsonencode(var.names)
This function produces compact JSON, so the result would be the following:
["ben","linda","john"]
Terraform provides a ready-to-use function for JSON because its a common need. If you need more control over the above decisions then you'd need to use more complex techniques to describe to Terraform what you need. For example, to produce a string where each input string is in quotes, the items are separated by commas, and the entire result is delimited by [ and ] markers, there are three steps:
Transform the list to add the quotes: [for s in var.names : format("%q", s)]
Join that result using , as the delimiter: join(", ", [for s in var.names : format("%q", s)])
Add the leading and trailing markers: "[ ${join(",", [for s in var.names : format("%q", s)])} ]"
The above makes the same decisions as the JSON encoding of a list, so there's no real reason to do exactly what I've shown above, but I'm showing the individual steps here as an example so that those who want to produce a different list serialization have a starting point to work from.
For example, if the spaces after the commas were important then you could adjust the first argument to join in the above to include a space:
"[ ${join(", ", [for s in var.names : format("%q", s)])} ]"
This appears to be pretty basic but I am unable to find a suitable pipeline expression function to achieve this.
I have set an array variable VAR1 with the following value, which is an output from a SQL Lookup activity in an ADF pipeline:
[
{
"Code1": "1312312"
},
{
"Code1": "3524355"
}
]
Now, I need to convert this into a comma separated string so I can pass it to a SQL query in the next activity - something like:
"'1312312','3524355'"
I am unable to find an expression function to iterate over the array elements, nor convert an array to a string. The only pipeline expression functions I see are to convert string to array and not the other way around.
Am I missing something basic? How can this be achieved?
Use 'join' function present in 'collection' functions in 'Add dynamic content'. For example:
join(variables('ARRAY_VARIABLE'), ',')
I had this same issue and was not totally satisfied just using the join function because it keeps the keys from the json object. Also, using an iterator approach can work but is needlessly expensive and slow if you have a long list. Here was my approach, using join and replace:
replace(replace(join(variables('VAR1'), ','), '{"Code1":', ''), '}', ''))
This will give you exactly the output you are looking for.
I got it working using a ForEach loop activity to iterate over my array and use a Set Variable task with a concat expression function to create my comma separated string.
Wish they had an iterator function in the expression language itself, that would have made it much easier.
In case, you just have two elements in the array, then you can do something like:
#concat(variables('variable_name')[0].Code1, ',', variables('variable_name')[1].Code1)
I'm new to Scala and unsure of how to achieve the following
I have the String
val output = "6055039\n3000457596\n3000456748\n180013\n"
I want to extract the numbers separated by \n and store them in an Array
output.split("\n").map(_.toInt)
Or only
output.split("\n")
if you want to keep the numbers in String format. Note that .toInt throws. You might want to wrap it accordingly.