I try to remove a json node when it contains specific value.
but I get an error.
Goal is to remove an element from my json by checking its path if it contains a prefix and a suffix
could you help me to make my code working ?
import groovy.json.JsonSlurper
import groovy.json.JsonBuilder
String pathPrefix = "/server_information/environment"
String pathSuffix = "/server_information/environment"
String diffOfApi = """[{op:replace, path:/server_information/environment, value:QCSGERFX023}, {op:replace, path:/json_detail/pick_batch/0/support_list/0/already_send, value:false}]""" JsonSlurper slurper = new JsonSlurper()
def slurped = slurper.parseText(diffOfApi)
def parsedJsonDiff = new JsonBuilder(slurped)
println "removeDiffByPath() - avant removeAll parsedJsonDiff : $parsedJsonDiff"
//parsedJsonDiff.removeAll { it.path == "/json_detail/preparation_list/0/consignee/update_date" }
parsedJsonDiff.removeAll { it.path.contains(pathPrefix) && it.path.contains(pathSuffix) }
println "removeDiffByPath() - apres removeAll parsedJsonDiff : $parsedJsonDiff"
println parsedJsonDiff.toString()
for the moment, I get this error :
Test Cases/_DEBUG SEB/TEST groovy FAILED. Reason:
groovy.json.JsonException: expecting '}' or ',' but got current char
'o' with an int value of 111
The current character read is 'o' with an int value of 111 expecting
'}' or ',' but got current char 'o' with an int value of 111 line
number 1 index number 2 [{op:replace,
path:/server_information/environment, value:QCSGERFX023}, {op:replace,
path:/json_detail/pick_batch/0/support_list/0/already_send,
value:false}] ..^ at TEST groovy.run(TEST groovy:27) at
com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194) at
com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at
com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:430)
at
com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:421)
at
com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:400)
at
com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:392)
at
com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:273)
at
com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:142)
at
com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:133)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown
Source) at
TempTestCase1637062445227.run(TempTestCase1637062445227.groovy:25)
thanks to cfrick, I corrected diffOfApi json which were malformed (missing "").
Then I used jsonSlurper instead of jsonBuilder in order to use removeAll()
Here is working code :
import groovy.json.JsonSlurper
import groovy.json.JsonBuilder
import groovy.json.JsonParserType
String pathPrefix = "/server_information/environment"
String pathSuffix = "/server_information/environment"
int i=0
//String diffOfApi = """[{op:replace, path:/server_information/environment, value:QCSGERFX023}, {op:replace, path:/json_detail/pick_batch/0/support_list/0/already_send, value:false}]"""
String diffOfApi = """[{"op":"replace", "path":"/server_information/environment", "value":"QCSGERFX023"}, {"op":"replace", "path":"/json_detail/pick_batch/0/support_list/0/already_send", "value":"false"}]"""
JsonSlurper slurper = new JsonSlurper()
//slurper.setType(JsonParserType.LAX)
def slurped = slurper.parseText(diffOfApi)
//def parsedJsonDiff = new JsonBuilder(slurped)
println "removeDiffByPath() - avant removeAll parsedJsonDiff : $slurped"
// on ne tient pas compte des modifs de date de consignee
slurped.each {println "slurped " + ++i + " "+it}
slurped.removeAll { it.path.contains(pathPrefix) && it.path.contains(pathSuffix) }
println "removeDiffByPath() - apres removeAll parsedJsonDiff : $slurped"
def parsedJsonDiff = new JsonBuilder(slurped)
println parsedJsonDiff.toPrettyString()
now I get this result :
2021-11-16 15:00:59.997 DEBUG testcase.TEST groovy - 12: println(parsedJsonDiff.toPrettyString())
[
{
"op": "replace",
"path": "/json_detail/pick_batch/0/support_list/0/already_send",
"value": "false"
}
]
Related
I have an requirement where I need to parse the data into the required format
Input:
{
"Message" : "\nRecord 1:\nRequired data is missing. \n\nRecord 2:\nprocessing failed\n"
}
Here the content and delimiters are not fixed. The fixed part is only /nRecord keyword on which I am writing the Script. But I am not getting desired Output using Groovy.
desired Output:
[
{
"Record 1": "nRequired data is missing"
},
{
"Record 2": "processing failed"
}
]
I have written Groovy Script for the same but I am getting empty array.
import org.apache.commons.io.IOUtils
import groovy.json.*
import java.util.ArrayList
import java.nio.charset.*
import java.nio.charset.StandardCharsets
import groovy.json.JsonSlurper
import groovy.json.JsonBuilder
def flowFile = session.get()
if(!flowFile) return
try {
flowFile = session.write(flowFile,
{ inputStream, outputStream ->
def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
splitted = text.split('\nRecord')
int j = splitted.size()
final1 = []
for (int i=0;i<j-1;i++)
{
k = "Record " + splitted[i+1]
valid = k.replaceAll("\\n|\"|\\n|}","")
final1.add("{\"" + valid.replaceFirst(":",'":"')+ "\"}" )
}
def json = JsonOutput.toJson(final1)
outputStream.write(JsonOutput.prettyPrint(json).getBytes(StandardCharsets.UTF_8))
} as StreamCallback)
session.transfer(flowFile, REL_SUCCESS)
} catch(Exception e) {
log.error('Error during JSON operations', e)
flowFile = session.putAttribute(flowFile, "error", e.getMessage())
session.transfer(flowFile, REL_FAILURE)
}
Can you please help me with the same.
Thank you.
I would use a regex with a simple trick:
import groovy.json.*
def json = new JsonSlurper().parseText '{ "Message" : "\nRecord 1:\nRequired data is missing. \n\nRecord 2:\nprocessing failed\nRecord 3:\nprocessing failed badly\n" }'
String msg = json.Message.replaceAll( /\n+(Record \d+:)/, '=$1' ) // THE trick!
List res = ( msg =~ /(?m)(Record \d+):([^=]+)*/ ).collect{ _, rec, text -> [ (rec):text.trim() ] }
assert JsonOutput.toJson( res ) == '[{"Record 1":"Required data is missing."},{"Record 2":"processing failed"},{"Record 3":"processing failed badly"}]'
I have txt file which I am reading using FetchSFTP in NiFi. Also, I have key and values in json format, as shown below received after REST call and JoltTransformJSON:
[{
"Key": "k2s2e2",
"Value": "Ottawa"
}, {
"Key": "60601",
"Value": "Chicago"
}, {
"Key": "",
"Value": "London"
}]
How can I replace all the occurrences of matching key from above to its value in txt file.
Example: abc.txt
000 apple stocks at k2s2e2 888
9000 samsung stocks at 60601 9990377
88 nokia devivces at 78889 790888071 hgj 7
Output:
000 apple stocks at Ottawa 888
9000 samsung stocks at Chicago 9990377
88 nokia devivces at 78889 790888071 hgj 7
My attempt using ExecuteGroovyScript:
import static groovy.json.JsonOutput.toJson
import java.nio.charset.StandardCharsets
import groovy.json.JsonBuilder
import groovy.json.JsonSlurper
class KeyValue{
String key
String value
}
private findAndReplace(KeyValueList) {
def response
KeyValueList.each {
def srcExp = it.key
def replaceText = it.value
def inputFilepath = "C:\\Project\\abc.txt"
def outputFilepath = "C:\\Project\\abc_output.txt"
new File(outputFilepath).withWriter { w ->
new File(inputFilepath).eachLine { line ->
w << line.replaceAll(srcExp , replaceText ) << '\n'
}
response = w
}
new File(inputFilepath).text= new File(outputFilepath).text
}
return response;
}
def flowFile = session.get()
if(!flowFile) return
def KeyValueList = []
//try {
def is = flowFile.read().withReader("UTF-8"){ new JsonSlurper().parse(it) }
is.each {
if(it.Key != "") {
KeyValue keyValue = new KeyValue(key:it.Key,value:it.Value)
KeyValueList.add(keyValue)
}
}
def retval = findAndReplace(KeyValueList)
flowFile = session.write(flowFile, {outputStream ->
outputStream.write(retval.toString().getBytes(StandardCharsets.UTF_8))
} as OutputStreamCallback)
session.transfer(flowFile, REL_SUCCESS)
//}catch(Exception e) {
// log.info(e.getMessage())
// REL_FAILURE << flowFile
//}
it's not a response to your question. just a try to fix your code.
if i understand correctly you are trying to
read json from flowfile
read text from some file path
write text with replacement to flowfile
code for ExecuteGroovyScript processor
import groovy.json.JsonSlurper
def ff = session.get()
if(!ff) return
ff.write{rawIn, rawOut->
def keyValueList = rawIn.withReader("UTF-8"){ new JsonSlurper().parse(it) }
new File('c:/Project/abc.txt').withReader("UTF-8"){reader->
rawOut.withWriter("UTF-8"){writer->
reader.eachLine{line->
keyValueList.each{ if(it.Key) line = line.replaceAll(it.Key, it.Value) }
writer << line << '\n'
}
}
}
}
REL_SUCCESS << ff
don't have time to test it...
In my json there is a field with the value "01.05.2021 00:00:00", I need the type to be defined as Date, but it is defined as "String" because of the quotes, apparently. How do I make sure the type is "Date" and not String? I can change String to Date, but how can I do it in the list obtained from JSON?
Example as now:
value 01.05.2021 00:00:00 typeMap String
value 6 typeMap Integer
def getTypeDef(def value) {
(value instanceof BigDecimal) ? "Double" : value.getClass().simpleName
}
def list = jsonSlurper.parseText JSON
def typeMap = [:].withDefault { key -> "String" }
list.each { map ->
map.each { key, values ->
if (values != null) {
typeMap[key] = getTypeDef(values)
println('value ' + values + ' typeMap ' + typeMap[key])
//typeMap[key] = values.getClass().simpleName
}
}
}
def types = names.collect { name -> typeMap[name] }
I can do it like this, but the problem is that the value will still be a string, not a date, but I will get Date in println.
def getTypeDef(def value) {
if (value ==~ /^(0?[1-9]|[12][0-9]|3[01])[\\/\-.](0?[1-9]|1[012])[\\/\-.]\d{4}\s[0-2]?[0-9]:[0-5][0-9]:[0-5][0-9]\u0024/){
(value instanceof String) ? "Date" : value.getClass().simpleName
}
else{
(value instanceof BigDecimal) ? "Double" : value.getClass().simpleName
}
}
Here is a method to convert a String being in a "2020.04.19 12:10:36" format to Date type.
Input:
def date = convertStringToDate("2020.04.19 12:10:36")
def sdf = new SimpleDateFormat("yyyy.MM.dd HH:mm:ss")
println sdf.format(date)
Output:
2020.04.19 12:10:36
Code:
import java.text.SimpleDateFormat;
import java.util.Date;
import java.time.*;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
def convertStringToDate(String str) {
//needed format: String dateStr = "2020.04.19 12:10:36";
DateTimeFormatter pattern = DateTimeFormatter.ofPattern("yyyy.MM.dd HH:mm:ss");
LocalDateTime dateTime = LocalDateTime.parse(str, pattern);
Date convertedDatetime = Date.from(dateTime.atZone(ZoneId.systemDefault()).toInstant());
assert convertedDatetime instanceof Date
return convertedDatetime
}
assert convertStringToDate("2020.04.19 12:10:36") instanceof Date
def date = convertStringToDate("2020.04.19 12:10:36")
def sdf = new SimpleDateFormat("yyyy.MM.dd HH:mm:ss")
println sdf.format(date)
I am iterating through an XML file and want to print the gpath for each node with a value. I spent the day reading Groovy API docs and trying things, but it seems that what I think is simple, is not implemented in any obvious way.
Here is some code, showing the different things you can get from a NodeChild.
import groovy.util.XmlSlurper
def myXmlString = '''
<transaction>
<payment>
<txID>68246894</txID>
<customerName>Huey</customerName>
<accountNo type="Current">15778047</accountNo>
<txAmount>899</txAmount>
</payment>
<receipt>
<txID>68246895</txID>
<customerName>Dewey</customerName>
<accountNo type="Current">16288</accountNo>
<txAmount>120</txAmount>
</receipt>
<payment>
<txID>68246896</txID>
<customerName>Louie</customerName>
<accountNo type="Savings">89257067</accountNo>
<txAmount>210</txAmount>
</payment>
<payment>
<txID>68246897</txID>
<customerName>Dewey</customerName>
<accountNo type="Cheque">123321</accountNo>
<txAmount>500</txAmount>
</payment>
</transaction>
'''
def transaction = new XmlSlurper().parseText(myXmlString)
def nodes = transaction.'*'.depthFirst().findAll { it.name() != '' }
nodes.each { node ->
println node
println node.getClass()
println node.text()
println node.name()
println node.parent()
println node.children()
println node.innerText
println node.GPath
println node.getProperties()
println node.attributes()
node.iterator().each { println "${it.name()} : ${it}" }
println node.namespaceURI()
println node.getProperties().get('body').toString()
println node.getBody()[0].toString()
println node.attributes()
}
I found a post groovy Print path and value of elements in xml that came close to what I need, but it doesn't scale for deep nodes (see output below).
Example code from link:
transaction.'**'.inject([]) { acc, val ->
def localText = val.localText()
acc << val.name()
if( localText ) {
println "${acc.join('.')} : ${localText.join(',')}"
acc = acc.dropRight(1) // or acc = acc[0..-2]
}
acc
}
Output of example code :
transaction/payment/txID : 68246894
transaction/payment/customerName : Huey
transaction/payment/accountNo : 15778047
transaction/payment/txAmount : 899
transaction/payment/receipt/txID : 68246895
transaction/payment/receipt/customerName : Dewey
transaction/payment/receipt/accountNo : 16288
transaction/payment/receipt/txAmount : 120
transaction/payment/receipt/payment/txID : 68246896
transaction/payment/receipt/payment/customerName : Louie
transaction/payment/receipt/payment/accountNo : 89257067
transaction/payment/receipt/payment/txAmount : 210
transaction/payment/receipt/payment/payment/txID : 68246897
transaction/payment/receipt/payment/payment/customerName : Dewey
transaction/payment/receipt/payment/payment/accountNo : 123321
transaction/payment/receipt/payment/payment/txAmount : 500
Besides help getting it right, I also want to understand why there isn't a simple function like node.path or node.gpath that prints the absolute path to a node.
You could do this sort of thing:
import groovy.util.XmlSlurper
import groovy.util.slurpersupport.GPathResult
def transaction = new XmlSlurper().parseText(myXmlString)
def leaves = transaction.depthFirst().findAll { it.children().size() == 0 }
def path(GPathResult node) {
def result = [node.name()]
def pathWalker = [hasNext: { -> node.parent() != node }, next: { -> node = node.parent() }] as Iterator
(result + pathWalker.collect { it.name() }).reverse().join('/')
}
leaves.each { node ->
println "${path(node)} = ${node.text()}"
}
Which gives the output:
transaction/payment/txID = 68246894
transaction/payment/customerName = Huey
transaction/payment/accountNo = 15778047
transaction/payment/txAmount = 899
transaction/receipt/txID = 68246895
transaction/receipt/customerName = Dewey
transaction/receipt/accountNo = 16288
transaction/receipt/txAmount = 120
transaction/payment/txID = 68246896
transaction/payment/customerName = Louie
transaction/payment/accountNo = 89257067
transaction/payment/txAmount = 210
transaction/payment/txID = 68246897
transaction/payment/customerName = Dewey
transaction/payment/accountNo = 123321
transaction/payment/txAmount = 500
Not sure that's what you want though, as you don't say why it "doesn't scale for deep nodes"
I have a problem where jsonBuilder strips quotes from the result string. How do I format the output to return a JSON response with quotes ?
import com.eviware.soapui.support.XmlHolder
import net.sf.*
import net.sf.json.*
import net.sf.json.groovy.*
import groovy.json.JsonSlurper
import groovy.json.JsonBuilder
import groovy.json.*
import groovy.json.JsonOutput
import net.sf.json.JSONObject
def ResponseMessage = testRunner.testCase.testSteps["MerchantEMS_POST"].testRequest.response.contentAsString
def jsonSlurper = new JsonSlurper().parseText(ResponseMessage)
log.info ResponseMessage
def merchantResult = ResponseMessage
def newMerchantID = "60300004055"
def entityID = jsonSlurper.entityId
jsonSlurper.merchantId = newMerchantID
def jsonBuilder = new groovy.json.JsonBuilder()
def updatedjson = jsonBuilder(jsonSlurper)
log.info "updated JSON = $updatedjson"
return updatedjson
ResponseMessage : { "entityId" : "93LSHLXW7BJ5K00MJALWZJMLL0", "creatorId" : "HPCDKMSV763K2VGHCKQQ09QSGM", "createdTimestamp" : "2015-09-02T00:26:34.015Z", "updaterId" : "HPCDKMSV763K2VGHCKQQ09QSGM", "updatedTimestamp" : "2015-09-02T00:26:34.015Z", "merchantId" : "L7QWKA0001F5W1RRZY4Z006153",
"createdBy" : "ralgg00", "isDeleted" : false }
updatedjson (no quotes) = [updatedTimestamp:2015-09-02T00:26:34.015Z, createdBy:ralgg00, createdTimestamp:2015-09-02T00:26:34.015Z, creatorId:HPCDKMSV763K2VGHCKQQ09QSGM, entityId:93LSHLXW7BJ5K00MJALWZJMLL0, merchantId:60300004055, isDeleted:false, updaterId:HPCDKMSV763K2VGHCKQQ09QSGM]
EDIT:
When you log the 'updatedjson' it recognises it as Map object and prints its fields. You need to use something that can convert a Map object to JSON and print it out. There are many ways to do this, for example:
def json = JsonOutput.toJson(updatedjson)
println json
Source: http://www.groovy-lang.org/json.html