Python Error: Cannot extract data from dictionary - python-3.x

I have a json output (y) like this below.
{
"WebACL":{
"Name":"aBlockKnownBadInputs-WebAcl",
"Id":"4312a5d0-9878-4feb-a083-09d7a9cfcfbb",
"ARN":"arn:aws:wafv2:us-east-1:100467320728:regional/webacl/aBlockKnownBadInputs-WebAcl/4312a5d0-9878-4feb-a083-09d7a9cfcfbb",
"DefaultAction":{
"Allow":{
}
},
"Description":"",
"Rules":[
{
"Name":"AWS-AWSManagedRulesKnownBadInputsRuleSet",
"Priority":500,
"Statement":{
"ManagedRuleGroupStatement":{
"VendorName":"AWS",
"Name":"AWSManagedRulesKnownBadInputsRuleSet"
}
},
"OverrideAction":{
"None":{
}
},
"VisibilityConfig":{
"SampledRequestsEnabled":true,
"CloudWatchMetricsEnabled":true,
"MetricName":"AWS-AWSManagedRulesKnownBadInputsRuleSet"
}
}
]
}
}
I want to extract "AWS-AWSManagedRulesKnownBadInputsRuleSet" from the section:-
"Name":"AWS-AWSManagedRulesKnownBadInputsRuleSet",
"Priority":500,
"Statement":{
"ManagedRuleGroupStatement":{
"VendorName":"AWS",
"Name":"AWSManagedRulesKnownBadInputsRuleSet"*
At the minute my code is returning a key error:
KeyError: 'Rules[].Statement[].ManagedRuleGroupStatement[].Name'
The format of this line is clearly wrong, but I don't know why.
ruleset = y['Rules[].Statement[].ManagedRuleGroupStatement[].Name']
My code block:
respons = client.get_web_acl(Name=(acl),Scope='REGIONAL',Id=(ids))
for y in response['WebACLs']:
ruleset = y['Rules[].Statement[].ManagedRuleGroupStatement[].Name']
Does anyone know what I'm doing wrong here?

UPDATE :- In case anyone else comes up against this problem, I fixed this by doing it a slightly different way.
#Requesting the info from AWS via get_web_acl request
respons = client.get_web_acl(Name=(acl),Scope='REGIONAL',Id=(ids))
#Converting the dict output to a string to make it searchable
result = json.dumps(respons)
#Defining what I want to search for
fullstring = "AWS-AWSManagedRulesKnownBadInputsRuleSet"
#Searching the output & printing the result: if = true / else = false
if fullstring in result:
print("Found WAF ruleset: AWS-AWSManagedRulesKnownBadInputsRuleSet!")
else:
print("WAF ruleset not found!")

Also, as part of my research I found a cool thing called jello.
(https://github.com/kellyjonbrazil/jello).
jello is similar to jq in that it processes JSON and JSON Lines data except it uses standard python dict and list syntax.
So, I copied my json into a file called file.json
Ran cat file.json | jello -s to print a grep-able schema by using the -s option
Found the bit I was interested in - in my case the name of the ManagedRuleGroupStatement and ran the following:
cat file.json | jello -s _.WebACL.Rules[0].Statement.ManagedRuleGroupStatement.Name
This gave me exactly what I wanted!
It doesn't work inside a python script but was something new and interesting to learn!

Related

Getting no response from ProcessBuilder's input stream in some specific case

So I am trying to get battery status from linux, and so far the first command (path variable) returns perfectly and I am able to get its response in form of Sequence from the input stream, but unfortunately the second command (of result variable) returns empty sequence.
fun getLinuxBatteryStatus(): Nothing? {
val path = """upower --enumerate""".runCommand() ?: return null
val parameters = listOf("present", "state", "energy-full", "energy", "energy-rate", "time to empty", "percentage")
val result = """upower -i ${path.first { "battery_BAT" in it }} | grep -E "${parameters.joinToString("|")}""""
.also { println(it) }
.runCommand() ?: return null
result.forEach(::println) // <- no ouput
// println(result.count()) // <- 0
/* Do other thing and return something (that is not related to problem) */
}
Ouput:
upower -i /org/freedesktop/UPower/devices/battery_BAT1 | grep -E "present|state|energy-full|energy|energy-rate|time to empty|percentage"
The above output is from the also block in the last command, just to preview the command's string for debugging. And if I run the above command directly into the terminal I am successfully getting the responses as follows:
present: yes
state: charging
energy: 47.903 Wh
energy-empty: 0 Wh
energy-full: 50.299 Wh
energy-full-design: 48.004 Wh
energy-rate: 17.764 W
percentage: 95%
Why is the last command not working (not returning any response) with the ProcessBuilder?
Note: the extension function runCommand has been taken from here
private fun String.runCommand(
workingDir: File = File("."),
timeoutAmount: Long = 60,
timeoutUnit: TimeUnit = TimeUnit.SECONDS
): Sequence<String>? = try {
ProcessBuilder(split("\\s".toRegex()))
.directory(workingDir)
.redirectOutput(ProcessBuilder.Redirect.PIPE)
.redirectError(ProcessBuilder.Redirect.PIPE)
.start()
.apply { waitFor(timeoutAmount, timeoutUnit) }
.inputStream.bufferedReader().lineSequence()
} catch (e: IOException) {
e.printStackTrace()
null
}
The problem here is the pipe.
You're trying to run a pipeline — a construction involving running multiple programs, that needs a shell to interpret.
But ProcessBuilder runs a single program.  In this case, it's running the program upower and passing it the parameters -i, /org/freedesktop/UPower/devices/battery_BAT1, |, grep, -E, and "present|state|energy-full|energy|energy-rate|time to empty|percentage".  Obviously upower won't know what to do with the | parameter or those after it.
You could use ProcessBuilder to run up a shell instance, which could then run your pipeline; see this answer.
But it would probably be simpler, safer, and more efficient to do the filtering in your own code, and avoid calling grep entirely.
I recommend capturing the process's error output, which would very probably have made the problem clear.

python glob.glob not working any more, return an empty list

I am running in a very, strange case... Yesterday I wrote a little script. It's goal is to check one condition in a file based on another file. It worked as intended. But since this morning, it doesn't. I haven't changed anything to the best of my knowledge. the code doesn't throw any error. I think the culprit is glob.glob
for file in glob.glob('*private.vcf.gz'):
seen = False
vcf = VCF(file)
print("test")
if not (file == "controlH.g.vcf.gz" or file == "output.g.vcf.gz"):
sample_name = file.split('.')[0]
out = "{}.FalsePositiveRefCallPurged.vcf".format(sample_name)
w = Writer(out, vcf)
for v in vcf:
seen = False
ref = VCF('output.g.vcf.gz')
for r in ref:
if seen :
break
if not seen:
if v.CHROM == r.CHROM:
if v.start == r.start or v.start > r.start and v.start < r.end:
if r.FILTER == "RefCall":
continue#print(str(v))
else:
w.write_record(v)
seen = True
w.close()
Indeed, when simply running
glob.glob('.*private.vcf.gz')
I get an empty list.
Here is the output of bash
ls *.private.vcf.gz
D2A1.private.vcf.gz D3A1.private.vcf.gz D5B3.private.vcf.gz H2C3.private.vcf.gz H4C2.private.vcf.gz
D2B3.private.vcf.gz D4A3.private.vcf.gz H2A3.private.vcf.gz H4A4.private.vcf.gz H5A3.private.vcf.gz
So I am sure the files are there ... I really don't understand why suddenly glob.glob has troubles finding them.
Any help would be greatly appreciated, thanks
Ok, it appearst that a direactory
/.ipynb_checkpoints
was created and my notebook using it as its default directory...

string.Contains function is not working as expected in my golang code. Though first string contains the second string, condition fails

I am checking some shell command output in golang. I need to check whether the output contains my input string.
//myinput is a valid string
stdoutput2,stderr2,err2 := Shellout("some valid shell command | grep "+myinput)
if(err2!=nil){
glg.Error("Not able to retrive information from system")
glg.Error(err2)
return false
} else if strings.Contains(strings.TrimSpace(stdoutput2),myinput){
glg.Info("able to retrive information from system")
return true
} else {
glg.Error("Not able to retrive information from system")
glg.Error("stdoutput2 after trimming is-->"+strings.TrimSpace(stdoutput2))
glg.Error("myinput is :"+myinput)
glg.Error("std error is :"+stderr2)
return false
}
Here though stdoutput2 contains the value of myinput, my else if condition is failing.
The final output is
Not able to retrive information from system.
stdoutput2 after trimming is--> Name = customcsi-b35f3ea733
myinput is :customcsi-b35f3ea733
Please note the space between Name and = symbol. That is part of output.
Please help in resolving this. My golang version is 1.13.5
Any help would be much appreciated.
Do you have a newline at the end of myinput?
You could try
stdoutput2, stderr2, err2 := Shellout("some valid shell command | grep " + myinput)
myinput = strings.TrimSpace(myinput)

Why does this String→List→Map conversion doesn't work in Groovy

I have input data of type
abc 12d
uy 76d
ce 12a
with the lines being separated by \n and the values by \t.
The data comes from a shell command:
brlist = 'mycommand'.execute().text
Then I want to get this into a map:
brmap = brlist.split("\n").collectEntries {
tkns = it.tokenize("\t")
[ (tkns[0]): tkns[1] ]
}
I also tried
brmap = brlist.split("\n").collectEntries {
it.tokenize("\t").with { [ (it[0]): it[1] ] }
}
Both ways gave the same result, which is a map with a single entry:
brmap.toString()
# prints "[abc:12d]"
Why does only the first line of the input data end up being in the map?
Your code works, which means the input String brlist isn't what you say it is...
Are you sure that's what you have? Try printing brlist, and then it inside collectEntries
As an aside, this does the same thing as your code:
brlist.split('\n')*.split('\t')*.toList().collectEntries()
Or you could try (incase it's spaces not tabs, this will expect both)
brlist.split('\n')*.split(/\s+/)*.toList().collectEntries()
This code works
// I use 4 spaces as tab.
def text = 'sh abc.sh'.execute().text.replaceAll(" " * 4, "\t")
brmap = text.split("\n").collectEntries {
tkns = it.tokenize("\t")
[(tkns[0]) : tkns[1]]
}
assert[abc:"12d", uy:"76d", ce:"12a"] == brmap
abc.sh
#!/bin/sh
echo "abc 12d"
echo "uy 76d"
echo "ce 12a
Also, I think your groovy code is correct. maybe your mycommand has some problem.
Ok, thanks for the hints, it is a bug in Jenkins: https://issues.jenkins-ci.org/browse/JENKINS-26481.
And it has been mentioned here before: Groovy .each only iterates one time

Pattern match data from file in Lua

I've been tasked with creating a new server modification for Crysis Wars. I have ran into a particular issue that it cannot read the old ban-file (this is required in order to keep the server consistent). The Lua code itself does not seem to have any errors, but it's just not getting any of the data.
Looking at the code I'm using for this below, can you find anything wrong with it?
This is the code I'm using for this:
function rX.CheckBanlist(player)
local Root = System.GetCVar("sys_root");
local File = ""..Root.."System/Bansystem/Raptor.xml";
local FileHnd = io.open(File, "r");
for line in FileHnd:lines() do
if (not string.find(line, "User:Read")) then
System.Log("[rX] File Read Error: System/Raptor/Banfile.xml, The contents are unexpected.");
return false;
end
local Msg, Date, Reason, Type, Domain = string.match(line, "User:Read( '(.*)', { Date='(.*)'; Reason='(.*)'; Typ='(.*)'; Info='(.*)'; } );");
local rldomain = g_gameRules.game:GetDomain(player.id);
if (Domain == rldomain) then
return true;
else
return false;
end
end
end
Also, the actual file reads as this, but I can't get the " to work in Lua properly. Could this be the issue?
User:Read( "Banned", { Date="31.03.2011"; Reason="WEBSTREAM"; Typ="Inetnum"; Info="COMPUTER.SED.gg"; } );
You may prefer using Lua's [[ for multiline string when you want to include quotes inside quotes etc.
Also, you'd have to escape the ( and ) while matching:
local Msg, Date, Reason, Type, Domain = line:match([[User:Read%( "(.-)", { Date="(.+)"; Reason="(.+)"; Typ="(.+)"; Info="(.+)"; } %);]])
And the results will be as expected: http://codepad.org/gN8kSL6H

Resources