Hi i'm trying to get to a log event by nestling a query in the "where" of another query. is this possible?
AzureDiagnostics
| where resource_workflowName_s == "[Workflow Name]"
| where resource_runId_s == (AzureDiagnostics | where trackedProperties_PayloadID_g == "[GUID]" | distinct resource_runId_s)
try:
AzureDiagnostics
| where resource_workflowName_s == "[Workflow Name]"
| where resource_runId_s in (
toscalar(AzureDiagnostics
| where trackedProperties_PayloadID_g == "[GUID]"
| distinct resource_runId_s))
Related
Resources
| where type has "microsoft.compute/disks"
| extend diskState = tostring(properties.diskState)
| where managedBy == ""
or diskState == 'Attached'
or diskState == 'Unattached'
| project name, diskState,managedBy,resourceGroup, location, subscriptionId, properties.diskSizeGB, properties.timeCreated
How do I convert this KQL Query into a az graph query command?
I'm from the Microsoft for Founders Hub team. I was able to run this and it worked as intended:
az graph query -q
"Resources
| where type has 'microsoft.compute/disks'
| extend diskState = tostring(properties.diskState)
| where managedBy == ''
or diskState == 'Attached'
or diskState == 'Unattached'
| project name, diskState,managedBy,resourceGroup, location, subscriptionId, properties.diskSizeGB, properties.timeCreated"
Upon reviewing your code block you submitted:
az graph query -q “
Resources
| where type =~ ‘microsoft.compute/disks’
| extend diskState = tostring(properties.diskState)
| where managedBy == "" or diskState == 'Attached' or diskState == 'Unattached'
| project name, diskState,managedBy,resourceGroup, location, subscriptionId, diskSize=properties.diskSizeGB, timeCreation=properties.timeCreated
”
--query ‘
data[].{Disk_Name:name, Disk_State:diskState, Managed_By:managedBy, Resource_Group:resourceGroup, Location:location, Subscription_Id:subscriptionId, Disk_Size:diskSize, Time_of_Creation:timeCreation}
’
-o tsv
I noticed you have two "query" parameters and you have double quotes within your query. Please convert the double quotes to single quotes and only use one query parameter.
Please review this for more information: https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/explore-resources
I'm running Cilium inside an Azure Kubernetes Cluster and want to parse the cilium log messages in the Azure Log Analytics. The log messages have a format like
key1=value1 key2=value2 key3="if the value contains spaces, it's wrapped in quotation marks"
For example:
level=info msg="Identity of endpoint changed" containerID=a4566a3e5f datapathPolicyRevision=0
I couldn't find a matching parse_xxx method in the docs (e.g. https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/parsecsvfunction ). Is there a possibility to write a custom function to parse this kind of log messages?
Not a fun format to parse... But this should work:
let LogLine = "level=info msg=\"Identity of endpoint changed\" containerID=a4566a3e5f datapathPolicyRevision=0";
print LogLine
| extend KeyValuePairs = array_concat(
extract_all("([a-zA-Z_]+)=([a-zA-Z0-9_]+)", LogLine),
extract_all("([a-zA-Z_]+)=\"([a-zA-Z0-9_ ]+)\"", LogLine))
| mv-apply KeyValuePairs on
(
extend p = pack(tostring(KeyValuePairs[0]), tostring(KeyValuePairs[1]))
| summarize dict=make_bag(p)
)
The output will be:
| print_0 | dict |
|--------------------|-----------------------------------------|
| level=info msg=... | { |
| | "level": "info", |
| | "containerID": "a4566a3e5f", |
| | "datapathPolicyRevision": "0", |
| | "msg": "Identity of endpoint changed" |
| | } |
|--------------------|-----------------------------------------|
With the help of Slavik N, I came with a query that works for me:
let containerIds = KubePodInventory
| where Namespace startswith "cilium"
| distinct ContainerID
| summarize make_set(ContainerID);
ContainerLog
| where ContainerID in (containerIds)
| extend KeyValuePairs = array_concat(
extract_all("([a-zA-Z0-9_-]+)=([^ \"]+)", LogEntry),
extract_all("([a-zA-Z0-9_]+)=\"([^\"]+)\"", LogEntry))
| mv-apply KeyValuePairs on
(
extend p = pack(tostring(KeyValuePairs[0]), tostring(KeyValuePairs[1]))
| summarize JSONKeyValuePairs=parse_json(make_bag(p))
)
| project TimeGenerated, Level=JSONKeyValuePairs.level, Message=JSONKeyValuePairs.msg, PodName=JSONKeyValuePairs.k8sPodName, Reason=JSONKeyValuePairs.reason, Controller=JSONKeyValuePairs.controller, ContainerID=JSONKeyValuePairs.containerID, Labels=JSONKeyValuePairs.labels, Raw=LogEntry
I have a query which results in a few columns but one of the columns, I am parsing JSON to retrieve the object value but there are multiple entries in it I want each entry in JSON to retrieve in a loop and display.
Below is the query,
let forEach_table = AzureDiagnostics
| where Parameters_LOAD_GROUP_s contains 'LOAD(AUTO)';
let ParentPlId = '';
let ParentPlName = '';
let commonKey = '';
forEach_table
| where Category == 'PipelineRuns'
| extend pplId = parse_json(Predecessors_s)[0].PipelineRunId, pplName = parse_json(Predecessors_s)[0].PipelineName
| extend dbMapName = tostring(parse_json(Parameters_getMetadataList_s)[0].dbMapName)
| summarize count(runId_g) by Resource, Status = status_s, Name=pipelineName_s, Loadgroup = Parameters_LOAD_GROUP_s, dbMapName, Parameters_LOAD_GROUP_s, Parameters_getMetadataList_s, pipelineName_s, Category, CorrelationId, start_t, end_t, TimeGenerated
| project ParentPL_ID = ParentPlId, ParentPL_Name = ParentPlName, LoadGroup_Name = Loadgroup, Map_Name = dbMapName, Status,Metadata = Parameters_getMetadataList_s, Category, CorrelationId, start_t, end_t
| project-away ParentPL_ID, ParentPL_Name, Category, CorrelationId
here in the above code,
extend dbMapName = tostring(parse_json(Parameters_getMetadataList_s)[0].dbMapName)
I am retrieving 0th element as default but I would like to retrieve all elements in sequence can somebody suggest me how to achieve this.
bag_keys() is just what you need.
For example, take a look at this query:
datatable(myjson: dynamic) [
dynamic({"a": 123, "b": 234, "c": 345}),
dynamic({"dd": 123, "ee": 234, "ff": 345})
]
| project keys = bag_keys(myjson)
Its output is:
|---------|
| keys |
|---------|
| [ |
| "a", |
| "b", |
| "c" |
| ] |
|---------|
| [ |
| "dd", |
| "ee", |
| "ff" |
| ] |
|---------|
If you want to have every key in a separate row, use mv-expand, like this:
datatable(myjson: dynamic) [
dynamic({"a": 123, "b": 234, "c": 345}),
dynamic({"dd": 123, "ee": 234, "ff": 345})
]
| project keys = bag_keys(myjson)
| mv-expand keys
The output of this query will be:
|------|
| keys |
|------|
| a |
| b |
| c |
| dd |
| ee |
| ff |
|------|
extend and mv-expand methods help in resolving this kind of scenario.
Solution:
extend rows = parse_json(Parameters_getMetadataList_s)
| mv-expand rows
| project Parameters_LOAD_GROUP_s,rows
The error I am getting:
invalid string interpolation: `$$', `$'ident or `$'BlockExpr expected
Spark SQL:
val sql =
s"""
|SELECT
| ,CAC.engine
| ,CAC.user_email
| ,CAC.submit_time
| ,CAC.end_time
| ,CAC.duration
| ,CAC.counter_name
| ,CAC.counter_value
| ,CAC.usage_hour
| ,CAC.event_date
|FROM
| xyz.command AS CAC
| INNER JOIN
| (
| SELECT DISTINCT replace(split(get_json_object(metadata_payload, '$.configuration.name'), '_')[1], 'acc', '') AS account_id
| FROM xyz.metadata
| ) AS QCM
| ON QCM.account_id = CAC.account_id
|WHERE
| CAC.event_date BETWEEN '2019-10-01' AND '2019-10-05'
|""".stripMargin
val df = spark.sql(sql)
df.show(10, false)
You added s prefix which means you want the string be interpolated. It means all tokens prefixed with $ will be replaced with the local variable with the same name. From you code it looks like you do not use this feature, so you could just remove s prefix from the string:
val sql =
"""
|SELECT
| ,CAC.engine
| ,CAC.user_email
| ,CAC.submit_time
| ,CAC.end_time
| ,CAC.duration
| ,CAC.counter_name
| ,CAC.counter_value
| ,CAC.usage_hour
| ,CAC.event_date
|FROM
| xyz.command AS CAC
| INNER JOIN
| (
| SELECT DISTINCT replace(split(get_json_object(metadata_payload, '$.configuration.name'), '_')[1], 'acc', '') AS account_id
| FROM xyz.metadata
| ) AS QCM
| ON QCM.account_id = CAC.account_id
|WHERE
| CAC.event_date BETWEEN '2019-10-01' AND '2019-10-05'
|""".stripMargin
Otherwise if you really need the interpolation you have to quote $ sign like this:
val sql =
s"""
|SELECT
| ,CAC.engine
| ,CAC.user_email
| ,CAC.submit_time
| ,CAC.end_time
| ,CAC.duration
| ,CAC.counter_name
| ,CAC.counter_value
| ,CAC.usage_hour
| ,CAC.event_date
|FROM
| xyz.command AS CAC
| INNER JOIN
| (
| SELECT DISTINCT replace(split(get_json_object(metadata_payload, '$$.configuration.name'), '_')[1], 'acc', '') AS account_id
| FROM xyz.metadata
| ) AS QCM
| ON QCM.account_id = CAC.account_id
|WHERE
| CAC.event_date BETWEEN '2019-10-01' AND '2019-10-05'
|""".stripMargin
Just getting into python, and so I decided to make a hangman game. Works good, but I was wondering if there was any kind of optimizations I could make or ways to clean up the code. Also, if anyone could recommend a project that I could do next that'd be cool.
import sys
import codecs
import random
def printInterface(lst, attempts):
""" Prints user interface which includes:
- hangman drawing
- word updater """
for update in lst:
print (update, end = '')
if attempts == 1:
print ("\n\n\n\n\n\n\n\n\n\n\n\t\t _____________")
elif attempts == 2:
print ("""
|
|
|
|
|
|
|
|
|
______|______""")
elif attempts == 3:
print ("""
______
|
|
|
|
|
|
|
|
|
______|______""")
elif attempts == 4:
print ("""
______
| |
| |
(x_X) |
|
|
|
|
|
|
______|______""")
elif attempts == 5:
print ("""
______
| |
| |
(x_X) |
| |
| |
| |
|
|
|
______|______""")
elif attempts == 6:
print ("""
______
| |
| |
(x_X) |
| |
/| |
| |
|
|
|
______|______""")
elif attempts == 7:
print ("""
______
| |
| |
(x_X) |
| |
/|\ |
| |
|
|
|
______|______""")
elif attempts == 8:
print ("""
______
| |
| |
(x_X) |
| |
/|\ |
| |
/ |
|
|
______|______""")
elif attempts == 9:
print ("""
______
| |
| |
(x_X) |
| |
/|\ |
| |
/ \ |
|
|
______|______""")
def main():
try:
wordlist = codecs.open("words.txt", "r")
except Exception as ex:
print (ex)
print ("\n**Could not open file!**\n")
sys.exit(0)
rand = random.randint(1,5)
i = 0
for word in wordlist:
i+=1
if i == rand:
break
word = word.strip()
wordlist.close()
lst = []
for h in word:
lst.append('_ ')
attempts = 0
printInterface(lst,attempts)
while True:
guess = input("Guess a letter: ").strip()
i = 0
for letters in lst:
if guess not in word:
print ("No '{0}' in the word, try again!".format(guess))
attempts += 1
break
if guess in word[i] and lst[i] == "_ ":
lst[i] = (guess + ' ')
i+=1
printInterface(lst,attempts)
x = lst.count('_ ')
if x == 0:
print ("You win!")
break
elif attempts == 9:
print ("You suck! You iz ded!")
break
if __name__ == '__main__':
while True:
main()
again = input("Would you like to play again? (y/n): ").strip()
if again.lower() == "n":
sys.exit(1)
print ('\n')
I didn't try the code, but here's some random tips:
Try to format your code accordingly to PEP 8 (use i += 1 instead of i+=1). PEP 8 is the standard style guide for Python.
Use
lst = ['_ '] * len(word)
instead of the for-loop.
Use enumerate as in:
for i, word in enumerate(wordlist)
instead of manually keeping track of i in the loop.
The default mode for opening files is 'r', there's no need to specify it. Are you using codecs.open instead of the built-in open in order to get Unicode strings back? Also, try to catch a more specific exception that Exception -- probably IOError.
First idea: ASCII art
The things special to Python are regular expression syntax and range() function, as well as [xxx for yyy in zzz] array filler.
import re
def ascii_art(attempt):
return re.sub(r'\d', '', re.sub('[0{0}].' \
.format(''.join([str(e) for e in range(attempt + 1, 10)])), ' ', """
3_3_3_3_3_3_
4| 2|
4| 2|
4(4x4_4X4) 2|
5| 2|
6/5|7\ 2|
5| 2|
8/ 9\ 2|
2|
2|
1_1_1_1_1_1_1|1_1_1_1_1_1_
"""))
for i in range(1, 10):
print(ascii_art(i))
Second idea: loops
Use enumerate for word reading loop. Use
for attempt in range(1, 10):
# inside main loop
...
print ('you suck!')
as the main loop. Operator break should be used with care and not as replacement for for!
Unless I miss something, the structure of
for letters in lst:
if guess not in word:
...
break
if guess in word[i]:
...
will be more transparent as
if guess not in word:
...
else:
index = word.find (guess)
...
I would use list instead of if .. else statement in printInterface.