Is there a way to draw arrows from/to table rows in PlantUML? - uml

If you had an ER-Diagram in PlantUML, stylized this way using tables, is there a way to connect the table rows with arrows?
#startuml
entity User {
<#transparent,#transparent>| <color:gold><&key> | <b>id |<r> bigint |
| | name |<r> varchar |
}
entity Email {
<#transparent,#transparent>| <color:gold><&key> | <b>id |<r> bigint |
| <color:purple><&link-intact> | <i>user_id |<r> bigint |
| | address |<r> varchar |
}
User::id -> Email::user_id
#enduml
I know you can get exactly what I want using fields, but the way it looks is a bit boring and simple.
Also, there is a bit of micromanagement by putting the tabs characters.
#startuml
entity User {
#id \t\t bigint
name \t varchar
}
entity Email {
#id \t\t bigint
+user_id \t bigint
address \t varchar
}
User::id -> Email::user_id
#enduml

There is a workaround that will not involve the tab micromanagement in the member/fields approach.
You can design a procedure, to take care of the tabs characters for you. The only thing left is to replace the member visibility icons (something I do not know how to do).
This approach requires the table data to be store as a JSON (one per each table).
For example:
table.iuml
#startuml
!global $tab_size = 4 ' The tab size in plantuml is 4 chars wide
' Returns the size in tabs, rounded down, that the $data has
!function $tab_size($data)
!return %strlen($data)/$tab_size
!endfunction
' Get the maximum number of tabs from the $data array
!function $get_max_tabs($data)
!local $tabs = 0
!foreach $field in $data
!local $field_tabs = $tab_size($field.name)
!if ($tabs < $field_tabs)
!local $tabs = $field_tabs
!endif
!endfor
!return $tabs
!endfunction
' Get the field name followed by the "perfect" number of tabs
!function $get_name($name, $max_tabs)
!local $tabs = "\t"
!local $length = $tab_size($name)
!while $max_tabs > $length
!local $tabs = $tabs + "\t"
!local $length = $length + 1
!endwhile
!return $name + $tabs
!endfunction
' Create a entity member for the table field
!procedure field($data, $max_tabs)
!local $field = $get_name($data.name, $max_tabs) + $data.type
!if %json_key_exists($data, "index")
!if $data.index == "primary"
# $field
!elseif $data.index == "unique"
~ $field
!elseif $data.index == "simple"
+ $field
!endif
!else
$field
!endif
!endprocedure
' Process the JSON data to create a plantuml entity
!procedure table($data)
!local $tabs = $get_max_tabs($data.fields)
entity $data.name {
!foreach $field in $data.fields
field($field, $tabs)
!endfor
}
!endprocedure
#enduml
database.puml
#startuml
!include table.iuml
!$user = %load_json("user.json")
table($user)
!$email = %load_json("email.json")
table($email)
user::id -> email::user_id
#enduml
user.json
{
"name": "user",
"fields": [
{
"name": "id",
"type": "bigint",
"index": "primary"
},
{
"name": "name",
"type": "varchar"
}
]
}
email.json
{
"name": "email",
"fields": [
{
"name": "id",
"type": "bigint",
"index": "primary"
},
{
"name": "user_id",
"type": "bigint"
},
{
"name": "address",
"type": "varchar"
}
]
}
Obtaining the following diagram

Related

How to (dynamically) join array with a struct, to get a value from the struct for each element in the array?

I am trying to parse/flatten a JSON data, containing an array and a struct.
For every "Id" in "data_array" column, I need to get the "EstValue" from "data_struct" column. Column name in "data_struct" is the actual id (from "data_array"). Tried my best to use a dynamic join, but getting error "Column is not iterable". Can't we use dynamic join conditions in PySpark, like we can in SQL? Is there any better way for achieving this?
JSON Input file:
{
"data_array": [
{
"id": 1,
"name": "ABC"
},
{
"id": 2,
"name": "DEF"
}
],
"data_struct": {
"1": {
"estimated": {
"value": 123
},
"completed": {
"value": 1234
}
},
"2": {
"estimated": {
"value": 456
},
"completed": {
"value": 4567
}
}
}
}
Desired output:
Id Name EstValue CompValue
1 ABC 123 1234
2 DEF 456 4567
My PySpark code:
from pyspark.sql.functions import *
rawDF = spark.read.json([f"abfss://{pADLSContainer}#{pADLSGen2}.dfs.core.windows.net/{pADLSDirectory}/InputFile.json"], multiLine = "true")
idDF = rawDF.select(explode("data_array").alias("data_array")) \
.select(col("data_array.id").alias("id"))
idDF.show(n=2,vertical=True,truncate=150)
finalDF = idDF.join(rawDF, (idDF.id == rawDF.select(col("data_struct." + idDF.Id))) )
finalDF.show(n=2,vertical=True,truncate=150)
Error:
def __iter__(self): raise TypeError("Column is not iterable")
Self joins create problems. In this case, you can avoid the join.
You could make arrays from both columns, zip them together and use inline to extract into columns. The most difficult part is creating array from "data_struct" column. Maybe there's a better way, but I only could think of first transforming it into map type.
Input:
s = """
{
"data_array": [
{
"id": 1,
"name": "ABC"
},
{
"id": 2,
"name": "DEF"
}
],
"data_struct": {
"1": {
"estimated": {
"value": 123
},
"completed": {
"value": 1234
}
},
"2": {
"estimated": {
"value": 456
},
"completed": {
"value": 4567
}
}
}
}
"""
rawDF = spark.read.json(sc.parallelize([s]), multiLine = "true")
Script:
id = F.transform('data_array', lambda x: x.id).alias('Id')
name = F.transform('data_array', lambda x: x['name']).alias('Name')
map = F.from_json(F.to_json("data_struct"), 'map<string, struct<estimated:struct<value:long>,completed:struct<value:long>>>')
est_val = F.transform(id, lambda x: map[x].estimated.value).alias('EstValue')
comp_val = F.transform(id, lambda x: map[x].completed.value).alias('CompValue')
df = rawDF.withColumn('y', F.arrays_zip(id, name, est_val, comp_val))
df = df.selectExpr("inline(y)")
df.show()
# +---+----+--------+---------+
# | Id|Name|EstValue|CompValue|
# +---+----+--------+---------+
# | 1| ABC| 123| 1234|
# | 2| DEF| 456| 4567|
# +---+----+--------+---------+

Replacing a value for a given key in Kusto

I am trying to use the .set-or-replace command to amend the "subject" entry below from sample/consumption/backups to sample/consumption/backup but I am not having much look in the world of Kusto.
I can't seem to reference the sub headings within Records, data.
"source_": CustomEventRawRecords,
"Records": [
{
"metadataVersion": "1",
"dataVersion": "",
"eventType": "consumptionRecorded",
"eventTime": "1970-01-01T00:00:00.0000000Z",
"subject": "sample/consumption/backups",
"topic": "/subscriptions/1234567890id/resourceGroups/rg/providers/Microsoft.EventGrid/topics/webhook",
"data": {
"resourceId": "/subscriptions/1234567890id/resourceGroups/RG/providers/Microsoft.Compute/virtualMachines/vm"
},
"id": "1234567890id"
}
],
Command I've tried to get to work;
.set-or-replace [async] CustomEventRawRecords [with (subject = sample/consumption/backup [, ...])] <| QueryOrCommand
If you're already manipulating the data, why not turn it into a columnar representation? that way you can easily make the corrections you want to make and also get the full richness of the tabular operators plus an intellisense experience that will help you formulate queries easily
here's an example query that will do that:
execute query in browser
datatable (x: dynamic)[dynamic({"source_": "CustomEventRawRecords",
"Records": [
{
"metadataVersion": "1",
"dataVersion": "",
"eventType": "consumptionRecorded",
"eventTime": "1970-01-01T00:00:00.0000000Z",
"subject": "sample/consumption/backups",
"topic": "/subscriptions/1234567890id/resourceGroups/rg/providers/Microsoft.EventGrid/topics/webhook",
"data": {
"resourceId": "/subscriptions/1234567890id/resourceGroups/RG/providers/Microsoft.Compute/virtualMachines/vm"
},
"id": "1234567890id"
}
]})]
| extend records = x.Records
| mv-expand record=records
| extend subject = tostring(record.subject)
| extend subject = iff(subject == "sample/consumption/backups", "sample/consumption/backup", subject)
| extend metadataVersion = tostring(record.metadataVersion)
| extend dataVersion = tostring(record.dataVersion)
| extend eventType = tostring(record.eventType)
| extend topic= tostring(record.topic)
| extend data = record.data
| extend id = tostring(record.id)
| project-away x, records, record

<h5>json sorted ascending order by value based</h5>

Here the Json data sorted:
{ designer : 5, tester :8,developer : 10, backend :7 }
I would like to the following sorted order
{ developer :10, tester :8, backend:7, designer :5 }
In order to sort JSON objects, we have to take help of Arrays. Arrays maintain an order of insertion. Moreover, it provides native sort() method to sort array elements.
Following sortByKey() function takes JSON Object as an input and returns JSON Array which is sorted by key.
var data = [
{ designer : 5, tester :8,developer : 10, backend :7 }
]
function sortByValue(jsObj){
var sortedArray = [];
for(var i in jsObj)
{
sortedArray.push([i,jsObj[i]]);
}
sortedArray.sort(function(k,v){ return k[1] - v[1]});
sortedArray.reverse();
}
var jsObj = {};
jsObj.designer = 5
jsObj.tester = 8
jsObj.developer = 10
jsObj.backend = 7
var sortedbyValueJSONArray = sortByValue(jsObj);
console.table(sortedbyValueJSONArray);
We will create JSON Array of [jsonValue, jsonKey] from JSON Object. Then, we will sort JSON Array using sort() function. Checkout folllwing example.
------------------------------
| 0 | 10 | 'developer' |
| 1 | 5 | 'designer' |
| 2 | 7 | 'backend' |
| 3 | 8 | 'tester' |
If the order of the data is relevant for your case, you should use an array, not an object.
e.g.
[
{ "name": "developer", "value": 10 },
{ "name": "tester", "value": 8 },
{ "name": "backend", "value": 7},
{ "name": "designer", "value": 5 }
]
Ordering the elements of the array is as simple as JSON.parse(data).sort(({value: a}, {value: b}) => b - a) (in JavaScript, ES6).

Replacing/Removing value for a given key in a dynamic value in Kusto

Is there any way in Kusto using which we can replace value for a specific key within a dynamic value in Kusto? Either replace value or even delete the whole key value pair if required?
UPDATE
Say we have the following dynamic value in a table:-
{
"SectionA":{
"Prop1":"abcd",
"Prop2":"efgh",
"Prop3":"ahd32",
"category":"main"
},
"Num1":1.33,
"Num2":33.8,
"City":"New York"
}
Now I want to get rid of the key value pair for the key Num1 so that the resulting output will be the following:-
{
"SectionA":{
"Prop1":"abcd",
"Prop2":"efgh",
"Prop3":"ahd32",
"category":"main"
},
"Num2":33.8,
"City":"New York"
}
If that is not possible even the following masking can work as a solution too, by masking I mean whenever key Num1 appears in the dynamic value , it will be assigned a fixed value (0 in this example) for all the rows:-
{
"SectionA":{
"Prop1":"abcd",
"Prop2":"efgh",
"Prop3":"ahd32",
"category":"main"
},
"Num1":0,
"Num2":33.8,
"City":"New York"
}
The value could be anything string or number, here in this example I have used number but this could be anything.
Update:
let t = datatable(mystring:string)
[
'
{ "SectionA":{
"Prop1":"abcd",
"Prop2":"efgh",
"Prop3":"ahd32",
"category":"main"
},
"Num1":180,
"Num2":33.8,
"City":"New York"
}
'
];
t
| project myjson = parse_json(mystring)
| project Num2=tostring(myjson.Num2), City=tostring(myjson.City), SectionA=tostring(myjson.SectionA)
| extend newColumn=strcat("\"City\":","\"",City,"\", \"Num2\":","", Num2,", \"SectionA\":","", SectionA)
The result:
Please try the code below to see if it can solves the issue:
let t = datatable(mystring:string)
[
'
{ "SectionA":{
"Prop1":"abcd",
"Prop2":"efgh",
"Prop3":"ahd32",
"category":"main"
},
"Num1":180,
"Num2":33.8,
"City":"New York"
}
'
];
t
| project myjson = parse_json(mystring)
| project Num2=tostring(myjson.Num2), City=tostring(myjson.City), SectionA=tostring(myjson.SectionA)
The result:
An alternative way for setting a value in a Kusto dynamic dictionary \ bag with unknown keys (or with too many keys to list in a query) is using the new bag_merge function:
let t = dynamic({
"SectionA":{
"Prop1":"abcd",
"Prop2":"efgh",
"Prop3":"ahd32",
"category":"main"
},
"Num1":1.33,
"Num2":33.8,
"City":"New York"
});
print bag_merge(dynamic({"Num1":0}), t)
The bag_remove_keys() function removes keys and associated values from a dynamic property-bag. The only limitation is the keys on the nested levels aren't supported.
datatable(input:dynamic) [
dynamic({
"SectionA":{
"Prop1":"abcd",
"Prop2":"efgh",
"Prop3":"ahd32",
"category":"main"
},
"Num1":1.33,
"Num2":33.8,
"City":"New York"
}) ] | extend result=bag_remove_keys(input, dynamic(['Num1']))

Convert name and Value to column and rows during Export to CSV in Powershell

I am trying to convert name and values returned from my API call into column and rows during export but so far I have no luck.
$searchResponse = Invoke-RestMethod -Method POST -Uri $searchUri -ContentType application/json -Header $requestHeader -Body $searchResultsRequest
$lists = $searchResponse.businessO.fields | Select name, value
The $list returns:
name value
---- ----
Name YY
Address ABC street
City Seattle
Zip 01256
Name XYZ
Address XYZ street
City XYZ
Zip 45456
Name Mike
Address 1256 Street
City New York
Zip 78965
I want to output this result as following in CSV:
Name Address City Zip
YY ABC street Seattle 01256
.
.
.
I tried looping through list and applied condition to check name and populate value based on it, but i end up getting either duplicates or all my output data are out of sysnc meaning Mike gets Address of YY and so on....
If($lists.count -ge 0){
ForEach($list in $lists){
if($list.name -eq "Name") {$name= $list.value }
}
I would really appreciate any help on this one. Thank You.
$searchResponse results
$searchResponse |ConvertTo-Json
{
"businessObjects": [
{
"busObId": "xxxxxxxxxxxxxx",
"busObPublicId": "abc345",
"busObRecId": "xxxxxxxxxxxxxxxxxxx",
"fields": " ",
"links": "",
"errorCode": null,
"errorMessage": null,
"hasError": false
},
{
"busObId": "xxxxxxxxxxxxx",
"busObPublicId": "rty567",
"busObRecId": "xxxxxxxxxxxxxxxx",
"fields": " ",
"links": "",
"errorCode": null,
"errorMessage": null,
"hasError": false
}
],
"hasPrompts": false,
"links": [
],
"prompts": [
],
"searchResultsFields": [
],
"totalRows": 2500
}
Fields has the name and value which I want to output.
$searchResponse.businessObjects.fields |ConvertTo-Json
[
{
"dirty": false,
"displayName": "Name",
"fieldId": "xxxxxxxxxxx",
"html": null,
"name": "Name",
"value": "Mike"
},
{
"dirty": false,
"displayName": "Address",
"fieldId": "456451212113132",
"html": null,
"name": "Address",
"value": "Seattle"
},
and so on.
This answer as a teaching exercise with the expectation, one will walk through the code in the VSCode or ISE debugger and learn more about the objects, members, and more.
There are a few ways to solve this. In this answer, I try to break down the steps in a way newer PowerShell user may learn more about the language.
# mock your data
$list1 = #(
[pscustomobject]#{Name='YY'},
[pscustomobject]#{Address='ABC street'},
[pscustomobject]#{City='Seattle'},
[pscustomobject]#{Zip='01256'},
[pscustomobject]#{Name='XYZ'},
[pscustomobject]#{Address='XYZ street'},
[pscustomobject]#{City='XYZ'},
[pscustomobject]#{Zip='45456'},
[pscustomobject]#{Name='Mike'},
[pscustomobject]#{Address='1256 Street'},
[pscustomobject]#{City='New York'},
[pscustomobject]#{Zip='78965'}
)
# mock your data
$list2 = #(
#{Name='YY'},
#{Address='ABC street'},
#{City='Seattle'},
#{Zip='01256'},
#{Name='XYZ'},
#{Address='XYZ street'},
#{City='XYZ'},
#{Zip='45456'},
#{Name='Mike'},
#{Address='1256 Street'},
#{City='New York'},
#{Zip='78965'}
)
# debuggng...
#$list1
#$list1.GetType()
#$list1[0].GetType()
#$list2
#$list2.GetType()
#$list2[0].GetType()
# seems your data looks like list1
$data = #()
$entry = [ordered]#{}
# transform the data into separate objects based on the field you want
foreach ($item in $list)
{
if ($item.Name) {$entry.Name = $item.Name}
elseif ($item.Address) {$entry.Address = $item.Address}
elseif ($item.City) {$entry.City = $item.City}
elseif ($item.Zip) {$entry.Zip = $item.Zip; $data += $entry; $entry = [ordered]#{}}
}
# data transformed now into an array of hashtables
# there are a few says to save as csv, one is to just roll your own rather
# than GetEnumerator(), etc.
$out = #()
$out += $data[0].Keys -join ', '
$out += $data | % {
#$_['Name']+','+$_['Address']+','+$_['City']+','+$_['Zip']
# or
#$_.Name+','+$_.Address+','+$_.City+','+$_.Zip
# or
$_.Name,$_.Address,$_.City,$_.Zip -join ', '
}
$out
# save $out to a file, uncomment to use after editing path
#$out | Out-File -FilePath 'c:\mycsvfile.csv'

Resources