What is the best way to define and initialize a Table? - stanza

In Python, we can do this.
board = {'us': {'name': 'USA', 'govern': 'good'},
'canada': {'name': 'Canada', 'govern': 'good'},
'uk': {'name': 'UK', 'govern': 'good', 'recruit': 3},
'spain': {'name': 'Spain', 'govern': 'good', 'schengen': True, 'recruit': 2},
'france': {'name': 'France', 'govern': 'good', 'schengen': True, 'recruit': 2},
'italy': {'name': 'italy', 'govern': 'good', 'schengen': True} }
to create a dictionary of name value pairs for easy lookup.
Can I the the same in Stanza language, like:
deftype Countries <: Table<String,Table<String,?>>
; value could be anything. we see Int|String|True|False here
val board : Countries = to-table( "us" => ( "name" => "USA", "govern" => "good" ), ....)
?

The closest data structure to a python dictionary in stanza is Hashtable, from collections. You can do something like :
; to-hashtable<K,V> can be found in collections, too!
val board = to-hashtable<String, HashTable<String, ?>> $ [
"us" => to-hashtable<String, ?> $ [
"name" => "USA"
],
"fr" => to-hashtable<String, ?> $ [
"name" => "France"
]
; .. etc ...
]
println(board["us"])
This will output :
HashTable(
"name" => "USA")
deftype Countries <: Table<...> doesn't create an alias for Table, it declares a new type. In order to use it like a table, you would need to implement the required methods for Table.
But normally we like to add more type information, not less!
defenum Government :
Democracy
Republic
Monarchy
defstruct Country :
name:String,
gov:Government
; ... etc ...
val board = to-hashtable<String,Country> $ [
"us" => Country(name, gov) where :
val name = "USA"
val gov = Republic
"fr" => Country(name, gov) where :
val name = "France"
val gov = Republic
]

I kind of come up with a solution:
defpackage labyrinth :
import core
import collections
deftype Countries <: HashTable<String,?>
defn print-deep ( xs : HashTable<String,?> ) :
for x in xs do :
val k = key(x)
val v = value(x)
print("%_ => " % [k])
match(v):
(v: HashTable<String,?>) : (print("[ "), print-deep(v), println("]"))
(v) : ( print(v), print(", ") )
defn to-hashtable ( t : HashTable<String,?>, kvs : Tuple<KeyValue<?,?>> ) -> False :
for kv in kvs do :
val k = key(kv)
val v = value(kv)
match(k) :
(k : String) :
if v is Tuple<?> :
var d : HashTable<String,?> = HashTable<String,?>()
to-hashtable(d, v)
set(t, k, d)
else :
t[k] = v
defn to-countries ( kvs : Tuple<KeyValue<String,?>> ) -> HashTable<String,?> :
val t : HashTable<String,?> = HashTable<String,?>()
to-hashtable(t, kvs)
t
defn test () -> HashTable<String,?> :
val c : Tuple<KeyValue<String,?>> =
[ "us" => ["name" => "us", "govern" => "good"]
"canada" => [ "name" => "Canada" "govern" => "good" ]
"uk" => [ "name" => "UK" "govern" => "good" "recruit" => 3 ]
"spain" => [ "name" => "Spain" "govern" => "good" "schengen" => true "recruit" => 2 ]
"france" => [ "name" => "France" "govern" => "good" "schengen" => true "recruit" => 2 ]
"italy" => [ "name" => "italy" "govern" => "good" "schengen" => true ]
]
val countries = to-countries(c)
countries
val board = test()
print-deep(board)

Related

convert and format list of dictionaries in Pyton by values

Hope someone can help me with dictionary:
data = [
{
"firstname" : "David",
"lastname" : "Brown",
"class" : [ "economy" ]
},
{
"firstname" : "Alina",
"lastname" : "Hoper",
"class" : [ "economy", "business" ]
},
{
"firstname" : "Bill",
"lastname" : "Flow",
"class" : [ "business" ]
},
{
"firstname" : "James",
"lastname" : "Frank",
"class" : [ "economy" ]
}
]
As output, I need to see who bought economy and who bought business class:
With sorting = ascending by class and inside class by firstname. So business comes first, then economy. and Alina comes first in both classes, because she bought both classes.
business: Alina Hoper, Bill Flow, ...
economy: Alina Hoper, David Brown, ...
I tried to write function, but can not understand right now where to start sorting and how to convert dictinory and group data by class:
def analyze(customers_data):
data = ""
data += "{} {} \n".format(customers_data["firstname"], customers_data["lastname"])
data += "{} \n".format(customers_data["aff"])
return data
for d in orders:
print(analyze(d))
Hope someone can help
You should first get the list of economy/business customers and then print the names from those lists:
data = [
{
"firstname" : "David",
"lastname" : "Brown",
"class" : [ "economy" ]
},
{
"firstname" : "Alina",
"lastname" : "Hoper",
"class" : [ "economy", "business" ]
},
{
"firstname" : "Bill",
"lastname" : "Flow",
"class" : [ "business" ]
},
{
"firstname" : "James",
"lastname" : "Frank",
"class" : [ "economy" ]
}
]
business_custom = [dic for dic in data if "business" in dic["class"]]
eco_custom = [dic for dic in data if "economy" in dic["class"]]
business_names = ', '.join(sorted([f"{dic['firstname']} {dic['lastname']}" for dic in business_custom]))
economy_names = ', '.join(sorted([f"{dic['firstname']} {dic['lastname']}" for dic in eco_custom]))
print(f"business: {business_names}")
print(f"economy: {economy_names}")
Output:
business : Alina Hoper, Bill Flow
economy : Alina Hoper, David Brown, James Frank
Edit: if you don't know the class names beforehand you can create a set of classes:
class_set = set()
for dic in data:
for c in dic['class']:
class_set.add(c)
for c in sorted(list(class_set)):
custom = [dic for dic in data if c in dic["class"]]
names = ', '.join(sorted([f"{dic['firstname']} {dic['lastname']}" for dic in custom]))
print(f"{c}: {names}")

Parsing a json output with dictionary values

Hello my JSON file looks like this :
{
'hits' : 3,
'results' : [{
'part' : {
'id' : '123',
'name' : 'to go',
'manu' :{
'name' :'xyz'
}
}
}]
}
how do i get :
hits : 3
results_id : 123
results_name : to go
manu_name : xyz
Looking to write a python key-value pair for a loop... Not getting please advise.
My keys are hits and results
Try this
dout = {}
#din = json.load("file.json") #read from json
din = {'hits' : 3, 'results' : [{'part' : {'id' : '123', 'name' : 'to go', 'manu' :{ 'name' :'xyz'}}}]}
for part in din['results']:
for p, data in part.items():
dout['hits'] = din['hits']
dout['results_id'] = data['id']
dout['results_name'] = data['name']
dout['manu_name'] = data['manu']['name']
print(dout)
First of all, update the string as follows to make it valid JSON:
{"hits" : 3, "results" : [{"part" : {"id" : "123", "name" : "to go", "manu" :{ "name" :"xyz"}}}]}
Assuming that's the content of your text file, proceed as follows:
import json
data = {}
with open("stackOverflow/brooklinite81.json") as json_file:
data = json.load(json_file)
data
which renders:
{'hits': 3, 'results': [{'part': {'id': '123', 'name': 'to go', 'manu': {'name': 'xyz'}}}]}
Now, you can create a dictionary the way you wanted it as follows:
res = {'hits' :data['hits'],
'results_id': data['results'][0]['part']['id'],
'manu': data['results'][0]['part']['manu']['name']
}
res
which renders as:
{'hits': 3, 'results_id': '123', 'manu': 'xyz'}

How to Parsing Nested Json with Python?

I am trying to parse this nested JSON file and I am running having trouble getting every element I need.
Here is the json example:
{
"sensor-time" : {
"timezone" : "New_York",
"time" : "2020-07-15T12:45:02-04:00"
},
"status" : {
"code" : "OK"
},
"content" : {
"element" : [ {
"element-id" : 0,
"element-name" : "Line 0",
"sensor-type" : "SINGLE_SENSOR",
"data-type" : "LINE",
"from" : "2020-07-15T12:30:00-04:00",
"to" : "2020-07-15T12:45:00-04:00",
"resolution" : "FIVE_MINUTES",
"measurement" : [ {
"from" : "2020-07-15T12:30:00-04:00",
"to" : "2020-07-15T12:35:00-04:00",
"value" : [ {
"value" : 1,
"label" : "fw"
}, {
"value" : 2,
"label" : "bw"
} ]
}, {
"from" : "2020-07-15T12:35:00-04:00",
"to" : "2020-07-15T12:40:00-04:00",
"value" : [ {
"value" : 3,
"label" : "fw"
}, {
"value" : 4,
"label" : "bw"
} ]
}, {
"from" : "2020-07-15T12:40:00-04:00",
"to" : "2020-07-15T12:45:00-04:00",
"value" : [ {
"value" : 5,
"label" : "fw"
}, {
"value" : 6,
"label" : "bw"
} ]
} ]
}, {
"element-id" : 1,
"element-name" : "Test Line",
"sensor-type" : "SINGLE_SENSOR",
"data-type" : "LINE",
"from" : "2020-07-15T12:30:00-04:00",
"to" : "2020-07-15T12:45:00-04:00",
"resolution" : "FIVE_MINUTES",
"measurement" : [ {
"from" : "2020-07-15T12:30:00-04:00",
"to" : "2020-07-15T12:35:00-04:00",
"value" : [ {
"value" : 7,
"label" : "fw"
}, {
"value" : 8,
"label" : "bw"
} ]
}, {
"from" : "2020-07-15T12:35:00-04:00",
"to" : "2020-07-15T12:40:00-04:00",
"value" : [ {
"value" : 9,
"label" : "fw"
}, {
"value" : 10,
"label" : "bw"
} ]
}, {
"from" : "2020-07-15T12:40:00-04:00",
"to" : "2020-07-15T12:45:00-04:00",
"value" : [ {
"value" : 11,
"label" : "fw"
}, {
"value" : 12,
"label" : "bw"
} ]
} ]
} ]
},
"sensor-info" : {
"serial-number" : "D7:40:1:7F:4A:72",
"ip-address" : "192.168.130.44",
"name" : "DemoNew",
"group" : "Internal Test Devices",
"device-type" : "PC2"
}
}
What I am trying to get is measurement data for each element name. Please see the example below:
Here is what I tried:
data = {} # element-name ↦ direction ↦ {readings ↦ (timestamp × value) list, meta ↦ name ↦ value}
for element in json_data['content']['element']:
element_name = element['element-name']
element_data = {}
# collect
for measurement in element['measurement']:
dt = datetime.strptime(measurement['to'][:-3]+'00', '%Y-%m-%dT%H:%M:%S%z')
t = mktime(dt.timetuple())
for pair in measurement['value']:
direction = pair['label']
value = pair['value']
if not direction in element_data: element_data[direction] = []
element_data[direction].append( (t, value) )
# insert
metadata = {}
for key in element:
if not key in ['measurement', 'from', 'to']:
metadata[key] = element[key]
data[element_name] = {}
for direction in element_data:
data[element_name][direction] = {'readings': element_data[direction], 'meta': metadata}
camera_metadata = {}
for key in json_data:
if not key in ['content']:
camera_metadata[key] = json_data[key]
And her is what I get as result:
{'Line 0': {'fw': {'readings': [(1594830900.0, 1),
(1594831200.0, 3),
(1594831500.0, 5)],
'meta': {'element-id': 0,
'element-name': 'Line 0',
'sensor-type': 'SINGLE_SENSOR',
'data-type': 'LINE',
'resolution': 'FIVE_MINUTES'}},
'bw': {'readings': [(1594830900.0, 2), (1594831200.0, 4), (1594831500.0, 6)],
'meta': {'element-id': 0,
'element-name': 'Line 0',
'sensor-type': 'SINGLE_SENSOR',
'data-type': 'LINE',
'resolution': 'FIVE_MINUTES'}}},
'GP Test CL.01': {'fw': {'readings': [(1594830900.0, 7),
(1594831200.0, 9),
(1594831500.0, 11)],
'meta': {'element-id': 1,
'element-name': 'GP Test CL.01',
'sensor-type': 'SINGLE_SENSOR',
'data-type': 'LINE',
'resolution': 'FIVE_MINUTES'}},
'bw': {'readings': [(1594830900.0, 8),
(1594831200.0, 10),
(1594831500.0, 12)],
'meta': {'element-id': 1,
'element-name': 'GP Test CL.01',
'sensor-type': 'SINGLE_SENSOR',
'data-type': 'LINE',
'resolution': 'FIVE_MINUTES'}}}}
What do I need to adjust to get the result to look as a screenshot example above?
You were trying to get the information one part at a time. But to parse your json to a dataframe you need to do it all in a nested loop.
result = []
for element in json_data['content']['element']:
for m in element['measurement']:
data = {}
for val in m['value']:
data['SERIAL_NUMBER'] = json_data['sensor-info']['serial-number']
data['IP'] = json_data['sensor-info']['ip-address']
data['name'] = json_data['sensor-info']['name']
data['Group'] = json_data['sensor-info']['group']
data['Device Type'] = json_data['sensor-info']['device-type']
data['element-id'] = element['element-id']
data['Line name'] = element['element-name']
data['From time'] = m['from']
data['to time'] = m['to']
data[val['label']] = val['value']
result.append(data)
df = pd.DataFrame(result)
Output:
SERIAL_NUMBER IP name Group \
0 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
1 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
2 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
3 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
4 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
5 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
Device Type element-id Line name From time \
0 PC2 0 Line 0 2020-07-15T12:30:00-04:00
1 PC2 0 Line 0 2020-07-15T12:35:00-04:00
2 PC2 0 Line 0 2020-07-15T12:40:00-04:00
3 PC2 1 Test Line 2020-07-15T12:30:00-04:00
4 PC2 1 Test Line 2020-07-15T12:35:00-04:00
5 PC2 1 Test Line 2020-07-15T12:40:00-04:00
to time fw bw
0 2020-07-15T12:35:00-04:00 1 2
1 2020-07-15T12:40:00-04:00 3 4
2 2020-07-15T12:45:00-04:00 5 6
3 2020-07-15T12:35:00-04:00 7 8
4 2020-07-15T12:40:00-04:00 9 10
5 2020-07-15T12:45:00-04:00 11 12
As you can see I didn't figure out your time format. Also, I think you switched "Group" and "Device Type".

Querying MongoDB collection by array intersection

What I have:
An array of strings that I wish to query with. ['a', 'b', 'c']
Data I'm querying against:
A collection of objects of type foo, all with a bar field. The bar field is an array of strings, the same type as the one I'm querying with, potentially with some of the same elements.
foo1 = { bar: ['a'] }
foo2 = { bar: ['d'] }
foo3 = { bar: ['a', 'c']}
What I need:
A query that returns all foo objects whose entire bar array is contained within the query array. In the example above, I'd want foo1 and foo3 to come back
using aggregation
you might need to use $setIsSubset in aggregate pipeline
db.col.aggregate(
[
{$project : { bar : 1 , isSubset: { $setIsSubset : [ "$bar" , ['a','b','c'] ] }}},
{$match : { isSubset : true}}
]
)
collection
> db.col.find()
{ "_id" : ObjectId("5a6420d984eeec7b0b2f767b"), "bar" : [ "a" ] }
{ "_id" : ObjectId("5a6420d984eeec7b0b2f767c"), "bar" : [ "a", "c" ] }
{ "_id" : ObjectId("5a6420d984eeec7b0b2f767d"), "bar" : [ "d" ] }
aggregate
> db.col.aggregate([{$project : { bar : 1 , isSubset: { $setIsSubset : [ "$bar" , ['a','b','c'] ] }}}, {$match : {isSubset : true}}])
{ "_id" : ObjectId("5a6420d984eeec7b0b2f767b"), "bar" : [ "a" ], "isSubset" : true }
{ "_id" : ObjectId("5a6420d984eeec7b0b2f767c"), "bar" : [ "a", "c" ], "isSubset" : true }
>
EDIT
using find with $expr
db.col.find({$expr : { $setIsSubset : [ "$bar" , ['a','b','c'] ] }})
result
> db.col.find({$expr : { $setIsSubset : [ "$bar" , ['a','b','c'] ] }})
{ "_id" : ObjectId("5a6420d984eeec7b0b2f767b"), "bar" : [ "a" ] }
{ "_id" : ObjectId("5a6420d984eeec7b0b2f767c"), "bar" : [ "a", "c" ] }
>

ReactiveMongo aggregate function

I'm trying to convert the following mongo query to Reactive Mongo Equivalent ( JSON )
db.media.aggregate( {$group : { "_id" : "$createdBy", "count" : { $sum : 1 }}}, {$sort : {"count" : -1}}, {$limit : 10} )
What I have come up with is this, but can't get around it.
override def getMediasCountByHandle(db:reactivemongo.api.DefaultDB): Future[JsObject] = {
val commandDoc = Json.obj(
"aggregate" -> "media", // we aggregate on collection orders
"pipeline" -> List(
Json.obj(
"$group" -> Json.obj(
"_id" -> "$createdBy",
"count" -> Json.obj("$sum" -> 1))),
Json.obj("$sort" -> Json.obj("total" -> -1)),
Json.obj("$limit" -> 10)
)
)
val runner = Command.run(JSONSerializationPack)
runner.apply(db, runner.rawCommand(commandDoc)).one[JsObject]
}
Please help

Resources