I have two json one is parent and another is child, I want to merge both. Please help. See the example below
Two tables are as below.
var parent = [{id : 1, name : 'India'},
{id : 2, name : 'USA'},
{id : 3, name : 'Japan'},
{id : 4, name : 'UK'}]
var child = [{id: 1, parentId: 1, city : 'Ahmedabad', population:100},
{id: 2, parentId: 1, city : 'Mumbai', population:200},
{id: 3, parentId: 2, city : 'NewYork', population:300},
{id: 4, parentId: 2, city : 'Chicago', population:400},
{id: 5, parentId: 3, city : 'Tokyo', population:500}]
I want to result like below
var result =
[{id : 1, name : 'India', city:[{name : 'Ahmedabad', population:100},{name : 'Mumbai', population:200}]},
{id : 2, name : 'USA', city:[{name : 'NewYork', population:300},{name : 'Chicago', population:400}]},
{id : 3, name : 'Japan', city:[{name : 'Tokyo', population:500}]},
{id : 4, name : 'UK', city:[]}
]
var result =JSON.parse(JSON.stringify(parent));
result.map(function (d, i) {
d.city = (child.filter(function (d1) {
if (d1.parentId == d.id) {
return d1;
}
})).map(function(d2){
return {city:d2.city,population:d2.population}
});
})
Now result variable contains your expected result.
Related
I have the following Structures:
MyDict = {'Team1': {'John' : 0, 'Mary' : 0},
'Team2': {'Steven' : 0, 'Gwen' : 0,'Fred' : 0},
'Team3': {'Louise' : 0, 'Alice' : 0, 'Jim' : 0},
'Team4': {'Paul' : 0, 'George' : 0},
'Team5': {'Tracey' : 0}}
MyList = ['John', 'Gwen', 'Jim', 'George', 'John', 'Gwen', 'Alice', 'Frank', 'John', 'Gwen', 'Frank', 'John', 'Gwen', 'Tracey']
The first is a nested structure with members in specific teams.
For each occurrence of the member in MyList, I need that value updated in MyDict.
Once complete, I need the printed total for each team.
So after iterating through MyList, the nested structure will look like this:
MyDict = {'Team1': {'John' : 4, 'Mary' : 0},
'Team2': {'Steven' : 0, 'Gwen' : 4,'Fred' : 0},
'Team3': {'Louise' : 0, 'Alice' : 1, 'Jim' : 1},
'Team4': {'Paul' : 0, 'George' : 1},
'Team5': {'Tracey' : 1}}
And the printed totals will look like this:
Team1: 4
Team2: 4
Team3: 2
Team4: 1
Team5: 1
use code below:
MyDict = {'Team1': {'John' : 0, 'Mary' : 0},
'Team2': {'Steven' : 0, 'Gwen' : 0,'Fred' : 0},
'Team3': {'Louise' : 0, 'Alice' : 0, 'Jim' : 0},
'Team4': {'Paul' : 0, 'George' : 0},
'Team5': {'Tracey' : 0}}
MyList = ['John', 'Gwen', 'Jim', 'George', 'John', 'Gwen', 'Alice', 'Frank', 'John', 'Gwen', 'Frank', 'John', 'Gwen', 'Tracey']
for name in MyList:
for key,val in MyDict.items():
if name in val:
val[name]+=1
for key,val in MyDict.items():
print(key,':',sum(val.values()))
have fun :)
I have a DataFrame that has a website, categories, and keywords for that website.
Url | categories | keywords
Espn | [sport, nba, nfl] | [half, touchdown, referee, player, goal]
Tmz | [entertainment, sport] | [gossip, celebrity, player]
Goal [ [sport, premier_league, champions_league] | [football, goal, stadium, player, referee]
Which can be created using this code:
data = [{ 'Url': 'ESPN', 'categories': ['sport', 'nba', 'nfl'] ,
'keywords': ["half", "touchdown", "referee", "player", "goal"] },
{ 'Url': 'TMZ', 'categories': ["entertainment", "sport"] ,
'keywords': ["gossip", "celebrity", "player"] },
{ 'Url': 'Goal', 'categories': ["sport", "premier_league", "champions_league"] ,
'keywords': ["football", "goal", "stadium", "player", "referee"]},
]
df =pd.DataFrame(data)
For all the word in the keywords column, I want to get the frequency of categories associated with it. The results might look like this:
{half: {sport: 1, nba: 1, nfl: 1}, touchdown : {sport: 1, nba: 1,
nfl: 1}, referee: {sport: 2, nba: 1, nfl: 1, premier_league: 1,
champions_league:1 }, player: {sport: 3, nba: 1, nfl: 1,
premier_league: 1, champions_league:1 }, gossip: {sport:1,
entertainment:1}, celebrity: {sport:1, entertainment:1}, goal:
{sport:2, premier_league:1, champions_league:1, nba: 1, nfl: 1},
stadium:{sport:1, premier_league:1, champions_league:1} }
Since the columns contain lists, you can explode them to repeat a row once for each element per list:
result = (
df.explode("keywords")
.explode("categories")
.groupby(["keywords", "categories"])
.size()
)
Hello my JSON file looks like this :
{
'hits' : 3,
'results' : [{
'part' : {
'id' : '123',
'name' : 'to go',
'manu' :{
'name' :'xyz'
}
}
}]
}
how do i get :
hits : 3
results_id : 123
results_name : to go
manu_name : xyz
Looking to write a python key-value pair for a loop... Not getting please advise.
My keys are hits and results
Try this
dout = {}
#din = json.load("file.json") #read from json
din = {'hits' : 3, 'results' : [{'part' : {'id' : '123', 'name' : 'to go', 'manu' :{ 'name' :'xyz'}}}]}
for part in din['results']:
for p, data in part.items():
dout['hits'] = din['hits']
dout['results_id'] = data['id']
dout['results_name'] = data['name']
dout['manu_name'] = data['manu']['name']
print(dout)
First of all, update the string as follows to make it valid JSON:
{"hits" : 3, "results" : [{"part" : {"id" : "123", "name" : "to go", "manu" :{ "name" :"xyz"}}}]}
Assuming that's the content of your text file, proceed as follows:
import json
data = {}
with open("stackOverflow/brooklinite81.json") as json_file:
data = json.load(json_file)
data
which renders:
{'hits': 3, 'results': [{'part': {'id': '123', 'name': 'to go', 'manu': {'name': 'xyz'}}}]}
Now, you can create a dictionary the way you wanted it as follows:
res = {'hits' :data['hits'],
'results_id': data['results'][0]['part']['id'],
'manu': data['results'][0]['part']['manu']['name']
}
res
which renders as:
{'hits': 3, 'results_id': '123', 'manu': 'xyz'}
I am trying to parse this nested JSON file and I am running having trouble getting every element I need.
Here is the json example:
{
"sensor-time" : {
"timezone" : "New_York",
"time" : "2020-07-15T12:45:02-04:00"
},
"status" : {
"code" : "OK"
},
"content" : {
"element" : [ {
"element-id" : 0,
"element-name" : "Line 0",
"sensor-type" : "SINGLE_SENSOR",
"data-type" : "LINE",
"from" : "2020-07-15T12:30:00-04:00",
"to" : "2020-07-15T12:45:00-04:00",
"resolution" : "FIVE_MINUTES",
"measurement" : [ {
"from" : "2020-07-15T12:30:00-04:00",
"to" : "2020-07-15T12:35:00-04:00",
"value" : [ {
"value" : 1,
"label" : "fw"
}, {
"value" : 2,
"label" : "bw"
} ]
}, {
"from" : "2020-07-15T12:35:00-04:00",
"to" : "2020-07-15T12:40:00-04:00",
"value" : [ {
"value" : 3,
"label" : "fw"
}, {
"value" : 4,
"label" : "bw"
} ]
}, {
"from" : "2020-07-15T12:40:00-04:00",
"to" : "2020-07-15T12:45:00-04:00",
"value" : [ {
"value" : 5,
"label" : "fw"
}, {
"value" : 6,
"label" : "bw"
} ]
} ]
}, {
"element-id" : 1,
"element-name" : "Test Line",
"sensor-type" : "SINGLE_SENSOR",
"data-type" : "LINE",
"from" : "2020-07-15T12:30:00-04:00",
"to" : "2020-07-15T12:45:00-04:00",
"resolution" : "FIVE_MINUTES",
"measurement" : [ {
"from" : "2020-07-15T12:30:00-04:00",
"to" : "2020-07-15T12:35:00-04:00",
"value" : [ {
"value" : 7,
"label" : "fw"
}, {
"value" : 8,
"label" : "bw"
} ]
}, {
"from" : "2020-07-15T12:35:00-04:00",
"to" : "2020-07-15T12:40:00-04:00",
"value" : [ {
"value" : 9,
"label" : "fw"
}, {
"value" : 10,
"label" : "bw"
} ]
}, {
"from" : "2020-07-15T12:40:00-04:00",
"to" : "2020-07-15T12:45:00-04:00",
"value" : [ {
"value" : 11,
"label" : "fw"
}, {
"value" : 12,
"label" : "bw"
} ]
} ]
} ]
},
"sensor-info" : {
"serial-number" : "D7:40:1:7F:4A:72",
"ip-address" : "192.168.130.44",
"name" : "DemoNew",
"group" : "Internal Test Devices",
"device-type" : "PC2"
}
}
What I am trying to get is measurement data for each element name. Please see the example below:
Here is what I tried:
data = {} # element-name ↦ direction ↦ {readings ↦ (timestamp × value) list, meta ↦ name ↦ value}
for element in json_data['content']['element']:
element_name = element['element-name']
element_data = {}
# collect
for measurement in element['measurement']:
dt = datetime.strptime(measurement['to'][:-3]+'00', '%Y-%m-%dT%H:%M:%S%z')
t = mktime(dt.timetuple())
for pair in measurement['value']:
direction = pair['label']
value = pair['value']
if not direction in element_data: element_data[direction] = []
element_data[direction].append( (t, value) )
# insert
metadata = {}
for key in element:
if not key in ['measurement', 'from', 'to']:
metadata[key] = element[key]
data[element_name] = {}
for direction in element_data:
data[element_name][direction] = {'readings': element_data[direction], 'meta': metadata}
camera_metadata = {}
for key in json_data:
if not key in ['content']:
camera_metadata[key] = json_data[key]
And her is what I get as result:
{'Line 0': {'fw': {'readings': [(1594830900.0, 1),
(1594831200.0, 3),
(1594831500.0, 5)],
'meta': {'element-id': 0,
'element-name': 'Line 0',
'sensor-type': 'SINGLE_SENSOR',
'data-type': 'LINE',
'resolution': 'FIVE_MINUTES'}},
'bw': {'readings': [(1594830900.0, 2), (1594831200.0, 4), (1594831500.0, 6)],
'meta': {'element-id': 0,
'element-name': 'Line 0',
'sensor-type': 'SINGLE_SENSOR',
'data-type': 'LINE',
'resolution': 'FIVE_MINUTES'}}},
'GP Test CL.01': {'fw': {'readings': [(1594830900.0, 7),
(1594831200.0, 9),
(1594831500.0, 11)],
'meta': {'element-id': 1,
'element-name': 'GP Test CL.01',
'sensor-type': 'SINGLE_SENSOR',
'data-type': 'LINE',
'resolution': 'FIVE_MINUTES'}},
'bw': {'readings': [(1594830900.0, 8),
(1594831200.0, 10),
(1594831500.0, 12)],
'meta': {'element-id': 1,
'element-name': 'GP Test CL.01',
'sensor-type': 'SINGLE_SENSOR',
'data-type': 'LINE',
'resolution': 'FIVE_MINUTES'}}}}
What do I need to adjust to get the result to look as a screenshot example above?
You were trying to get the information one part at a time. But to parse your json to a dataframe you need to do it all in a nested loop.
result = []
for element in json_data['content']['element']:
for m in element['measurement']:
data = {}
for val in m['value']:
data['SERIAL_NUMBER'] = json_data['sensor-info']['serial-number']
data['IP'] = json_data['sensor-info']['ip-address']
data['name'] = json_data['sensor-info']['name']
data['Group'] = json_data['sensor-info']['group']
data['Device Type'] = json_data['sensor-info']['device-type']
data['element-id'] = element['element-id']
data['Line name'] = element['element-name']
data['From time'] = m['from']
data['to time'] = m['to']
data[val['label']] = val['value']
result.append(data)
df = pd.DataFrame(result)
Output:
SERIAL_NUMBER IP name Group \
0 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
1 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
2 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
3 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
4 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
5 D7:40:1:7F:4A:72 192.168.130.44 DemoNew Internal Test Devices
Device Type element-id Line name From time \
0 PC2 0 Line 0 2020-07-15T12:30:00-04:00
1 PC2 0 Line 0 2020-07-15T12:35:00-04:00
2 PC2 0 Line 0 2020-07-15T12:40:00-04:00
3 PC2 1 Test Line 2020-07-15T12:30:00-04:00
4 PC2 1 Test Line 2020-07-15T12:35:00-04:00
5 PC2 1 Test Line 2020-07-15T12:40:00-04:00
to time fw bw
0 2020-07-15T12:35:00-04:00 1 2
1 2020-07-15T12:40:00-04:00 3 4
2 2020-07-15T12:45:00-04:00 5 6
3 2020-07-15T12:35:00-04:00 7 8
4 2020-07-15T12:40:00-04:00 9 10
5 2020-07-15T12:45:00-04:00 11 12
As you can see I didn't figure out your time format. Also, I think you switched "Group" and "Device Type".
Below if a document from my collection of over 20,000,000 documents.
I need to find documents by a particular zip, out of these documents I need to select one record from each postal address (ADDR, CITY, STATE, ZIP, APT) and which has a age value of 18 or higher.
The results need to be limited to a number as well which is entered by the end-user.
{
"_id" : ObjectId("55e86e98f493590878bb45d7"),
"RecordID" : 84096380,
"FN" : "Michael",
"MI" : "",
"LN" : "Horn",
"NAME_PRE" : "MR",
"ADDR" : "160 Yankee Camp Rd",
"CITY" : "Telford",
"ST" : "TN",
"ZIP" : 37690,
"APT" : "",
"Z4" : 2200,
"DPC" : 605,
"CAR_RTE" : "R001",
"WALK_SEQ" : 228,
"LOT" : "0136A",
"FIPS_ST" : 47,
"FIPS_CTY" : 179,
"LATITUDE" : 36.292787,
"LONGITUDE" : -82.568171,
"ADDR_TYP" : 1,
"MSA" : 3660,
"CBSA" : 27740,
"ADDR_LINE" : 3,
"DMA_SUPPR" : "",
"GEO_MATCH" : 1,
"CENS_TRACT" : 61900,
"CENS_BLK_GRP" : 1,
"CENS_BLK" : 17,
"CENS_MED_HOME_VALUE" : 953,
"CENS_MED_HH_INCOME" : 304,
"CRA" : "",
"Z4_TYP" : "S",
"DSF_IND" : 1,
"DPD_IND" : "N",
"PHONE_FLAG" : "Y",
"PHONE" : NumberLong("4237730233"),
"TIME_ZN" : "E",
"GENDER" : "M",
"NEW_TO_BLD" : "",
"SOURCES" : 19,
"BASE_VER_DT" : 20101,
"COMP_ID" : NumberLong("3769001836"),
"IND_ID" : 1,
"INF_HH_RANK" : 1,
"HOME_OWNR_SRC" : "V",
"DOB_YR" : 1975,
"DOB_MON" : 7,
"DOB_DAY" : 10,
"EXACT_AGE" : 39,
"AGE" : 39,
"HH_INCOME" : "D"
}
if you are using mongoose, we can chain the operations by dot(.) operator. Since i see all your needs is conditional here is the example -
Person.
find({
ZIP: "37690",
ADDR : "",
STATE : "", //so on
AGE: { $gt: 18 }
}).
limit(10).
exec(callback);
more info - http://mongoosejs.com/docs/queries.html
You need to use aggregate operation.
var pipeline = [
{
$match: {ZIP: 37690, AGE: {$gt: 18}}
}, {
$group: {
_id: {ADDR: '$ADDR', CITY: '$CITY', STATE: '$STATE', ZIP: '$ZIP', APT: '$APT'},
PHONE: {$first: '$PHONE'}
}
},
{$limit: 10}
];
db.mycoll.aggregate(pipeline)
enhance the above to project whatever fields you require in results
I think This query will solve your problem.
Person.find({
ZIP: "37690",
AGE: { $gt: 18 }
}).
limit(50).
exec(callback);