I have a list of dictionaries that looks like this:
[
{
"name": "hello",
"value": "world"
},
{
"name": "foo",
"value": "bar"
}
]
What's the pythonic way of fetching the dictionary where name = "foo" from a list of dictionaries?
Assuming your list of dicts is stored as variable l, you can use next() with a generator expression like this, which returns the first dict whose name key is foo:
next(d for d in l if d['name'] == 'foo')
This will otherwise raise StopIteration if there is no dict in l with a name key that equals foo.
Try this simple example.
data = [
{
"name": "hello",
"value": "world"
},
{
"name": "foo",
"value": "bar"
}
]
for item in data:
if item['name'] == 'foo':
print(item)
Output:
{'name': 'foo', 'value': 'bar'}
Related
I want to as the title says create a group of attributes a, b and c, such that any combination can be supplied as long as one is given. I have managed to achieve the functionality but it is not reflected in the schema which is what I can't manage to do.
from pydantic import BaseModel, root_validator
class Foo(BaseModel):
a: str | None = None
b: str | None = None
c: str | None = None
#root_validator
def check_at_least_one_given(cls, values):
if not any((values.get('a'), values.get('b'), values.get('c'))):
raise ValueError("At least of a, b, or c must be given")
return values
# Doesn't have required fields
print(Foo.schema_json(indent=2))
{
"title": "Foo",
"type": "object",
"properties": {
"a": {
"title": "A",
"type": "string"
},
"b": {
"title": "B",
"type": "string"
},
"c": {
"title": "C",
"type": "string"
}
}
}
# No error
print(Foo(a="1"))
>>> a='1' b=None c=None
print(Foo(b="2"))
>>> a=None b='2' c=None
print(Foo(c="3"))
>>> a=None b=None c='3'
print(Foo(a="1", b="2"))
>>> a='1' b='2' c=None
print(Foo(a="1", c="3"))
>>> a='1' b=None c='3'
print(Foo(b="2", c="3"))
>>> a=None b='2' c='3'
print(Foo(a="1", b="2", c="3"))
>>> a='1' b='2' c='3'
# Invalid
Foo()
>>> Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pydantic\main.py", line 342, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Foo
__root__
At least of a, b, or c must be given (type=value_error)
I want the schema to output something like
{
"title": "Foo",
"type": "object",
"properties": {
"a": {
"title": "A",
"type": "string"
},
"b": {
"title": "B",
"type": "string"
},
"c": {
"title": "C",
"type": "string"
}
},
"required": [
["a", "b", "c"]
]
}
or something else that (probably) more clearly expresses the intent of at least one of these is required.
Is this possible and if so how is it done?
As far as I can tell, Pydantic does not have a built in mechanism for this. And the validation logic you provide in a custom validator will never find its way into the JSON schema. You could try and search through and if unsuccessful post a feature request for something like this in their issue tracker.
The JSON schema core specification defines the anyOf keyword, which takes subschemas to validate against. This allows specifying the required keyword once for each of your fields in its own subschema. See this answer for details and an example.
In the Pydantic Config you can utilize schema_extra to extend the auto-generated schema. Here is an example of how you can write a corresponding workaround:
from typing import Any
from pydantic import BaseModel, root_validator
class Foo(BaseModel):
a: str | None = None
b: str | None = None
c: str | None = None
#root_validator
def check_at_least_one_given(cls, values: dict[str, Any]) -> dict[str, Any]:
if all(
(v is None for v in (values.get("a"), values.get("b"), values.get("c")))
):
raise ValueError("Any one of `a`, `b`, or `c` must be given")
return values
class Config:
#staticmethod
def schema_extra(schema: dict[str, Any]) -> None:
assert "anyOf" not in schema, "Oops! What now?"
schema["anyOf"] = [
{"required": ["a"]},
{"required": ["b"]},
{"required": ["c"]},
]
for prop in schema.get("properties", {}).values():
prop.pop("title", None)
if __name__ == "__main__":
print(Foo.schema_json(indent=2))
Output:
{
"title": "Foo",
"type": "object",
"properties": {
"a": {
"type": "string"
},
"b": {
"type": "string"
},
"c": {
"type": "string"
}
},
"anyOf": [
{
"required": [
"a"
]
},
{
"required": [
"b"
]
},
{
"required": [
"c"
]
}
]
}
This conforms to the specs and expresses your custom validation.
But note that I put in that assert to indicate that I have no strong basis to assume that the automatic schema will not provide its own anyOf key at some point, which would greatly complicate things. Consider this an unstable solution.
Side note:
Be careful with the any check in your validator. An empty string is "falsy" just like None, which might lead to unexpected results, depending on whether you want to consider empty strings to be valid values in this context or not. any(v for v in ("", 0, False, None)) is False.
I adjusted your validator in my code to explicitly check against None for this reason.
I have below unstructured dictionary list which contains values of other keys in a list .
I am not sure if the question i ask is strange. this is the actual dictionary payload that we receive from source which not aligned with respective entry
[
{
"dsply_nm": [
"test test",
"test test",
"",
""
],
"start_dt": [
"2021-04-21T00:01:00-04:00",
"2021-04-21T00:01:00-04:00",
"2021-04-21T00:01:00-04:00",
"2021-04-21T00:01:00-04:00"
],
"exp_dt": [
"2022-04-21T00:01:00-04:00",
"2022-04-21T00:01:00-04:00",
"2022-04-21T00:01:00-04:00",
"2022-04-21T00:01:00-04:00"
],
"hrs_pwr": [
"14",
"12",
"13",
"15"
],
"make_nm": "test",
"model_nm": "test",
"my_yr": "1980"
}
]
"the length of list cannot not be expected and it could be more than 4 sometimes or less in some keys"
#Expected:
i need to check if the above dictionary are in proper structure or not and based on that it should return the proper dictionary list associate with each item
for eg:
def get_dict_list(items):
if type(items == not structure)
result = get_associated_dict_items_mapped
return result
else:
return items
#Final result
expected_dict_list=
[{"dsply_nm":"test test","start_dt":"2021-04-21T00:01:00-04:00","exp_dt":"2022-04-21T00:01:00-04:00","hrs_pwr":"14"},
{"dsply_nm":"test test","start_dt":"2021-04-21T00:01:00-04:00","exp_dt":"2022-04-21T00:01:00-04:00","hrs_pwr":"12","make_nm": "test",model_nm": "test","my_yr": "1980"},
{"dsply_nm":"","start_dt":"2021-04-21T00:01:00-04:00","exp_dt":"2022-04-21T00:01:00-04:00","hrs_pwr":"13"},
{"dsply_nm":"","start_dt":"2021-04-21T00:01:00-04:00","exp_dt":"2022-04-21T00:01:00-04:00","hrs_pwr":"15"}
]
in above dictionary payload, below part is associated with the second dictionary items and have to map respectively
"make_nm": "test",
"model_nm": "test",
"my_yr": "1980"
}
Can anyone help on this?
Thanks
Since customer details is a list
dict(zip(customer_details[0], list(customer_details.values[0]())))
this yields:
{'insured_details': ['asset', 'asset', 'asset'],
'id': ['213', '214', '233'],
'dept': ['account', 'sales', 'market'],
'salary': ['12', '13', '14']}
I think a couple of list comprehensions will get you going. If you would like me to unwind them into more traditional for loops, just let me know.
import json
def get_dict_list(item):
first_value = list(item.values())[0]
if not isinstance(first_value, list):
return [item]
return [{key: item[key][i] for key in item.keys()} for i in range(len(first_value))]
cutomer_details = [
{
"insured_details": "asset",
"id": "xxx",
"dept": "account",
"salary": "12"
},
{
"insured_details": ["asset", "asset", "asset"],
"id":["213","214","233"],
"dept":["account","sales","market"],
"salary":["12","13","14"]
}
]
cutomer_details_cleaned = []
for detail in cutomer_details:
cutomer_details_cleaned.extend(get_dict_list(detail))
print(json.dumps(cutomer_details_cleaned, indent=4))
That should give you:
[
{
"insured_details": "asset",
"id": "xxx",
"dept": "account",
"salary": "12"
},
{
"insured_details": "asset",
"id": "213",
"dept": "account",
"salary": "12"
},
{
"insured_details": "asset",
"id": "214",
"dept": "sales",
"salary": "13"
},
{
"insured_details": "asset",
"id": "233",
"dept": "market",
"salary": "14"
}
]
I have the following structure of JSON/Dict.
[
{
"childrens": [
{
"childrens": [
{
"name": "somenam1"
}
],
"name": "B999"
}
],
"name": "11111"
},
{
"childrens": [
{
"childrens": [
{
"name": "somename2"
},
{
"name": "somename3"
}
],
"name": "B5555"
},
{
"childrens": [
{
"name": "somename4"
}
],
"name": "B2222"
}
],
"name": "2222"
}
]
I want to iterate over all dictionaries and list inside root list and create single string for each dictionary inside root list.
Output will look like this (two lines):
1111|B999|somename1
2222|B5555|somename2|somename3|B2222|somename4
Also this is just an example i can have N nested childrens.
Looks like a good candidate for recursion:
def flatten(child):
if not child:
return child
return [child['name']] + [name for c in child.get('childrens', []) for name in flatten(c)]
In []:
for child in data:
print('|'.join(flatten(child)))
Out[]:
11111|B999|somenam1
2222|B5555|somename2|somename3|B2222|somename4
Sure you can just pass add a level arg and return that:
def flatten(child, level=0):
if not child:
return child
return [level] + [l for c in child.get('childrens', []) for l in flatten(c, level+1)]
In []:
for child in data:
print('|'.join(str(level) for level in flatten(child)))
Out[]:
0|1|2
0|1|2|2|1|2
Here's a solution by recursion
data_json = '[{"childrens":[{"childrens":[{"name":"somenam1"}],"name":"B999"}],"name":"11111"},{"childrens":[{"childrens":[{"name":"somename2"},{"name":"somename3"}],"name":"B5555"},{"childrens":[{"name":"somename4"}],"name":"B2222"}],"name":"2222"}]'
data = json.loads(data_json)
def get_names(data_dict):
if ("childrens" in data_dict):
ret_dict = "|".join(map(get_names, data_dict["childrens"]))
return data_dict["name"] + "|" + ret_dict
else:
return data_dict["name"]
def get_all_name(data):
for i in data:
print(get_names(i))
get_all_name(data)
I have a dataframe and have to convert it into nested JSON.
countryname name text score
UK ABC Hello 5
Right now, I have some code that generates JSON, grouping countryname and name.
However, I want to firstly group by countryname and then group by name. Below is the code and output:
cols = test.columns.difference(['countryname','name'])
j = (test.groupby(['countryname','name'])[cols]
.apply(lambda x: x.to_dict('r'))
.reset_index(name='results')
.to_json(orient='records'))
test_json = json.dumps(json.loads(j), indent=4)
Output:
[
{
"countryname":"UK"
"name":"ABC"
"results":[
{
"text":"Hello"
"score":"5"
}
]
}
]
However, I am expecting an output like this:
[
{
"countryname":"UK"
{
"name":"ABC"
"results":[
{
"text":"Hello"
"score":"5"
}
]
}
}
]
Can anyone please help in fixing this?
This would be the valid JSON. Note the comma , usage, is required as you may check here.
[
{
"countryname":"UK",
"name":"ABC",
"results":[
{
"text":"Hello",
"score":"5"
}
]
}
]
The other output you try to achieve is also not according to the standard:
[{
"countryname": "UK",
"you need a name in here": {
"name": "ABC",
"results": [{
"text": "Hello",
"score": "5"
}]
}
}]
I improved that so you can figure out what name to use.
For custom JSON output you will need to use custom function to reformat your object first.
l=df.to_dict('records')[0] #to get the list
print(l, type(l)) #{'countryname': 'UK', 'name': 'ABC', 'text': 'Hello', 'score': 5} <class 'dict'>
e = l['countryname']
print(e) # UK
o=[{
"countryname": l['countryname'],
"you need a name in here": {
"name": l['name'],
"results": [{
"text": l['text'],
"score": l['score']
}]
}
}]
print(o) #[{'countryname': 'UK', 'you need a name in here': {'name': 'ABC', 'results': [{'text': 'Hello', 'score': 5}]}}]
I have a List of Map with nested Map as well as below :-
def list = [
[
"description": "The issue is open and ready for the assignee to start work on it.",
"id": "1",
"name": "Open",
"statusCategory": [
"colorName": "blue-gray",
"id": 2,
"key": "new",
"name": "To Do",
]
],
[
"description": "This issue is being actively worked on at the moment by the assignee.",
"id": "3",
"name": "In Progress",
"statusCategory": [
"colorName": "yellow",
"id": 4,
"key": "indeterminate",
"name": "In Progress",
]
]
]
I have a task to get List of subMap with nested subMap. I'm doing some thing like as below :-
def getSubMap = { lst ->
lst.findResults { it.subMap(["id", "name", "statusCategory"])}
}
println getSubMap(list)
But its give me output as below :-
[
[
"id":1,
"name":"Open",
"statusCategory":[
"colorName":"blue-gray",
"id":2,
"key":"new",
"name":"To Do"
]
],
[
"id":"3",
"name":"In Progress",
"statusCategory":[
"colorName":"yellow",
"id":"4",
"key":"indeterminate",
"name":"In Progress"
]
]
]
As you can see I'm unable to get subMap of statusCategory key Map. Actually I want to get further subMap for nested Maps something like as below :-
[
[
"id":1,
"name":"Open",
"statusCategory":[
"id":"2",
"name":"To Do"
]
],
[
"id":"3",
"name":"In Progress",
"statusCategory":[
"id":"4",
"name":"In Progress"
]
]
]
To achieve this I'm trying as below :-
def getSubMap = { lst ->
lst.findResults { it.subMap(["id", "name", "statusCategory":["id","name"]])}
}
def modifiedList = getSubMap(list)
But it throws me Excpetion. And If I'm doing as below :-
def getSubMap = { lst ->
lst.findResults { it.subMap(["id", "name", "statusCategory"]).statusCategory.subMap(["id","name"])}
}
println getSubMap(list)
It gives only nested subMap as :-
[["id":"2", "name":"To Do"], ["id":"4", "name":"In Progress"]]
could anyone suggest me how to recurselvely find List of subMap with nested subMap if exist?
If your Map nesting is arbitrary, then you might want to consider something like this:
def nestedSubMap
nestedSubMap = { Map map, List keys ->
map.subMap(keys) + map.findAll { k, v -> v instanceof Map }.collectEntries { k, v -> [(k):nestedSubMap(v, keys)] }
}
Given your input and this closure, the following script:
def result = list.collect { nestedSubMap(it, ["id", "name"]) }
println '['
result.each { print it; println ',' }
println ']'
Produces this output:
[
[id:1, name:Open, statusCategory:[id:2, name:To Do]],
[id:3, name:In Progress, statusCategory:[id:4, name:In Progress]],
]
Given the original list, consider this:
def resultList = list.collect {
def fields = ["id", "name"]
def m = it.subMap(fields)
m["statusCategory"] = it["statusCategory"].subMap(fields)
return m
}
which supports these assertions:
assert 1 == resultList[0]["id"] as int
assert "Open" == resultList[0]["name"]
assert 2 == resultList[0]["statusCategory"]["id"] as int
assert "To Do" == resultList[0]["statusCategory"]["name"]
assert 3 == resultList[1]["id"] as int
assert "In Progress" == resultList[1]["name"]
assert 4 == resultList[1]["statusCategory"]["id"] as int
assert "In Progress" == resultList[1]["statusCategory"]["name"]