Protect a nested object against flattening when using pandas.json_normalize - python-3.x

Question

In pandas >= 1.1.4 / Python 3, I would like to protect a nested element against flattening when using json_normalize().
I cannot figure out such thing in the documentation.
Actual example
Here's a concrete example to figure out the main idea:
res='''
{
"results": [
{
"geometry": {
"type": "Polygon",
"crs": 4326,
"coordinates":
[[
[6.0, 49.0],
[6.0, 40.0],
[7.0, 40.0],
[7.0, 49.0],
[6.0, 49.0]
]]
},
"attribute": "layer.metadata",
"bbox": [6, 40, 7, 49],
"featureName": "Coniferous_Trees",
"layerName": "State_Forests",
"type": "Feature",
"id": "17",
"properties": {
"resolution": "100",
"Year": "2020",
"label": "Coniferous"
}
}
]
}
'''
This is a single JSON record from an API response. Here, there is only one element in the top level list, but there my be more, each following the same structure as the one shown here. I'd like to import this into a DataFrame without columns containing structured element, namely, I want to flatten / normalize them all. Well,... almost all. json_normalize() is a doing an amazing job in doing that:
import pandas as pd
data = json.loads(res)['results']
df = pd.DataFrame(pd.json_normalize(data))
And here are the columns of the DataFrame:
>>> print(df.columns)
Index(['attribute', 'bbox', 'featureName', 'layerName', 'type', 'id',
'geometry.type', 'geometry.crs', 'geometry.coordinates', # <-- the geometry has been flattened
'properties.resolution', 'properties.Year', 'properties.label'],
dtype='object')
Wanted behaviour
But I need to, let's say, "protect" the geometry object in the input JSON response against flattening so that I end up with these columns instead:
# e.g. something like this:
df = pd.DataFrame(pd.json_normalize(data, protect="results.geometry"))
# or this if there isn't two objects with the same name:
df = pd.DataFrame(pd.json_normalize(data, protect="geometry"))
which would lead to:
>>> print(df.columns)
Index(['attribute', 'bbox', 'featureName', 'layerName', 'type', 'id',
'geometry', 'properties.resolution', # <-- the geometry element has been protected!
'properties.Year', 'properties.label'],
dtype='object')
Is there a way of doing that properly?

Answer 1

Consider max_level=0. Per pandas.json_normalize docs:
max_level : int, default None
Max number of levels(depth of dict) to normalize. if None, normalizes all levels.
data = json.loads(response)["results"]
df = pd.DataFrame(pd.json_normalize(data, max_level=0))
print(df.T)
# 0
# geometry {'type': 'Polygon', 'crs': 4326, 'coordinates'...
# attribute layer.metadata
# bbox [6, 40, 7, 49]
# featureName Coniferous_Trees
# layerName State_Forests
# type Feature
# id 17
# properties {'resolution': '100', 'Year': '2020', 'label':...
print(df.columns)
# Index(['geometry', 'attribute', 'bbox', 'featureName', 'layerName', 'type', 'id',
# 'properties'], dtype='object')
And since all nested objects are not normalized, use data wrangling to unwind needed columns like properties:
df = (
df.drop(['properties'], axis="columns")
.join(df["properties"].dropna().apply(pd.Series))
)
print(df.T)
# 0
# geometry {'type': 'Polygon', 'crs': 4326, 'coordinates'...
# attribute layer.metadata
# bbox [6, 40, 7, 49]
# featureName Coniferous_Trees
# layerName State_Forests
# type Feature
# id 17
# resolution 100
# Year 2020
# label Coniferous
print(df.columns)
# Index(['geometry', 'attribute', 'bbox', 'featureName', 'layerName', 'type', 'id',
# 'resolution', 'Year', 'label'], dtype='object')