i have an array of JSON objects that arrived from the source I can't control, and sometimes there are values, like this:
[{"name": "HDD", "brand": "Samsung", "price": "$100"},
<NULL>,
{"name": "Mouse", "brand": "Logitech", "price": "$10"}]
is there any way to handle it in python? I'm getting a syntax error on reading the value.
i tried to put it this way:
try:
products = sorted(products, key=lambda k: k['price'], reverse=True)
except SyntaxError:
print("Error")
but no luck.
I have a json file with list of dicts. I want to modify its content by adding key:value in every dict with index as the value
Note that the json file is malform, so I need to remove the extra '[]'
file.json
[
{
"sample1": 1,
"sample2": "value"
}
[]
{
"sampleb": "123",
"some": "some"
}
...............
]
code
""" open the files"""
with open("list1.json", "r") as f:
data = f.read()
data = data.replace("][", ",")
data = json.loads(data)
for v in data:
for i, c in v.items():
c["rank"] = i + 1
""" put back to file"""
with open("list1.json", "w") as file:
file.write(data)
So what I am trying to achieve is something like
[
{
"rank": 1
"sample1": 1,
"sample2": "value"
},
{
"rank": 2,
"sampleb": "123",
"some": "some"
}
...............
]
But I got error
c["rank"] = i
TypeError: 'str' object does not support item assignment
printing the index print 1 shows
0,
1,
.....
0,
1,
........
0,
1
But it should be
0,
1,
2,
3,
4
5
...
100
Any ideas?
I have a nested dictionary which comprises of multiple lists and dictionaries. The "Stations" key contains contains the values which I want to convert to CSV file. I am only after the certain values. A snippet of the dictionary is as below:
data = { "brands": {...},
"fueltypes": {...},
"stations": {"items": [
{
"brandid": "",
"stationid": "",
"brand": "Shell",
"code": "2126",
"name": "Cumnock General Store",
"address": "31 Obley St, CUMNOCK NSW 2867",
"location": {
"latitude": -32.928744,
"longitude": 148.755153
},
"state": "NSW"
},
{
"brandid": "",
"stationid": "",
"brand": "Shell",
"code": "2200",
"name": "Tea Tree Cafe",
"address": "160 Mount Darragh Rd, SOUTH PAMBULA NSW 2549",
"location": {
"latitude": -36.944277,
"longitude": 149.845399
},
"state": "NSW"
}....]}}
In order to obtain certain values in "Stations" key, I created blank lists for each of those values and appended accordingly. After that I used the ZIP function to combine the list and converted to a CSV. The Code that I have used is as below:
Station_Code = []
Station_Name = []
Latitude = []
Longitude = []
Address = []
Brand = []
for k,v in data["stations"].items():
for item in range(len(v)):
Station_Code.append(v[item]["code"])
Station_Name.append(v[item]["name"])
Latitude.append(v[item]["location"]["latitude"])
Longitude.append(v[item]["location"]["longitude"])
Address.append(v[item]["address"])
Brand.append(v[item]["brand"])
#print(f'{v[item]["code"]} - {v[item]["name"]} - {v[item]["location"]["latitude"]}')
rows = zip(Station_Code, Station_Name, Latitude, Longitude, Address, Brand )
with open("Exported_File.csv", "w") as f:
writer = csv.writer(f)
for row in rows:
writer.writerow(row)
Is there any other alternate/short ways of extracting this information?
If you're using pandas, there's a fairly easy way to do this.
import pandas as pd
# Convert dict to a pandas DataFrame
df = pd.DataFrame(data["stations"]["items"])
# 'location' is a dict, so we need to extract the 'latitude' and 'longitude'.
df['latitude'] = df['location'].apply(lambda x: x['latitude'])
df['longitude'] = df['location'].apply(lambda x: x['longitude'])
# Select subset of columns for final csv
df = df[['code', 'name', 'latitude', 'longitude', 'address', 'brand']]
df.to_csv('exported-file.csv', index=False, header=False)
I have this CSV:
color,property,type,id
red,house,building,02
I'm trying to convert a csv to dictionary with the following structure:
{
"0": {"val1": 1, "val2": 2, "val3": 3, ..., "valn": n},
"1": {"val1": 45, "val2": 7, "val3": None, ..., "valn": 68},
}
Where as val1, val2 and so on are the header names of the columns and "0" and "1" are the number of rows.
So we should have:
CSV content is like this:
color,property,type,id
red,house,building,02
blue,department,flat,04
{
"0": {"color": "red", "property": "house", "type": "building", ..., "valn": n},
"1": {"color": "blue", "property": "farm", "type": "area", ..., "valn": n},
}
How can I achieve this result without using any library? I'd like to implement it from the scratch and don't use CSV library or the like.
Thank you.
Try this approach:
inp = """color,property,type,id
red,house,building,02
blue,department,flat,04
cyan,,flat,10
"""
lines = inp.split('\n')
colnames = list(map(lambda x: x.strip(), lines[0].split(',')))
lines = lines[1:]
res = {}
for i, line in enumerate(lines[:-1]):
res[i] = {
colname: val if val != '' else None
for colname, val in zip(colnames, map(lambda x: x.strip(), line.split(',')))
}
print(res)
However for additional features like type deduction code will be more complex: you can follow answers to this question
I got an assignment to import a CSV file with some fields, and I need to create a new CSV file with different fields that contains the original fields (in a different order).
original csv:
full name,Posiotion,Phone,Email,LinkedIn,Source,Comment
I tried to look up online and this is as far as i got:
import csv
with open("mobileTL.csv", 'r') as csv_file:
reader = csv.reader(csv_file)
newcsvdict = {"First name": [], "Middle name": [], "Last name": [], "Email": [], "Creation date": [], "Status": [],
"Position": [], "ID/SSN": [], "Source": [], "Source type": [], "Availability": [], "Salary expectations": [],
"Phone": [], "Mobile": [], "Street Adress": [], "City": [], "State": [], "Country": [], "Zip": [],
"LinkedIn URL": [], "Resume file name": [], "Migration ID": [], "Comment": [], "Comment2": []}
next(reader)
for row in reader:
first = ""
last = ""
if row[0] != "":
first = row[0].split()[0]
last = row[0].split()[1]
newcsvdict["First name"].append(first)
newcsvdict["Last name"].append(last)
newcsvdict["Phone"].append(row[2])
newcsvdict["Position"].append(row[1])
newcsvdict["Email"].append(row[3])
newcsvdict["Source"].append(row[5])
newcsvdict["Comment"].append(row[6])
newcsvdict["LinkedIn URL"].append(row[4])
with open('new.csv', 'w') as csv_file:
w = csv.DictWriter(csv_file, newcsvdict.keys())
w.writeheader()
w.writerows(newcsvdict)
It does create a new file but for some reason only the header is written.
First, the reason why it's only writing the header is because you'll get an error:
Traceback (most recent call last):
File "test.py", line 29, in <module>
w.writerows(newcsvdict)
...
wrong_fields = rowdict.keys() - self.fieldnames
AttributeError: 'str' object has no attribute 'keys'
You need to learn not to ignore error messages. The cause of that problem is that you were using writerows (note plural rows, which expects an iterable of rows) instead of writerow (note singular row, which expects just one row). To use writerows, you need to pass a list of dicts like this:
w.writerows([newcsvdict, newcsvdict, newcsvdict])
You should be using writerow, since you seem to only have 1 row, newcsvdict. Though, when I went ahead and did that, the output does not seem to be what you need:
First name,Middle name,Last name,Email,Creation date,Status,Position,ID/SSN,Source,Source type,Availability,Salary expectations,Phone,Mobile,Street Adress,City,State,Country,Zip,LinkedIn URL,Resume file name,Migration ID,Comment,Comment2
"['aaa', 'bbb', 'ccc']",[],"['AAA', 'BBB', 'CCC']","['aaa#email.com', 'bbb#email.com', 'ccc#email.com']",[],[],"['Pos1', 'Pos2', 'Pos3']",[],"['aaa', 'bbb', 'ccc']",[],[],[],"['123', '456', '789']",[],[],[],[],[],[],"['aaa', 'bbb', 'ccc']",[],[],"['aaa', 'bbb', 'ccc']",[]
That looks weird, because you created a dict with a list for each value (ex. "First name": []). Maybe that's what you want... but my understanding of your requirement is that you want for the new CSV is to have the same number of rows but different columns.
For that, it does not make sense to store the values as a list. One solution is to read one row, create a dict for it, then writerow it, then just repeat for the steps for all the rows. You can also use DictReader to easily access the values from the old CSV as a dict.
with open("new.csv", "w") as new_file:
new_row = dict.fromkeys([
"First name", "Middle name", "Last name", "Email",
"Creation date", "Status", "Position", "ID/SSN",
"Source", "Source type", "Availability", "Salary expectations",
"Phone", "Mobile", "Street Adress", "City",
"State", "Country", "Zip", "LinkedIn URL",
"Resume file name", "Migration ID", "Comment", "Comment2"
])
writer = csv.DictWriter(new_file, fieldnames=new_row.keys())
writer.writeheader()
with open("old.csv", 'r') as old_file:
old_csv = csv.DictReader(old_file)
for row in old_csv:
first = ""
last = ""
if row["full name"] != "":
first, last = row["full name"].split()
new_row["First name"] = first
new_row["Last name"] = last
new_row["Phone"] = row["Phone"]
new_row["Position"] = row["Position"]
new_row["Email"] = row["Email"]
new_row["Source"] = row["Source"]
new_row["Comment"] = row["Comment"]
new_row["LinkedIn URL"] = row["LinkedIn"]
writer.writerow(new_row)