I have a web service written in Python 3.4 that uses the Falcon framework. One particular method accepts a post of json values. My code:
try:
raw_json = req.stream.read()
except Exception as ex:
raise falcon.HTTPError(falcon.HTTP_400, 'Error', ex.message)
try:
result_json = json.loads(raw_json.decode('utf-8'))
except ValueError:
raise falcon.HTTPError(falcon.HTTP_400,
'Malformed JSON', 'Could not decode the request body. The JSON was incorrect.')
clientIp = result_json['c']
wpIp = result_json['w']
lan = result_json['l']
table = int(result_json['t'])
This code was working fine 9 months ago but currently throws the error: "list indices must be integers or slices, not str." I think it likely broke after a Python or Falcon package update.
The ouput of raw_json.decode('utf-8') looks ok, returning [{"w": "10.191.0.2", "c": "10.191.0.3", "l": "255.255.255.0", "t": "4"}]. I think json.loads() is the root of my problem. len(result_json) is returning 1 where I would expect 4. Is there an additional parameter needed for json.loads() to help it parse correctly? Or am I missing something else entirely?
Thanks,
Greg (Python noob)
The returned result [{"w": "10.191.0.2", "c": "10.191.0.3", "l": "255.255.255.0", "t": "4"}] is a json array, which is parsed into a python list. Therefore
result_json['c']
produces the the mentioned error. Maybe there was a change in the API and it now returns an array where it previously returned a json object.
This should work:
clientIp = result_json[0]['c']
...
Related
I'm using regex to look through a file to check for a username and whether their action caused an error or not (ERROR for error, INFO for successful action)
I am using a nested dictionary to keep track of actions, with the username as the primary key and the nested dictionary of how many INFO and ERROR lines they generate.
#!/usr/bin/env python3
import re
users = {}
with open('logfile.txt') as f:
for line in f:
regex_user = r"(INFO|ERROR): .* \((.+)\)$"
"""searches for users and if there message was info or error"""
user = re.serach(regex_user, line)
if user is None:
continue
name = user[2]
cat = user[1]
try:
# Method 1?
users[name][cat] = users[name].get(cat, 0) + 1
# Method 2?
users[name][cat] = users.get(name, {}).get(cat, 0) + 1
except KeyError:
print("Where are my keys?")
I am wondering which of the two methods (if any) are correctly modifying the dictionary to increase the count of the respective nested key.
Output should look like:
{'John': {'INFO': 22, 'ERROR': 3}}
if the log contained 22 lines of INFO and 3 lines of ERROR for user John.
You probably want to solve this with a nested collections.defaultdict or alternatively using setdefault() though the latter is not a clean in my opinion. collections.defaultdict() will allow you to reference a dictionary key that does not yet exist which is the crux of your issue.
Try something like:
import collections
import json
import random
users = collections.defaultdict(lambda: collections.defaultdict(int))
for i in range(100):
user = random.choice(["John", "Jane", "Sanjay"])
cat = random.choice(["INFO", "ERROR"])
users[user][cat] += 1
print(json.dumps(users, indent=2))
That will print out something like:
{
"Jane": {
"INFO": 16,
"ERROR": 19
},
"John": {
"ERROR": 21,
"INFO": 10
},
"Sanjay": {
"INFO": 18,
"ERROR": 16
}
}
though each run will be different.
As we noted by #Ryukashin, a defaultdict() does not print() as nicely as a dict(). So casting back is possilbe if it is really needed for some reason. The easiest (no not the most efficient) manner for that might be:
users = json.loads(json.dumps(users))
why do i get this error message: ValueError: malformed node or string: <_ast.Name object at 0x0000016835AEE3D0>. i dont understand please help me..
import ast
def convert_text_list(text):
texts = ast.literal_eval(str(text))
return text
TWEET_DATA["tweet_list"] = TWEET_DATA["tweet"].apply(convert_text_list)
print(TWEET_DATA["tweet_list"][90])
print("\ntype : ", type(TWEET_DATA["tweet_list"][90]))```
It has been some months since you asked, so you might have found the answer for yourself, I got this problem yesterday, all you have to do is to get rid os every "null" value in the dataset.
Sending Client request to get data from API.
request =client.get_income('ET',incomeType='PNL',startTime=1611287550000)
The API returns the following data:
[{"symbol":"ET","incomeType":"R","income":"-2.4","time":1611287909000,"info":"50538270"},{"symbol":"ET","incomeType":"R","income":"1.68","time":1611287909000,"info":"50538271"}]
It's a dictionary inside the list. When I try to access the items through a for loop or any of the following methods, It returns the OBJECT.
Methods I tried:
for item in request:
print(item['income'])
returns this : <model.income.Income object at 0x000000684EFB4580>
print(request[0]['income']
ERROR: TypeError: 'Income' object is not subscriptable
None of them works.
I have fixed it myself.
request =client.get_income('ET',incomeType='PNL',startTime=1611287550000)
for x in range(len(request)):
print(request[x].__dict__['income'])
request = [{"symbol":"ET","incomeType":"R","income":"-2.4","time":1611287909000,"info":"50538270"},{"symbol":"ET","incomeType":"R","income":"1.68","time":1611287909000,"info":"50538271"}]
for item in request:
print(item['income'])
it's working fine on my pc, I'm using python 3.9
-2.4
1.68
The data structure you have is a JSON. Use Pandas to parse it and access it elements:
import pandas as pd
request = [{"symbol":"ET","incomeType":"R","income":"-2.4","time":1611287909000,"info":"50538270"},
{"symbol":"ET","incomeType":"R","income":"1.68","time":1611287909000,"info":"50538271"}]
df = pd.DataFrame(request)
print(df)
print(f'Incomes are: {df["income"].tolist()}')
Output:
I am new to Python, stupid question ahead
I need to compare the entries from a MySQL database with ldap. I created a dictionary to hold the corresponding values, when I try to loop through the dictionary and pass them to ldap3 entries to show the results it takes the variable as literal.
for x in values_dict:
value2=values_dict[x]
try:
ldap_conn.entries[0].value2
except Exception as error:
print(error)
else:
print(value2)
attribute 'value2' not found
If I replace value2 with 'sn' or any of the other attributes I have pulled it works fine. I have also played around with exec() but this returns nothing.
for x in values_dict:
value2=values_dict[x]
test1='ldap_conn.entries[0].{}'.format(value2)
try:
result1=exec(test1)
except Exception as error:
print(error)
else:
print(result1)
None
Any ideas?
EDIT 1 : As requested here are the values for values_dict. As stated previously the loop does parse these correctly, ldap does return the attributes, but when I try to use a variable to lookup the attributes from entries the variable is taken literally.
values_dict = {
"First_Name": "givenname",
"Middle_Name": "middlename",
"Last_Name": "sn",
"Preferred_Name": "extensionattribute2",
"Work_Location": "physicaldeliveryofficename",
"Work_City": "l",
"Work_State": "st",
"Work_Postal": "postalcode",
"Work_Email": "mail"
}
The syntax somevariable.someattr, which you are using here:
ldap_conn.entries[0].value2
Always means "access an attribute named someattr of somevariable". It's always interpreted as a literal string. If you need to dynamically access an attribute, use the getattr function:
getattr(ldap_conn.entries[0], value2)
You're not currently assigning that that result anywhere, so you probably want something like:
result1 = getattr(ldap_conn.entries[0], value2)
I'm querying Github's Jobs API with python3, using the requests library, but running into an error parsing the response.
Library: http://docs.python-requests.org/en/latest/
Code:
import requests
import json
url = 'https://jobs.github.com/positions.json?'
response = requests.get(url)
print(response.json())
Error:
UnicodeEncodeError: 'ascii' codec can't encode character '\u2019' in
position 321: ordinal not in range(128)
Using this API in the past with Ruby, I've never run into this issue.
I also tried converting it to a dictionary but it resulted in the same errors.
There's other questions on SO about the UnicodeEncodeError (mostly re: opening files), but I'm not familiar with Python and didn't find them helpful.
First, check that the response is actually JSON. Try printing response.text and see if it looks like a valid JSON object.
Assuming it is JSON: it's very "hack"-ey, but you can replace the non ASCII characters with their escaped Unicode representation:
def escape_unicode(c):
return c.encode('ascii', 'backslashreplace').decode('ascii')
response = ...
text = response.text
escaped = re.sub(r'[^\x00-\x7F]', lambda m: escape_unicode(m.group(0)), text)
json_response = json.loads(escaped)