Sending Client request to get data from API.
request =client.get_income('ET',incomeType='PNL',startTime=1611287550000)
The API returns the following data:
[{"symbol":"ET","incomeType":"R","income":"-2.4","time":1611287909000,"info":"50538270"},{"symbol":"ET","incomeType":"R","income":"1.68","time":1611287909000,"info":"50538271"}]
It's a dictionary inside the list. When I try to access the items through a for loop or any of the following methods, It returns the OBJECT.
Methods I tried:
for item in request:
print(item['income'])
returns this : <model.income.Income object at 0x000000684EFB4580>
print(request[0]['income']
ERROR: TypeError: 'Income' object is not subscriptable
None of them works.
I have fixed it myself.
request =client.get_income('ET',incomeType='PNL',startTime=1611287550000)
for x in range(len(request)):
print(request[x].__dict__['income'])
request = [{"symbol":"ET","incomeType":"R","income":"-2.4","time":1611287909000,"info":"50538270"},{"symbol":"ET","incomeType":"R","income":"1.68","time":1611287909000,"info":"50538271"}]
for item in request:
print(item['income'])
it's working fine on my pc, I'm using python 3.9
-2.4
1.68
The data structure you have is a JSON. Use Pandas to parse it and access it elements:
import pandas as pd
request = [{"symbol":"ET","incomeType":"R","income":"-2.4","time":1611287909000,"info":"50538270"},
{"symbol":"ET","incomeType":"R","income":"1.68","time":1611287909000,"info":"50538271"}]
df = pd.DataFrame(request)
print(df)
print(f'Incomes are: {df["income"].tolist()}')
Output:
Related
My code accesses a light sensor via a python request:
address = 'https://api.particle.io/v1/devices/my_device_id/analogvalue'
headers = {'Authorization':'Bearer {0}'.format(access_token)}
vals = requests.get(address, headers=headers)
The code returns the following values:
{"cmd":"VarReturn","name":"analogvalue","result":171,"coreInfo":{"last_app":"","last_heard":"2019-06-13T21:55:57.387Z","connected":true,"last_handshake_at":"2019-06-13T20:51:02.691Z","deviceID":"my_device_id","product_id":6}}
Python tells me that this is a 'requests.models.Response' class and not a dictionary like I thought.
When I try to access the 'result' value, I get error messages. Here are the various ways I have tried along with their error messages.
print(vals[2])
TypeError: 'Response' object does not support indexing
print(vals['result'])
TypeError: 'Response' object is not subscriptable
print(vals[2].json())
TypeError: 'Response' object does not support indexing
print(vals['result'].json())
TypeError: 'Response' object is not subscriptable
I got the last two approaches (.json) from a answer here on stack overflow.
Can anyone tell me how to access this result value or am I going to be forced to use regular expression?
EDIT: With help from Sebastien D I added the following and was able to get the result I was looking for.
import json
new_vals = json.loads(vals.content)
print(new_vals['result'])
Just do :
import json
### your code ###
json.loads(vals.content)
I am trying to filter through ny gov open database with their SODA API. I am following the docs on how to filter, but it is returning an empty dataframe.
# noinspection PyUnresolvedReferences
import numpy as np
# noinspection PyUnresolvedReferences
import pandas as pd
# noinspection PyUnresolvedReferences
from sodapy import Socrata
clientNYgov = Socrata('data.ny.gov', None)
Here is where I am trying to find only results in NY.
databaseM = clientNYgov.get('yg7h-zjbf.csv?business_city=NEW+YORK')
dfDatabaseM = pd.DataFrame.from_records(databaseM)
dfDatabaseM.to_csv('Manhattan Agents.csv')
print(dfDatabaseM)
But here is the Empty Output:
0 1 ... 9 10
0 business_address_1 business_address_2 ... license_number license_type
[1 rows x 11 columns]
Process finished with exit code 0
Please let me know if there's a problem with how I am filtering, not quite sure what is going wrong here. Thanks so much in advance!
Socrata uses a json endpoint to export the files via the API. This is found in the top right hand corner of the dataset when selecting API. For this solution I am using just requests to retrieve the data. The Soda module is nice to use, but works the same as a request.
import pandas as pd
import requests
data=requests.get('http://data.ny.gov/resource/yg7h-zjbf.json?$limit=50000&business_city=NEW YORK').json()
df=pd.DataFrame.from_records(data)
df
There are two approaches to do this with filters.
Method 1
This can be done using Socrata() by passing the filters using SQL to the query keyword in the get() method of the instantiated Socrata client. You will need an application token. If you do not use a token, then your requests will be subjected to throttling. To avoid throttling, sign up for a socrata account and create your app token
query = f"""SELECT * WHERE business_city="NEW YORK" LIMIT 50000"""
client = Socrata("data.ny.gov", <YOUR-APP-TOKEN-HERE>)
results = client.get("yg7h-zjbf", query=query)
df_socrata = pd.DataFrame.from_records(results)
Method 2
Using the JSON endpoint (same as #Joseph Gattuso's answer)
data = requests.get(
"http://data.ny.gov/resource/yg7h-zjbf.json?"
"$limit=50000&"
"business_city=NEW YORK"
).json()
df = pd.DataFrame.from_records(data)
Comparison of output - Verify that the two methods return the same result
assert df_socrata.equals(df)
I am trying to use python to access pull and plot data from a server (ERRDAP if you're interested).
Here is the code that I am trying to use:
url = 'https://scoos.org/erddap/tabledap/autoss.csv?query'
values = {'time>=' : '2016-07-10T00:00:00Z',
'time<' : '2017-02-10T00:00:00Z',
'temperature' : 'temperature',
'time' :'time',
'temperature_flagPrimary'}
data = urllib.parse.urlencode(values)
data.encode('utf-8')
resp = urllib.request.urlopen(req)
The error message that I get is: "POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str." I think this either has something to do with me trying to request a csv file or improperly using urllib. Any help would be greatly appreciated!
I have question. I try to get info from function and pack result into dictionary like this:
import requests
class CommonUtils():
def get_data_list(self, url, listType):
c_list = requests.get(url)
data = {"json": c_list.json(), "text": c_list.text}
return data.get(listType)
I debugged function and I know, that requests retrieves info from passed url.
And when I try to return data, I get error with message: "AttributeError: 'dict' object has no attribute 'json'"
What I do wrong?
I am using aiohttp get request to download some content from another web api
but i am receiving:
exception = TypeError('not a valid non-string sequence or mapping object',)
Following is the data which i am trying to sent.
data = "symbols=LGND-US&exprs=CS_EVENT_TYPE_CD_R(%27%27,%27now%27,%271D%27)"
How to resolve it?
I tried it in 2 ways:
r = yield from aiohttp.get(url, params=data) # and
r = yield from aiohttp.post(url, data=data)
At the same time i am able to fetch data using:
r = requests.get(url, params=data) # and
r = requests.post(url, data=data)
But i need async implementation.
And also suggest me some way if i can use import requests library instead of import aiohttp to make async http request, because in many cases aiohttp post & get request are not working but the same are working for requests.get & post requests.
The docs use bytes (i.e. the 'b' prefix) for the data argument.
r = await aiohttp.post('http://httpbin.org/post', data=b'data')
Also, the params argument should be a dict or a list of tuples.