I have been spending some time learning how APIs work for a project I am working on.
I have been using the requests library to do so, and it has been very helpful. Right now I am working with the github api - https://api.github.com/
I ran into a problem when trying to make a call from this specific section of the API - Github-API-Call
The following is what I tried:
r = requests.get('https://api.github.com/users/{user}/repos{?type,page,per_page,sort}', auth = ('user', 'pass'))
When I make an attempt at checking the status code though with,
r.status_code
I get returned 404. I noticed that the URL looks strange, I was hoping that somebody might be able to let me know exactly what is going wrong, or explain why the URL looks strange in this example.
Really appreciate somebody taking the time out to explain it.
Related
I have been trying to externally log data to my home server with a little GSM modem and InfluxDB2 HTTP API- it is far away and needs to be external, just checking water levels and other system stuff.
I am struggeling to understand the correct payload and keys to give it in order for it to accept my data.
I am using an ESP32 and the requests module on Micro Python, using MicroPython_ESP32_psRAM_LoBo.
The GSM library I am using makes everything work as if I was connected to Wifi, I am pretty certian that my problem has nothing to do with the GSM side of things.
The system uses InfluxDB2 and Python3 ( Micro Python to be exact )
Current Setup
payload = {"header":"Authorization: Token CrAzYlOnG_ToKeN", "data-raw":"WaterWatcher1 Level=532,Volume=752.22,BattV=3.768,Temperature=25.36 1621434110"}
response = requests.post("http://xxx.xxx.xxx.xxx:8086/api/v2/write?org=Home_Org&bucket=BackYardInfo&precision=s", params=payload)
( xxx Replaced with my correct IP - Yes I have made sure this is correct and got my ISP to set it to static )
This gives me a constant Auth error :
W (3390149) HTTP_CLIENT: This request requires authentication, but does not provide header information for that
I have read over the InfluxDB2 docs and I still cant seem to get it correct, I seem to think that if I did this in InfluxDB Ver 1 it would have been easier but now I just want to try learn it this way.
I have entered the exact same data manually and that works fine ( changed the time stamps though )
I am pretty sure it boils down to me just not knowing how to send the data correctly with the requests module, any pointers and explanations would be a great help.
I managed to find a solution after days of searching and trying new things.
Basically I used the mrequests lib
( https://github.com/SpotlightKid/mrequests )
With this I managed to get the whole thing working as expected using the following code - Done in Micropython
I did make a free account with No-IP to set my ISP external IP to a URL, I dont think this has a massive change but it is worth noting.
I did try the older methods with this new URL and they still did not work.
Here is the new code with the mrequest lib.
import mrequests
url = "http://ni-ip-address:8086/api/v2/write?org=Home_org&bucket=backYardData&precision=s"
token = "Token CrazyLongToken"
DataString = "WaterWatcher1 Level=888,Volume=888.88,BattV=8.888,Temperature=25.36 1621694039"
res = mrequests.request("POST", url, headers={"Authorization":token}, data=DataString)
This works perfectly for me.
On a side note, I can also upload multiple points if I read them from a file but I had to put the newline char "\n" at the front of each of my lines not the end or Influx would not pars the data.
I think it boiled down to the urequest lib not being able to do headers as I needed them in post requests.
I may have just been using it wrong but I feel that I did give it more than a fair go at getting it right where as this mrequests lib just worked perfectly.
Hope this helps anyone else out there one day.
I want to save the data column from this url to an array in python. I tried it with, for instance, pandas.save_table:
import pandas as pd
pd.read_table('https://adventofcode.com/2019/day/1/input', sep='')
but I get HTTPError: HTTP Error 400: Bad Request and I think this is not the right way to do that.
Can someone help me with that?
If you try to open the link in your question (in a browser using incognito mode or something similar i.e. delete your cookies) you'll see that you need login into the website to access the page. This is why the you're getting a 400 Bad Request error as a response from the server.
From the FAQ section of the website that you're trying to access:
How does authentication work? Advent of Code uses OAuth to confirm
your identity through other services. When you log in, you only ever
give your credentials to that service - never to Advent of Code. Then,
the service you use tells the Advent of Code servers that you're
really you. In general, this reveals no information about you beyond
what is already public; here are examples from Reddit and GitHub.
Advent of Code will remember your unique ID, names, URL, and image
from the service you use to authenticate.
The website uses OAuth to handle logins to the url that you create will need these access tokens. You can use a library like python-oauth2 to help you with this (there are others so you can read around and decide which you'd like to use). Creating and understanding how to make http requests is beyond the scope of this answer. I'd suggest you have a look around on the internet for some explanations and try again, if you have get stuck please ask another question. Otherwise it'll probably be easier to save the file from your browser...But I'll leave this answer here for the next person who runs into the same problem.
I was trying to do some simple authorization for ameritrade's developer platform. I was attempting.
According to the platform, the Endpoint I need to access is is:
https://auth.tdameritrade.com/auth?response_type=code&redirect_uri={uri}&client_id={client_id}}%40AMER.OAUTHAP
https://developer.tdameritrade.com/content/simple-auth-local-apps
When looking at the client_id, for the dev application, I was noticing that they may actually be referencing the Applications, Consumer Key instead? So i did just that, but when attempting to query the information, it returns: A third-party application may be attempting to make unauthorized access to your account. The reason why i think it is the consumer key, is listed at: https://developer.tdameritrade.com/content/getting-started
So I ended up doing something like:
from urllib.parse import urlencode, quote_plus
url = "https://auth.tdameritrade.com/auth?response_type=code&redirect_uri={uri}&client_id={client_id}}%40AMER.OAUTHAP".format(
uri=urlencode("http://localhost", quote_via=quote_plus),
client_id="JHBDFGJH45OOUDFHGJKSDBNG" #Sample
)
I dont think this is because I am currently in a different country currently, I think that something else is wrong here.
It doesnt follow through with it, but instead returns a 400 error with that information. Im not sure whats wrong though.
This happens when you copied the callback URI incorrectly. Imagine if this were a client application, and TD detected that the application is trying to send the user to a different URL than the app is configured with. If they send the callback request to that application, it will receive the token and gain full control over your account.
Have you double and triple checked that you're copying the callback URL correctly, protocol name, ports, and trailing slashes and everything? Also, consider using an API library instead of writing your own. You can find documentation about this specific error here.
I had this issue and I solved it using simply using http://127.0.0.1 on the call back URI of the App.
I then used below URL and it worked as expected.
https://auth.tdameritrade.com/auth?response_type=code&redirect_uri=http%3A%2F%2F127.0.0.1&client_id={MyConsumerKey}%40AMER.OAUTHAP
Just in case anyone is still having this problem, make sure the callback URI is spelled EXACTLY the same as you specified when creating the app. I was having this problem because I set the callback on the TD developer website to "https://localhost/" and used "https://localhost" in the URL instead (missing the slash at the end). As soon as I added the slash at the end, it worked.
I found out that the issue is caused by the way the callback URL is set. It have to be exactly the same as the callback URL you have typed in at the apps details on the TD developer API page. I tried several permutations and indeed to get the authorization to work both have to be the same. eg. https or http.. end with '/' or does not, it matters. There is also no need to URL encode it.
when trying to post a WakeUp event with a JSON body to the Alexa events API using nodejs with axios or request-promise, the API always returns an error 500.
I posted to an online endpoint to actually see what gets posted and learned that the post body gets truncated which obviously results in invalid json. I abstracted the problem and tried to run it from a virgin nodejs installation by using repl.it and the result is the same.
Interestingly enough, there seems to be a relation between the length of the header and the body. So when I shorten the auth token in the header, more characters of the body get transferred. If I shorten the long tokens in the body to about 450 to 500 characters (it seems to vary) the whole request gets through. Obviously this is not a solution, because the tokens are needed for authentication.
When I experimented with the axios version used lowering it to 0.10 I once got a result but posting again lead to another 500. If I post often enough some requests get trough complete, even on the current axios version. I also tried using request-promise with the same outcome.
I got the feeling that I made a really stupid mistake but I can't find it and I really couldn't find anything on this topic, so it's driving me crazy. Any help would be greatly appreciated!
This looks like a tricky one.. first of all, I don't think you're making a really stupid mistake. It looks to me like one of the low-level modules doesn't like something in the POST body for some reason (really weird.).. I've played about with this and I'm getting exactly the same behaviour with both Axios and Request.. if I comment out the tokens (correlationToken and bearer token ) everything works fine.
If I test this locally, everything works as it should (e.g. set up express server and log POST body).
Also posting to https://postman-echo.com/post works as expected (with the original post data)..
I've created this here: https://repl.it/repls/YoungPuzzlingMonad
It looks to me like the original request to http://posthere.io is failing because of the request size only. If you try a very basic POST with a large JSON body you get the same result.
I get the same result with superagent too.. this leads me to believe this is something server side...
This was not related to the post request at all. The reason for the error after sending the WakeUp event was the missing configuration parameter containing the MACAdresses in the Alexa.WakeOnLANController interface.
I used the AlexaResponse class to add the capability via createPayloadEndpointCapability which had not been modified to support the "new" WakeOnLANController interface yet.
It's a pity that the discovery was accepted and my WOL-capable device was added to my smart home devices although a required parameter was missing :(
posthere.io cutting off long post bodys cost me quite a few hours... On the upside, I go to know many different ways of issuing a post request in node ;)
Thanks again Terry for investigating!
So.. Back again with another flask question:
My application is set up largely around the use of subdomains for various portions instead of prefixes. As such, typical urls for my app look like this:
domain.tld/
domain.tld/about
admin.domain.tld/
admin.domain.tld/login
etc.
I'm composing unit tests and I've seem to run into a problem. I've done a lot of searching of the Flask and Werkzeug documentation but I can't seem to figure out how to fix my issue.
Basically, when I'm testing I'm trying to simulate a GET request to various urls.. and a lot of those URLs will fall on subdomains. I've tried the following which would seem to be logical:
with app.app_context():
url = url_for("admin.login") # returns http://admin.domain.tld:80/login
with app.test_client() as c:
resp = c.get(url) # Fails: returns 404
assert resp.data == "Expected test response", "Bad Response"
Now.. my c.get should return a response containing expected data at the url, but instead I'm being given the default 404 handler. This makes it quite hard to test many of my routes.
I've done some reading into Flask and found a github issue that notes that the test client expects a relative url... well.. I'm not sure how to provide that since I need to specify the subdomain.
Additionally, I've done further reading and discovered that the Flask test client is built on the Werkzeug test client and the Werkzeug test client contains an option for "allow_subdomain_redirects" that is normally false. Sadly, attempting to configure my Flask test client to have that behavior so far has failed (I might be doing it wrong).
Anyways, does anyone know how I can simulate a request in my flask app and target a subdomain? At the moment, I'm all out of ideas. Thanks for any suggestions. :)
So.. After a lot of debugging I have found the problem... and it seems to be a Flask issue after all, though not the one I linked above.
First, after LOTS of searching I found this archive here which seemed to give guidance on how to properly make the request for subdomains:
with app.test_client() as c:
c.get("/target/path.html", base_url="admin.domain.tld/")
There is a caveat, however... Depending on what app.config["SERVER_NAME"] is set to this might not work... Specifically, as of flask 0.11.1 if app.config["SERVER_NAME"] has an explicit port of 80 the above c.get will FAIL. (i.e. if SERVER_NAME looks like domain.tld:80 ) However.. if SERVER_NAME contains any other port it will function properly. (i.e. if SERVER_NAME looks like domain.tld:5000 everything will work fine)
I don't know if this is intended or not.. but it has caused me a lot of headache and work.. I'll be posting a github issue on it shortly just to make sure it is indeed intended.