I want to save the data column from this url to an array in python. I tried it with, for instance, pandas.save_table:
import pandas as pd
pd.read_table('https://adventofcode.com/2019/day/1/input', sep='')
but I get HTTPError: HTTP Error 400: Bad Request and I think this is not the right way to do that.
Can someone help me with that?
If you try to open the link in your question (in a browser using incognito mode or something similar i.e. delete your cookies) you'll see that you need login into the website to access the page. This is why the you're getting a 400 Bad Request error as a response from the server.
From the FAQ section of the website that you're trying to access:
How does authentication work? Advent of Code uses OAuth to confirm
your identity through other services. When you log in, you only ever
give your credentials to that service - never to Advent of Code. Then,
the service you use tells the Advent of Code servers that you're
really you. In general, this reveals no information about you beyond
what is already public; here are examples from Reddit and GitHub.
Advent of Code will remember your unique ID, names, URL, and image
from the service you use to authenticate.
The website uses OAuth to handle logins to the url that you create will need these access tokens. You can use a library like python-oauth2 to help you with this (there are others so you can read around and decide which you'd like to use). Creating and understanding how to make http requests is beyond the scope of this answer. I'd suggest you have a look around on the internet for some explanations and try again, if you have get stuck please ask another question. Otherwise it'll probably be easier to save the file from your browser...But I'll leave this answer here for the next person who runs into the same problem.
Related
I already set up the application proxy and it works fine. The problem is I need to pass info about the Microsoft user to the application that the internal URL is pointing to, and I haven't found a way to change the url dynamically, much less create a custom body, to pass on said information.
If there isn't any way to send said info over url or request body, would it be possible over cookies? If so I'd love it if you could link me to some sources to read up on it as I have no prior experience working with cookies and have absolutely no idea about how they work:\
I'd really appreciate any help you can give me, I googled a bunch and am yet to find an answer. Also please don't crucify me if the answer is obvious lol, I'm trying:(
Most of the site sources, opens with a simple request, usualy by file_gets_contents() or curl_init().
I've tried a lot of combinations of stream_context_create() and curl_setopt(), and none returned any thing different of 400 bad request.
Is there an explanation for why some server-sites ( like https://phys.org/ ) do not return de source code by quoted methods?
obs.: if you were able to get the source of the exemple ( https://phys.org/ ), using file_gets_contents() or curl_init(), or any other method with php, please post the code, thanks.
Some Website's are validating the request if it comes from a real/allowed client (bot/user).
This can have multiple reasons.
Maybe the bots are sending to many requests, or the specific site is blocked behind a paywall/firewall. But there are many other people who can explain it to you better then me.
Here are some known Examples how they did it:
Some Site's are supporting request with an API-Token.
Google API's are an great example.
Some Site's are validing the User-Agent.
It looks like that your example site is doing this.
When I'm sending a custom User-Agent Header the result is returning to an error.
And Of Course can some site's check for the User IP Address :)
I believe in your example there should be a good solution to get a result.
I was trying to do some simple authorization for ameritrade's developer platform. I was attempting.
According to the platform, the Endpoint I need to access is is:
https://auth.tdameritrade.com/auth?response_type=code&redirect_uri={uri}&client_id={client_id}}%40AMER.OAUTHAP
https://developer.tdameritrade.com/content/simple-auth-local-apps
When looking at the client_id, for the dev application, I was noticing that they may actually be referencing the Applications, Consumer Key instead? So i did just that, but when attempting to query the information, it returns: A third-party application may be attempting to make unauthorized access to your account. The reason why i think it is the consumer key, is listed at: https://developer.tdameritrade.com/content/getting-started
So I ended up doing something like:
from urllib.parse import urlencode, quote_plus
url = "https://auth.tdameritrade.com/auth?response_type=code&redirect_uri={uri}&client_id={client_id}}%40AMER.OAUTHAP".format(
uri=urlencode("http://localhost", quote_via=quote_plus),
client_id="JHBDFGJH45OOUDFHGJKSDBNG" #Sample
)
I dont think this is because I am currently in a different country currently, I think that something else is wrong here.
It doesnt follow through with it, but instead returns a 400 error with that information. Im not sure whats wrong though.
This happens when you copied the callback URI incorrectly. Imagine if this were a client application, and TD detected that the application is trying to send the user to a different URL than the app is configured with. If they send the callback request to that application, it will receive the token and gain full control over your account.
Have you double and triple checked that you're copying the callback URL correctly, protocol name, ports, and trailing slashes and everything? Also, consider using an API library instead of writing your own. You can find documentation about this specific error here.
I had this issue and I solved it using simply using http://127.0.0.1 on the call back URI of the App.
I then used below URL and it worked as expected.
https://auth.tdameritrade.com/auth?response_type=code&redirect_uri=http%3A%2F%2F127.0.0.1&client_id={MyConsumerKey}%40AMER.OAUTHAP
Just in case anyone is still having this problem, make sure the callback URI is spelled EXACTLY the same as you specified when creating the app. I was having this problem because I set the callback on the TD developer website to "https://localhost/" and used "https://localhost" in the URL instead (missing the slash at the end). As soon as I added the slash at the end, it worked.
I found out that the issue is caused by the way the callback URL is set. It have to be exactly the same as the callback URL you have typed in at the apps details on the TD developer API page. I tried several permutations and indeed to get the authorization to work both have to be the same. eg. https or http.. end with '/' or does not, it matters. There is also no need to URL encode it.
I am trying to automate some simple updating of a Google spreadsheet and I'm using the gspread library to do so. One task that is critical and not currently supported by gspread is the ability to add comments to a specific cell (there's an open issue for this and even a gist solution but I was getting a 404 error when trying to use it).
I know that the Google Drive API (v3) supports adding comments as described here, but I'm having issues with authenticating and could use some help.
What I have/know:
I have already setup the OAuth 2.0 and registered for the API through Google, as well as have the client_secret.json in my directory, but my knowledge of web requests and responses is limited so going through the Drive API documentation hardly makes sense. I know in order to create the comments I will have to make use of anchors and specify the cell location using column/row numbers.
What I'm stuck on:
When using the Google API Explorer, I'm getting a 400 error with the message: The 'fields' parameter is required for this method. How can I make the POST request using my authentication? I think from there I'd be able to actually add the comments myself.
I'm getting a 400 error with the message: The 'fields' parameter is required for this method
The error is asking for a property which you want returned (these properties are listed in Drive API files resource).
You can just place ' * ' to indicate you want it to return a complete response. That's the quick fix.
I'm trying to create a script to generate Spotify playlists with Node (using the spotify-web-api-node module), and I'm having problems getting authentication. I'm terrible with OAuth, so bear with me through this question.
If I understand correctly, I need to use the authorization code flow to get an access token, and to do it that way, I would need to visit a webpage to get the code. So my question is, is there any way to get the code without having to actually visit the webpage?
I tried using request to see if it would show a redirect, but it just shows me a website from Spotify. If I have to manually allow access to an account the first time, that's fine. I'm just trying to have to take away as much repetition as I can. Also, I'm trying to have this be specific to one account, if it could make a difference.