I need to login to a website with username and password, and then download a file. The url of the file is static. How do I automate the above process with Linux/Unix scripts? Thanks a lot.
Jiangzhe
well, it's not that simple.
what you need to do is the following:
send an HTTP POST request containing your username and password to the login form's URL.
you will get a cookie (probably containing a session ID).
send an HTTP GET request for the file, sending your cookie details in the HTTP headers.
you probably should use some scripting language with an HTTP library (python's httplib and urllib2 are great options).
Just use CURL to send POST or GET request with login data to site and then do second request to download file.
Related
I want to send some HTTP POST requests to a website. There's no problem with doing it by my browser, since I'm logged in and I have my authenticated session.
I don't know how to make POST requests by Python while using that authenticated session (so importing data from the browser), and I don't even know if it's possibile, so I thought that I could trigger these POST requests in browser from the Python script.
I know there exists a library called "webbrowser" that allows to make requests by opening new tabs in browser. My problem is that I need a POST request, not a GET one.
So I ask you for 2 suggestions: is there a way to import authenticated session such that Pyhon can make requests for websites that require auth?
If not, is there a way of doing something like this:
webbrowser.open(url, new=0)
but with POST requests?
Thanks!
I agree with Mr. C, but alternatively...
Checkout Selenium. You can't make a direct POST request, but you can do anything that you would usually do as a normal user.
Check out the requests library. Seems like you need more functionality than the webbrowser module provides.
I am trying to login to a website and based on the REST API guide provided, I will be able to receive some data for an application.
The two steps required for me to enact are:
1. Send a HTTP post command for authentication.
2. Send a GET command for receiving the data.
When I send the post command using python requests, i receive the required json response showing my login rights. e.g. role admin.
However, when I perform the get command after, it doesn't retrieve the data but sends an HTML form showing that I require authentication even though I have authenticated already.
Has anyone encountered this and how will I be able to solve it?
I am working on this for a customer and as a result, cannot post the actual login I am using and url and will hence replace this with my name with the code I will display.
Thanks
enter image description here
You can take a look at the session object in the requests library. It's used to keep a session across multiple requests.
https://3.python-requests.org/user/advanced/#session-objects
import requests
s = requests.Session()
s.get('http://httpbin.org/cookies/set?authcookie=123')
r = s.get('http://httpbin.org/cookies')
print(r.text)
This is a sample that uses a request session object to call a url that sets a cookie and then does another call with the same session (including the cookie). Just some basic example using http://httpbin.org/.
can any explain me about how the server responds if we click the button in browser using PHP script then which situation we use JAVASCRIPT and which situation we have to use PHP .
If you're asking about how it works, I'll suggest to google about HTTP protocol, POST and GET methods.
Usually, you use JavaScript to send request to a server when you want it to be AJAX, ie when you want to send request to server and receive response without reloading page. And you use PHP when you have form in your HTML (like login or registration form), and you send it to server in one request to go to another page
How to do following actions using Node.js http POST, GET?
Log in to abc.com(It is a login page with login form)
In the page after log in, fill another form using POST and GET the result after posting the form.
Use the request module: https://www.npmjs.com/package/request. Inspect the login form, look at it's method and action, the using request send a request of the method type to the action with the needed form data. Then use the response's origin to see what the route of the next page is, and then POST to that route. You shouldn't need the GET then, it should respond with the result.
But instead of all this, if the site has supports oauth, you'd be much better off authenticating doing that with passport.js and using their api.
I am looking for an easy way to re-send POST request to the server within the browser mainly for debug purposes. Say you have a XHR request which contains POST parameters that is to be send to the server. After having changed the script on the server side, you would like to resent the very same request for analyzing the output.
What tool could help? I guess it is a browser's extension.
I already tried extension Tamper Data for Firefox which does the job as you can "Replay in browser". But for my taste, it is not enough straight forward, as there are 3 - 4 clicks to get the result of the request.
Unfortunately, curl would not be suitable for my needs as my application has a session's cookie.
You can use cookies with cUrl. If the cookies is created on login, you can login using cUrl, saving the cookie to a file:
curl -c cookies.txt -d "user=username&pass=password" http://example.com/login
And then to load them for the subsequent requests:
curl -b cookies.txt ...
If for some reason the cookie can't be created using a cUrl request, you can use the Firefox extension Cookie Exporter, which produces files that can be read by curl using the -b flag.