I have a GET request fetching data that is redirected (302).
Using IE 11 the re-direct does not hit the server and the result of the first request (with identical query string parameters) is returned. Chrome, Firefox and Edge do not have the problem.
My solution was to simply append a &counter=[counter] parameter to the query string and have the [counter] increase by one on every call.
Example:
first call: [query string]&counter=1
second call: [query string]&counter=2
third call: [query string]&counter=3
etc.
Related
Good day,
I am trying to consume a get method on a server hosted locally.
rc = http_url_get('http://sss:13013/cgi-bi/sendms?'
+ 'username=cous&pas'
+ 'to=' + vCell + '&text=' + %TRIM(vSMSText):
'/tmp/httptest.html');
I am using a library called HTTPAPI its from https://www.scottklement.com/httpapi/
I am trying to figure out how to get the response of the call as a string.
The Variable rc contains a integer response code but no response, I am assuming it gets downloaded to the /tmp/httptest.html parameter specified. But I want it as a string as I want to process the response as a string. I do not know if there is a extra parameter that I can use to store the result of the call.
I have tried the http_string method but I get an error that the method is undefined, I think the my client is using an old version of the HTTPAPI lib.
Is it possible to use the http_url_get function to return the response as a string?.
if you want to get a string response, use Klement's HTTP_req(). It is flexible enough to get a file or a string (and use a file or a string to send a request). You can play with *omit to select the way you can make a GET (on other actions).
I am trying to handle an API response with pagination. The first page provides a pagination token to reach the next one, but when I try to feed this back into the params parameter of the requests.get method it seems to slightly encode the token in the wrong way.
My attempt to retrieve the next page (using the response output of the first requests.get method):
# Initial request
response = requests.get(url=url, headers=headers, params=params)
params.update({"paginationToken": response.json()["paginationToken"]})
# Next page
response = requests.get(url=url, headers=headers, params=params)
This fails with status 500: Internal Server Error and message Padding is invalid and cannot be removed.
An example pagination token:
gyuqfh%2bqyNrV9SI1%2bXulE6MXxJgb1VmOu68eH4YZ6dWUgRItb7yJPnO9bcEXdwg6gnYStBuiFhuMxILSB2gpZCLb2UjRE0pp9RkDdIP226M%3d
The url attribute of response seems to show a slightly different token if you look carefully, especially around the '%' signs:
https://www.wikiart.org/en/Api/2/DictionariesByGroup?group=1&paginationToken=gyuqfh%252bqyNrV9SI1%252bXulE6MXxJgb1VmOu68eH4YZ6dWUgRItb7yJPnO9bcEXdwg6gnYStBuiFhuMxILSB2gpZCLb2UjRE0pp9RkDdIP226M%253d
For example, the pagination token and url end differently: 226M%3d and 226M%253d. When I manually copy the first part of the url and add in the correct pagination token it does retrieve the information in a browser.
Am I missing some kind of encoding I should apply to the request.get parameters before feeding them back into a new request?
You are right it is some form of encoding, percentage encoding to be precise. It is frequently used to encode URLs. It is easy to decode:
from urllib.parse import unquote
pagination_token="gyuqfh%252bqyNrV9SI1%252bXulE6MXxJgb1VmOu68eH4YZ6dWUgRItb7yJPnO9bcEXdwg6gnYStBuiFhuMxILSB2gpZCLb2UjRE0pp9RkDdIP226M%253d"
pagination_token = unquote(pagination_token)
print(pagination_token)
Outputs:
gyuqfh%2bqyNrV9SI1%2bXulE6MXxJgb1VmOu68eH4YZ6dWUgRItb7yJPnO9bcEXdwg6gnYStBuiFhuMxILSB2gpZCLb2UjRE0pp9RkDdIP226M%3d
But I expect that is half your problem, use a requests session object https://requests.readthedocs.io/en/master/user/advanced/#session-objects to make the requests as there is most likely a cookie which will be sent with the request to be used in conjunction with the pagination token. I can not tell for sure as the website is currently down.
I'm trying to make a query to Here Maps API with JavaScript to calculate a route with waypoints, where the waypoints are of type "passThrough", the actual produced URL is (I just removed the API key):
https://router.hereapi.com/v8/routes?xnlp=CL_JSMv3.1.21.3&apikey={API_KEY_HERE}&routingMode=fast&transportMode=truck&origin=25.900672%2C-80.253709&destination=40.213615%2C-97.188347&unit=imperial&truck=%5Bobject%20Object%5D&return=polyline%2CtravelSummary&via=40.052839%2C-87.410475!passThrough%3Dtrue
This query returns an error response, even when I'm following the documentation. Here is the problem I found,
If I paste this URL in the browser and remove "%3D" after "passThrough" from the URL, and explicitly change it to "=", the API then returns the expected response. Have to clarify that the URL from above works with curl -X GET. So I really think that the Here Maps API is not decoding the URL, even when they say that special characters have to be encoded.
Any clue on this?
Am I wrong?
Im getting a different URL from what was initially displayed when tried on a browser
Facebook's docs say that a
Login Request
should have a format like this so using requests and urllib.parse I tried getting the response URL
import requests, facebook, logging
# REQUIRED AUTHENTICATION PARAMS
APP_ID = '1976346389294466'
APP_SECRET = '*************************'
REDIRECT_URI = 'https://www.facebook.com/connect/login_success.html/'
logging.basicConfig(level=logging.DEBUG)
perms = ['manage_pages','publish_pages']
fb_login_url = facebook.auth_url(app_id=APP_ID, canvas_url=REDIRECT_URI, perms=perms)
logging.debug("-----LOGIN URL:" + fb_login_url)
response = requests.get(fb_login_url, params={'response_type':'token'}, allow_redirects=True)
try:
response.raise_for_status()
except Exception as exec:
print("%(There was a problem)s" % (exec))
response = requests.get(response.url)
logging.debug("-----Response URL: "+response.url)
I'm expecting a Expected Return URL in the format of
https://www.facebook.com/connect/login_success.html#
access_token=ACCESS_TOKEN...
However, I'm only getting the correct response when I use a browser, on my program the response returns a URL of an entirely different format
https://www.facebook.com/login.php?skip_api_login=1&api_key=xxxxxxxxx&signed_next=1&next=https%3A%2F%2Fwww.facebook.com%2Fv2.11%2Fdialog%2Foauth%3Fredirect_uri%3Dhttps%253A%252F%252Fwww.facebook.com%252Fconnect%252Flogin_success.html%252F%26scope%3Dmanage_pages%252Cpublish_pages%26response_type%3Dtoken%26client_id%xxxxxxxxxxx%26ret%3Dlogin%26logger_id%xxxxxxxxxxxxxxx&cancel_url=https%3A%2F%2Fwww.facebook.com%2Fconnect%2Flogin_success.html%2F%3Ferror%3Daccess_denied%26error_code%3D200%26error_description%3DPermissions%2Berror%26error_reason%3Duser_denied%23_%3D_&display=page&locale=en_US&logger_id=xxx-xxxxx-xxxxx-xxxxxxxx
When I GET from the last redirect url from response.history,
the response returns a url to itself, so I'm not sure how to go about capturing
the initial value of the url such as when I use the browser
the thing is, Im not looking for anything else from the response besides the URL itself.
Additional Notes:
-in the browser after getting the response url I think javascript also changes the url to blank after a brief moment for security reasons
-When I enter the wrong formatted url to the browser, it redirects to the right value so is there something thats handling the response differently when I'm using the browser. If so, how do grab the right url?
Simply put
When I enter fb_login_url in browser I get...
https://www.facebook.com/connect/login_success.html#access_token=ACCESS_TOKEN...
which is what I want, but
when I do it in the app with requests...
either with requests.get(fb_login_url).url
OR (because of a 303) something like
for r in response.history:
requests.get(r.url).url
i get the wrong url which is
https://www.facebook.com/login.php?skip_api_login=1&api_key=xxxxxxxxx&signed_next=1&n....
So here are 2 requests:
http://example.com/someUrl/
http://example.com/someUrl/index.xhtml (xhtml extension is not relevant just an example)
When the <welcome-file>index.xhtml</welcome-file> is been set, request 1 is handled by the server as 2.
However, in both cases the request.getRequestURI() returns the complete URI: someUrl/index.xhtml.
According to documentation it shouldn't but in most cases it's what we want so it seems fine it does.
I'm working with JSF under JBoss Wildfly (Undertow webservice) and I don't know which one is responsible.
I don't necessarily want to change how it works but I'm looking for a way of getting the original URI as the enduser sees in browser address bar, thus without the index.xhtml part in case of 1.
To be more precise, I have to get the exact same URL as returned by document.location.href in JavaScript.
The welcome file is been displayed by a forward which is under the server's covers been performed by RequestDispatcher#forward(). In that case, the original request URI is available as a request attribute with a key as identified by RequestDispatcher#FORWARD_REQUEST_URI, which is javax.servlet.forward.request_uri.
So, this should do:
String originalURI = request.getAttribute(RequestDispatcher.FORWARD_REQUEST_URI);
if (originalURI == null) {
originalURI = request.getRequestURI();
}
// ...