I'm trying to make a query to Here Maps API with JavaScript to calculate a route with waypoints, where the waypoints are of type "passThrough", the actual produced URL is (I just removed the API key):
https://router.hereapi.com/v8/routes?xnlp=CL_JSMv3.1.21.3&apikey={API_KEY_HERE}&routingMode=fast&transportMode=truck&origin=25.900672%2C-80.253709&destination=40.213615%2C-97.188347&unit=imperial&truck=%5Bobject%20Object%5D&return=polyline%2CtravelSummary&via=40.052839%2C-87.410475!passThrough%3Dtrue
This query returns an error response, even when I'm following the documentation. Here is the problem I found,
If I paste this URL in the browser and remove "%3D" after "passThrough" from the URL, and explicitly change it to "=", the API then returns the expected response. Have to clarify that the URL from above works with curl -X GET. So I really think that the Here Maps API is not decoding the URL, even when they say that special characters have to be encoded.
Any clue on this?
Am I wrong?
Related
I am trying to handle an API response with pagination. The first page provides a pagination token to reach the next one, but when I try to feed this back into the params parameter of the requests.get method it seems to slightly encode the token in the wrong way.
My attempt to retrieve the next page (using the response output of the first requests.get method):
# Initial request
response = requests.get(url=url, headers=headers, params=params)
params.update({"paginationToken": response.json()["paginationToken"]})
# Next page
response = requests.get(url=url, headers=headers, params=params)
This fails with status 500: Internal Server Error and message Padding is invalid and cannot be removed.
An example pagination token:
gyuqfh%2bqyNrV9SI1%2bXulE6MXxJgb1VmOu68eH4YZ6dWUgRItb7yJPnO9bcEXdwg6gnYStBuiFhuMxILSB2gpZCLb2UjRE0pp9RkDdIP226M%3d
The url attribute of response seems to show a slightly different token if you look carefully, especially around the '%' signs:
https://www.wikiart.org/en/Api/2/DictionariesByGroup?group=1&paginationToken=gyuqfh%252bqyNrV9SI1%252bXulE6MXxJgb1VmOu68eH4YZ6dWUgRItb7yJPnO9bcEXdwg6gnYStBuiFhuMxILSB2gpZCLb2UjRE0pp9RkDdIP226M%253d
For example, the pagination token and url end differently: 226M%3d and 226M%253d. When I manually copy the first part of the url and add in the correct pagination token it does retrieve the information in a browser.
Am I missing some kind of encoding I should apply to the request.get parameters before feeding them back into a new request?
You are right it is some form of encoding, percentage encoding to be precise. It is frequently used to encode URLs. It is easy to decode:
from urllib.parse import unquote
pagination_token="gyuqfh%252bqyNrV9SI1%252bXulE6MXxJgb1VmOu68eH4YZ6dWUgRItb7yJPnO9bcEXdwg6gnYStBuiFhuMxILSB2gpZCLb2UjRE0pp9RkDdIP226M%253d"
pagination_token = unquote(pagination_token)
print(pagination_token)
Outputs:
gyuqfh%2bqyNrV9SI1%2bXulE6MXxJgb1VmOu68eH4YZ6dWUgRItb7yJPnO9bcEXdwg6gnYStBuiFhuMxILSB2gpZCLb2UjRE0pp9RkDdIP226M%3d
But I expect that is half your problem, use a requests session object https://requests.readthedocs.io/en/master/user/advanced/#session-objects to make the requests as there is most likely a cookie which will be sent with the request to be used in conjunction with the pagination token. I can not tell for sure as the website is currently down.
I'm using simple-oauth2 in this example to query Microsoft Graph. All works well so far. But when I try to refresh the access token var newToken = await storedToken.refresh();, I get an error:
The content-type is not JSON compatible
This is thrown in wreck's index.js and it seems like there is no content-type set in the headers, while the mode is set to strict. The problem is, that I have no idea how to change this or why this is happening. It only happens on refresh().
I figured this is a configuration problem. The sample provides the config as follows
OAUTH_AUTHORITY=https://login.microsoftonline.com/common
OAUTH_ID_METADATA=/v2.0/.well-known/openid-configuration
OAUTH_AUTHORIZE_ENDPOINT=/oauth2/v2.0/authorize
OAUTH_TOKEN_ENDPOINT=/oauth2/v2.0/token
wreck uses Url.URL to combine OAUTH_AUTHORITY with OAUTH_TOKEN_ENDPOINT which results in https://login.microsoftonline.com/oauth2/v2.0/token and therefore loses common. This results in a 404 and therefore no JSON response anymore.
I changed the config slightly and removed the leading slashes from the relative paths and added a trailing slash to the base URL.
OAUTH_AUTHORITY=https://login.microsoftonline.com/common/
OAUTH_ID_METADATA=/v2.0/.well-known/openid-configuration
OAUTH_AUTHORIZE_ENDPOINT=oauth2/v2.0/authorize
OAUTH_TOKEN_ENDPOINT=oauth2/v2.0/token
So that OAUTH_TOKEN_ENDPOINT is relative. I have not figured why it worked for authorize though, but still works.
I'm having an issue with a NodeJS REST api created using express.
I have two calls, a get and a post set up like this:
router.get('/:id', (request, response) => {
console.log(request.params.id);
});
router.post('/:id', (request, response) => {
console.log(request.params.id);
});
now, I want the ID to be able to contain special characters (UTF8).
The problem is, when I use postman to test the requests, it looks like they are encoded very differently:
GET http://localhost:3000/api/â outputs â
POST http://localhost:3000/api/â outputs â
Does anyone have any idea what I am missing here?
I must mention that the post call also contains a file upload so the content type will be multipart/form-data
You should encode your URL on the client and decode it on the server. See the following articles:
What is the proper way to URL encode Unicode characters?
Can urls have UTF-8 characters?
Which characters make a URL invalid?
For JavaScript, encodeURI may come in handy.
It looks like postman does UTF-8 encoding but NOT proper url encoding. Consequently, what you type in the request url box translates to something different than what would happen if you typed that url in a browser.
I'm requesting: GET localhost/ä but it encodes it on the wire as localhost/ä
(This is now an invalid URL because it contains non ascii characters)
But when I type localhost/ä in to google chrome, it correctly encodes the request as localhost/%C3%A4
So you could try manually url encoding your request to http://localhost:3000/api/%C3%A2
In my opinion this is a bug (perhaps a regression). I am using the latest version of PostMan v7.11.0 on MacOS.
Does anyone have any idea what I am missing here?
yeah, it doesn't output â, it outputs â, but whatever you're checking the result with, think you're reading something else (iso-8859-1 maybe?), not UTF-8, and renders it as â
Most likely, you're viewing the result in a web browser, and the web server is sending the wrong Content-Type header. try doing header("Content-type: text/plain;charset=utf-8"); or header("Content-type: text/html;charset=utf-8"); , then your browser should render your â correctly.
I wrote a simple API which will return request.query as a response.
The behavior is little different than what I am expecting.
redirectto -- I am getting the only name as part of response redirectto param.
id -- I am getting an array in response.
Why is this behaviour?
Query parameters that contain reserved characters should be URL encoded or they will fail to parse correctly.
The properly encoded URL should look something like this:
http://localhost:8082/redirect?requesttype=click&id=79992&redirectto=http%3A%2F%2Flocalhost%3A8081%2Fredirect%3Fname%3Djohn%26id%3D123
Im getting a different URL from what was initially displayed when tried on a browser
Facebook's docs say that a
Login Request
should have a format like this so using requests and urllib.parse I tried getting the response URL
import requests, facebook, logging
# REQUIRED AUTHENTICATION PARAMS
APP_ID = '1976346389294466'
APP_SECRET = '*************************'
REDIRECT_URI = 'https://www.facebook.com/connect/login_success.html/'
logging.basicConfig(level=logging.DEBUG)
perms = ['manage_pages','publish_pages']
fb_login_url = facebook.auth_url(app_id=APP_ID, canvas_url=REDIRECT_URI, perms=perms)
logging.debug("-----LOGIN URL:" + fb_login_url)
response = requests.get(fb_login_url, params={'response_type':'token'}, allow_redirects=True)
try:
response.raise_for_status()
except Exception as exec:
print("%(There was a problem)s" % (exec))
response = requests.get(response.url)
logging.debug("-----Response URL: "+response.url)
I'm expecting a Expected Return URL in the format of
https://www.facebook.com/connect/login_success.html#
access_token=ACCESS_TOKEN...
However, I'm only getting the correct response when I use a browser, on my program the response returns a URL of an entirely different format
https://www.facebook.com/login.php?skip_api_login=1&api_key=xxxxxxxxx&signed_next=1&next=https%3A%2F%2Fwww.facebook.com%2Fv2.11%2Fdialog%2Foauth%3Fredirect_uri%3Dhttps%253A%252F%252Fwww.facebook.com%252Fconnect%252Flogin_success.html%252F%26scope%3Dmanage_pages%252Cpublish_pages%26response_type%3Dtoken%26client_id%xxxxxxxxxxx%26ret%3Dlogin%26logger_id%xxxxxxxxxxxxxxx&cancel_url=https%3A%2F%2Fwww.facebook.com%2Fconnect%2Flogin_success.html%2F%3Ferror%3Daccess_denied%26error_code%3D200%26error_description%3DPermissions%2Berror%26error_reason%3Duser_denied%23_%3D_&display=page&locale=en_US&logger_id=xxx-xxxxx-xxxxx-xxxxxxxx
When I GET from the last redirect url from response.history,
the response returns a url to itself, so I'm not sure how to go about capturing
the initial value of the url such as when I use the browser
the thing is, Im not looking for anything else from the response besides the URL itself.
Additional Notes:
-in the browser after getting the response url I think javascript also changes the url to blank after a brief moment for security reasons
-When I enter the wrong formatted url to the browser, it redirects to the right value so is there something thats handling the response differently when I'm using the browser. If so, how do grab the right url?
Simply put
When I enter fb_login_url in browser I get...
https://www.facebook.com/connect/login_success.html#access_token=ACCESS_TOKEN...
which is what I want, but
when I do it in the app with requests...
either with requests.get(fb_login_url).url
OR (because of a 303) something like
for r in response.history:
requests.get(r.url).url
i get the wrong url which is
https://www.facebook.com/login.php?skip_api_login=1&api_key=xxxxxxxxx&signed_next=1&n....