I am trying to make a post call to IPv6 server using python-requests module on linux. This gives me a 502 Bad Gateway error.
Here are the details of the versions:
Python version: 2.7
Requests module version: 2.22
But I have tried to make the same request from a windows machine(using requests 2.21), and the request is successful.
payload = {
'username' : "abc",
'password' : "xxxx"
}
response = requests.post('https://[2606:b400:605:b857::489]/xyz/auth/tokens', data=payload, verify=False)
print response
I get a 502 Bad Gateway Response Code
This is probably because you didn't add headers required to make a request. Sometimes the request may accept only specific Content-Type
Try capturing the request and have a look to find out what's the issue.
Related
I have a curl command. I want to execute in in Python and fetch the response to pass it through other code.
curl https://api.box.com/oauth2/token -d 'grant_type=refresh_token' -d 'refresh_token=Ew38UXVKS3kc0axFt6MdDklUnoHpipxTzDWBmKlXAG9ImvGafbbaUhQndv89e677' -d 'client_id=3qkm5h4wh765b3khiws0z8hdpkc56jhs' -d 'client_secret=h9AeXzZL3KATJuaHmimFFBRBDZQrp9tr' -X POST
How can I execute the script in Python and get the response and pass it through other code?
When I am executing the curl script in CMD, I am getting this response:
{"access_token":"uz843jpIiEWnu0CcuT9as2XbA3UEQTR67","expires_in":4261,"restricted_to":[],"refresh_token":"GsDaP6VyUpHN8vDHbz9ktAjLfMLN0dFL6PMIK4fmDH8eKRqR360vDhQTBhIMZxy67","token_type":"bearer"}
From the above response I need to take the access_token value.
Like avloss said - try out requests.
Another great resource is the application Postman
It will let you try out any http calls you'd like, and then can also translate that into/out of curl and/or python requests code for you (and a bunch of other languages).
I find it really useful when trying to figure how to use requests for anything more than simple http calls.
you should use requests library (pip install requests)
import requests
url = 'https://api.box.com/oauth2/token'
data = {
'grant_type':'refresh_token',
'refresh_token':'***',
'client_id':'***',
'client_secret':'***'
}
response = requests.post(url, data).json()
print(response)
this might not be an issue per say but here is my problem : I'm using childprocess.execSync to execute curl request on pages I need the html content of. It works perfectly fine for valid pages but if the curl request to a page encounters pretty much any error code such as 404 or 403, then the result of the execSync is empty and I have no way to know what error code the curl encountered.
Is there any way to know the curl error code that happens during the childprocess.execSync ?
node.js version : 8.16.2
Well, I found a way, look like I just had to pass "-i" with my curl request to return the response headers too and not just the body, now I can just parse that and see the error code I got.
I have a python script which does a GET request to GitLab and stores the data from the response in an excel file using tablib library.
This script works fine in Windows when I execute it using python3.
I have tried to execute the same script in the Windows Subsystem for Linux (WSL) I have enabled and the script fails.
The output when I execute with python3 script.py in WSL is the following:
RESPONSE {"message":"404 Project Not Found"}
When I execute from Windows using python .\gitlab.py where python is python3:
RESPONSE [{"id":567,"iid":22}, {"id":10,"iid":3}]
I think the problem could be related to the GET api call I am doing because in WSL it returns Project Not Found.
I executed that request using curl in WSL to see if the unix in general has this issue, but I get back the expected response instead of the not found response. This was the request:
curl -X GET 'https://URL/api/v4/projects/server%2Fproducts%2FPROJECT/issues?per_page=100' -H 'Content-Type: application/json' -H 'PRIVATE-TOKEN: TOKEN' --insecure
Why is python failing in unix using Python if unix is able to execute the get request using curl? Should I enable/disable something in the request perhaps?
This is the request I am doing in my python script:
def get_items():
url = "https://URL/api/v4/projects/server%2Fproducts%2FPROJECT/issues"
payload = {}
querystring = {"state": "closed", "per_page": "100"}
headers = {
'Content-Type': "application/json",
'PRIVATE-TOKEN': os.environ.get("GITLAB_KEY") # enviromental variable added in windows
}
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
response = requests.request(
"GET", url, headers=headers, data=payload, params=querystring, verify=False)
print("RESPONSE " + response.text)
return json.loads(response.text)
UPDATE:
I have tried using the project id as well instead of the path but it didn't work
REF: https://docs.gitlab.com/ee/api/projects.html#get-single-project
GET /projects/:id
Change this:
url = "https://URL/api/v4/projects/server%2Fproducts%2FPROJECT/issues"
To
projectId = 1234 # or whatever your project id is ... Project Page, Settings -> General
url = "https://URL/api/v4/projects/" + projectId + "/issues"
Based on an answer I got in the post I did in Reddit, I found the problem.
In the python script, I am using an environmental variable which is not accessible in that way ( os.environ.get("GITLAB_KEY") ) from the WSL.
For now, I have replaced it with the hard-coded value just to check that this was really the issue. The script now works as expected.
I will find a way to access the env var again now that I know what the problem was.
I'm trying to authenticate to a server using the requests_kerberos package, following the instructions here:
https://github.com/requests/requests-kerberos
import requests
from requests_kerberos import HTTPKerberosAuth
kerberos_auth = HTTPKerberosAuth()
r = requests.get(<myserver>, auth=kerberos_auth)
r.text
And here is the response:
'Apache Tomcat/6.0.53 - Error report HTTP Status 401 - Authentication requiredtype Status reportmessage Authentication requireddescription This request requires HTTP authentication.Apache Tomcat/6.0.53'
klist shows that I have a valid TGT.
I have tried setting the principal directly, but that didn't help. I can authenticate using curl:
curl -i -L --negotiate -u : "<server>"
I'm not sure what else to try; everything is happening "behind the scenes" so I don't know what I'm doing wrong.
So I'm trying to download a file through nodejs that opens fine in the browser, and even downloads fine in tools like curl.
But nodejs just fails for some reason to download the file. I tried downloading the file through the request module in node and through a node cli module called download-cli. Both of them fail with either a 400 or 404 response yet the file downloads fine through regular tools like curl.
What could be the issue? I have tried setting the user-agent to that of Firefox (where it opens just fine) but that doesn't do the trick. I'm assuming the problem isn't about the user-agent anyway since curl doesn't have its own user-agent.
The url in question can be any url from alicdn but lets take this one as an example:
https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-часы-поддержка-синхронизации-notifier-sim-карты-подключение-bluetooth-для-android-apple-iphone.jpg_640x640.jpg
Here's the response by running the above url through the node download-cli tool and the Invoke-WebRequest tool in powershell.
PS C:\code> download https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-часы-поддержка-син
хронизации-notifier-sim-карты-подключение-bluetooth-для-android-apple-iphone.jpg_640x640.jpg
Couldn't connect to https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-часы-поддержка-синхронизации-notifier-sim-карты-подключение-bluetooth-для-android-apple-iphone.jpg_640x640.jpg (404)
PS C:\code> curl https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-часы-поддержка-синхрон
изации-notifier-sim-карты-подключение-bluetooth-для-android-apple-iphone.jpg_640x640.jpg
StatusCode : 200
StatusDescription : OK
Content : {255, 216, 255, 224...}
RawContent : HTTP/1.1 200 OK
X-Application-Context: fileserver2-download:prod:7001
From-Req-Dns-Type: NA,NA
SERVED-FROM: 72.247.178.95
Connection: keep-alive
Network_Info: DE_FRANKFURT_16509
Timing-Allow-Ori...
Headers : {[X-Application-Context, fileserver2-download:prod:7001], [From-Req-Dns-Type, NA,NA], [SERVED-FROM, 72.247.178.95],
[Connection, keep-alive]...}
RawContentLength : 114927
Okay so I tried downloading the file through node's native http module, I tried downloading through the popular request module AND I tried downloading through a node based cli tool called download-cli. Everyone of them had the same response.
So I fired up Wireshark and tried to see exactly where the requests are different and it turns out that tools like curl and Invoke-WebRequest escape the path before making a GET request but node's native module doesn't do that. That was the only difference. Using the escaped url works fine.
Invoke-WebRequest's GET path:
GET /kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-%D1%87%D0%B0%D1%81%D1%8B-%D0%BF%D0%BE%D0%B4%D0%B4%D0%B5%D1%80%D0%B6%D0%BA%D0%B0-%D1%81%D0%B8%D0%BD%D1%85%D1%80%D0%BE%D0%BD%D0%B8%D0%B7%D0%B0%D1%86%D0%B8%D0%B8-notifier-sim-%D0%BA%D0%B0%D1%80%D1%82%D1%8B-%D0%BF%D0%BE%D0%B4%D0%BA%D0%BB%D1%8E%D1%87%D0%B5%D0%BD%D0%B8%D0%B5-bluetooth-%D0%B4%D0%BB%D1%8F-android-apple-iphone.jpg_640x640.jpg HTTP/1.1
Node's GET path:
GET /kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-G0AK-?>445#6:0-A8=E#>=870F88-notifier-sim-:0#BK-?>4:;NG5=85-bluetooth-4;O-android-apple-iphone.jpg_640x640.jpg HTTP/1.1
why you didnt do it :
$url='https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-%D1%87%D0%B0%D1%81%D1%8B-%D0%BF%D0%BE%D0%B4%D0%B4%D0%B5%D1%80%D0%B6%D0%BA%D0%B0-%D1%81%D0%B8%D0%BD%D1%85%D1%80%D0%BE%D0%BD%D0%B8%D0%B7%D0%B0%D1%86%D0%B8%D0%B8-notifier-sim-%D0%BA%D0%B0%D1%80%D1%82%D1%8B-%D0%BF%D0%BE%D0%B4%D0%BA%D0%BB%D1%8E%D1%87%D0%B5%D0%BD%D0%B8%D0%B5-bluetooth-%D0%B4%D0%BB%D1%8F-android-apple-iphone.jpg_640x640.jpg'
Invoke-WebRequest -Uri $url -OutFile C:\temp\android-apple-iphone.jpg_640x640.jpg