this might not be an issue per say but here is my problem : I'm using childprocess.execSync to execute curl request on pages I need the html content of. It works perfectly fine for valid pages but if the curl request to a page encounters pretty much any error code such as 404 or 403, then the result of the execSync is empty and I have no way to know what error code the curl encountered.
Is there any way to know the curl error code that happens during the childprocess.execSync ?
node.js version : 8.16.2
Well, I found a way, look like I just had to pass "-i" with my curl request to return the response headers too and not just the body, now I can just parse that and see the error code I got.
Related
I have a curl command. I want to execute in in Python and fetch the response to pass it through other code.
curl https://api.box.com/oauth2/token -d 'grant_type=refresh_token' -d 'refresh_token=Ew38UXVKS3kc0axFt6MdDklUnoHpipxTzDWBmKlXAG9ImvGafbbaUhQndv89e677' -d 'client_id=3qkm5h4wh765b3khiws0z8hdpkc56jhs' -d 'client_secret=h9AeXzZL3KATJuaHmimFFBRBDZQrp9tr' -X POST
How can I execute the script in Python and get the response and pass it through other code?
When I am executing the curl script in CMD, I am getting this response:
{"access_token":"uz843jpIiEWnu0CcuT9as2XbA3UEQTR67","expires_in":4261,"restricted_to":[],"refresh_token":"GsDaP6VyUpHN8vDHbz9ktAjLfMLN0dFL6PMIK4fmDH8eKRqR360vDhQTBhIMZxy67","token_type":"bearer"}
From the above response I need to take the access_token value.
Like avloss said - try out requests.
Another great resource is the application Postman
It will let you try out any http calls you'd like, and then can also translate that into/out of curl and/or python requests code for you (and a bunch of other languages).
I find it really useful when trying to figure how to use requests for anything more than simple http calls.
you should use requests library (pip install requests)
import requests
url = 'https://api.box.com/oauth2/token'
data = {
'grant_type':'refresh_token',
'refresh_token':'***',
'client_id':'***',
'client_secret':'***'
}
response = requests.post(url, data).json()
print(response)
I have come into a trouble that when using execSync in node.js, it's not working as directly type the command in the shell.
Here is my issue:
I use a curl to request for some data from a server, and I need to do that with a cookie because there is a login requirement.
It's easy to handle the login process and get the cookie, but it's weird that using the cookie with a curl in node.js would cause the server an "internal error". And since I don't have the permission to change the server-code, I'm looking for help about the difference of calling curl in Node.js and directly use curl.
Here is the code:
var command = 'curl --cookie cookie.txt ' + getURL();
console.log(command);
// output: curl --cookie cookie.txt http://example.com/getdata
var result = child_process.execSync(command).toString();
// will cause an internal error and the "result" is an error-reporting page.
Directly calling this in the shell:
curl --cookie cookie.txt http://example.com/getdata
Everything works, I got the data I need.
I tried to find some plots, for instance, changing the code to:
var command = 'curl --cookie cookie-bad.txt ' + getURL();
I put some wrong cookie in the cookie-bad.txt, I will get a "you are not log in" result.
So there must be something wrong with:
sending a cookie to the server to request some data with curl running inside a nodejs script with execSync.
Is there any way I can improve the code or something?
What is your Node.js version? I don't have any problem with 10.16.0.
I am trying to convert speech to text using Nuance so i am trying to send this request
curl "https://dictation.nuancemobility.net:443/NMDPAsrCmdServlet/dictation?appId=[INSERT YOUR APP ID]&appKey=[INSERT YOUR 128-BYTE STRING APP KEY]&id=C4461956B60B" -H "Content-Type: audio/x-wav;codec=pcm;bit=16;rate=16000" -H "Accept-Language: ENUS" -H "Transfer-Encoding: chunked" -H "Accept: application/xml" -H "Accept-Topic: Dictation" -k --data-binary #audio_16k16bit.pcm
need to upload audio file (.pcm) format.
I am using okhttp3 library following is the builder
RequestBody requestBody = new MultipartBody.Builder().setType(MultipartBody.FORM)
.addFormDataPart("file", "audio_16k16bit.pcm", RequestBody.create(MEDIA_TYPE_PNG, "audio_16k16bit"))
.build();
httpBuider.addQueryParameter("appId", "NMDPTRIAL_XXXXXXX_XXX_com20161122071457").addQueryParameter("appKey", "fadaed7b801e10d84272c0a75317d8cee13ab86ae902ab322cd6e1219fcbe79aa5d41526f225fe3497bfdbead6b4b9b7ee7122d773cd0a9fa3ebc042b7a7dc5c");
Request request = new Request.Builder().addHeader("Content-Type","audio/x-wav;codec=pcm;bit=16;rate=16000").addHeader("Accept-Language","eng-GBR").addHeader("Transfer-Encoding","chunked").addHeader("Accept","application/xml").addHeader("Accept-Topic","Dictation").post(requestBody).url(httpBuider.build()).build();
I am getting following log
HTTP ERROR 500
Problem accessing /NMDPAsrCmdServlet/dictation. Reason:
Server Error
Missing anything?
I have experienced the same just now using Jersey. Problem for me was, that Jersey was overwriting the content type header (I tried with Application_Octet_Stream).
Here is the request with which I finally got it to work:
Response myResponse = target.
request().
accept(MediaType.TEXT_PLAIN_TYPE).
header("Accept-Language","DEDE").
header("Accept-Topic", "Dictation").
header("Transfer-Encoding","chunked").
post(Entity.entity(speechStream, "audio/x-wav;codec=pcm;bit=16;rate=16000"));
I suggest using something like fiddler to find out what is really posted and comparing that to the curl post. This is how I finally found out what was wrong with my request.
Remove the header "Transfer-Encoding","chunked". My error was fixed by removing it.
So I'm trying to download a file through nodejs that opens fine in the browser, and even downloads fine in tools like curl.
But nodejs just fails for some reason to download the file. I tried downloading the file through the request module in node and through a node cli module called download-cli. Both of them fail with either a 400 or 404 response yet the file downloads fine through regular tools like curl.
What could be the issue? I have tried setting the user-agent to that of Firefox (where it opens just fine) but that doesn't do the trick. I'm assuming the problem isn't about the user-agent anyway since curl doesn't have its own user-agent.
The url in question can be any url from alicdn but lets take this one as an example:
https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-часы-поддержка-синхронизации-notifier-sim-карты-подключение-bluetooth-для-android-apple-iphone.jpg_640x640.jpg
Here's the response by running the above url through the node download-cli tool and the Invoke-WebRequest tool in powershell.
PS C:\code> download https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-часы-поддержка-син
хронизации-notifier-sim-карты-подключение-bluetooth-для-android-apple-iphone.jpg_640x640.jpg
Couldn't connect to https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-часы-поддержка-синхронизации-notifier-sim-карты-подключение-bluetooth-для-android-apple-iphone.jpg_640x640.jpg (404)
PS C:\code> curl https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-часы-поддержка-синхрон
изации-notifier-sim-карты-подключение-bluetooth-для-android-apple-iphone.jpg_640x640.jpg
StatusCode : 200
StatusDescription : OK
Content : {255, 216, 255, 224...}
RawContent : HTTP/1.1 200 OK
X-Application-Context: fileserver2-download:prod:7001
From-Req-Dns-Type: NA,NA
SERVED-FROM: 72.247.178.95
Connection: keep-alive
Network_Info: DE_FRANKFURT_16509
Timing-Allow-Ori...
Headers : {[X-Application-Context, fileserver2-download:prod:7001], [From-Req-Dns-Type, NA,NA], [SERVED-FROM, 72.247.178.95],
[Connection, keep-alive]...}
RawContentLength : 114927
Okay so I tried downloading the file through node's native http module, I tried downloading through the popular request module AND I tried downloading through a node based cli tool called download-cli. Everyone of them had the same response.
So I fired up Wireshark and tried to see exactly where the requests are different and it turns out that tools like curl and Invoke-WebRequest escape the path before making a GET request but node's native module doesn't do that. That was the only difference. Using the escaped url works fine.
Invoke-WebRequest's GET path:
GET /kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-%D1%87%D0%B0%D1%81%D1%8B-%D0%BF%D0%BE%D0%B4%D0%B4%D0%B5%D1%80%D0%B6%D0%BA%D0%B0-%D1%81%D0%B8%D0%BD%D1%85%D1%80%D0%BE%D0%BD%D0%B8%D0%B7%D0%B0%D1%86%D0%B8%D0%B8-notifier-sim-%D0%BA%D0%B0%D1%80%D1%82%D1%8B-%D0%BF%D0%BE%D0%B4%D0%BA%D0%BB%D1%8E%D1%87%D0%B5%D0%BD%D0%B8%D0%B5-bluetooth-%D0%B4%D0%BB%D1%8F-android-apple-iphone.jpg_640x640.jpg HTTP/1.1
Node's GET path:
GET /kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-G0AK-?>445#6:0-A8=E#>=870F88-notifier-sim-:0#BK-?>4:;NG5=85-bluetooth-4;O-android-apple-iphone.jpg_640x640.jpg HTTP/1.1
why you didnt do it :
$url='https://ae01.alicdn.com/kf/HTB1ftVmPVXXXXXUXVXXq6xXFXXXG/Langtek-smart-watch-gt12-%D1%87%D0%B0%D1%81%D1%8B-%D0%BF%D0%BE%D0%B4%D0%B4%D0%B5%D1%80%D0%B6%D0%BA%D0%B0-%D1%81%D0%B8%D0%BD%D1%85%D1%80%D0%BE%D0%BD%D0%B8%D0%B7%D0%B0%D1%86%D0%B8%D0%B8-notifier-sim-%D0%BA%D0%B0%D1%80%D1%82%D1%8B-%D0%BF%D0%BE%D0%B4%D0%BA%D0%BB%D1%8E%D1%87%D0%B5%D0%BD%D0%B8%D0%B5-bluetooth-%D0%B4%D0%BB%D1%8F-android-apple-iphone.jpg_640x640.jpg'
Invoke-WebRequest -Uri $url -OutFile C:\temp\android-apple-iphone.jpg_640x640.jpg
I am trying to install SBT following the instructions mentioned in:
here
But I am getting an error while running command:
wget https://dl.bintray.com/sbt/native-packages/sbt/0.13.8/sbt-0.13.8.tgz
The error is:
--2016-08-16 11:39:16-- https://dl.bintray.com/sbt/native-packages/sbt/0.13.8/sbt-0.13.8.tgz
Resolving dl.bintray.com... 108.168.243.150, 75.126.118.188 Connecting
to dl.bintray.com|108.168.243.150|:443... connected. HTTP request
sent, awaiting response... 302 Location:
https://akamai.bintray.com/15/155d6ff3bc178745ad4f951b74792b257ed14105?gda=exp=1471366276~hmac=1332caeed34aa8465829ba9f19379685c23e33ede86be8d2b10e47ca4752f8f0&response-content-disposition=attachment%3Bfilename%3D%22sbt-0.13.8.tgz%22&response-content-type=application%2Foctet-stream&requestInfo=U2FsdGVkX19rkawieFWSsqtapFvvLhwJbzqc8qYcoelvh1%2BUW9ffT9Q4RIJPf%2B2WqkegCpNt2tOXFO9VlWuoGzk1Wdii9dr2HpibwrTfZ92pO8iqdjNbL%2BDzZTYiC826
[following]
--2016-08-16 11:39:16-- https://akamai.bintray.com/15/155d6ff3bc178745ad4f951b74792b257ed14105?gda=exp=1471366276~hmac=1332caeed34aa8465829ba9f19379685c23e33ede86be8d2b10e47ca4752f8f0&response-content-disposition=attachment%3Bfilename%3D%22sbt-0.13.8.tgz%22&response-content-type=application%2Foctet-stream&requestInfo=U2FsdGVkX19rkawieFWSsqtapFvvLhwJbzqc8qYcoelvh1%2BUW9ffT9Q4RIJPf%2B2WqkegCpNt2tOXFO9VlWuoGzk1Wdii9dr2HpibwrTfZ92pO8iqdjNbL%2BDzZTYiC826
Resolving akamai.bintray.com... 23.193.25.35 Connecting to
akamai.bintray.com|23.193.25.35|:443... connected. HTTP request sent,
awaiting response... 200 OK Length: 1059183 (1.0M)
[application/octet-stream]
155d6ff3bc178745ad4f951b74792b257ed14105?gda=exp=1471366276~hmac=1332caeed34aa8465829ba9f19379685c23e33ede86be8d2b10e47ca4752f8f0&response-content-disposition=attachment;filename="sbt-0.13.8.tgz"&response-content-type=application%2Foctet-stream&requestInfo=U2FsdGVkX19rkawieFWSsqtapFvvLhwJbzqc8qYcoelvh1+UW9ffT9Q4RIJPf+2WqkegCpNt2tOXFO9VlWuoGzk1Wdii9dr2HpibwrTfZ92pO8iqdjNbL+DzZTYiC826:
File name too long
Cannot write to
“155d6ff3bc178745ad4f951b74792b257ed14105?gda=exp=1471366276~hmac=1332caeed34aa8465829ba9f19379685c23e33ede86be8d2b10e47ca4752f8f0&response-content-disposition=attachment;filename="sbt-0.13.8.tgz"&response-content-type=application%2Foctet-stream&requestInfo=U2FsdGVkX19rkawieFWSsqtapFvvLhwJbzqc8qYcoelvh1+UW9ffT9Q4RIJPf+2WqkegCpNt2tOXFO9VlWuoGzk1Wdii9dr2HpibwrTfZ92pO8iqdjNbL+DzZTYiC826”
(Success).
I referred StackOverflow Link with similar issue but I am not able to figure out what is the problem.
I don't think it's really an issue with the filename. I was able to use that same command without a problem. If the filename was an issue, you could always use this to save it as a different filename:
wget -O newname.tgz https://dl.bintray.com/sbt/native-packages/sbt/0.13.8/sbt-0.13.8.tgz
The other option is to use bitly and get a URL if the URL is just too long.
But it could be a space issue. Do you have enough disk space? Check with df -h to see your available space.