Node Libcurl (node-libcurl) - trouble with multipart/form-data request - node.js

I have a particularly stubborn api that I've only been able to get to work with the following curl request:
curl --request PUT -H "Content-Type: multipart/form-data" -H "Authorization: Bearer abcd" -F functionConfig='{"parallelism":2};type=application/json' http://test-pulsar:8080/admin/v3/functions/test/ingest/test-entity-FeedTransformer
Here is the --libcurl output
#include <curl/curl.h>
int main(int argc, char *argv[])
{
CURLcode ret;
CURL *hnd;
curl_mime *mime1;
curl_mimepart *part1;
struct curl_slist *slist1;
mime1 = NULL;
slist1 = NULL;
slist1 = curl_slist_append(slist1, "Content-Type: multipart/form-data");
slist1 = curl_slist_append(slist1, "Authorization: Bearer abcd");
hnd = curl_easy_init();
curl_easy_setopt(hnd, CURLOPT_BUFFERSIZE, 102400L);
curl_easy_setopt(hnd, CURLOPT_URL, "http://test-pulsar.com:8080/admin/v3/functions/test/ingest/test-entity-FeedTransformer");
curl_easy_setopt(hnd, CURLOPT_NOPROGRESS, 1L);
mime1 = curl_mime_init(hnd);
part1 = curl_mime_addpart(mime1);
curl_mime_data(part1, "{\"parallelism\":2}", CURL_ZERO_TERMINATED);
curl_mime_name(part1, "functionConfig");
curl_mime_type(part1, "application/json");
curl_easy_setopt(hnd, CURLOPT_MIMEPOST, mime1);
curl_easy_setopt(hnd, CURLOPT_HTTPHEADER, slist1);
curl_easy_setopt(hnd, CURLOPT_USERAGENT, "curl/7.79.1");
curl_easy_setopt(hnd, CURLOPT_MAXREDIRS, 50L);
curl_easy_setopt(hnd, CURLOPT_HTTP_VERSION, (long)CURL_HTTP_VERSION_2TLS);
curl_easy_setopt(hnd, CURLOPT_CUSTOMREQUEST, "PUT");
curl_easy_setopt(hnd, CURLOPT_FTP_SKIP_PASV_IP, 1L);
curl_easy_setopt(hnd, CURLOPT_TCP_KEEPALIVE, 1L);
/* Here is a list of options the curl code used that cannot get generated
as source easily. You may select to either not use them or implement
them yourself.
CURLOPT_WRITEDATA set to a objectpointer
CURLOPT_INTERLEAVEDATA set to a objectpointer
CURLOPT_WRITEFUNCTION set to a functionpointer
CURLOPT_READDATA set to a objectpointer
CURLOPT_READFUNCTION set to a functionpointer
CURLOPT_SEEKDATA set to a objectpointer
CURLOPT_SEEKFUNCTION set to a functionpointer
CURLOPT_ERRORBUFFER set to a objectpointer
CURLOPT_STDERR set to a objectpointer
CURLOPT_HEADERFUNCTION set to a functionpointer
CURLOPT_HEADERDATA set to a objectpointer
*/
ret = curl_easy_perform(hnd);
curl_easy_cleanup(hnd);
hnd = NULL;
curl_mime_free(mime1);
mime1 = NULL;
curl_slist_free_all(slist1);
slist1 = NULL;
return (int)ret;
}
I believe the main part I'm stumped on is how to convert the following:
curl_mime_data(part1, "{\"parallelism\":2}", CURL_ZERO_TERMINATED);
curl_mime_name(part1, "functionConfig");
curl_mime_type(part1, "application/json");
What i've tried:
curl.setOpt(Curl.option.HTTPPOST, [
{
name: "functionConfig",
contents: '{"parallelism":1};type=application/json',
},
]);
But the api server doesn't recognize the functionConfig
{"reason":"Function config is not provided"}
How to resolve?

node-libcurl author here. I will post the same answer I posted in the original issue:
The curl mime APIs are not available, see https://github.com/JCMais/node-libcurl/issues/112 for the feature request.
What you can use is the HTTPPOST option (which has been deprecated, but is still working).
It would be something like this:
curl.setOpt(Curl.option.HTTPPOST, [
{ name: 'functionConfig', contents: '{"parallelism":2}'}
]);
You do lose the content-type in the above example. If that is important to the server you are using, the only way to set it is by saving the data to a temporary file and using that file as the source of the data:
curl.setOpt(Curl.option.HTTPPOST, [
{ name: 'functionConfig', file: '/path-to-temp-file', type: 'application/json' }
]);
Let me know if that does not work.

Related

Azure: Get TTS File with Curl -Cognitive Speech

How do I get a correct TTS Response back, with an audio file?
The token works, but the TTS request not!
{"code":"404","message": "Resource not found"}
For the Token iam using this URL: https://westeurope.api.cognitive.microsoft.com/sts/v1.0/issuetoken
i found this Code Example on Azure this site, but its not working :-/ (https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/rest-text-to-speech#sample-request-1)
$token=getToken();
$cont="<speak version='1.0' xml:lang='en-US'><voice xml:lang='en-US' xml:gender='Male'
name='en-US-ChristopherNeural'>
Microsoft Speech Service Text-to-Speech API
</voice></speak>";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://westeurope.api.cognitive.microsoft.com/cognitiveservices/v1');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, ($cont));
$headers = array();
#$headers[] = 'Ocp-Apim-Subscription-Key: xxxxxxxxxxxx';
$headers[] = 'X-Microsoft-OutputFormat: raw-24khz-16bit-mono-pcm';
$headers[] = 'Content-Type: application/ssml+xml';
#$headers[] = 'Host: westus.tts.speech.microsoft.com';
$headers[] = 'Content-Length: '.strlen($cont);
$headers[] = 'Authorization: Bearer '.$token;//Token okay
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
print_r($result); //{"code":"404","message": "Resource not found"}
OK, now it works,
i put the option curl_setopt($ch, CURLOPT_HEADER, 1);
to get some more information about the response, and it shows me this error
Required header User-Agent missing.
so i add one more headerParameter to it and now it works:
$headers[] = 'User-Agent: myApp/1.0.0';

Connection to Amazon Selling Partner API with curl

I have to add more details in my question, so here is my try to get orders from my seller central account:
$host = 'sellingpartnerapi-na.amazon.com'; // is this the right URL for REST API to get Orders from my seller central account?
$accessKey = 'AK...YOX.........';
$secretKey = 'K...........';
$region = 'us-east-1';
$service = 'execute-api';
$requestUrl = 'https://sandbox.sellingpartnerapi-na.amazon.com/orders/v0/orders'; // '<full url>';
$uri = '/orders/v0/orders'; // '<method path>';
$httpRequestMethod = 'GET'; // '<http verb>';
I used this sample files: https://github.com/avi-wish/aws4-signature-php
unfortunately there is no example URL for variable $host
If I test the connection, this is my response:
Response:403
{
"errors": [
{
"message": "Access to requested resource is denied.",
"code": "Unauthorized",
"details": "Access token is missing in the request header."
}
]
}
I found a possible solution here:
https://github.com/amzn/selling-partner-api-docs/issues/52#issuecomment-713522351
but it's not working for me.
Is that necessary to create a IAM User before? Here is the way I created it:
https://github.com/amzn/selling-partner-api-docs/blob/main/guides/developer-guide/SellingPartnerApiDeveloperGuide.md#registering-your-selling-partner-api-application
I think a lot of people have the same problem. Hope someone can give me a hint.
after getting the access token, the curl return the error :
[message] => Access to requested resource is denied. [code] => MissingAuthenticationToken
function get_orders(){
$this->host = 'https://sellingpartnerapi-na.amazon.com/orders/v0/orders';
debug('>>get list of orders');
$data = "x-amz-date&access_token={$this->access_token}";
$curl = curl_init($this->host . '/orders/v0/orders');
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "GET");
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json;charset=UTF-8')
);
$result = curl_exec($curl);
curl_close($curl);
$obj = json_decode($result,true);
debug($obj);
}
Unfortunately you give very little information about what you want to do. There are two ways to authorize the SP-API that is set via the grant-type.
I suppose you want to authorize yourself and not a vendor.
Then you have to send your request to the following endpoint:
api.amazon.com/auth/o2/token
You have to specify the $host with api.amazon.com.
The request method will be POST.
You pass the parameters to the http Body:
grant_type, refresh_token, client_id, client_secret
Please have a look at the following link:
Connecting to the Selling Partner API
After you have done all this the same way, you will get the following answer:
{
"access_token":"Atza|IwEBIJR-9fxxxxxxxxxxxxxxxxxxxxxx",
"refresh_token":"Atzr|IwEBIxxxxxxxxxxxxxxxxxxxxxxxx",
"token_type":"bearer",
"expires_in":3600
}
Now you can use the access_token to sign your requests.
In the following, I also recommend you to use my post from another answer to sign a request and successfully send a request.
Answer: 64858116
public function GetAcessToken()
{
$RefeshToken = $this->RefreshToken;
$data = "grant_type=refresh_token"
."&refresh_token=". $RefeshToken
."&client_id=".$this->ClientID
."&client_secret=".$this->ClientSecret;
$curl = curl_init($this->UrlForAmazonToken);
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
'Content-Type: application/x-www-form-urlencoded;charset=UTF-8')
);
$result = curl_exec($curl);
curl_close($curl);
$obj = json_decode($result,true);
$this->AccessToken = $obj['access_token'] ;
return $this->AccessToken;
}
It works for me.

multipart/form-data http post request curl vc++

first time curl user, working on a curl fileupload with http multipart/form-data request with vc++.
After researching I found two ways to make a multipart/form-data request. Either use "CURLOPT_POSTFIELDS" set a value as an array or use formadd and pass the pointer to CURLOPT_POSTFIELDS. I decided to do the second and configure the different fields of the request with "formdata" (two fields, one file (application/sla) the other one origin (text/plain)
I used wireshark to catch the request and saw that CURL returned CURLE_READ_ERROR (26). Then I added CURLFORM_FILECONTENT, "tmp_3DButton.stl" and got CURL_OK but the server I was sending the request to returned "Bad Media Type" and I saw that CURL didnt send my request as multipart..
(win8 64 bit, visual studio 2015, curl, c++
CURL *curl;
CURLcode res;
char error[CURL_ERROR_SIZE];
struct curl_httppost *formpost = NULL;
struct curl_httppost *lastptr = NULL;
struct curl_slist *headerlist = NULL;
curl_global_init(CURL_GLOBAL_ALL);
// Fill in the file upload field
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "file",
CURLFORM_CONTENTTYPE, "application/sla",
CURLFORM_FILECONTENT, "tmp.stl",
CURLFORM_FILE, filePathSTL,
CURLFORM_END);
// Fill in the filename field
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "origin",
CURLFORM_CONTENTTYPE, "text/plain",
CURLFORM_FILENAME, "se_1_0",
CURLFORM_END);
curl = curl_easy_init();
headerlist = curl_slist_append(headerlist, "Accept: text/plain");
if (curl) {
// URL that receives this POST
curl_easy_setopt(curl, CURLOPT_URL, "http://blabla.com/upload");
//multipart/form-data
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);
curl_easy_setopt(curl, CURLOPT_HTTPPOST, formpost);
2);
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, error);
error[0] = 0;
// Perform the request, res will get the return code
res = curl_easy_perform(curl);
CString str;
// Check for errors
if (res != CURLE_OK)
{
str.Format("curl_easy_perform return %s [%d]", curl_easy_strerror(res), res);
size_t len = strlen(error);
if (len)
{
_bstr_t bstrt(error);
AfxMessageBox(error);
}
}
curl_easy_cleanup(curl);
filePathSTL is bstrt for example C:\..\..\tmp.stl (with doubled slashes..)
Found an answer, "CURLFORM_FILE" obviously wants a string!

How to use libcurl to HTTP PUT Xively

I want to make HTTPS PUT request to put a csv file. Below is the code used to upload data to Xively. Earlier I was getting 411 length required error. I referred to the code available here (Send string in PUT request with libcurl) to resolve this, where CURLOPT_CUSTOMREQUEST is made. Now I am getting HTTP 500 Internal Server error.
void upload()
{
CURL *curl;
CURLcode res;
FILE * hd_src;
struct stat file_info;
struct curl_slist* header = NULL;
char * csvfile = "123.csv";
/* get the file size */
stat(csvfile, &file_info);
hd_src = fopen("123.csv","rb");
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if(curl)
{
header = curl_slist_append(header,"X-ApiKey: 123123123"); /* API KEY HERE - sample only*/
header = curl_slist_append(header,"Accept: text/csv");
header = curl_slist_append(header, "Host: api.xively.com");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
/* Actual Xively feed is here. For demonstration purpose the feed is listed as 123 */
curl_easy_setopt(curl, CURLOPT_URL, "https://api.xively.com/v2/feeds/123.csv");
curl_easy_setopt(curl, CURLOPT_READDATA, hd_src);
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, (curl_off_t)file_info.st_size);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT" );
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, hd_src);
res = curl_easy_perform(curl);
if(res != CURLE_OK)
{
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
}
curl_easy_cleanup(curl);
}
fclose(hd_src);
curl_slist_free_all(header);
curl_global_cleanup();
}
Can anyone make suggestions? If I remove curl_easy_setopt(curl, CURLOPT_POSTFIELDS, hd_src), I'll get 411 length required error. I removed this and added Content-Length to header using file_info.st_size. Again 411 length required error is received. With curl_easy_setopt(curl, CURLOPT_POSTFIELDS, hd_src) 411 is solved but Xively gives 500 internal server error.
Thanks in advance.
Try using CURLOPT_UPLOAD instead, like the documentation says to:
CURLOPT_PUT - make a HTTP PUT request
...
This option is deprecated since version 7.12.1. Use CURLOPT_UPLOAD!
CURLOPT_UPLOAD - enable data upload
...
If the protocol is HTTP, uploading means using the PUT request unless you tell libcurl otherwise
You have to be very careful with CURLOPT_CUSTOMREQUEST:
CURLOPT_CUSTOMREQUEST - custom string for request
...
When you change the request method by setting CURLOPT_CUSTOMREQUEST to something, you don't actually change how libcurl behaves or acts in regards to the particular request method, it will only change the actual string sent in the request
...
Many people have wrongly used this option to replace the entire request with their own, including multiple headers and POST contents. While that might work in many cases, it will cause libcurl to send invalid requests and it could possibly confuse the remote server badly.
You are also not sending a Content-Type header, you are sending an Accept header instead. Content-Type tells the server the type of data you are sending. Accept tells the server what type(s) of data you are willing to receive in reply.
And lastly, you are not doing error handling on stat() or fopen().
Try this instead:
void upload()
{
CURL *curl;
CURLcode res;
FILE * hd_src;
struct stat file_info = {0};
struct curl_slist* header = NULL;
char * csvfile = "C:\\full path to\\123.csv";
/* get the file size */
if (stat(csvfile, &file_info) == -1)
{
fprintf(stderr, "stat() failed: %s\n", strerror(errno));
return;
}
hd_src = fopen(csvfile, "rb");
if (!hd_src)
{
fprintf(stderr, "fopen() failed: %s\n", strerror(errno));
return;
}
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (!curl)
{
fprintf(stderr, "curl_easy_init() failed\n");
}
else
{
header = curl_slist_append(header, "X-ApiKey: 123123123"); /* API KEY HERE - sample only*/
header = curl_slist_append(header, "Content-Type: text/csv");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, header);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
/* Actual Xively feed is here. For demonstration purpose the feed is listed as 123 */
curl_easy_setopt(curl, CURLOPT_URL, "https://api.xively.com/v2/feeds/123.csv");
curl_easy_setopt(curl, CURLOPT_READDATA, hd_src);
curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, (curl_off_t)file_info.st_size);
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L );
res = curl_easy_perform(curl);
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
curl_slist_free_all(header);
curl_easy_cleanup(curl);
}
fclose(hd_src);
curl_global_cleanup();
}

Need Working Example of Nest REST API without using Firebase API

I'm struggling trying to find a working example of writing data to the Nest Thermostat API using plain rest. Attempting to write a C# app and cannot use Firebase. The multiple Curl examples posted so far do not work. I have a valid auth_token and can read data without issues. Finding the correct post url is elusive. Can anyone assist?
Examples like
curl -v -X PUT "https://developer-api.nest.com/structures/g-9y-2xkHpBh1MGkVaqXOGJiKOB9MkoW1hhYyQk2vAunCK8a731jbg?auth=<AUTH_TOKEN>" -H "Content-Type: application/json" -d '{"away":"away"}'
don't change any data.
Two things. First, follow redirects with -L. Second, put directly to the away data location, like
curl -v -L -X PUT "https://developer-api.nest.com/structures/g-9y-2xkHpBh1MGkVaqXOGJiKOB9MkoW1hhYyQk2vAunCK8a731jbg/away?auth=<AUTH_TOKEN>" -H "Content-Type: application/json" -d '"away"'
The PUT overwrites all data at a location. The previous command would logically be setting the structure's data to just {"away":"away"}.
user3791884,
Any luck with your C# PUT? Here is C# code that works:
using System.Net.Http;
private async void changeAway()
{
using (HttpClient http = new HttpClient())
{
string url = "https://developer-api.nest.com/structures/" + structure.structure_id + "/?auth=" + AccessToken;
StringContent content = new StringContent("{\"away\":\"home\"}"); // derived from HttpContent
HttpResponseMessage rep = await http.PutAsync(new Uri(url), content);
if (null != rep)
{
Debug.WriteLine("http.PutAsync2=" + rep.ToString());
}
}
}
Debug.WriteLine writes this to the Output window:
"http.PutAsync2=StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
Access-Control-Allow-Origin: *
Cache-Control: no-cache, max-age=0, private
Content-Length: 15
Content-Type: application/json; charset=UTF-8
}"
These two methods return a valid structure of my data.
1/ command line
curl -v -k -L -X GET "https://developer-api.nest.com/structures/Za6hCZpmt4g6mBTaaA96yuY87lzLtsucYjbxW_b_thAuJJ7oUOelKA/?auth=c.om2...AeiE"
2/ C#
private bool getStructureInfo()
{
bool success = false;
try
{
// Create a new HttpWebRequest Object to the devices URL.
HttpWebRequest myHttpWebRequest=(HttpWebRequest)WebRequest.Create("https://developer-api.nest.com/structures/?auth=" + AccessToken);
// Define the request access method.
myHttpWebRequest.Method = "GET";
myHttpWebRequest.MaximumAutomaticRedirections=3;
myHttpWebRequest.AllowAutoRedirect=true;
myHttpWebRequest.Credentials = CredentialCache.DefaultCredentials;
using(HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse())
{
if (null != myHttpWebResponse)
{
// Store the response.
Stream sData = myHttpWebResponse.GetResponseStream();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader (sData, Encoding.UTF8);
Debug.WriteLine("Response Structure stream received.");
string data = readStream.ReadToEnd();
Debug.WriteLine(data);
readStream.Close();
success = deserializeStructure(data);
}
}
}
catch (Exception ex)
{
Debug.WriteLine("getStructure Exception=" + ex.ToString());
}
return success;
}

Resources