INVALID_SIGNATURE error while trying to retrieve list of assemblies in transloadit - signature

I am using Transloadit API to merge audio file and series of images.
At some point, I need to retrieve list of assemblies (videos generated till now) for which transloadit provides a get API endpoint but that endpoint accepts two query strings, signature and params(to configure the list)
I am generating signature of the same params which is being sent as query string to the API along with it's signature but it is returning an error that signature doesn't match.
Transloadit have proper docs of how to create signature for each major language here https://transloadit.com/docs/#signature-authentication
Also the docs (https://transloadit.com/docs/api/#assemblies-get) doesn't state whether the signature will be generated of the same params or not.
Please help if anyone have used transloadit and had a same problem before and solved it now

I believe what your problem may be is that you're not URL encoding the JSON before passing it in your GET request. Here's a small snippet in Python showing how to turn a dictionary of values into JSON to generate a signature, and then into a URL encoded object for the GET request.
params = {
'auth': {
'key': auth_key,
'expires': expires
},
'template_id': template_id
}
# Converts the dictionary into JSON
message = json.dumps(params, separators=(',', ':'))
signature = hmac.new(auth_secret.encode('utf-8'),
message.encode('utf-8'),
hashlib.sha1).hexdigest()
# URL encodes it
params_encoded = urllib.parse.quote_plus(message)
url = f'https://api2.transloadit.com/assemblies?signature={signature}&params={params_encoded}'
response = requests.get(url)

Related

Google Vision text detection for Node.js using base64 encoding

Just started exploring Google Cloud Vision APIs. From their guide:
const client = new vision.ImageAnnotatorClient();
const fileName = 'Local image file, e.g. /path/to/image.png';
const [result] = await client.textDetection(fileName);
However, I wanna use base64 representation of binary image data, since they claim that it's possible to use.
I found this reference on SO:
Google Vision API Text Detection with Node.js set Language hint
Instead of imageUri I used "content": stringas mentioned here. But the SO sample uses const [result] = await client.batchAnnotateImages(request);method. I tried using same technique on const [result] = await client.textDetection( method and it gave me an error.
So my question is: Is it possible to use base64 encoded string to represent image in order to perform TEXT_DETECTION ? And if so, how?
Any kind of help is highly appreciated.
You can use the quickstart guide and from there edit the lines after the creation of the client for the following:
// Value of the image in base64
const img_base64 = '/9j/...';
const request = {
image: {
content: Buffer.from(img_base64, 'base64')
}
};
const [result] = await client.textDetection(request);
console.log(result.textAnnotations);
console.log(result.fullTextAnnotation);
You can take a look at the function here, read the description of the request parameter, in particular the following part:
A dictionary-like object representing the image. This should have a
single key (source, content).
If the key is content, the value should be a Buffer.
Which leads to the structure used in the sample code from before. Opposed to when using imageUri or filename, which have to be inside of another object which key is source, as shown in the sample.
content field need to be Buffer.
You use the nodejs client library. The library use the grpc API internally, and grpc API expect bytes type at content field.
However, JSON API expect base64 string.
References
https://cloud.google.com/vision/docs/reference/rpc/google.cloud.vision.v1#image
https://googleapis.dev/nodejs/vision/latest/v1.ImageAnnotatorClient.html#textDetection

How do I escape a '/' in the URI for a GET request?

I'm trying to use Groovy to script a GET request to our GitLab server to retrieve a file. The API URI format is:
https://githost/api/v4/projects/<namespace>%2F<repo>/files/<path>?ref=<branch>
Note that there is an encoded '/' between namespace and repo. The final URI needs to look like the following to work properly:
https://githost/api/v4/projects/mynamespace%2Fmyrepo/files/myfile.json?ref=master
I have the following code:
File f = HttpBuilder.configure {
request.uri.scheme = scheme
request.uri.host = host
request.uri.path = "/api/v4/projects/${apiNamespace}%2F${apiRepoName}/repository/files/${path}/myfile.json"
request.uri.query.put("ref", "master")
request.contentType = 'application/json'
request.accept = 'application/json'
request.headers['PRIVATE-TOKEN'] = apiToken
ignoreSslIssues execution
}.get {
Download.toFile(delegate as HttpConfig, new File("${dest}/myfile.json"))
}
However, the %2F is re-encoded as %252F. I've tried multiple ways to try and create the URI so that it doesn't encode the %2F in between the namespace and repo, but I can't get anything to work. It either re-encodes the '%' or decodes it to the literal "/".
How do I do this using Groovy + http-builder-ng to set the URI in a way that will preserve the encoded "/"? I've searched but can't find any examples that have worked.
Thanks!
As of the 1.0.0 release you can handle requests with encoded characters in the URI. An example would be:
def result = HttpBuilder.configure {
request.raw = "http://localhost:8080/projects/myteam%2Fmyrepo/myfile.json"
}.get()
Notice, the use of raw rather than uri in the example. Using this approach requires you to do any other encoding/decoding of the Uri yourself.
Possible Workaround
The Gitlab API allows you to query via project id or project name. Look up the project id first, then query the project.
Lookup the project id first. See https://docs.gitlab.com/ee/api/projects.html#list-all-projects
def projects = // GET /projects
def project = projects.find { it['path_with_namespace'] == 'diaspora/diaspora-client' }
Fetch Project by :id, See https://docs.gitlab.com/ee/api/projects.html#get-single-project
GET /projects/${project.id}

Swagger generated API for the Microsoft Cognative Services Recommendations

I can't seem to figure out how to include the CSV file content when calling the Swagger API generated methods for the Microsoft Cognitive Services Recommendations API method Uploadacatalogfiletoamodel(modelID, catalogDisplayName, accountKey);. I've tried setting the catalogDisplayName to the full path to the catalog file, however I'm getting "(EXT-0108) Passed argument is invalid."
When calling any of the Cog Svcs APIs that require HTTP body content, how do I include the body content when the exposed API doesn't have a parameter for the body?
I guess, Swagger can't help you testing functions that need to pass data thru a form. And I guess sending the CSV content in the form data shall do the trick, if you know the proper headers.
I work with nuGet called "Microsoft.Net.Http" and code looks like
HttpContent stringContent = new StringContent(someStringYouWannaSend);
HttpContent bytesContent = new ByteArrayContent(someBytesYouWannaSend);
using (var client = new HttpClient())
using (var formData = new MultipartFormDataContent())
{
formData.Add(stringContent, "metadata", "metadata");
formData.Add(bytesContent, "bytes", "bytes");
HttpResponseMessage response = client.PostAsync(someWebApiEndPoint.ToString(), formData).Result;
if (!response.IsSuccessStatusCode)
{
return false; //LOG
}
string responseContent = response.Content.ReadAsStringAsync().Result;
jsonResult= JsonConvert.DeserializeObject<someCoolClass>(responseContent);
return true;
}
Sorry about that someVariables that can't compile. Hope you'll figure this out.
When you are basing your code on the Swagger definition you depend on the good will of the person that created that Swagger definition. Maybe it is not complete yet.
If you are working on C#, try looking at the Samples repo.
Particularly for the Uploading of the catalog there are several functions on the ApiWrapper class that might be helpful, one has this signature: public CatalogImportStats UploadCatalog(string modelId, string catalogFilePath, string catalogDisplayName), another has this other signature public UsageImportStats UploadUsage(string modelId, string usageFilePath, string usageDisplayName) (where it seems like you can point to a public url).
In your case I'd probably try the second one.
Download the sample and use the Wrapper code defined there in your project.

ServiceStack: Send JSON string instead DTO via POST

I would like to send a string (JSON formatted) to my webservice instead using a DTO.
var client = new JsonServiceClient(absoluteUrl);
client.Post<T>(absoluteUrl, data);
But, after to do change my data (DTO object) to a JSON string, I'm getting a ServiceStack Exception: Internal Server Error. Looks like the ServiceStack Post/Send method changes my JSON request.
Is it possible or is there any way to avoid it? Someone else had the same issue?
UPDATE 1: I'm using this approach because of the OAuth authentication. I'm generating a HMAC by request.
Thanks.
You can use HTTP Utils to send raw JSON, e.g:
var response = absoluteUrl.PostJsonToUrl(data)
.FromJson<T>();
ServiceStack's .NET Service Clients provide Typed API's to send Typed Request DTO's, it's not meant for POST'ing raw strings but I've just added support for sending raw string, byte[] and Stream in this commit so now you can send raw data with:
var requestPath = request.ToPostUrl();
string json = request.ToJson();
var response = client.Post<GetCustomerResponse>(requestPath, json);
byte[] bytes = json.ToUtf8Bytes();
response = client.Post<GetCustomerResponse>(requestPath, bytes);
Stream ms = new MemoryStream(bytes);
response = client.Post<GetCustomerResponse>(requestPath, ms);
This change is available from v4.0.43+ that's now available on MyGet.
Sharing Cookies with HttpWebRequest
To have different HttpWebRequests share the same "Session" you just need to share the clients Cookies, e.g. after authenticating with a JsonServiceClient you can share the cookies with HTTP Utils by assigning it's CookieContainer, e.g:
var response = absoluteUrl.PostJsonToUrl(data,
requestFilter: req => req.CookieContainer = client.CookieContainer)
.FromJson<T>();

Is this API signed request methodology secure?

I'm working on authentication for my JSON-RPC API and my current working strategy is using signed requests sent via POST over SSL.
I'm wondering if anyone can see any vulnerabilities that I haven't taken into consideration with the following signature method.
All communication between the client and the server is done via POST requests sent over SSL. Insecure http requests are denied outright by the API server.
Dependencies
var uuid = require('node-uuid');
var crypto = require('crypto');
var moment = require('moment');
var MyAPI = require('request-json').newClient('https://api.myappdomain.com');
Dependency Links: node-uuid, crypto, moment, request-json
Vars
var apiVersion = '1.0';
var publicKey = 'MY_PUBLIC_KEY_UUID';
var secretKey = 'MY_SECRET_KEY_UUID';
Request Object
var request = {
requestID : uuid.v4(),
apiVersion : apiVersion,
nonce : uuid.v4(),
timestamp : moment.utc( new Date() ),
params : params
}
Signature
var signature = crypto.createHmac('sha512',secretKey).update(JSON.stringify(request)).digest('hex');
Payload Packaging (Sent as cleartext via POST over SSL)
var payload = {
request: request,
publicKey : publicKey,
signature : signature
}
Resultant Payload JSON Document
{
"request" : {
"requestID" : "687de6b4-bb02-4d2c-8d3a-adeacd2d183e",
"apiVersion" : "1.0",
"nonce" : "eb7e4171-9e23-408a-aa2b-cd437a78af22",
"timestamp" : "2014-05-23T01:36:52.225Z",
"params" : {
"class" : "User"
"method" : "getProfile",
"data" : {
"id" : "SOME_USER_ID"
}
}
},
"publicKey" : "PUBLIC_KEY",
"signature" : "7e0a06b560220c24f8eefda1fda792e428abb0057998d5925cf77563a20ec7b645dacdf96da3fc57e1918950719a7da70a042b44eb27eabc889adef95ea994d1",
}
POST Request
MyAPI.post('/', payload, function(response){
/// Handle any errors ...
/// Do something with the result ...
/// Inspect the request you sent ...
});
Server-Side
And then on the server-side the following occurs to authenticate the request:
PUBLIC_KEY is used to lookup the SECRET_KEY in the DB.
SECRET_KEY is used to create an HMAC of the request object from the payload.
The signature hash sent in the payload is compared to the hash of the request object created on the server. If they match, we move on to authenticating the timestamp.
Given that we can now trust the timestamp sent in the cleartext request object since it was included in the signature hash sent from the client, the timestamp is evaluated and the authentication is rejected if the request is too old. Otherwise, the request is authenticated.
So far as I understand, this is a secure method for signing and authentication requests sent over SSL. Is this correct?
Thanks in advance for any help.
Update on JSON Property Order
The order of properties when using JSON.stringify is essentially random, which could cause signature mis-matches.
Using this signing process over the past few weeks I haven't run into any hash mis-match issues due to the order of the properties in the JSON request object. I believe it's because I only stringify the request object literal once, right before the client-side hash is calculated. Then, the request object is in JSON format as part of the payload. Once received by the server, the hash is created directly from the JSON object received in the payload, there's no second JSON.stringify method invoked, so the signature always matches because the order of the properties is determined once, by the client. I'll keep looking into this though as it seems like a weak point, if not a security concern.
JSON.stringify does not guarantee order of properties. For example, object
{
a: 1,
b: 2
}
could be serialized in two ways: {"a":1,"b":2} or {"b":2,"a":1}. They are the same from JSON point of view but they will result it different HMACs.
Imaging, that for signings your JSON.stringify produced first form, but for checking signature second one. Your signature check will fail although signature was valid.
The only fishy thing I see here would be the JSON.stringify as posted in other comments, but you can use:
https://www.npmjs.com/package/json-stable-stringify
That way you can have a deterministic hash for you signs.

Resources