I have an application (headless CMS) running locally. It has an option to send a webhook to another application. I have been trying to interpret this webhook via express using posts. I have not been able to even get it to register a request coming in from the application. I tested this route using postman and found that it is working when I post to it.
router.post('/', (req, res) => {
console.log("Recieved");
console.log(req.body);
res.status(200).send('ok')
});
Thus, when I send a post to: http://localhost:3000/recall via postman. I get the following header back:
Access-Control-Allow-Credentials →true
Access-Control-Allow-Headers →X-Requested-With,content-type
Access-Control-Allow-Methods →GET, POST, OPTIONS, PUT, PATCH, DELETE
Access-Control-Allow-Origin →http://localhost:3000
Connection →keep-alive
Content-Length →2
Content-Type →text/html; charset=utf-8
Date →Mon, 03 Sep 2018 21:23:22 GMT
ETag →W/"2-eoX0dku9ba8cNUXvu/DyeabcC+s"
X-Powered-By →Express
With the body:
ok
My script also prints the body of the post.
I can verify the webhook is working by testing it with request bin. I get back the following:
FORM/POST PARAMETERS
None
HEADERS
Cloudfront-Forwarded-Proto: http
Cloudfront-Is-Mobile-Viewer: false
Cloudfront-Is-Desktop-Viewer: true
Connect-Time: 1
Via: 1.1 3566cbcd49f71967b52a565888e4d272.cloudfront.net (CloudFront), 1.1 vegur
Content-Length: 387
Connection: close
Accept: */*
Content-Type: application/json
Cloudfront-Viewer-Country: US
X-Amz-Cf-Id: dRe5CvkLFJZJNcpZbhmeEHo0ar_taj6guvN8utwkyVXM7ZMJc5BZTw==
Cloudfront-Is-Smarttv-Viewer: false
X-Request-Id: 4b6d2cdc-5c45-495b-b358-2e808e1bfeb4
Cloudfront-Is-Tablet-Viewer: false
Total-Route-Time: 0
Host: requestbin.fullcontact.com
BODY
{"event":"singleton.remove","hook":"Save After Sington","backend":1,"args":[{"name":"Wonder","label":"Wonder","_id":"Wonder5b8cef36a0097","fields":[{"name":"Best","label":"","type":"text","default":"","info":"","group":"","localize":false,"options":[],"width":"1-1","lst":true,"acl":[]}],"template":"","data":null,"_created":1535962934,"_modified":1535962934,"description":"","acl":[]}]}
I tried enabling cross-origin requests. How could I fix this problem? My thought is it has something to do with the fact this request is originating and ending locally.
For an application to consume webhooks it needs to have a publicly accessible URL. Basically, the rest of the world (internet) doesn't know your localhost:3000 endpoints exist.
An easy way to fix this is to use a lightweight tool like ngrok to expose your local server; in turn allowing other applications to communicate with yours.
You will need to define the specific callback route that you want to consume the webhook POST request. Examples below.
Run your node script
Turn on ngrok
send webhook POST requests to your endpoint using the NGROK https address
Now, instead of sending your webhook to localhost:8000/MyWebhookConsumingEndpoint
you send it to
https://95e26af4.ngrok.io/MyWebhookConsumingEndpoint
Related
General context:
I am working on an IoT application where I upload images from an ESP32 connected to an SBC.
The uploading is done through an API provided by a third-party backend developer.
The upload API works through other mediums (such as Postman, python requests library, python http client library)
The ESP32 is connected to the SBC through UART.
I construct/generate the HTTP request on the SBC and send it as bytes. I have written a function on ESP32 that can send the bytes as a generic HTTP request, to the URL specified.
Then it sends the response string back to the SBC.
All of this works. For small requests, I am facing no issues. I am able to download images, etc.
However, when uploading an image, I don't get a response and I end up timing out after 30s. I checked without timeout FYI, but no response.
I checked from the server-side. It appears my request has succeeded and the server is sending me 200 with the URL of the image. Using that URL, I was able to verify that the image was uploaded successfully.
However, I do not receive this response on the microcontroller.
Not sure what the issue is. Any suggestions as to what I can do?
I can't give out the code but I'll send a general structure:
ESP32
-> Receives URL, port, length of request
-> Connects to server and reads the request from UART and writes to server
-> Wait for response after response is sent
Python raw http
POST (server path) HTTP/1.1
Host: (url)
correlation-id: test5
Content-Type: multipart/form-data; boundary=WebKitFormBoundary7MA4YWxkTrZu0gW
Authorization: Bearer (access token)
Content-Length: 268
--WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="portraits"; filename="name"
Content-Type: image/jpeg
(data)
--WebKitFormBoundary7MA4YWxkTrZu0gW--
Edit 1:
So, turns out it is not only "upload image", some other requests are also behaving similarly. Our server has many microservices. The services written in nodeJS which have more than 1 redirects are not working...?
I figured out what the issue is and hopefully, it will help anyone else facing the same issue. Also, some of my requests to the backend server which used a different authentication method worked
I had been generating the raw HTTP using postman code generation but it turns out Postman doesn't add a few headers which are needed for communicating with more complex servers.
What I mean is that if I host a local server, the above code will work. I had already tested it that way
What solved my problem is adding these headers:
POST (server path) HTTP/1.1
Host: (server URL)
User-Agent: ESP32
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
correlation-id: test
Authorization: Bearer (access_token)
Content-Length: 146360
Content-Type: multipart/form-data; boundary=af59ef02d60cd0efefb7bc03db1f4ffc
--af59ef02d60cd0efefb7bc03db1f4ffc
Content-Disposition: form-data; name="portraits"; filename="(name)"
Content-Type: image/jpeg
(data)
--af59ef02d60cd0efefb7bc03db1f4ffc--
I have troubles gettting cookies to work in my production environment which uses a Nest.js API Backend on Azure WebServices (with functions). I basically followed this guide: https://dev.to/azure/build-your-first-serverless-app-with-angular-nestjs-and-azure-108h
I also added an angular frontend app, also hosted on Azure. But as far as my testing concerns this does not matter.
I want to add authorization by JWT stored in a cookie. However, the Backend does not add the cookie to the header in production. In development (localhost) everything works like a charm. It works with the local/dev frontend calling the local/dev backend as well as when I use the VSC Rest Client to just call the API.
However in production I won't receive the cookie nor other header I test. I configured CORS in Azure and this kind of looks good as I do get a HTTP200 and not a CORS Error back. Just the header info is missing.
I already read a lot of advice, but none helped. I do set 'width credentials' (also in azure). Do you experts have any advice what to try or what might be the problem?
Thanks
Controller for testing purposes ...
#Get("server-check")
#HttpCode(HttpStatus.OK)
#Header('Set-Cookie', 'cookieName = 12345; secure; SameSite=none"') // "Usin header decorator"
#Header('Access-Control-Expose-Headers', 'set-cookie, authorization') //
async serverCheck(
#Res() response: Response) {
response.cookie('rememberme', '1') // Using express res object.
return response.send('Cookie has been set! :)')
}
Response from Production env (no cookie in application tab)
Screen from Network-Response in Browser
Response from REST call in VSC
HTTP/1.1 200 OK
Connection: close
Date: Thu, 05 May 2022 11:51:27 GMT
Transfer-Encoding: chunked
Request-Context: appId=cid-v1:b8d7a5c0-962f-40ec-b128-b47139939cf4
Cookie has been set! :)
There are a few questions similar to this on Stack Overflow, and none of the proposed solutions worked, so I'll walk through the case and what I've tried.
I have a server application hosted on Cloud Run, which can only be accessed with the appropriate Bearer token in the request Authorization header. I've tried accessing it via Postman and an Axios request from a local Nodejs server, with the Authorization header, and it worked fine. With React (create-react-app specifically), I get the following error: Access to XMLHttpRequest at 'https://myserver-lhp5a9xp5a-ue.a.run.app/api/rules' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
On the server side, I get the 403 error that Cloud Run gives when the incorrect Authorization token is passed. Also, when I allow unauthenticated access from the Cloud Run side (so remove the need for an Authorization header), the request works fine, so it looks like this is indeed an issue with the Authorization header and not CORS.
In addition, I'm handling CORS on the server side. Here's my server-side code:
var express = require('express');
var router = express.Router();
const cors = require('cors');
router.options('/api/rules', cors());
router.get('/api/rules', cors(), (req, res, next) => {
res.status(200).send()
});
Here's my React code:
const axiosInstance = axios.create({
baseURL: process.env.REACT_APP_API_BASE_URL
});
const buttonClickHandler = async (event) => {
const resp = await axiosInstance.get('/api/rules'
, {
headers: {
'Authorization': 'Bearer eyJhbGciOiJSUzI1NiIsImtpZ...' // I used this token within the same minute when trying the request via Postman or from my Nodejs app, so a token expiry isn't the issue.
}
}
)
console.log(resp.data)
}
Here's what I tried so far:
Using fetch instead of axios - same error
Using the same token, within the same 5 seconds, to send the request from Postman or a Nodejs server - it worked fine.
Using an axios interceptor to set the Authorization - same error
Removing the single quotes around Authorization - same error
Sending the request to my Nodejs server instead and doing a console.log of the header to make sure the Authorization token is being passed correctly (it is)
Not using an an axios instance but spelling out the full URL in the request - same error
Trying a different endpoint on my Cloud Run server - same error
Deploying my React app to be served from a https endpoint and sending the request from there - same error
Adding Accept: '*/*' to the headers
Adding 'Accept': '*/*' to the headers
Adding 'Content-Type': 'application/json' to the headers
All combinations of the three above points
I found the answer after some digging, thanks #aniket-kolekar for pointing me in the right direction.
When Postman or a Nodejs server query an endpoint like GET, POST, PUT, DELETE, they send the call without checking the OPTIONS first. Create-React-App does.
The service I was querying is hosted on Cloud Run and doesn't allow unauthenticated invocations. So while I was including the authorization header to make my GET call, it wasn't being included in the pre-flight OPTIONS call. In fact, CORS prevents auth headers from being included in an OPTIONS call.
A Cloud Run PM replied in this post that this is a known issue with Cloud Run. The way I'll get around it for now is to host two services on Cloud Run - one that doesn't require authentication, and effectively acts as a proxy server to route calls from the client service to the shielded server service.
TLDR;
CORS is a mechanism built into the web browser. It’s not a UI code issue.
To fix CORS problems, you need to make changes on the API (server) side.
Here is the behind the scenes working:
Browser: Sends OPTIONS call to check the server type and getting the headers before sending any new request to the API endpoint. Where it checks for Access-Control-Allow-Origin. Taking this into account Access-Control-Allow-Origin header just specifies which all CROSS ORIGINS are allowed, although by default browser will only allow the same origin.
Postman: Sends direct GET, POST, PUT, DELETE etc. request without checking what type of server is and getting the header Access-Control-Allow-Origin by using OPTIONS call to the server.
You will have to configure Access-Control-Allow-Origin header in your server to resolve the CORS issue.
i am facing an issue with azure proxy call.
Created a azure function app.
Created a basic get function and sending response body in json format back.
response body example{url: "https://www.google.com"}
If i configure direct function endpoint in one of my company application everything working fine able launch response url coming from azure function call.
I created proxy for the same get function and configured proxy end point in my application.Now application is failing to launch response url.(but back end same function is hitting and logs created no error in logs and ended with status 200)
Unfortunately i don't have control on application code to verify what exact cause in response.
I verified azure function call and proxy call from postman both are giving same response body. I don't know why it is failing in my application
i didn't understand.
One more point i verified response headers in postman in both cases
For function app end point call response headers:
content-type →application/json; charset=utf-8
date →Wed, 09 Jan 2019 12:39:23 GMT
server →Microsoft-IIS/10.0
transfer-encoding →chunked
x-powered-by →ASP.NET
For proxy endpoint call response headers:
content-encoding →gzip
content-length →208
content-type →application/json; charset=utf-8
date →Wed, 09 Jan 2019 12:41:16 GMT
server →Microsoft-IIS/10.0
vary →Accept-Encoding
x-powered-by →ASP.NET, ASP.NET
is gzip encoding creating problem in proxy call. How to disable it in azure proxy.
my application should able to launch even if i use proxy end point.
I'm currently building out an application that utilizes Shippo's webhooks functionality, but wondering if it's possible to ensure that the HTTP POST requests are coming from Shippo's servers rather than some malicious third party.
Adding a webhook to Shippo:
https://shippo-static.s3.amazonaws.com/img/illustrations/webhooks.png
When recieving HTTP requests, it appears as though Shippo doesn't provide a key or anything that I can use on my end to verify that the request came from shippo themselves. Is there any way that I would be able to retroactively and securely ensure that the request came from Shippo's servers?
Sample headers:
User-Agent: python-requests/2.9.1
Via: 1.1 vegur
Accept: /
Connection: close
Content-Length: 2203
Total-Route-Time: 0
Cf-Connecting-Ip: 54.87.248.176
Cf-Ipcountry: US
X-Request-Id: cec0a1aa-6a1a-47e9-ac9d-c685a893591d
Cf-Ray: 3d0718455e149fea-IAD
Connect-Time: 1
Accept-Encoding: gzip
Host: requestb.in
Shippo-Api-Version: 2017-08-01
Cf-Visitor: {"scheme":"https"}
Content-Type: application/json
Sample JSON Response:
https://goshippo.com/docs/tracking
You need to do two things:
Have your webhook include some sort of token, i.e. have shippo call https://yourapp.com/webhookroute/?secure_token=123abc. Parse and check the token server side. Can add multiple tokens, i.e. https://yourapp.com/webhookroute/?secure_token_1=123abc&secure_token_2=456def
Use SSL/TLS certs when deploying your app, which will encrypt the URL/tokens when shippo sends the webhook. Make sure the webhook calls https:// [rest of url], not http://. This will hide the tokens from the rest of the internet.
If the tokens are set as env vars server side, this should be secure.