Total noob here, and currently learning node.js and all the fun it can procure.
I am wrote a REST API with node/express, with a database in mongoDB (and mongoose in my code).
The database is a free one hosted on AWS (through the mongoDB website).
I am calling the following endpoint :https://rewaer-backend.herokuapp.com/api/users/ with postman.
Here is how the route look like:
// GET all users
router.get("/", async (req, res) => {
try {
const user = await User.find();
res.json(user);
}
catch (err) {
res.status(400).json({ message: err });
}
});
but I a getting the following error from heroku:
2020-10-07T14:39:58.684608+00:00 heroku[router]: at=error code=H12
desc="Request timeout" method=POST path="/api/users/"
host=rewaer-backend.herokuapp.com
request_id=94d50342-514e-45e7-94ad-5cb1acde44fa fwd="37.120.11.242"
dyno=web.1 connect=1ms service=30001ms status=503 bytes=0
protocol=https
My collection "users" has only two set of data. Also, really small. When running the server in local (connected to the DB on AWS) this works fine. As you can see in my code, I should be returning something, even if an err happens. So no reason to timeout here.
The Heroku logs show that the connection to the db is successful.
I have no idea what is not working here, and google was not able to help me any further: hence me asking the real wise people ;)
Thanks a lot for any help,
Clément
The default server selection timeout in MongoDB drivers is 30 seconds. Heroku also has a 30 second timeout in its routing layer for the app server to produce the response.
Change server selection timeout to a lower value.
Then fix MongoDB connectivity issues you have (likely IP whitelist).
Note: Updated 2019 JUN 04; see below
This is not a question about a particular snippet of code; rather, I am seeking help properly architecting the proxy for an Angular client communicating with a back-end API using the MEAN stack (MongoDB, Express, Angular, Node). I’m an old school C programmer who is self-teaching himself MEAN. Old dog, new trick. I’ve scrubbed several resources, including Stack Overflow, but I haven’t found the answer. Several users on Stack Overflow are reporting the same problem (I’ve listed those under SOURCES). I also have a help ticket with Heroku.
Thank you to everyone who reads this and considers it. An extra thanks (in advance) to all those who post helpful suggestions and solutions.
QUESTION: How do you correctly architect an Angular client’s calls to a back-end API to avoid the problems inherent in CORS (Cross-Origin Resource Sharing)? The solution I drew and developed from Angular documentation, proxies, works when running on my development system (i.e. localhost), but it fails when uploaded to my PaaS (Heroku). Is this because Angular’s proxies are a tool only for development? If so, what is the proper method for calling a back-end API?
CONFIGURATION: I’m developing on Darwin OS (macOS 10.14.4), using Node 10.13.0, npm 6.9.0, Express 4.16.2, and Angular 7.2.9. I’m using a git repository on Bitbucket which pipelines deployments to Heroku. I have separate deployments for the client (Angular) and the API (Node/Express). Right now the client is running on a Hobby dyno and the API on a Free dyno on Heroku. My database is hosted on mLab in a Sandbox (free database).
Client: https://www.markwilx.com
API: http://markwilx-api.herokuapp.com/api/test
DISCUSSION: The following question in Stack Overflow is the same problem I’m experiencing:
Angular proxy.conf.json to call API works locally but not on Heroku
There is no solution given but Massimiliano Sartoretto makes a comment that states:
“That proxy is supposed to work only for the development server. It has nothing to do with Heroku and it's shouldn't even be deployed on Heroku. Its goal is to help you proxy external API calls while serving the app from localhost”
I have a problem with this comment. The official Angular documentation (see SOURCES, below) makes no mention of this. I’m not challenging Mr. Sartoretto’s claim, but I certainly assert that his claim is not widely known nor documented. If anyone is aware of this fact, I’d appreciate a reference where I can read about it.
Likewise, Mr. Chenkie, who uses the same method, does not mention this in his book (see SOURCES, below). I have tried several times over several months to ask Mr. Chenkie, via his website, but he has not responded. When/if he does, I will update this post.
The good team at Heroku have responded to my help ticket. Heroku does not provide the native proxying capability, so any proxy must be handled in the Angular client. They are inclined to believe the problem lies somewhere within Mr. Sartoretto’s claim. Describing how I should architect my client and API is, of course, outside the scope of their service.
Finally, I found the following on Stack Overflow
Proxy server with Node.js on Heroku
This, again, is the same problem I’m having. The person asking the question, Andrea Reginato, is not using Angular, but he posted his work-around for Node. I’m wondering if this is the preferred method and I just need to port it into Angular.
I’m really stuck and I want to make sure I build my system with the industry best-practice techniques.
SOURCES: The official Angular documentation on configuring a proxy to a back-end server is given at this link:
https://angular.io/guide/build#proxying-to-a-backend-server
I’ve been reading and following “Securing Angular Applications” by Ryan Chenkie. Beginning on pages 41, he begins the process of demonstrating how to build a proxy for his examplar application. His example is consistent with the methods in the above Angular documentation.
Here are unsolved queries on Stack Overflow that are strongly related to the problem I am having:
[I had to remove all of them. With them included, Stack Overflow decided my post was spam. I put the most important ones in the text above.]
Update: 2019 JUN 04
I continue to research this problem with little success. Though I've found nothing beyond Massimiliano Sartoretto's comment stating the proxies are only for development servers, I've removed the proxy and am attempting to address the external API by other means.
At present, I've configured the Content Security Policy to address the Cross-Origin Resource sharing, and I'm using Express to redirect the API call using the following code:
app.get('/api/test', function(req, res) {
request.get({
url: 'https://markwilx-api.herokuapp.com/api/test'
}, function(error, response, body) {
if(!error && response.statusCode == 200) {
res.send(body);
}
});
});
Once again, this works seamlessly on my development environment (the front-end server running on localhost and the back-end server running on Heroku), but it fails once uploaded to Heroku.
The front-end server running on Heroku gives the following errors when the API is called:
Jun 04 05:08:02 markwilx heroku/router: at=info method=GET path="/api/test" host=www.markwilx.com request_id=2f789dad-7161-4269-bf92-64db8060eadd fwd="184.170.243.167" dyno=web.1 connect=0ms service=10ms status=500 bytes=404 protocol=https
Jun 04 05:08:02 markwilx app/web.1: ReferenceError: request is not defined
Jun 04 05:08:02 markwilx app/web.1: at /app/server.js:35:3
Jun 04 05:08:02 markwilx app/web.1: at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
Jun 04 05:08:02 markwilx app/web.1: at next (/app/node_modules/express/lib/router/route.js:137:13)
Jun 04 05:08:02 markwilx app/web.1: at Route.dispatch (/app/node_modules/express/lib/router/route.js:112:3)
Jun 04 05:08:02 markwilx app/web.1: at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
Jun 04 05:08:02 markwilx app/web.1: at /app/node_modules/express/lib/router/index.js:281:22
Jun 04 05:08:02 markwilx app/web.1: at Function.process_params (/app/node_modules/express/lib/router/index.js:335:12)
Jun 04 05:08:02 markwilx app/web.1: at next (/app/node_modules/express/lib/router/index.js:275:10)
Jun 04 05:08:02 markwilx app/web.1: at SendStream.error (/app/node_modules/serve-static/index.js:121:7)
Jun 04 05:08:02 markwilx app/web.1: at emitOne (events.js:116:13)
Jun 04 05:08:07 markwilx heroku/router: at=info method=GET path="/api/test" host=www.markwilx.com request_id=99922c46-a529-4f7e-8190-670cedf3a33d fwd="184.170.243.167" dyno=web.1 connect=0ms service=5ms status=500 bytes=404 protocol=https
Jun 04 05:08:07 markwilx app/web.1: ReferenceError: request is not defined
Jun 04 05:08:07 markwilx app/web.1: at /app/server.js:35:3
Jun 04 05:08:07 markwilx app/web.1: at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
Jun 04 05:08:07 markwilx app/web.1: at next (/app/node_modules/express/lib/router/route.js:137:13)
Jun 04 05:08:07 markwilx app/web.1: at Route.dispatch (/app/node_modules/express/lib/router/route.js:112:3)
Jun 04 05:08:07 markwilx app/web.1: at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
Jun 04 05:08:07 markwilx app/web.1: at /app/node_modules/express/lib/router/index.js:281:22
Jun 04 05:08:07 markwilx app/web.1: at Function.process_params (/app/node_modules/express/lib/router/index.js:335:12)
Jun 04 05:08:07 markwilx app/web.1: at next (/app/node_modules/express/lib/router/index.js:275:10)
Jun 04 05:08:07 markwilx app/web.1: at SendStream.error (/app/node_modules/serve-static/index.js:121:7)
Jun 04 05:08:07 markwilx app/web.1: at emitOne (events.js:116:13)
I'll continue providing updates as I work to resolve this problem.
I've not found any definitive answer, but the best practice seems to be using CORS (Cross-Origin Resource Sharing).
In his book, "Securing Angular Applications," Ryan Chenkie guides the reader through setting up a user signup route using a proxy. On page 43 he states, "The difference might not seem that important but the distinction makes a world of difference and allows us to get around things like cross-origin resource sharing (CORS) and also makes it possible for us to set cookies more easily."
Because of this, I avoided using CORS in attempting to deploy my angular application on Heroku. CORS, however, seems to be an industry standard for building an API. It seems that proxies are not, as Massimiliano Sartoretto claims (see above), for production environments. I decided CORS was not something one could simply "get around" and learned CORS through other sources.
I attempted, several times, to contact Mr.Chenkie to gain an understanding of his approach via his own website contact page. He never replied. Thus I feel justified in stating I'm very disappointed that his book, which is about securing angular applications -- ostensibly for deploying them, avoids this essential security issue in web applications. I do not know what other critical security aspects he may have also omitted from his book; therefore, I do not recommend his book. Go elsewhere for learning about angular application web security.
Here are some very useful explanations about CORS:
https://www.codecademy.com/articles/what-is-cors
https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
https://auth0.com/blog/cors-tutorial-a-guide-to-cross-origin-resource-sharing/
Note that the preflight request (http OPTION) is an important aspect to learn for Angular applications.
There is a useful CORS package in npm, which you can find here:
https://www.npmjs.com/package/cors
Good luck in your programming!
Mark
#MarkWilx the CORS method also do not serve the purpose. our Backend call should be properly hit the backend URL, such as https://backend-app.herokuapp.com/viewProduct. but cors don't do such things.
I wrote a simple GraphQL server using Apollo Express and deployed it to Heroku. After some messing around with Procfiles and the like, it built OK.
When I hit the main URL https://limitless-atoll-59109.herokuapp.com I get the error
Cannot GET /
OK, then I thought the Express server must just be looking for a get on the graphql endpoint. But when I hit https://limitless-atoll-59109.herokuapp.com/graphql I get
GET query missing.
Do I need to include a port in the url? I've got the port set correctly in the code
const PORT = process.env.PORT || 4000
app.listen({port: PORT}, () =>
console.log(`Server ready at http://localhost:${PORT}${server.graphqlPath}`)
);
but I don't think I need to include it when accessing the server on Heroku, do I?
For what it's worth, this is the error in the error logs
019-05-08T17:07:41.492327+00:00 heroku[router]: at=info method=GET
path="/graphql" host=limitless-atoll-59109.herokuapp.com
request_id=b6171835-aac4-4b45-8a7b-daebbb3167ed fwd="139.47.21.74"
dyno=web.1 connect=0ms service=900ms status=400 bytes=196
protocol=https
Thanks for any help!
The answer is that it is the url that ends in /graphql that one wants. However, when you hit that url from the browser, it attempts to load the playground, and fails, as the playground is disabled by default in production.
However, you can make graphql calls against that url from your client app and it works fine.
If you are using apollo server you can just enable it in production:
const server = new ApolloServer({
typeDefs,
resolvers,
introspection: true,
playground: true,
});
My node app is receiving some mysterious CONNECT requests and I'm trying to get to the bottom of where these are coming from (see separate post here).
I'm using the express module and until now, incoming CONNECT requests would just be fended off automatically with a 503 response.
But now, in order to examine the headers of these CONNECT requests, I'm trying to implement handling of such requests using the express module, if only just to dump the headers to the console and then send a 503 response myself.
Given the express seems to provide handling of connect as well as get and post (see here), this is what I've tried so far:
var connectHandler = function(req, res) {
console.log(JSON.stringify(req.headers));
res.sendStatus(503);
};
var app = express();
app.connect('*', connectHandler);
But the behaviour of the node app doesn't change... i.e. nothing is dumped to console when an incoming CONNECT request is received, with the CONNECT request being fended off and with the following entry in the log (just as before):
Oct 27 14:14:25 example heroku/router: at=error code=H13 desc="Connection closed without response" method=CONNECT path="example.herokuapp.com:443" host=example.herokuapp.com request_id=353e623x-dec4-42x5-bcfb-452add02ecef fwd="111.22.333.4" dyno=web.1 connect=0ms service=1ms status=503 bytes=0
Any help appreciated.
Scaling my web app to 2 web proceses on Heroku breaks Nowjs/Sockets.io.
I'm using full stack that is new to me and everything works great until I add a 2nd web process. Node.js still responds fine but Nowjs stops responding. I get 503 responds in the browser and
GET musicbacon.com/socket.io/1/?t=1339117661910 dyno=web.2 queue=0 wait=0ms service=2ms status=200 bytes=82
2012-06-08T01:07:42+00:00 heroku[router]: Error H13 (Connection closed without response) -> GET musicbacon.com/socket.io/1/websocket/12401614301555103827 dyno=web.2 queue= wait= service= status=503 bytes=
on the server.
I think that Nowjs (which is an abstraction layer on top of Socket.io) creates it's own server along side my Node.js server. Scaling Heroku spins up a new Node.js server VM but since Now.js isn't yet compatible with distributed environments it just dies.
Hopefully I'm missing something and I can scale Now.js because 1 Heroku web process can only handle about 60 users at a time in my experience.
Full stake is:
Heroku
Node.js
Express
Jade
Now.js
Postgres
Error H13 reference: https://devcenter.heroku.com/articles/error-codes#h13__connection_closed_without_response
WebSockets don't work on Heroku, you need to change your Socket.IO configuration to disable WS:
https://devcenter.heroku.com/articles/using-socket-io-with-node-js-on-heroku
https://devcenter.heroku.com/articles/request-timeout#longpolling_and_streaming_responses
https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO