I'm creating a website using React and Express GraphQL, with a MariaDB database. I've come across a problem with making frequent requests/subscriptions to the GraphQL API, however.
After the page in question loads in React, I call this:
componentDidMount() {
this.setState({
waitingTimer: setInterval(this.queryWaiting.bind(this), 1000)
});
}
where queryWaiting is a function that carries out a fetch request to my GraphQL server at localhost:3000/graphql. That URL itself is a proxy defined in my setupProxy.js file which proxies the URL from localhost:4000/graphql, so that I don't need to use CORS.
It's worth noting that I also clearInterval this waitingTimer in componentWillUnmount.
In my GraphQL server file, which I'm simply running with node, I set up my server like this:
var app = express();
app.use("/graphql", graphqlHTTP({
schema: schema,
rootValue: root,
graphiql: true,
}));
app.listen(4000);
Where schema is a valid schema, and root is my root resolver.
The API query that I'm repeatably calling is called getWaitingCount, and it acts like this:
getWaitingCount: () => {
return new Promise((resolve, reject) => {
var currentTime = Date.now() / 1000;
if (cachedWaitingTime + cachedWaitingInterval > currentTime) {
return cachedWaiting;
}
connection.query("SELECT COUNT(*) FROM profiles WHERE isWaiting=1;", function (error, results, fields) {
if (error)
reject(error);
else {
// Update cache values
cachedWaiting = results[0]["COUNT(*)"];
cachedWaitingTime = Date.now() / 1000;
resolve(cachedWaiting);
}
});
});
}
I've implemented some caching to reduce server load. The caching variables are defined elsewhere.
The problem I experience is that, after between 2-10 seconds, the POST requests for this specific query from my React app stop getting resolved, and I have to reload the page for them to start getting accepted again. Weirdly, other requests go through fine.
I've tried various solutions, such as switching to using the proxy method I'm using currently, and I've also considered the possibility of a built-in anti-DDOS mechanism in the Express code, but I can't find any evidence for that.
Any help with this issue is very much appreciated.
Turns out the best solution is to type out and explain your code :)
This code:
if (cachedWaitingTime + cachedWaitingInterval > currentTime) {
return cachedWaiting;
}
is inside a promise, so really, it should do:
resolve(cachedWaiting);
instead of returning it.
That is all. I am currently kicking myself.
Related
I'm coding for an API connection area, that's predominately graphql but needs to have some REST connections for certain things, and have equivalent to the following code:
foo.js
module.exports = {
routes: () => {
return [
{
method: 'GET',
path: '/existing_endpoint',
handler: module.exports.existing_endpoint
},
{
method: 'POST',
path: '/new_endpoint',
handler: module.exports.new_endpoint // <--- this not passing variables
}
]
},
existing_endpoint: async () => {
/* endpoint that isn't the concern of this */
},
new_endpoint: async (req, res) => {
console.log({req, res})
return 1
}
}
The existing GET endpoint works fine, but my POST endpoint always errors out with the console of {} where {req, res} should have been passed in by the router, I suspect because the POST isn't receiving. I've tried changing the POST declaration in the routes to module.exports.new_endpoint(req, res), but it tells me the variables aren't found, and the lead-in server.js does have the file (it looks more like this...), and doing similar with the server.js, also getting similar results, implying that's probably wrong too. Also, we have a really strict eslint setup, so I can't really change the format of the call.
Every example I've seen online using these libraries is some short form, or includes the function in the routes call, and isn't some long form like this. How do I do a POST in this format?
/* hapi, environment variables, apollog server, log engine, etc. */
/* preceeding library inclusions */
const foo = require('./routes/foo')
const other_route = require('./routes/other_route')
const startServer = async () => {
const server = Hapi.server({port, host})
server.route(other_route.routes())
server.route(foo.routes())
}
This is a bug with Hapi in node v16. I just opened an issue.
Your current solutions are either:
Upgrade to Hapi v20
Use n or another method to downgrade to node v14.16 for this project. I can confirm that POST requests do not hang in this version.
I'm using Next.js for my side project. I have a PostrgeSQL database hosted on ElephantSQL. Inside the Next.js project, I have a GraphQL API set up, using the apollo-server-micro package.
Inside the file where the GraphQL API is set up (/api/graphql), I import a database helper-module. Inside that, I set up a pool connection and export a function which uses a client from the pool to execute a query and return the result. This looks something like this:
// import node-postgres module
import { Pool } from 'pg'
// set up pool connection using environment variables with a maximum of three active clients at a time
const pool = new Pool({ max: 3 })
// query function which uses next available client to execute a single query and return results on success
export async function queryPool(query) {
let payload
// checkout a client
try {
// try executing queries
const res = await pool.query(query)
payload = res.rows
} catch (e) {
console.error(e)
}
return payload
}
The problem I'm running into, is that it appears as though the Next.js API doesn't (always) keep the connection alive but rather opens up a new one (either for every connected user or maybe even for every API query), which results in the database quickly running out of connections.
I believe that what I'm trying to achieve is possible for example in AWS Lambda (by setting context.callbackWaitsForEmptyEventLoop to false).
It is very possible that I don't have a proper understanding of how serverless functions work and this might not be possible at all but maybe someone can suggest me a solution.
I have found a package called serverless-postgres and I wonder if that might be able to solve it but I'd prefer to use the node-postgres package instead as it has much better documentation. Another option would probably be to move away from the integrated API functionality entirely and build a dedicated backend-server, which maintains the database connection but obviously this would be a last resort.
I haven't stress-tested this yet, but it appears that the mongodb next.js example, solves this problem by attaching the database connection to global in a helper function. The important bit in their example is here.
Since the pg connection is a bit more abstract than mongodb, it appears this approach just takes a few lines for us pg enthusiasts:
// eg, lib/db.js
const { Pool } = require("pg");
if (!global.db) {
global.db = { pool: null };
}
export function connectToDatabase() {
if (!global.db.pool) {
console.log("No pool available, creating new pool.");
global.db.pool = new Pool();
}
return global.db;
}
then in, eg, our API route, we can just:
// eg, pages/api/now
export default async (req, res) => {
const { pool } = connectToDatabase();
try {
const time = (await pool.query("SELECT NOW()")).rows[0].now;
res.end(`time: ${time}`);
} catch (e) {
console.error(e);
res.status(500).end("Error");
}
};
so may be this is very basic question so please bear with me. Let me explain what I am doing and what I really need.
EXPLANATION
I have created a graphql server by using ApolloGraphql (apollo-server-express npm module).
Here is the code snippet to give you an idea.
api.js
import express from 'express'
import rootSchema from './root-schema'
.... // some extra code
app = express.router()
app.use(jwtaAuthenticator) // --> this code authenticates Authorization header
.... // some more middleware's added
const graphQLServer = new ApolloServer({
schema: rootSchema, // --> this is root schema object
context: context => context,
introspection: true,
})
graphQLServer.applyMiddleware({ app, path: '/graphql' })
server.js
import http from 'http'
import express from 'express'
import apiRouter from './api' // --> the above file
const app = express()
app.use([some middlewares])
app.use('/', apiRouter)
....
....
export async function init () {
try {
const httpServer = http.createServer(app)
httpServer
.listen(PORT)
.on('error', (err) => { setTimeout(() => process.exit(1), 5000) })
} catch (err) {
setTimeout(() => process.exit(1), 5000)
}
console.log('Server started --- ', PORT)
}
export default app
index.js
require('babel-core')
require('babel-polyfill')
require = require('esm')(module/* , options */)
const server = require('./server.js') // --> the above file
server.init()
PROBLEM STATEMENT
I am using node index.js to start the app. So, the app is expecting Authorization header (JWT token) to be present all the times, even for the introspection query. But this is not what I want, I want that introspection query will be resolvable even without the token. So that anyone can see the documentation.
Please shed some light and please guide what is the best approach to do so. Happy coding :)
.startsWith('query Introspection') is insecure because any query can be named Introspection.
The better approach is to check the whole query.
First import graphql and prepare introspection query string:
const { parse, print, getIntrospectionQuery } = require('graphql');
// format introspection query same way as apollo tooling do
const introspectionQuery = print(parse(getIntrospectionQuery()));
Then in Apollo Server configuration check query:
context: ({ req }) => {
// allow introspection query
if (req.body.query === introspectionQuery) {
return {};
}
// continue
}
There's a ton of different ways to handle authorization in GraphQL, as illustrated in the docs:
Adding middleware for express (or some other framework like hapi or koa)
Checking for authorization inside individual resolvers
Checking for authorization inside your data models
Utilizing custom directives
Adding express middleware is great for preventing unauthorized access to your entire schema. If you want to allow unauthenticated access to some fields but not others, it's generally recommended you move your authorization logic from the framework layer to the GraphQL or data model layer using one of the methods above.
So finally I found the solution and here is what I did.
Let me first tell you that there were 2 middle-wares added on base path. Like this:
app //--> this is express.Router()
.use(jwtMw) // ---> these are middlewares
.use(otherMw)
The jwtMw is the one that checks the authentication of the user, and since even introspection query comes under this MW, it used to authenticate that as well. So, after some research I found this solution:
jwtMw.js
function addJWTMeta (req, res, next) {
// we can check for null OR undefined and all, then check for query Introspection, with better condition like with ignore case
if (req.body.query.trim().startsWith('query Introspection')) {
req.isIntrospection = true
return next()
}
...
...
// ---> extra code to do authentication of the USER based on the Authorization header
}
export default addJWTMeta
otherMw.js
function otherMw (req, res, next) {
if (req.isIntrospection) return next()
...
...
// ---> extra code to do some other context creation
}
export default otherMw
So here in jwtMw.js we are checking that if the query is Introspection just add a variable in req object and move forward, and in next middleware after the jwtMw.js whosoever wants to check for introspection query just check for that variable (isIntrospection, in this case) and if it is present and is true, please move on. We can add this code and scale to every middleware that if req.isIntrospection is there just carry on or do the actual processing otherwise.
Happy coding :)
I have made a web application with a simple API. The code for the front-end and and the API are both served from the same host. The front end consumes the API by making basic http requests. While developing, I have been making these requests within the front-end using port 3000 from the locally run server.
What is the best way to do this on a production server (An AWS EC2 instance)?
How do I easily generalize this in the development code so I don't have to change it from
axios.get("localhost:3000" + otherParams)
.then(response => {
//use the response to do things
});
})
to
axios.get("http://99.999.999.999:80" + otherParams)
.then(response => {
//use the response to do things
});
})
every time I push an update to the live server? Is this just something that web developers have to put up with? Sorry if this is a dumb question..
We definitely don't have to put up with changing our code like that every time! (Thank the coding gods)
So I think what you are after is environment variables
For example: You could setup an environment variable called SERVER_URL
Then when you are running locally that variable is localhost:3000 but when you deploy to amazon it can be set to http://99.999.999.999:80
in node you consume the variable like this
process.env.WHATEVER_YOUR_VARIABLE_NAME_IS
So in your case it would be
axios.get(process.env.SERVER_URL + otherParams)
a popular module to help create these variables is dotenv, which is worth looking at.
As a little bonus answer to help (and hopefully not confuse you too much), axios lets you create your own instance of axios so that you don't have to repeat yourself. Their example is
const instance = axios.create({
baseURL: 'https://some-domain.com/api/',
timeout: 1000,
headers: {'X-Custom-Header': 'foobar'}
});
So you could do something like
const api = axios.create({
baseURL: process.env.SERVER_URL
});
then you can replace your axios calls with your new instance of axios (api)
Something like this.
api.get(otherParams)
Hope that makes some sense and gets you back on track!
You can create a config.js file:
var configs = {};
configs.appPort = 3000;
configs.host = '192.168.99.100';
module.exports = configs;
Importing the configure file:
var configs = require('./config');
Axios:
axios.get(configs.host + ":" + configs.appPort + "/" + otherParams)
.then(response => {
//use the response to do things
});
})
You can also create environment variables like this:
configs.isProduction = false;
configs.localHost = "localhost";
configs.productionHost = "192.168.99.100";
And then you can check in your app if it is production, use productionHost, otherwise, use localHost.
I have an existing Node.js/Express app which connects to 2 separate databases, it has a MySQL DB for all the relational and a MongoDB store for the non-relational vertical data.
It uses Sequelize and Mongoose and works absolutely swimmingly.
I've been looking at Next.js today and I'm pretty impressed, one of my pet peeves with React is actually how much bootstrapping there is and how much code it takes to achieve something simple. Next.js seems to solve some of those issues for me, so I'm willing to embrace it.
First issue - Is it possible to connect Next.js to existing DB's and read their objects directly in the view?
e.g. ./server.js:
const mongoDb = mongoose.connect(configDB.url); // MongoDB connection
const models = require('./models'); // Sequelize connection
app.prepare().then(() => {
server.use((req, res, next) => {
req.mongodb = mongoDb
req.mysqldb = models
// Logging req.mysqldb/req.mongodb at this point gives the correct result.
next()
});
server.get('*', (req, res) => {
return handle(req, res)
})
})
./pages/index.js:
Index.getInitialProps = async function(req) {
console.log(req.mongodb);
console.log(req.mysqldb)
// Example of what I want: req.mysqldb.users.findAll()....... to populate collection for this view
}
When the console statements are executed in the index.js page, they are logged as undefined.
Ideally I want to use the objects/ORM layer directly in the next.js templates, I do not want to have to call my own API internally, it seems like a huge waste of resources!
Any help, greatly appreciated.
Just for future reference. getInitialProps gets passed in an object with one of the keys being req. So you're meant to do something like the following instead
// add the curly braces around req
Index.getInitialProps = async function({ req }) {
// code
}
This is known as Function Parameter Destructuring and was introduced in ES6. What this accomplishes is similar to the following code
Index.getInitialProps = async function(_ref) {
var req = _ref.req;
}
Meaning, it takes the value of req of the object that gets passed and uses that value.
Well apparently by the time the request gets to the template it has changed a bit! Namely, it is nested within another request object.
req.req.mongodb and req.req.mysqldb both work fine :).