Beginner to redis here.
I'm trying to CRUD data from a distant redis server but I don't seem to make it work :
First we connect to the server :
const client = redis.createClient("the PORT", "the host", {auth_pass: "the password"});
(async () => {
try {
await client.connect();
console.log('Connected to redis');
} catch (err) {
console.error(err)
}
})()
The connection seems to work, as I receive a
Connected to redis in my terminal
But when I try to add data, it doesn't seem to work
app.get('/redis', (req, res) => {
// set name to antoine on redis
client.set('name', 'antoine', redis.print);
// get name from redis
client.get('name', (err, name) => {
res.send(name);
}
);
});
The endpoint just load indefinitely as if it doesn't do the
client.get('name', (err, name) => {
res.send(name);
}
);
And in my redis GUI, it shows No keys to display., so it didn't create the key either.
What am I missing here ?
EDIT : I just realized that instead of connecting to the distant server, I was connecting to my local one somehow.
Then, it seems that it doesn't connect to the distant.
EDIT2 : I forgot the async connect function, this is why it didn't work !
Related
I am trying to add a set to a redis database in a node.js app like this:
let redisConnect = async () => {
redisClient.on('error', (err) => {
console.log('Redis Client Error', err);
});
redisClient.on('ready', () => console.log('Redis is ready'));
await redisClient.connect();
redisClient.sadd(['tags', 'angularjs', 'reactjs', 'nodejs'], function(err, reply) {
console.log(reply);
});
};
redisConnect();
This error is thrown:
TypeError: redisClient.sadd is not a function
I am able to set other Redis database types on this client like list or string.
I do not understand fully how I solved this problem. I switched from the redis library to ioredis and got rid of:
await redisClient.connect();
and added 'await' to
await redisClient.sadd(['tags', 'angularjs', 'reactjs', 'nodejs'], function(err, reply) {
console.log(reply);
});
I think the redisClient.connect( ) might have been redundant. This threw an error when I switched to ioredis :
Error: Redis is already connecting/connected
Possibly someone will have an explanation for this solution.
I am trying to implement a Websocket connection from a React TypeScript app using RTK query. At the moment I am just trying to connect to a local socket.io server BUT ultimately it will be an AWS API Gateway with Cognito auth. In any case I having some problems getting this to work as a simple starting point. I have a few elements at play that may be causing the issue/s:-
MSW is being used to intercept http requests to mock a restful API locally. I wonder if this is one of the issues
I am adding the Websocket as a query to an RTK Query createApi object with other queries and mutations. In reality the Websocket query will need to hit a different API Gateway to the one that is being set as the baseQuery baseUrl currently. Do I need to create a new and separate RTK Query api using createApi() for the Websocket query?
Anyhow, here is the server code:-
// example CRA socket.io from https://github.com/socketio/socket.io/blob/main/examples/create-react-app-example/server.js
const getWebsocketServerMock = () => {
const io = require('socket.io')({
cors: {
origin: ['http://localhost:3000']
}
});
io.on('connection', (socket: any) => {
console.log(`connect: ${socket.id}`);
socket.on('hello!', () => {
console.log(`hello from ${socket.id}`);
});
socket.on('disconnect', () => {
console.log(`disconnect: ${socket.id}`);
});
});
io.listen(3001);
setInterval(() => {
io.emit('message', new Date().toISOString());
}, 1000);
console.log('Websocket server file initialised');
};
getWebsocketServerMock();
export {};
My RTK Query api file looks like this:-
reducerPath: 'someApi',
baseQuery: baseQueryWithReauth,
endpoints: (builder) => ({
getWebsocketResponse: builder.query<WebsocketResult, void>({
query: () => ``,
async onCacheEntryAdded(arg, { updateCachedData, cacheDataLoaded, cacheEntryRemoved }) {
try {
// wait for the initial query to resolve before proceeding
await cacheDataLoaded;
const socket = io('http://localhost:3001', {});
console.log(`socket.connected: ${socket.connected}`);
socket.on('connect', () => {
console.log('socket connected on rtk query');
});
socket.on('message', (message) => {
console.log(`received message: ${message}`);
// updateCachedData((draft) => {
// draft.push(message);
// });
});
await cacheEntryRemoved;
} catch {
// no-op in case `cacheEntryRemoved` resolves before `cacheDataLoaded`,
// in which case `cacheDataLoaded` will throw
}
}
}),
getSomeOtherQuery(.....),
getSomeOtherMutation(....),
Any advice or thoughts would be greatly appreciated! I guess my main question is should I be able to combine the websocket query in the same createApi function with other queries and mutations that need to use a different baseQuery url as they need to hit different API Gateways on AWS?
Much thanks,
Sam
You can circumvent the baseQuery from being used by specifying a queryFn instead of query on your endpoint.
In the most simple version, that just returns null as data so you can modify it later - but if you have an initial websocket request you can also do that in the queryFn.
queryFn: async () => { return { data: null } },
I have a lambda function using Node 12.
I need to add a new connection to a Redis database hosted in AWS ElastiCache.
Both are in one private VPC and the security groups/subnets are configured properly.
Solution:
globals.js:
const redis = require('redis');
const redisClient = redis.createClient(
`redis://${process.env.REDIS_HOST}:${process.env.REDIS_PORT}/${process.env.REDIS_DB}`,
);
redisClient.on('error', (err) => {
console.log('REDIS CLIENT ERROR:' + err);
});
module.exports.globals = {
REDIS: require('../helpers/redis')(redisClient),
};
index.js (outside handler):
const { globals } = require('./config/globals');
global.app = globals;
const lambda_handler = (event, context, callback) => { ... }
exports.handler = lambda_handler;
helpers/redis/index.js:
const get = require('./get');
module.exports = (redisClient) => {
return {
get: get(redisClient)
};
};
helpers/redis/get.js:
module.exports = (redisClient) => {
return (key, cb) => {
redisClient.get(key, (err, reply) => {
if (err) {
cb(err);
} else {
cb(null, reply);
}
});
};
};
Function call:
app.REDIS.get(redisKey, (err, reply) => {
console.log(`REDIS GET: ${err} ${reply}`);
});
Problem:
When increasing lambda timeout to a value greater than Redis timeout, I get this error:
REDIS CLIENT ERROR:Error: Redis connection to ... failed - connect ETIMEDOUT ...
Addition:
I tried quiting/closing the connection after each transaction:
module.exports = (redisClient) => {
return (cb) => {
redisClient.quit((err, reply) => {
if (err) {
cb(err);
} else {
cb(null, reply);
}
});
};
};
app.REDIS.get(redisKey, (err, reply) => {
console.log(`REDIS GET: ${err} ${reply}`);
if (err) {
cb(err);
} else {
if (reply) {
app.REDIS.quit(() => {
cb()
});
}
}
})
Error:
REDIS GET: AbortError: GET can't be processed. The connection is already closed.
Extra Notes:
I have to use callbacks, this is why I pass ones in the above examples
I'm using "redis": "^3.0.2"
It's not a configuration issue as the cache was accessed hundred of times in a small period of time but it then started giving the timeout errors.
Everything works normally locally
It's not a configuration issue as the cache was accessed hundred of times in a small period of time but it then started giving the timeout errors.
i think it is origin of issue, probably redis database size hit the size limit, and it cannot process new data?
Can you delete old data in it?
Also it is possible Elastic Cache has limits on new TCP clients' connections, and if its depleted, new connections are refused with similar error message you mentioned.
If redis client in aws lambda function cannot establish connection, aws lambda function fails - and new one is started. New lambda function makes one more connection to redis, redis cannot process it, and one more lambda function is started...
So, at one moment, we hit the limit on active redis connections, and system is in deadlock.
I think you can temporary stop all lambda functions, and scale up Elastic Cache redis database.
I'd like to get the number of connections of a few servers running on my local machine.
I've successfully used server.getConnections() on a server created via net.createServer(), however I don't know how to use it on an already started server.
I tried obtaining the server instance by connecting to it, using net.connect(). The connection is created successfully and I get a new net.Socket object, however my understanding is that I actually need a net.Server in order to use getConnections().
So my question is, how do I get a net.Server instance of an already running server?
I realize my question is an XY problem, so apologies to all who tried to answer it.
I suspect the answer to the literal question, "how do I get an instance of an existing server", is: "you can't".
I should have added more details to the question, especially what I was trying to achieve.
My application is a load balancer / reverse proxy server. Initially I was able to use getConnections() because I would start the proxy server and a few dummy servers from the same script. However I wanted to make the dummy servers and the proxy separate from each other, so even though I did have complete control over them, I needed to pretend that I didn't actually own the servers.
The solution I found to my specific case, in the end, was to keep a hash list of servers I can connect to (via the reverse proxy), and increment the connection counters every time I connect to a specific server:
let servers = [
{ port: 4000, connectionsCounter: 0 },
{ port: 5000, connectionsCounter: 0 },
{ port: 6000, connectionsCounter: 0 },
];
let myProxyServer = net.createServer((socket) => {
// Open a connection to the first server in the list
net.connect(servers[0].port, () => {
// Once connected, increment the connections counter
socket.on('connect', () => {
servers[0].connectionsCounter++;
});
// When the connection ends, decrement the counter
socket.on('close', () => {
servers[0].connectionsCounter--;
});
});
});
I hope this will be helpful to someone.
If you want to just use the server, you can probably store it as a variable when you call net.createServer()
const my_server = net.createServer();
// do what you want with it
my_server.getConnections();
my_server.listen();
you can make instance of net.createServer() and then get your number of connections from server.on('connection', <callback>):
server.on('connection', (socket) => {
// someone connected
console.log("New active connection");
server.getConnections((err, count) => {
if(err){
console.log(err);
} else {
console.log("Currently " + count + " active connection(s)");
}
});
});
i hope this complete example code help you:
const net = require('net');
const uuid = require('uuid/v1');
const server = net.createServer((socket) => {
socket.uuid = uuid();
socket.on('data', (data) => {
//const response = JSON.parse(data.toString('utf8'));
});
socket.on('error', (err) => {
console.log('A client has left abruptly !');
server.getConnections((err, count) => {
if(err){
console.log(err);
} else {
console.log("Currently " + count + " active connection(s)");
}
});
});
socket.on('end', () => {
console.log("A client has left");
server.getConnections((err, count) => {
if(err){
console.log(err);
} else {
console.log("Currently " + count + " active connection(s)");
}
});
});
});
server.on('error', (err) => {
// handle errors here
console.log("Error:", err);
});
server.on('connection', (socket) => {
// someone connected
console.log("New active connection");
server.getConnections((err, count) => {
if(err){
console.log(err);
} else {
console.log("Currently " + count + " active connection(s)");
}
});
});
// port number.
server.listen(3000, () => {
console.log('opened server on', server.address());
});
or you can use netstat for get number of connection, https://www.npmjs.com/package/node-netstat nodejs module for this solution:
const netstat = require('node-netstat');
myObject = {
protocol: 'tcp',
};
setInterval(function () {
let count = 0;
netstat({
filter: {
local: {port: 3000, address: '192.168.1.1'}
}
}, item => {
// console.log(item);
count++;
console.log(count);
});
}, 1000);
Your application is a bit unclear.
Only the server socket can report how many connections it has. This is why if you create the net.Server you can access that information from it.
If you want to connect to an application and query the number of clients connected to it, the application that you connect to needs to provide that information to you when you ask. This is not information that the socket provides - the application itself has to provide that information.
If you are writing the application that created the net.Server, you can create another net.Server on a different port that you can then connect to and query it for information about the other clients on its other sockets.
If you are trying to generically find the number of connections to a particular application that has a socket, that application needs to be able to tell you, or, as #root mentioned, you need to ask the OS the application is running on. This function will be OS dependent and will likely require elevated privileges. But consider connecting to a socket on a router or IoT device: that application may not be running on any OS at all.
I want my application (lets say a simple node file for now) to work as it is even if redis is not available. I'm not able to do it the correct way. This is what I've tried.
var redis = require('redis');
var redisClient = null;
var getRedisClient = function(){
if(redisClient){
return redisClient;
}
try {
redisClient = redis.createClient({connect_timeout : 5000, max_attempts : 1});
redisClient.on("error", function(err) {
console.error("Error connecting to redis", err);
redisClient = null;
});
return redisClient;
} catch(ex){
console.log("error initialising redis client " + ex);
return null;
}
};
try {
var client = getRedisClient();
console.log("done!");
} catch (ex){
console.log("Exception");
}
However, with this code my application exits if redis is not available (it shouldn't because i've not given a process.exit() command).
How can I solve this?
Checking for Successful Connection on Start
Using a promise, you could guarantee that at least initially, you were able to connect to redis without error within a specified time period:
const redis = require('redis');
const Promise = require('bluebird');
function getRedisClient(timeoutMs){
return new Promise((resolve, reject) => {
const redisClient = redis.createClient();
const timer = setTimeout(() => reject('timeout'), timeoutMs);
redisClient.on("ready", () => {
clearTimeout(timer);
resolve(redisClient);
});
redisClient.on("error", (err) => {
clearTimeout(timer);
reject(err);
});
});
};
const redisReadyTimeoutMs = 10000;
getRedisClient(redisReadyTimeoutMs)
.then(redisClient => {
// the client has connected to redis sucessfully
return doSomethingUseful();
}, error => {
console.log("Unable to connect to redis", error);
});
You Need Proper Error Handling
The redis client being non-null does NOT guarantee using it won't throw an error.
you could experience infrastructure misfortune e.g. crashed redis process, out of memory or network being down.
a bug in your code could cause an error e.g. invalid or missing arguments to a redis command.
You should be handling redis client errors as a matter of course.
DON'T null the Redis Client on Error
It won't give you much but it will force you to check for null every time you try and use it.
The redis client also has inbuilt reconnect and retry mechanisms that you'll miss out on if you null it after the first error. See the redis package docs, look for retry_strategy.
DO Wrap your redis client code with try .. catch ... or use .catch in your promise chain.
DO Make use of a retry_strategy.