I need create public queue with Microsoft Message Queuing(MSMQ) and send/receive messages from node.js server. What is best way to use node.js and MSMQ?
I'm interested in the "Fire-and-forget" model.
I too have the same use case. Whilst searching I stumbled across the node-msmq package on NPM/GitHub.
It is relatively new (May 2016) but I have had no issues with it to date. Simple API with a clear usage guide - worth a look.
Note: I'm not very sure this will be the answer but it helps
Try BusMQ is message bus and queueing system for node.js. Message queues are backed by Redis.
Code: https://github.com/capriza/node-busmq
var Bus = require('busmq');
var bus = Bus.create({redis: ['redis://127.0.0.1:6379']});
bus.on('online', function() {
var q = bus.queue('foo');
q.on('attached', function() {
console.log('attached to queue');
});
q.attach();
q.push({hello: 'world'});
q.push('my name if foo');
});
bus.connect();
Related
I am using this bitcore npm package.
https://bitcore.io/api/lib
And i want to monitor all the transactions over the blockchain, and read the input address, output address and amount associated with that transaction.
But i am unable to find the javascript method to invoke to accomplish this.
Even i am not able to find a example for this.
I am looking for as short as something like
var someLib = require('some-bitcore-lib')
someLib.on('transaction-found', function(){
// print everything
console.log(arguments);
// do something else;
})
Any help?
Where can i find that some-bitcore-lib or how can i create that in nodejs?
Using a third-party API, as the accepted answers suggests, will work in the short-term. But if you're looking for a long-term, reliable, not-rate-limited solution; you should run your own bitcoin node. It, of course, depends on your project's requirements.
For a robust solution to the OP's question, I suggest the following:
Run a pruned bitcoin node using bitcoind
Enable the ZeroMQ interface of bitcoind with the configuration option zmqpubrawtx=tcp://127.0.0.1:3600. This will enable streaming of raw transaction data to your node.js application
Use the ZeroMQ node.js module to subscribe to the bitcoind's ZeroMQ interface
Use bitcoinjs-lib to decode the raw transaction data
The following node.js example will use zeromq to subscribe to bitcoind's zeromq interface. Then bitcoinjs-lib is used to decode those raw transactions.
var bitcoin = require('bitcoinjs-lib');
var zmq = require('zeromq');
var sock = zmq.socket('sub');
var addr = 'tcp://127.0.0.1:3600';
sock.connect(addr);
sock.subscribe('rawtx');
sock.on('message', function(topic, message) {
if (topic.toString() === 'rawtx') {
var rawTx = message.toString('hex');
var tx = bitcoin.Transaction.fromHex(rawTx);
var txid = tx.getId();
tx.ins = tx.ins.map(function(in) {
in.address = bitcoin.address.fromOutputScript(in.script, bitcoin.networks.bitcoin);
return in;
});
tx.outs = tx.outs.map(function(out) {
out.address = bitcoin.address.fromOutputScript(out.script, bitcoin.networks.bitcoin);
return out;
});
console.log('received transaction', txid, tx);
}
});
For more details, please have a look at this guide
If you don't have your own node you can use blockchain.info APIs as described in here (https://github.com/blockchain/api-v1-client-node/tree/master/Socket)
const Socket = require('blockchain.info/Socket');
const mySocket = new Socket();
mySocket.onTransaction(function() {
console.log(arguments);
});
You can always watch transactions by running your own node without the need to depend on a service like blockchain.info... For example, if you are using btcd (Golang) (https://github.com/btcsuite/btcd) then you can get notified on transactions like in here (http://godoc.org/github.com/btcsuite/btcrpcclient#Client.NotifyNewTransactions)
I think this is what you're looking for. The tutorial helps the user set up a local btc node and demonstrates how to use a zmq subscription along with RPC comms to accomplish sending and receiving transactions as well as notifications and other functionality.
#c.hill's response is correct but leaves out the more complicated functionality described here :)
I'm using Google's Pub/Sub queue to handle messages between services. Some of the subscribers connect to rate-limit APIs.
For example, I'm pushing street addresses onto a pub/sub topic. I have a Cloud function which subscribes (via push) to that topic, and calls out to an external rate-limited geocoding service. Ideally, my street addresses could be pushed onto the topic with no delay, and the topic would retain those messages - calling the subscriber in a rate-limited fashion.
Is there anyway to configure such a delay, or a message distribution rate limit? Increasing the Ack window doesn't really help: I've architected this system to prevent long-running functions.
Because there's no answer so far describing workarounds, I'm going to answer this now by stating that there is currently no way to do this. There are workarounds (see the comments on the question that explain how to create a queueing system using Cloud Scheduler), but there's no way to just set a setting on a pull subscription that creates a rate limit between it and its topic.
I opened a feature request for this though. Please speak up on the tracked issue if you'd like this feature.
https://issuetracker.google.com/issues/197906331
An aproach to solve your problem is by using: async.queue
There you have a concurrency attribute wich you can manage the rate limit.
// create a queue object with concurrency 2
var q = async.queue(function(task, callback) {
console.log('hello ' + task.name);
callback();
}, 2);
// assign a callback
q.drain = function() {
console.log('all items have been processed');
};
// add some items to the queue
q.push({name: 'foo'}, function(err) {
console.log('finished processing foo');
});
// quoted from async documentation
GCP cloud task queue enables you to limit the number of tasks. Check this doc
Folks,
I would like to set up a message queue between our Java API and NodeJS API.
After reading several examples of using aws-sdk, I am not sure how to make the service watch the queue.
For instance, this article Using SQS with Node: Receiving Messages Example Code tells me to use the sqs.receiveMessage() to receive and sqs.deleteMessage() to delete a message.
What I am not clear about, is how to wrap this into a service that runs continuously, which constantly takes the messages off the sqs queue, passes them to the model, stores them in mongo, etc.
Hope my question is not entirely vague. My experience with Node lies primarily with Express.js.
Is the answer as simple as using something like sqs-poller? How would I implement the same into an already running NodeJS Express app? Quite possibly I should look into SNS to not have any delay in message transfers.
Thanks!
For a start, Amazon SQS is a pseudo queue that guarantees availability of messages but not their sequence in FIFO fashion. You have to implement sequencing logic into your app if you want it to work that way.
Coming back to your question, SQS has to be polled within your app to check if there are new messages available. I implemented this in an app using setInterval(). I would poll the queue for items and if no items were found, I would delay the next call and in case some items were found, the next call would be immediate bypassing the setInterval(). This is obviously a very raw implementation and you can look into alternatives. How about a child process on your server that pings your NodeJS app when a new item is found in SQS ? I think you can implement the child process as a watcher in BASH without using NodeJS. You can also look into npm modules if there is already one for this.
In short, there are many ways you can poll but polling has to be done one way or the other if you are working with Amazon SQS.
I am not sure about this but if you want to be notified of items, you might want to look into Amazon SNS.
When writing applications to consume messages from SQS I use sqs-consumer:
const Consumer = require('sqs-consumer');
const app = Consumer.create({
queueUrl: 'https://sqs.eu-west-1.amazonaws.com/account-id/queue-name',
handleMessage: (message, done) => {
console.log('Processing message: ', message);
done();
}
});
app.on('error', (err) => {
console.log(err.message);
});
app.start();
See the docs for more information (well documented):
https://github.com/bbc/sqs-consumer
I don't understand the way DNode uses websocket communication.
Some say it uses socket.io others say sockjs.
Which one is it? Or is it possible to choose?
I'm trying to use DNode, but I also need access to the connections for (semi-)broadcasting in reaction to RPC calls. How do I do this?
Is there a more extensive manual on dnode somewhere?
Your question is kind of vague. I'm not exactly sure whether DNode uses socket.io or sockjs, not sure it even uses one of those based on their dependencies list, but that is not really important when you program it.
As for using connections with DNode, it is pretty straight forward. Here's an example:
var server = dnode({
pushMessageNotification: function(message, cb) {
contact = getClientFromId(message.receiver);
contact.socket.emit('messageNotification', {
message: message.message,
sender: message.sender,
time: message.time
});
cb('success');
}
});
So as you can see, pushMessageNotification is a method that I binded with DNode-PHP and the message is encoded in JSON through PHP. Afterward, all you need is a method to find the socket of the client based on its id.
I'm trying to implement an API that interacts with a NodeJS server for realtime messaging. Now when that NodeJS app is deployed to a scalable environment like Heroku, multiple instances of this app may be running.
Is it possible to design the node app so that all clients subscribed to a "message channel" will receive this message, although multiple node instances are running - and therefore multiple copies of this channel?
Check out zeromq, it should provide some simple, high performance IPC abstractions to do what you want. In particular, the pub/sub example will be useful.
The main challenge as I imagine it, without knowing anything about how Heroku spawns multiple server instances, will be the logic to determine who is the publisher (the rest of the instances will be subscribers). So let's say, for argument's sake, that your hosting provider gives you an environment variable called INSTANCE_NUM which is an integer in [0,1024] indicating the instance number of the process; so we'll say that instance zero is the message publisher.
var zmq = require('zeromq')
if (process.env['INSTANCE_NUM'] === '0') { // I'm the publisher.
var emitter = getEventEmitter(); // e.g. an HttpServer.
var pub = zmq.createSocket('pub');
pub.bindSync('tcp://*:5555');
emitter.on('someEvent', function(data) {
pub.send(data);
});
} else { // I'm a subscriber.
var sub = zmq.createSocket('sub');
sub.subscribe('');
sub.on('message', function(data) {
// Handle the event data...
});
sub.connect('tcp://localhost:5555');
}
Note that I'm new to zeromq and the above code is totally untested, just for demonstration.