I am trying to create a simple text editor that has Operational Transform multi tenant support and while it was reasonably easy to get the editor working and syncing across clients using shareJS, my problem is I would like to sync the shareJS doc's with a Folder structure on the Server side (this will eventually be a git repo)
I am completely new to sharejs and Operational Transforms and found the shareJS documentation a little tough to follow for more complex example.
Any suggestions on How might I approach this problem?
What I have tried to do is to implement a client on the server side that could get the entire doc text on update but (and this is the lack of experience I'm sure) the only way I can think to accomplice is to use the client api to cycle through all documents and write each to a file. But to me this sounds horribly inefficient. Can anyone point me to any resources that might help or offer some advice as to how I could approach this?
This is a bit late thought but you can still call getSnapshop method on the server side and dump that into a file on your file system. If it is not run locally you can create a tiny router with express on your local machine that listen for post request and you post the dumped file into the post request body on your sharejs server and then on your machine dump the post request body to a file, that should work.
Beware of security considerations if you use a auth system on your server.
Related
Bots are amazing, unless you're Google Analytics
After many months of learning to host my own Discord bot, I finally figured it out! I now have a node server running on my localhost that sends and receives data from my Discord server; it works great. I can do all kinds of the things I want to with my Discord bot.
Given that I work with analytics everyday, one project I want to figure out is how to send data to Google Analytics (specifically GA4) from this node server.
NOTE: I have had success in sending data to my Universal Analytics property. However, as awesome as that was to finally see pageviews coming into, it was equally heartbreaking to recall that Google will be getting rid of Universal Analytics in July of this year.
I have tried the following options:
GET/POST requests to the collect endpoint
This option presented itself as impossible from the get-go. In order to send a request to the collection endpoint, a client_id must be sent along with the request itself. And this client_id is something that must be generated using Google's client id algorithm. So, I can't just make one up.
If you consider this option possible, please let me know why.
Install googleapis npm package
At first, I thought I could just install the googleapis package and be ready to go, but that idea fell on its face immediately too. With this package, I can't send data to GA, I can only read with it.
Find and install a GTM npm package
There are GTM npm packages out there, but I quickly found out that they all require there to be a window object, which is something my node server would not have because it isn't a browser.
How I did this for Universal Analytics
My biggest goal is to do this without using Python, Java, C++ or any other low level languages. Because, that route would require me to learn new languages. Surely it's possible with NodeJS alone... no?
I eventually stumbled upon the idea of actually hosting a webpage as some sort of pseudo-proxy that would send data from the page to GA when accessed by something like a page scraper. It was simple. I created an HTML file that has Google Tag Manager installed on it, and all I had to do was use the puppeteer npm package.
It isn't perfect, but it works and I can use Google Tag Manager to handle and manipulate input, which is wonderful.
Unfortunately, this same method will not work for GA4 because GA4 automatically excludes all identified bot traffic automatically, and there is no way to turn that setting off. It is a very useful feature for GA4, giving it quite a bit more integrity than UA, and I'm not trying to get around that fact, but it is now the Bane of my entire goal.
https://support.google.com/analytics/answer/9888366?hl=en
Where to go from here?
I'm nearly at the end of my wits on figuring this one out. So, either an npm package exists out there that I haven't found yet, or this is a futile project.
Does anyone have any experience in sending data from NodeJS to GA4? (or even GTM?) How did you do it?
...and this client_id is something that must be generated using Google's client id algorithm. So, I can't just make one up...
Why, of course you can. GA4 generates it pretty much the same as UA does. You don't need anything from google to do it.
Besides, instead of mimicking just requests to the collect endpoint, you may just wanna go the MP route right away: https://developers.google.com/analytics/devguides/collection/protocol/ga4 The links #dockeryZ gave, work perfectly fine. Maybe try opening them in incognito, or in a different browser? Maybe you have a plugin blocking analytics urls.
Moreover, you don't really need to reinvent the bicycle. Node already has a few packages to send events to GA4, here's one looking good: https://www.npmjs.com/package/ga4-mp?activeTab=readme
Or you can just use gtag directly to send events. I see a lot of people doing it even on the front-end: https://www.npmjs.com/package/ga-gtag Gtag has a whole api not described in there. Here's more on gtag: https://developers.google.com/tag-platform/gtagjs/reference Note how the library allows you to set the client id there.
The only caveat there is that you'll have to track client ids and session ids manually. Shouldn't be too bad though. Oh, and you will have to redefine the concept of a pageview, I guess. Well, the obvious one is whenever people post in the chan that is different from the previous post in a session. Still, this will have to be defined in the code.
Don't worry about google's bot traffic detection. It's really primitive. Just make sure your useragent doesn't scream "bot" in it. Make something better up.
I have a huge symfony app and I wanted to add some feature that I could only do with a nodeJS server .
So I have a big JSON file which result from my nodeJS run, this file have to go in Symfony.
And symfony have to be able to send some pdf file to the node server (the one which will be transform in JSON by my node server).
Is anyone have some starting idea ?
thansk for help :D
No one is going to be able to provide a full answer with so few details, but generally speaking messaging and remote procedure calls are excellent for interop between parts of a large app.
You could send a message from Symfony (which includes the path of the PDF, or the contents itself), and node will provide the result. You can encode that as JSON, and send it as an answer.
RabbitMq is widely supported, allows both produce-consume or RPC-style use.
I'm using an external service to create images. I'd like my users to be able to hit my API and ask for the image. Then my Express server would retrieve it from the external service, then serve it to the user. Sort of like a proxy I suppose, but not exactly.
Is there an easy way to do this, preferably one that doesn't involve downloading the image to the hard drive, then reading it back in and serving it?
Using the request library, I was able to come up with this:
var request = require("request");
exports.relayImage = function(req, res){
request(req.params.url).pipe(res);
}
That seems to work. If there is a more efficient way to do this (meaning on server resources, not in terms of lines of code), speak up!
What you are doing is exactly what you should be doing, and is the most efficient method. Using pipe, the data is sent as it comes in, requiring no additional resources than are needed to buffer and transmit.
Also be mindful of content type and other response headers that you may want to relay. Finally, realize that you've effectively built an open proxy where anyone can request anything they want through your servers. This is a bit dangerous, so be sure to lock it down in your final application.
You should be able to use the http module to make a request to the external image service with a callback that returns the image as the response. It won't write to disk unless you explicitly tell it to.
I am trying to allow users to upload images to a site I have built with Meteor, which requires a server-side route I can POST data to. Is there a way to set up a route server side such that I can call Meteor code from it (for example, a route that I can call this.userId or Meteor.userId() from)?
Server side routing is on the roadmap, but not yet available the way you need it. (At that link, Server-side rendering is set for version 1.0).
In the interim, you can do some server side rendering with Tom Coleman's excellent meteor-router mrt package. It's unclear to me how much of this will make it into Meteor core.
More likely, however, for the file upload problem, this issue describes the problem, and it looks like people have had luck implementing imgur's xhr api or perhaps even better is this smart idea.
Hope this helps.
I'm trying to get my node.js server to server up audio files for the HTML5 audio tag. Currently I'm referencing audio files via an express staticProvider, but this will allow the sound to play once and then never again.
From what I've been able to gather in order for the sound to work properly with seeking or looping I need to provide the "Content-Range" header when my audio file is requested. I highly doubt that the express staticProvider is doing this for me, so it would seem that I need to serve the file up using custom code. Unfortunately I'm fairly new to node.js and things like serving files are still a little beyond me. Can anyone offer some advice on how I can provide the appropriate metadata for my audio files?
[EDIT] (Removed old server code since it won't do anyone any good!)
Sorry, I don't typically ask questions this broad, but I'm really at a loss as to where to start with this one. Any suggestions?
[SOLUTION]
So it appears that the solution is just to use updated software. I was trying express rather than the built-in connect HTTP middleware because I thought connect wasn't doing static file serving right. Turns out, I was just looking at the documentation for a newer version of connect while the version that comes with node is a bit older. Once I updated my connect library (I just used npm to install the latest, for those that are curious), the following worked beautifully:
var connect = require('connect');
var server = connect.createServer(
// If your server errors on this line, saying it doesn't know what
// "static" is you need to get the latest connect!
connect.static(__dirname + '/public')
);
server.listen(PORT);
The static serving logic is done by the underlying connect server (using its static middleware). There is code for 'Content-Range' headers (see here, about line 148), but these headers are only set when the underlying client (in your case, the browser's HTML5 implementation) sends the correct request headers.
Maybe this post from google groups can help you out. The topic is slightly different (video streaming), but i think the core of the problem is about the same.