I'm building my first node.js app, and trying to integrate it with the Instapaper API. Instapaper only uses XAuth, and I'm a little confused how it works.
Should I use the node-oauth module and try to customize it for xauth? Or should I roll my own xauth? Or something else, maybe?
I'd checkout something like EveryAuth, see how they are handling the various options out there, forking it, and then contributing back with a new implementation.
Good luck man.
Here is how to get it working with the oauth module.
https://stackoverflow.com/a/9645033/186101
You'd still need to handle storing the tokens for your users, but that answer should get you u and running for testing the API in node.
Related
Bots are amazing, unless you're Google Analytics
After many months of learning to host my own Discord bot, I finally figured it out! I now have a node server running on my localhost that sends and receives data from my Discord server; it works great. I can do all kinds of the things I want to with my Discord bot.
Given that I work with analytics everyday, one project I want to figure out is how to send data to Google Analytics (specifically GA4) from this node server.
NOTE: I have had success in sending data to my Universal Analytics property. However, as awesome as that was to finally see pageviews coming into, it was equally heartbreaking to recall that Google will be getting rid of Universal Analytics in July of this year.
I have tried the following options:
GET/POST requests to the collect endpoint
This option presented itself as impossible from the get-go. In order to send a request to the collection endpoint, a client_id must be sent along with the request itself. And this client_id is something that must be generated using Google's client id algorithm. So, I can't just make one up.
If you consider this option possible, please let me know why.
Install googleapis npm package
At first, I thought I could just install the googleapis package and be ready to go, but that idea fell on its face immediately too. With this package, I can't send data to GA, I can only read with it.
Find and install a GTM npm package
There are GTM npm packages out there, but I quickly found out that they all require there to be a window object, which is something my node server would not have because it isn't a browser.
How I did this for Universal Analytics
My biggest goal is to do this without using Python, Java, C++ or any other low level languages. Because, that route would require me to learn new languages. Surely it's possible with NodeJS alone... no?
I eventually stumbled upon the idea of actually hosting a webpage as some sort of pseudo-proxy that would send data from the page to GA when accessed by something like a page scraper. It was simple. I created an HTML file that has Google Tag Manager installed on it, and all I had to do was use the puppeteer npm package.
It isn't perfect, but it works and I can use Google Tag Manager to handle and manipulate input, which is wonderful.
Unfortunately, this same method will not work for GA4 because GA4 automatically excludes all identified bot traffic automatically, and there is no way to turn that setting off. It is a very useful feature for GA4, giving it quite a bit more integrity than UA, and I'm not trying to get around that fact, but it is now the Bane of my entire goal.
https://support.google.com/analytics/answer/9888366?hl=en
Where to go from here?
I'm nearly at the end of my wits on figuring this one out. So, either an npm package exists out there that I haven't found yet, or this is a futile project.
Does anyone have any experience in sending data from NodeJS to GA4? (or even GTM?) How did you do it?
...and this client_id is something that must be generated using Google's client id algorithm. So, I can't just make one up...
Why, of course you can. GA4 generates it pretty much the same as UA does. You don't need anything from google to do it.
Besides, instead of mimicking just requests to the collect endpoint, you may just wanna go the MP route right away: https://developers.google.com/analytics/devguides/collection/protocol/ga4 The links #dockeryZ gave, work perfectly fine. Maybe try opening them in incognito, or in a different browser? Maybe you have a plugin blocking analytics urls.
Moreover, you don't really need to reinvent the bicycle. Node already has a few packages to send events to GA4, here's one looking good: https://www.npmjs.com/package/ga4-mp?activeTab=readme
Or you can just use gtag directly to send events. I see a lot of people doing it even on the front-end: https://www.npmjs.com/package/ga-gtag Gtag has a whole api not described in there. Here's more on gtag: https://developers.google.com/tag-platform/gtagjs/reference Note how the library allows you to set the client id there.
The only caveat there is that you'll have to track client ids and session ids manually. Shouldn't be too bad though. Oh, and you will have to redefine the concept of a pageview, I guess. Well, the obvious one is whenever people post in the chan that is different from the previous post in a session. Still, this will have to be defined in the code.
Don't worry about google's bot traffic detection. It's really primitive. Just make sure your useragent doesn't scream "bot" in it. Make something better up.
I am new to Node and web development. I am working on a simple application to get started and want to limit the use of my endpoints to server side calls, i.e. anyone else trying to access them gets a 403.
However, after researching, I found PassportJS with examples of how to protect endpoints, but they all seem to be overkill especially for a small application like mine.
Am I overthinking this, or should I be looking for a more basic solution?
How do you need to consume these end points yourself? I don't know if all you are looking to do is to call one file from the other. You need not expose them as restful end points in that case. Create the methods as module methods (as in node modules). Then just use require syntax to reference that file as a dependency.
You need restful end points for server side only if it needs to be called from another web application.
I am using Express to expose a REST API in my Node.js application. When a REST request comes in, I extract the user information from the HTTP headers. I would like this information to be available throughout the life of this request, no matter what function I am in. An obvious but kludgy way is to pass around the user information as parameters to all function calls. Is there a better way? So far I have found the following solutions, but I am not sure if they are ready for prime time:
StrongLoop Zone Library: Docs say "The zone library and documentation are still under development: there are bugs, missing features, and limited documentation."
Continuation-Local Storage: Not sure if this is slated to be part of Node.js. This issue at the end recommends looking at StrongLoop zone.
Node.js Domains: Does not look like this actually took off.
I'm not sure if you're still looking, but a while back I built express-http-context for this very purpose. It's an express middleware that sits on top of cls-hooks (formerly continuation-local-storage).
Full disclosure: this is an npm package that I built and maintain.
I'm working on a restAPI using Express.js and I'm wondering if there is a way for me to generate API documents that allow a user to view API definitions or possibly even try out the API call?
What you are looking for is a good JavaScript documentation generator. I found a decent one here http://apidocjs.com/example/.
This will allow you to use just Express as OP asked.
Swagger isn't for generating docs, it's for making APIs. So you are going to need to learn a full system to get that feature.
I found a couple more here that I haven't fully looked into yet which seem promising:
Docco here JSDoc here and an article on Documentor for Node
Swagger is an amazing project for auto generating API documentation. It includes an Express module.
http://swagger.io/
https://github.com/wordnik/swagger-node-express
I found this library to be very useful when it comes to keeping your documentation and route declaration logic close together:
https://www.npmjs.com/package/swagger-jsdoc
If you use Postman while developing you might already have a Postman collection containing the most relevant calls.
If that is the case Postman has a docs feature https://learning.postman.com/docs/publishing-your-api/documenting-your-api/
I'm trying to get NTLM Authentication working w/ Node.js. I've been reading this ( http://davenport.sourceforge.net/ntlm.html#theNtlmMessageHeaderLayout ). I send the header and get a Base64 authentication header.
I tried converting it from Base64 to UTF8 by making a new Buffer with base64 encoding and then calling toString('utf8') which returns a string something like
NTLMSSP\u0000\u0001\u0000\u0000\u0000\u0007�\b�\u0000
This is where I need help. I understand the NTLMSSP\u0000 is the null terminated signature, but and what the rest is supposed to indicate, but to me it's just garbage. It's unicode characters, but how am I supposed to get actual data out of that? I may be converting it incorrectly, which may be adding to my troubles, but I'm hoping someone can help.
Have a look at http://www.innovation.ch/personal/ronald/ntlm.html
What you receive is a Type-2 Message. The pages explains it in a very practical way. You have to extract the server challenge (nonce) and the server flags.
I just implemented a module for node.js to do just that: https://github.com/SamDecrock/node-http-ntlm
Have you looked at NTLMAPS?
You may be able to solve your problem by using it as a proxy server, but if you really want to implement NTLM auth in Javascript, then NTLMAPS provides lots of working code to study.
Sam posted the best resource I've seen for understanding what's going on.
jclulow on GitHub seems to have implemented it in a Samba library he built.
Take a look here:
https://github.com/jclulow/node-smbhash
under lib\ntlm.js you can see how he's handled the responses.
I've built client a couple of months ago using javascript, ntlm.js. Maybe that can help you get along. It was based on the documentation # innovation.ch and Microsofts own official documentation (see the references on the github page).