I'm using MongoDB with NodeJS and am wondering if I need to sanitize data before inserting/updating database documents. Its hard to find definite answer and I'm wondering if there are any Node modules that do it nicely or I need to strip all occurences of $ in strings or simply no need to worry about this. I know that PHP has holes but I'm using Node/Mongo (native driver) combo but still not sure if I need to do any cleaning of user input.
If you store your data as String and you are not parsing it to execute Mongo command, then there is nothing much to worry about it.
Nice article on security
http://cr.yp.to/qmail/guarantee.html
The only problem occurs when you are retrieving the user input, and you parse that input to execute the Mongo command, here you will need to take care to sanitize the input, or else you will get attack.
There is a npm package to do that for you
https://www.npmjs.com/package/mongo-sanitize
and nice article on this too
https://thecodebarbarian.wordpress.com/2014/09/04/defending-against-query-selector-injection-attacks/
Yes, you do.
For more information check this out; https://www.npmjs.com/package/content-filter
Also native escape() method might be used for to protect the database.
Run the code snippet below to see the results.
let a = "{$gt:25}"
console.log(a)
console.log(escape(a))
Related
I have an excel file that I want to only call in once and make available to all of my tests, at the moment it's being called on each test, I have tried storing it in the cache using https://www.npmjs.com/package/node-cache but when I tried to get it from cache it's saying undefined, so then I tried using onPrepare hook with no luck. Can someone point me in the right direction thanks in advance.
Assuming that excel has data for test automation and it does not include any writing operations, I would suggest reading the content and store it in a constant. If you declare that variable globally, it will be available to all your tests. The complexity of this READ function depends on how diverse is your data. You can use libraries like https://www.npmjs.com/package/xlsx, https://www.npmjs.com/package/exceljs, etc..,
Is it possible to force file processing on one node without splitting? I tried to use AtomicFileProcessing set on true, but it doesn't work.
Setting
[SqlUserDefinedExtractor(AtomicFileProcessing = true)]
should work. Can you please contact me directly (mrys at msft) and provide more information on what does not, so we can investigate?
In one of my job, I am using custom user defined operators for data processing, and it does not split data. You can also try this option.
What I mean by this is:
I have a program. The end user is currently using it. I submit a new piece of source code and expect it to run as if it were always there?
I can't find an answer that specifically answers the point.
I'd like to be able to say, "extend" or add new features (rather than fix something that's already there on the fly) to the program without requiring a termination of the program (eg. Restart or exit).
Yes, you can definitely do that in python.
Although, it opens a security hole, so be very careful.
You can easily do this by setting up a "loader" class that can collect the source code you want it to use and then call the exec builtin function, just pass some python source code in and it will be evaluated.
Check the package
http://opensourcehacker.com/2011/11/08/sauna-reload-the-most-awesomely-named-python-package-ever/ . It allows to overcome certain raw edges of plain exec. Also it may be worth to check Dynamically reload a class definition in Python
Not having any experience with couch and redis, this is becoming more than I can handle at this point.
The npm website allows you to search for packages by a keyword - https://npmjs.org/browse/keyword/awesome
However, it doen't provide any way of obtaining this information in json format - ideally, we could just do https://npmjs.org/browse/keyword/awesome.json but that is not the case :(
I know that the npm website is powered by couchdb and a local redis instance. The remote couchdb installation is http://registry.npmjs.org/ and powered by https://github.com/isaacs/npmjs.org
However, spending the day looking into this, I just cannot figure out how to get all packages of a particular keyword. Does anyone know how? Bonus points if you also explain the process that you went about finding out how to do it so I know for next time :)
Great question!
This will give you what you're looking for about a specific module:
npm view request
To get what you want for all modules you can hit the URL:
https://registry.npmjs.org/-/all/
After pouring through these two files:
https://github.com/isaacs/npm/blob/master/lib/search.js
https://github.com/isaacs/npm-www/blob/master/models/browse.js
I came to the following conclusions:
I'm super surprised there's not a better way to do search without hitting couchdb directly.
The command-line NPM client does search inside of node.js by sorting and filtering through the full results of that /all/ search listed above.
The website doesn't even bother with real search as it pawns it to a search engine
The search by keyword thing you want won't get the same results as command-line NPM. It's really limited in scope to the keyword attribute, other search options may be available through (see search.js above)
The query is going to look really weird.
Try this:
https://registry.npmjs.org/-/_view/byKeyword?startkey=["keyword"]&endkey=["keyword",{}]&group_level=3
Also, one quick note, this is the kind of question that would probably get answered in the node.js chat room or mailing list in about 4 seconds :)
Hope that helps.
Based on this answer I wrote a small lib for node, https://github.com/wires/npm-keywordsearch.
npm install npm-keywordsearch
then
var search = require('npm-keywordsearch')
search('my-plugin', function (error, packages) {
packages.forEach(function (pkg) {
console.log(pkg.name + ': ' + pkg.description)
})
})
Maybe useful for you
Sometime in 2018, npm retired the /-/_view/byKeyword API:
https://registry.npmjs.org/-/_view/byKeyword?startkey=%5B%22docpad-plugin%22%5D&endkey=%5B%22docpad-plugin%22,%7B%7D%5D&group_level=2
The new API is now: /-/v1/search:
https://registry.npmjs.org/-/v1/search?text=keywords:docpad-plugin&size=250
Documentation for it is here:
https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md#get-v1search
I've updated the pluginclerk package accordingly. It is a node package that provides an API to hit that call, while providing caching, as well as dependency compatibility resolution.
Has anyone successuflly used Russian/Chinese/Japanese characters with redis-node or any other redis library (using nodejs) to store/receive messages to & from Redis (2.0 and above)? I believe encoding/decoding of these messages is upto the client, not Redis. If so, do any of the libraries provide this encoding/decoding or is there another library one can use for this purpose? Any working code examples would be greatly appreciated.
using utf-8 with redis works - if you use redis-cli, make sure that you pass in raw as the parameter to read the russian/chinese/japanese characters (otherwise those characters will not appear correctly in redis-cli). You can retrieve them correctly using an API like Jedis/redis-node etc.
Hope this helps!