What is the exact procudure to add a CommonJs module on couchdb ?
I've read tutorials like:
https://caolan.org/posts/commonjs_modules_in_couchdb.html
from official doc:
http://docs.couchdb.org/en/1.6.1/query-server/javascript.html#commonjs-modules
The CommonJS module can be added to a design document, like so:
{
"views": {
"lib": {
"security": "function user_context(userctx, secobj) { ... }"
}
},
"validate_doc_update": "function(newdoc, olddoc, userctx, secobj) {
user = require('lib/security').user(userctx, secobj);
return user.is_admin();
}"
"_id": "_design/test"
}
but where I copy or paste that code? must I save to the file and add it with curl ? On fauxton i don'see where.
Managing CouchDB design documents is usually best accomplished via a tool like couchapp. It allows you to package up a directory of files and outputs a CouchDB design document.
You can manually edit that JSON in the futon/fauxton editor, but it's a pain and there are other tools out there depending on your toolchain. An external tool like this also aids in deployment, particularly across different environments.
Related
I am new in JSON schema and trying to validate my request with JSON schema. So i created schema. But i don't know why i am not able to give refrence to any another schema which located in same folder can any one help me to how to give $ref
{
"$id":"file:/schemas/ActionType",
"title":"ActionType",
"type":"object",
"properties":{
"ActionType":{
"enum":["fullLoad","update","insert","delete"]
}
},
"additionalProperties": false,
"required": ["ActionType"]
}
{
"$id":"file:/schemas/ERoaming",
"title":"ERoaming",
"type":"object",
"properties":{
"ActionType":{
"type":"object",
"$ref":"/schemas/ActionType.json"
}
},
"additionalProperties": false,
"required": ["ActionType"],
"$defs": {
}
}
I tried with without $id, relative path, path.
First of all, I don't think that's a valid file URI. It should look like file:///path/to/schemas/ActionType with three /s after the : and the full system path. You can't use a relative path.
Otherwise, everything looks fine with the schema. Another likely problem is that not all JSON Schema libraries understand files that reference the file system. You'll want to make sure the one you're using supports that feature.
When you're working with schemas in files, it's unnecessary to use $id. If the library properly handles file system references, the full path and filename should be automatically be used as the schema identifier. Then you can use relative references like I'm sure you're familiar with, { "$ref": "./ActionType.json" }.
I would like to store a value in the config file and look it up in the design document for comparing against update values. I'm sure I have seen this but, for the life of me, I can't seem to remember how to do this.
UPDATE
I realize (after the first answer) that there was more than one way to interpret my question. Hopefully this example clears it up a little. Given a configuration:
curl -X PUT http://localhost:5984/_config/shared/token -d '"0123456789"'
I then want to be able to look it up in my design document
{
"_id": "_design/loadsecrets",
"validate_doc_update": {
"test": function (newDoc,oldDoc) {
if (newDoc.supersecret != magicobject.config.shared.token){
throw({unauthorized:"You don't know the super secret"});
}
}
}
}
It's the abilitly to do something like the magicobject.config.shared.token that I am looking for.
UPDATE 2
Another potentially useful (contrived) scenario
curl -X PUT http://trustedemployee:5984/_config/eventlogger/detaillevel -d '"0"'
curl -X PUT http://employee:5984/_config/eventlogger/detaillevel -d '"2"'
curl -X PUT http://vicepresident:5984/_config/eventlogger/detaillevel -d '"10"'
Then on devices tracking employee behaviour:
{
"_id": "_design/logger",
"updates": {
"logger": function (doc,req) {
if (!doc) {
doc = {_id:req.id};
}
if(req.level < magicobject.config.eventlogger.detaillevel ){
doc.details = req.details;
}
return [doc, req.details];
}
}
}
Here's a follow-up to my last answer with more general info:
There is no general way to use configuration, because CouchDB is designed with scalability, stability and predictability in mind. It has been designed using many principles of functional programming and pure functions, avoiding side effects as much as possible. This is a Good Thing™.
However, each type of function has additional parameters that you can use, depending on the context the function is called with:
show, list, update and filter functions are executed for each request, so they get the request object. Here you have the req.secObj and req.userCtx to (ab)use for common configuration. Also, AFAIK the this keyword is set to the current design document, so you can use the design doc to get common configuration (at least up to CouchDB 1.6 it worked).
view functions (map, reduce) don't have additional parameters, because the results of a view are written to disk and reused in subsequent calls. map functions must be pure (so don't use e.g. Math.random()). For shared configuration across view functions within a single design doc you can use CommonJS require(), but only within the views.lib key.
validate doc update functions are not necessarily executed within a user-triggered http request (they are called before each write, which might not be triggered only via http). So they have the userCtx and secObj added as separate parameters in their function signature.
So to sum up, you can use the following places for configuration:
userCtx for user-specific config. Use a special role (e.g. with a prefix) for storing small config bits. For example superLogin does this.
secObj for database-wide config. Use a special member name for small bits (as you should normally use roles instead of explicit user names, secObj.members.names or secObj.admins.names is a good place).
the design doc itself for design-doc-wide config. Best use the this.views.lib.config for this, as you can also read this key from within views. But keep in mind that all views are invalidated as soon as you change this key. So if the view results will stay the same no matter what the config values are, it might be better to use a this.config key.
Hope this helps! I can also add examples if you wish.
I think I know what you're talking about, and if I'm right then what you are asking for is no longer possible. (at least in v1.6 and v2.0, I'm not sure when this feature was removed)
There was a lesser-known trick that allowed a view/show/list/validation/etc function to access the parent design document as this in your function. For example:
{
"_id": "_design/hello-world",
"config": {
"PI": 3.14
},
"views": {
"test": {
"map": "function (doc) { emit(this.config.PI); })"
}
}
}
This was a really crazy idea, and I imagine it was removed because it created a circular dependency between the design document and the code of the view that made the process of invalidating/rebuilding a view index a very tricky affair.
I remember using this trick at some point in the distant past, but the feature is definitely gone now. (and likely to never return)
For your special use-case (validating a document with a secret token), there might be a workaround, but I'm not sure if the token might leak in some place. It all depends what your security requirements are.
You could abuse the 4th parameter to validate_doc_update, the securityObject (see the CouchDB docs) to store the secret token as the first admin name:
{
"test": "function (newDoc, oldDoc, userCtx, secObj) {
var token = secObj.admins.names[0];
if (newDoc.supersecret != token) {
throw({unauthorized:"You don't know the super secret"});
}
}"
}
So if you set the db's security object to {admins: {names: ["s3cr3t-t0k3n"], roles: ["_admin"]}}, you have to pass 's3cr3t-t0k3n' as the doc's supersecret property.
This is obviously a dirty hack, but as far as I remember, the security object may only be read or modified by admins, you wouldn't immediately leak your token to the public. But consider adding a separate layer between the CouchDB and your caller if you need "real" security.
I am using the following code for partial update
POST /website/blog/1/_update
{
"script" : "ctx._source.views+=1"
}
is there any alternative way I can achieve the same thing. because I don't want to change anything in
groovy script because last time I changed the settings and my server was compromised.
So someone please help me with the solution or some security measures if there is no work around.
No, you cannot dynamically change a field value without using a script.
You can use file-based scripts though, which means that you can disable dynamic scripting (default in ES 1.4.3+) while still using scripting in a safe, trusted way.
config/
elasticsearch.yml
logging.yml
scripts/
your_custom_script.groovy
You could have the script store:
ctx._source.views += your_param
Once stored, you can then access the script by name, which bypasses dynamic scripting.
POST /website/blog/1/_update
{
"script": "your_custom_script",
"params" : {
"your_param" : 1
}
}
Depending on the version of Elasticsearch, the script parameter is better named (e.g., ES 2.0 uses "inline" for dynamic scripts), but this should get you off the ground.
I am developing an applications using MongoDB, Sails JS and ElasticSearch.
MongoDB is used to write records that are retrieve for the application. ElasticSearch is used for search text and geo locations distance search etc.
I am using mongo-connector to keep my data in sync from MongoDB to ElasticSearch.
Issue is, i am not able to maintain my mappings for geo_point for the fields that store lat and lon or parent/child or analyzer etc. Every time sails server is lifted i see in elasticsearch logs that all the mappings are removed, created and updated, and i lose my mapping for geo_point that is have created manually via the REST after every thing is up and running or even if i have created mapping at bootstrap time of sails js(as a work around).
I have also tried to create a mapping file and placed it in elasticsearch/config/mappings/index/mymapping.json but i get an error
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Root type mapping not empty after parsing! Remaining fields: ...
Here i tried all the combinations to make this work but no success eg
{"mappings" : {
"locations" : {
"dynamic": "false",
"properties":{
"location": {
"type": "geo_point"
}
}
}
}
}
Also tried using a template to create the mapping but after that mongo-connector kick in and overrides the mapping.
As of now i am only able to make this work is to stop mongo-connector, delete the oplog.timestamp file, start the sails server(Here at bootstrap time i delete and recreate the mapping for that document) and then start mongo-connector. But this create accidents if we forgots to do a step.
Am i doing any thing wrong or is there a better way to sync the mongodb to elasticsearch without losing the custom mapping or an alternative mongo-connector.
According to the documentation, if you install a mapping on the filesystem, the file must be named <your_mapping>.json so in your case it should be named locations.json and be placed either in
elasticsearch/config/mappings/_default/locations.json
or
elasticsearch/config/mappings/<your_index_name>/locations.json
Moreover, you mapping file shouldn't contain the mappings keyword, it should instead look like this:
{
"locations" : {
"dynamic": "false",
"properties":{
"location": {
"type": "geo_point"
}
}
}
}
You should try again after correctly naming your mapping file and folders.
I'm trying to get notifications in a CouchDB change poll as soon as pre-defined field is set or changed. I've already had a look at filters that can be used for filtering change events(db/_changes?filter=myfilter). However, I've not yet found a way to include this temporal information, because you can only get the current version of the document in this filter functions.
Is there any possibility to create such a filter?
If it does not work, I could export my field to a separate database and the only poll for changes in that db, but I'd prefer to keep together my data for obvious reasons.
Thanks in advance!
You are correct: filters and _changes feeds can only see snapshots of a document. What you need is a function which can see the old document and the new document and act correctly. But that is unavailable in _filters and _changes.
Obviously your client code knows if it updates that field. You might update your client code however there is a better solution.
Update functions can access both documents. I suggest you make an _update
function which notices the field change and flags that in the document. Next you
have a simple filter checking for that flag. The best part is, you can use a
rewrite function to make the HTTP API exactly the same as before.
1. Create an update function to flag interesting updates
Your _design/myapp would be {"updates", "smart_updater": "(see below)"}.
Update functions are very flexible (see my recent update handlers
walkthrough). However we only want to mimic the normal HTTP/JSON API.
Your updates.smart_updater field would look like this:
function (doc, req) {
var INTERESTING = 'dollars'; // Set me to the interesting field.
var newDoc = JSON.parse(req.body);
if(newDoc.hasOwnProperty(INTERESTING)) {
// dollars was set (which includes 0, false, null, undefined
// values. You might test for newDoc[INTERESTING] if those
// values should not trigger this code.
if((doc === null) || (doc[INTERESTING] !== newDoc[INTERESTING])) {
// The field changed or created!
newDoc.i_was_changed = true;
}
}
if(!newDoc._id) {
// A UUID generator would be better here.
newDoc._id = req.id || Math.random().toString();
}
// Return the same JSON the vanilla Couch API does.
return [newDoc, {json: {'id': newDoc._id}}];
}
Now you can PUT or POST to /db/_design/myapp/_update/[doc_id] and it will feel
just like the normal API except if you update the dollars field, it will add
an additional flag, i_was_changed. That is how you will find this change
later.
2. Filter for documents with the changed field
This is very straightforward:
function(doc, req) {
return doc.i_was_changed;
}
Now you can query the _changes feed with a ?filter= parameter. (Replication
also supports this filter, so you could pull to your local system all documents
which most recently changed/created the field.
That is the basic idea. The remaining steps will make your life easier if you
already have lots of client code and do not want to change the URLs.
3. Use rewriting to keep the HTTP API the same
This is available in CouchDB 0.11, and the best resource is Jan's blog post,
nice URLs in CouchDB.
Briefly, you want a vhost which sends all traffic to your rewriter (which itself
is a flexible "bouncer" to all design doc functionality based on the URL).
curl -X PUT http://example.com:5984/_config/vhosts/example.com \
-d '"/db/_design/myapp/_rewrite"'
Then you want a rewrites field in your design doc, something like (not
tested)
[
{
"comment": "Updates should go through the update function",
"method": "PUT",
"from": "db/*",
"to" : "db/_design/myapp/_update/*"
},
{
"comment": "Creates should go through the update function",
"method": "POST",
"from": "db/*",
"to" : "db/_design/myapp/_update/*"
},
{
"comment": "Everything else is just like normal",
"from": "*",
"to" : "../../../*"
}
]
(Once again, I got this code from examples and existing code I have laying
around but it's not 100% debugged. However I think it makes the idea very clear.
Also remember this step is optional however the advantage is, you never have to
change your client code.)