Couchdb and Sofa Help - couchdb

I'm new to Couchdb just a few week back I git clone the couchdb app called sofa [what and app] . Over the week It was going great but suddenly today I stumble upon something.
Here what I meant
when I browse the Sofa app and try to create a Post without the title
it prompt with and alert box "The document could not be saved: The database could not be created, the file already exists." which was weird as looking at the source I found that the require (in validate_doc_update.js return its custom json error) something like in this format {"forbidden" : message }) with forbidden as key
v.forbidden = function(message) {
throw({forbidden : message})
};
v.require = function() {
for (var i=0; i < arguments.length; i++) {
var field = arguments[i];
message = "The '"+field+"' field is required.";
if (typeof newDoc[field] == "undefined") v.forbidden(message);
};
};
in validate_doc_update.js
if (newDoc.type == 'post') {
if (!v.isAuthor()) {
v.unauthorized("Only authors may edit posts.");
}
v.require("created_at", "author", "body", "format", "title");
inspecting the response state that the json returned was found to be different from the json had it would have been return by the above require function in validate_doc_update.js
here is the json
{"error":"file_exists","reason":"The database could not be created, the file already exists."}
This make be believe that the validation in validation_doc_update.js only execute during updating of document
to Prove this point I try to update a document without the title, expecting that it would return the error but surprisingly the document just got saved
so Here are my Question regarding all the Point I mention above
Does validate_doc_update.js "validate" work only during updation of document
if YES
then
how can I manage to succeed in updating a post without the error [Weird bypassing the Validation Completely] . + How can execute validation on create of a document
if NO
then
What is the Error {"error":"file_exists","reason":"The database could not be created, the file already exists."} that is prevent a document to be saved
Can anyone please share light on all the questions listed here

Yes, the validate_doc_update functions are run only when updating documents (include creation and deletion).
The function you show here will allow a document without a title as long as its type is not "post". If you could include the actual request you attempted, I could confirm it.
Finally, the ""The database could not be created" is because you are attempting to create the database (by doing PUT /dbname/ instead of PUT /dbname/docid, I would guess) when it already exists. Again, if you would include the actual request, I could confirm that too.

Related

Table-Scraper Not Returning Data Before Function Returns

I am currently working on a Discord bot, which uses node.js, to scrape info from Dododex.com, a database of creatures from the game ARK. I found a very useful library called table-scraper which scrapes tables from web pages and returns the data as an object (or array of objects for multiple tables). In the background, table-scraper uses another library called x-ray, along with the common web request-maker request. Keep in mind that I don't know that much about node or how request works.
What I have working is asking the user for the name of a creature to get the data for. I then check that name against a list of creatures (that I have in a file/JSON object) to see if that name is valid.
Now, what I want to get working is to scrape the relevant data from the dododex page belonging to that creature using table-scraper. Table-scraper is definitely working with dododex, but when I call it to gather the data and put it into an object, it seems to start scraping, but then lets the function return with an empty object. If the program didn't crash right after the return (caused by the attempt to access an empty object), I know from testing that it would finish scraping, even though the function the scraping was called in has already finished.
Here's where the relevant code starts:
const creature = getDinoFromName(args[0].toLowerCase(), arkdata); //Function explained below
if(creature.health.base == 404) //If the creature's name is invalid, getDinoFromName() will return a creature with health.base = 404
{
console.log("unknown");
unknownCreature(args, msg); //Tells the user that the creature is unknown
return;
}
else
{
//Outputs the data
}
And here's where the data gets gathered:
function getDinoFromName(name, arkdata)
{
for(let i = 0; i < arkdata.creatures.length; i++) //Loops through arkdata, a json object with the possible names of all the creatures
{
if(arkdata.creatures[i].possible_names.includes(name)) //If the name the user provided matches one of the possible names of a creature (creatures can have multiple possible names, i.e. rex, t-rex, tyrannosaurus rex)
{
var creature;
console.log("found");
tableScraper.get("http://www.dododex.com/taming/" + arkdata.creatures[i].possible_names[0]).then(function(tableData)
{
console.log("scraped");
var stats = tableData[1]; //tableData is an array of 3 tables matching the 3 html tables found on the site, tableData[1] is the one with the creature's stats. It looks like this: http://ronsoros.github.io/?31db928bc662538a80bd25dd1207ac080e5ebca7
creature = //Now the actual creature object, parsed from tableData[1]
{
"health":
{
"base": stats[0]["Base (Lvl 1)"], //
"perwild": stats[0]["Increase Per Lvl (Wild)"],
"pertamed": stats[0]["Increase Per Lvl (Tamed)"]
}
//There is more data to parse but I only included hp as an example
}
});
return creature;
}
}
//If the creature name is not found to be one of the possible names, returns this to signify the creature was not found
return {"health":{"base":404}};
}
Running the code result in node crashing and the error
/app/server.js:320
if(creature.health.base == 404)
TypeError: Cannot read property 'health' of undefined
at stats (/app/server.js:320:15)
at Client.client.on.msg (/app/server.js:105:7)
Something wonky is going on here, and I really can't figure it out. Any help is appreciated.

MongoDB update object and remove properties?

I have been searching for hours, but I cannot find anything about this.
Situation:
Backend, existing of NodeJS + Express + Mongoose (+ MongoDB ofcourse).
Frontend retrieves object from the Backend.
Frontend makes some changes (adds/updates/removes some attributes).
Now I use mongoose: PersonModel.findByIdAndUpdate(id, updatedPersonObject);
Result: added properties are added. Updated properties are updated. Removed properties... are still there!
Now I've been searching for an elegant way to solve this, but the best I could come up with is something like:
var properties = Object.keys(PersonModel.schema.paths);
for (var i = 0, len = properties.length; i < len; i++) {
// explicitly remove values that are not in the update
var property = properties[i];
if (typeof(updatedPersonObject[property]) === 'undefined') {
// Mongoose does not like it if I remove the _id property
if (property !== '_id') {
oldPersonDocument[property] = undefined;
}
}
}
oldPersonDocument.save(function() {
PersonModel.findByIdAndUpdate(id, updatedPersonObject);
});
(I did not even include trivial code to fetch the old document).
I have to write this for every Object I want to update. I find it hard to believe that this is the best way to handle this. Any suggestions anyone?
Edit:
Another workaround I found: to unset a value in MongoDB you have to set it to undefined.
If I set this value in the frontend, it is lost in the REST-call. So I set it to null in the frontend, and then in the backend I convert all null-values to undefined.
Still ugly though. There must be a better way.
You could use replaceOne() if you want to know how many documents matched your filter condition and how many were changed (I believe it only changes one document, so this may not be useful to know). Docs: https://mongoosejs.com/docs/api/model.html#model_Model.replaceOne
Or you could use findOneAndReplace if you want to see the document. I don't know if it is the old doc or the new doc that is passed to the callback; the docs say Finds a matching document, replaces it with the provided doc, and passes the returned doc to the callback., but you could test that on your own. Docs: https://mongoosejs.com/docs/api.html#model_Model.findOneAndReplace
So, instead of:
PersonModel.findByIdAndUpdate(id, updatedPersonObject);, you could do:
PersonModel.replaceOne({ _id: id }, updatedPersonObject);
As long as you have all the properties you want on the object you will use to replace the old doc, you should be good to go.
Also really struggling with this but I don't think your solution is too bad. Our setup is frontend -> update function backend -> sanitize users input -> save in db. For the sanitization part, we use a helper function where we integrate your approach.
private static patchModel(dbDocToUpdate: IModel, dataFromUser: Record<string, any>): IModel {
const sanitized = {};
const properties = Object.keys(PersonModel.schema.paths);
for (const key of properties) {
if (key in dbDocToUpdate) {
sanitized[key] = data[key];
}
}
Object.assign(dbDocToUpdate, sanitized);
return dbDocToUpdate;
}
That works smoothly and sets the values to undefined. Hence, they get removed from the document in the db.
The only problem that remains for us is that we wanted to allow partial updates. With that solution that's not possible and you always have to send everything to the backend.
EDIT
Another workaround we found is setting the property to an empty string in the frontend. Mongo then also removes the property in the database

XPages SSJS Code giving error "One or more of the source document's attachment are missing"

This line
publishedDoc.save(true, false, true);
causes the above stated error once in a while, on documents saved on the web where inline images where added via CKEditor. I can't pinpoint the circumstances that causes this error, so any hints or comments are greatly appreciated.
Here is a bit more code, so you get the context of that call:
function postSavePage(doc) {
if(doc.getItemValueString("status")=="To Be Published" || doc.getItemValueString("status")=="Save as Current Version" ) {
var saveAsCurrent = doc.getItemValueString("status")=="Save as Current Version";
var publishedDoc:NotesDocument = getCurrentlyPublishedDoc(doc);
if(!publishedDoc) {
publishedDoc = doc;
publishedDoc.replaceItemValue("status", "Published");
PublishedDoc.replaceItemValue("VERNUMBER", 1);
} else {
//******
//copy draft to temp doc
var tmpDoc:NotesDocument = database.createDocument();
doc.copyAllItems(tmpDoc, true);
//copy published to draft (for archiving)
publishedDoc.copyAllItems(doc, true);
if(saveAsCurrent) {
doc.replaceItemValue("status", "Archive (Saved As Current)");
} else {
doc.replaceItemValue("status", "Archive");
}
//copy temp (newly published) to published doc
tmpDoc.copyAllItems(publishedDoc, true);
//Make sure we set the version number if saved as current version
if(saveAsCurrent){
publishedDoc.replaceItemValue("VERNUMBER", doc.getItemValueInteger("VERNUMBER"));
}
publishedDoc.replaceItemValue("status", "Published");
updateRevisionData(doc);
if(!saveAsCurrent) {
setVersion(doc);
}
//save docs
doc.save(true, false, true);
publishedDoc.save(true, false, true);
}
}
}
Basically, this code manages verioning in a CMS-type application. Since many doc links are already present in existing content, I need to keep the UNID of the published document. That explains the nice little dance between the published doc, the temp doc and the doc, which is the draft: Draft content goes to the published version, published version goes to thte archives.
I have persistence set to "keep pages on disk" and persistence mode to "Entire page content". Not sure it makes a difference though...
Any clues? :D
Wrong event. If you want to amend document data when writing back to disk use QuerySave and don't do a document.save() on the current document.
So you might need to split your code.
Background: XPages uploads attachments (inline images are attachments) to a temp location and keeps track of them. When you save they are added into the note and discarded. The pointer to the temp files is only reset after the event chain (QuerySave, actual write to disk, PostSave). So by trying to save again you point to files that are gone.
Btw. Saving the current document (again) in a PostSave (or for that matter: premature save in QuerySave) is a popular Anti-pattern in Notes development

Docuemt postopen event not operating on profile document

I need to save serial number of the document in a profile document and here is a code of action Execute Script:
if (document1.isNewNote()){
var pdoc:NotesDocument=database.getProfileDocument("LastNumber","")
var lnm=pdoc.getItemValue("lastNumber")[0];
var inputText6:com.ibm.xsp.component.xp.XspInputText = getComponent("inputText6");
inputText6.setValue(lnm);
pdoc.replaceItemValue("lastNumber",lnm);
pdoc.save();
}
This code is not opening profile document at all. Any thing wrong in the code?
"LastNumber" is the name of the form used to create Profile Document ?
this profile document already exist ?
there are no reader fields in this profile document ?
you have an error on this line : var pdoc:NotesDocument=database.getProfileDocument("LastNumber","") ?
or you have debug it and see that pdoc is null ?
instead of pdoc.getItemValue("lastNumber")[0] you can use pdoc.getItemValueInteger("lastNumber") to get a typed result
I supposed that this field contains a number and you want to increment it
instead of using inputText field you can set value directly with document1.setValue("NumberField", lnm);
I second the caution Per is suggesting. Profile documents can be a beast. You should abstract access to the "next number" into a SSJS function call. Btw. in your code snippet you don't actually increment the last number. Also: if your input text control is bound, go after the data source, not the UI.
A crude way (I would use a managed application bean for better isolation) for a better function could be this:
if(document1.isNewNote() {
document1.setValue("DocumentNumber",applicationTools.getNextNumber());
}
Then in a SSJS library you would have:
var applicationTools = {
"getNextNumber" : function() {
synchronized(applicationScope){
var pdoc:NotesDocument=database.getProfileDocument("LastNumber","");
if (!applicationScope.lastNumber) {
applicationScope.lastNumber = pdoc.getItemValueInteger("lastNumber");
}
applicationScope.lastNumber++;
pdoc.replaceItemValue("lastNumber",applicationScope.lastNumber);
pdoc.save(); //Make sure pdoc is writeable by ALL!!!!
pdoc.recycle();
return applicationScope.lastNumber;
}
},
"someOtherUtility" : function(nameToLookup, departments) {
// more stuff here
}
}
Which, in some way has been asked before, but not for a profile field. Someone still could simply go after the applicationScope.lastNumber variable, which is one of the reasons why I rather use a bean. The other: you could do the saving asynchronously, so it would be faster.
Note: in any case the number generation only works when you have a non-replicating database. But abstracting the function opens the possibility to replace fetching the number from the profile with a call to a central number generator ... or any other mechanism ... without changing your form again.

How to display arbitrary, schemaless data in HTML with node.js / mongodb

I'm using mongodb to store application error logs as json documents. I want to be able to format the error logs as HTML rather than returning the plain json to the browser. The logs are properly schemaless - they could change at any time, so it's no use trying to do this (in Jade):
- var items = jsonResults
- each item in items
h3 Server alias: #{item.ServerAlias}
p UUID: #{item.UUID}
p Stack trace: #{item.StackTrace}
h3 Session: #{item.Session}
p URL token: #{item.Session.UrlToken}
p Session messages: #{item.Session.SessionMessages}
as I don't know what's actually going to be in the JSON structure ahead of time. What I want is surely possible, though? Everything I'm reading says that the schema isn't enforced by the database but that your view code will outline your schema anyway - but we've got hundreds of possible fields that could be removed or added at any time so managing the views in this way is fairly unmanageable.
What am I missing? Am I making the wrong assumptions about the technology? Going at this the wrong way?
Edited with extra info following comments:
The json docs look something like this
{
"ServerAlias":"GBIZ-WEB",
"Session":{
"urltoken":"CFID=10989&CFTOKEN=f07fe950-53926E3B-F33A-093D-3FCEFB&jsessionid=84303d29a229d1",
"captcha":{
},
"sessionmessages":{
},
"sessionid":"84197a667053f63433672873j377e7d379101"
},
"UUID":"53934LBB-DB8F-79T6-C03937JD84HB864A338",
"Template":"\/home\/vagrant\/dev\/websites\/g-bis\/code\/webroot\/page\/home\/home.cfm, line 3",
"Error":{
"GeneratedContent":"",
"Mailto":"",
"RootCause":{
"Message":"Unknown tag: cfincflude.",
"tagName":"cfincflude",
"TagContext":[
{
"RAW_TRACE":"\tat cfhome2ecfm1296628853.runPage(\/home\/vagrant\/dev\/websites\/nig-bis\/code\/webroot\/page\/home\/home.cfm:3)",
"ID":"CFINCLUDE",
"TEMPLATE":"\/home\/vagrant\/dev\/websites\/nig-bis\/code\/webroot\/page\/home\/home.cfm",
"LINE":3,
"TYPE":"CFML",
"COLUMN":0
},
{
"RAW_TRACE":"\tat cfdisplay2ecfm1093821753.runPage(\/home\/vagrant\/dev\/websites\/nig-bis\/code\/webroot\/page\/display.cfm:6)",
"ID":"CFINCLUDE",
"TEMPLATE":"\/home\/vagrant\/dev\/websites\/nig-bis\/code\/webroot\/page\/display.cfm",
"LINE":6,
"TYPE":"CFML",
"COLUMN":0
}
]
}
}
... etc, but is likely to change depending on what the individual project that generates the log is configured to trigger.
What I want to end up with is a formatted HTML page with headers for each parent and the children listed below, iterating right through the data structure. The Jade sample above is effectively what we need to output, but without hard-coding that in the view.
Mike's analysis in the comments of the problem being that of creating a table-like structure from a bunch of collections that haven't really got a lot in common is bang-on. The data is relational, but only within individual documents - so hard-coding the schema into anything is virtually impossible as it requires you to know what the data structure looks like first.
The basic idea is what #Gates VP described. I use underscore.js to iterate through the arrays/objects.
function formatLog(obj){
var log = "";
_.each(obj, function(val, key){
if(typeof(val) === "object" || typeof(val) === "array"){
// if we have a new list
log += "<ul>";
log += formatLog(val);
log += "</ul>";
}
else{
// if we are at an endpoint
log += "<li>";
log += (key + ": " + val);
log += "</li>";
}
});
return log;
}
If you call formatLog()on the example data you gave it returns
ServerAlias: GBIZ-WEBurltoken: CFID=10989&CFTOKEN=f07fe950-53926E3B-F33A-093D-3FCEFB&jsessionid=84303d29a229d1sessionid: 84197a667053f63433672873j377e7d379101UUID: 53934LBB-DB8F-79T6-C03937JD84HB864A338Template: /home/vagrant/dev/websites/g-bis/code/webroot/page/home/home.cfm, line 3GeneratedContent: Mailto: Message: Unknown tag: cfincflude.tagName: cfincfludeRAW_TRACE: at cfhome2ecfm1296628853.runPage(/home/vagrant/dev/websites/nig-bis/code/webroot/page/home/home.cfm:3)ID: CFINCLUDETEMPLATE: /home/vagrant/dev/websites/nig-bis/code/webroot/page/home/home.cfmLINE: 3TYPE: CFMLCOLUMN: 0RAW_TRACE: at cfdisplay2ecfm1093821753.runPage(/home/vagrant/dev/websites/nig-bis/code/webroot/page/display.cfm:6)ID: CFINCLUDETEMPLATE: /home/vagrant/dev/websites/nig-bis/code/webroot/page/display.cfmLINE: 6TYPE: CFMLCOLUMN: 0
How to format it then is up to you.
This is basically a recursive for loop.
To do this with Jade you will need to use mixins so that you can print nested objects by calling the mixin with a deeper level of indentation.
Note that this whole thing is a little ugly as you won't get guaranteed ordering of fields and you may have to implement some logic to differentiate looping on arrays vs. looping on JSON objects.
You can try util.inspect. In your template:
pre
= util.inspect(jsonResults)

Resources