CouchDB many-to-many fetching with dynamic sorting - couchdb

I have a CouchDB index where documents are music files with fields artist, album, title, etc. I also want to store playlists of tracks and be able to fetch all of a playlist's tracks and optionally sort the results by arbitrary fields.
I've read through the documentation on joins and the blog post/comment example is basically the same architecture as my playlist/track one, so I already have the ability to fetch a playlist's tracks and sort them by some predetermined key.
Is there any way to combine a view (like in those docs) with post-hoc sorting by some other arbitrary field (artist, title, etc.)?
I don't mind using multiple queries, but collecting all of the returned document IDs and using them in a subsequent /_find query like {"_id": {"$in": [...]}} query is extremely slow. Sorting them client side is always an option, but then I'd need to implement sorting logic twice in my app: a server-side version for when I'm using /_find on all tracks and a client-side version for tracks returned from a playlist view.
(This would be trivially easy to accomplish with an RDBMS but my use case involves syncing data between mobile and desktop versions of the same app (which might be offline), which is what led me to CouchDB+PouchDB.)

This belongs in a comment but would be an ugly mess.
When not to use map/reduce is a must read blog post by Nolan Lawson. There the point made is to leverage document _id's as much as possible, and demonstrates that denormalization isn't always the answer.
I believe if you are able to refactor your id's and documents, it will be evident how a secondary index (or two!) will fill in the gaps.
I won't go further because it would be plagiarism - it can't be explained in simpler terms. Read that article, toy with this code.
const g_view_result = 'view_result';
const getAllArtists = async() => {
const options = {
include_docs: true,
startkey: 'artist_',
endkey: 'artist_\uffff'
};
allDocs(["name"], options);
}
const getAllAlbums = async() => {
const options = {
include_docs: true,
startkey: 'album_',
endkey: 'album_\uffff'
};
allDocs(["title"], options);
}
const getAllBowieAlbums = async() => {
const options = {
include_docs: true,
startkey: 'album_bowie_',
endkey: 'album_bowie_\uffff'
};
allDocs(["title"], options);
}
const allDocs = async(fields, options) => {
let html = [];
const results = getEl('results');
results.innerText = '';
let docs = await db.allDocs(options);
docs.rows.forEach(row => {
html.push(fields.map(f => row.doc[f]).join(", "));
})
results.innerText = html.join('\n');
}
// canned test documents
const getDocsToInstall = () => {
return [
/*---- artist ----*/
{
_id: 'artist_bowie',
type: 'artist',
name: 'David Bowie',
age: 67
},
{
_id: 'artist_dylan',
type: 'artist',
name: 'Bob Dylan',
age: 72
},
{
_id: 'artist_joni',
type: 'artist',
name: 'Joni Mitchell',
age: 70
},
/*---- albums ----*/
{
_id: 'album_bowie_1971_hunky_dory',
artist: 'artist_bowie',
title: 'Hunky Dory',
type: 'album',
year: 1971
},
{
_id: 'album_bowie_1972_ziggy_stardust',
artist: 'artist_bowie',
title: 'The Rise and Fall of Ziggy Stardust and the Spiders from Mars',
type: 'album',
year: 1972
},
{
_id: 'album_dylan_1964_times_they_are_changin',
artist: 'artist_dylan',
title: 'The Times They Are a-Changin\'',
type: 'album',
year: 1964
},
{
_id: 'album_dylan_1965_highway_61',
artist: 'artist_dylan',
title: 'Highway 61 Revisited',
type: 'album',
year: 1965
},
{
_id: 'album_dylan_1969_nashville_skyline',
artist: 'artist_dylan',
title: 'Nashville Skyline',
type: 'album',
year: 1969
},
{
_id: 'album_joni_1974_court_and_spark',
artist: 'artist_joni',
title: 'Court and Spark',
type: 'album',
year: 1974
}
]
}
//
// boilerplate code
//
let db;
// init example db instance
const initDb = async() => {
db = new PouchDB('test', {
adapter: 'memory'
});
await db.bulkDocs(getDocsToInstall());
}
initDb().then(() => {
document.querySelectorAll('.hide').forEach(el => el.classList.remove('hide'));
});
const getEl = id => document.getElementById(id);
.hide {
display: none
}
.label {
text-align: right;
margin-right: 1em;
}
.hints {
font-size: smaller;
}
<script src="https://cdn.jsdelivr.net/npm/pouchdb#7.1.1/dist/pouchdb.min.js"></script>
<script src="https://github.com/pouchdb/pouchdb/releases/download/7.1.1/pouchdb.memory.min.js"></script>
<table id='actions' class='hide'>
<tr>
<td>
<button onclick='getAllArtists()'>All artists</button>
</td>
<td>
<button onclick='getAllAlbums()'>All albums</button>
</td>
<td>
<button onclick='getAllBowieAlbums()'>All albums by David Bowie</button>
</td>
</tr>
</table>
<div style='margin-top:2em'></div>
<pre id='results'></pre>

Related

How to show list of field values of a collection of objects

I am making an API that shows a collection of ads with MongoDB and Node.js
I need to display the list of collection tags in a JSON string.
Example: 'home', 'mobile', 'motor'
This is the API initializer code:
const readline = require('readline');
const Product = require('./models/Product');
async function main() {
const advance = await question('Are you sure to continue with the deletion of the database? (yes or no) ')
if (!advance) {
process.exit();
}
const connection = require('./lib/connectMongoose')
await initProducts();
connection.close();
}
async function initProducts() {
const deleted = await Product.deleteMany();
console.log(`Remove ${deleted.deletedCount} products.`);
const inserted = await Product.insertMany([
{name: 'Table', sale: true, price: 150, photo: 'Table.png', tags: ['home']},
{name: 'Iphone 13 pro', sale: false, price: 950, photo: 'Iphone 13 pro.png', tags: ['mobile']},
{name: 'Car Mini Cooper', sale: true, price: 1000, photo: 'Car Mini Cooper.png', tags: ['motor']}
]);
console.log(`Create ${inserted.length} products.`)
}
main().catch(err => console.log('Hubo un error', err))
function question(text) {
return new Promise((resolve, reject) => {
const interface = readline.createInterface({
input: process.stdin,
output: process.stdout
});
interface.question(text, answer => {
interface.close();
if (answer.toLowerCase() === 'yes') {
resolve(true);
return;
}
resolve(false);
})
})
}
I need to find a MongoDB method that allows me to show when the API route calls the list that shows in JSON format all the tags that the collection includes
If I've understood correctly, one option is $unwind the tags array to get all tags as strings and be able to $group adding to a set to avoid duplicates.
db.collection.aggregate([
{
"$unwind": "$tags"
},
{
"$group": {
"_id": null,
"tags": {
"$addToSet": "$tags"
}
}
}
])
I think this works but $unwind and $group the entire collection is not always a good idea. It may be a slow process.
Example here

Vuejs- How to show name values in a <v-select> but send the ID in the axios POST request

EDIT: I fixed it by adding the return-object prop to v-select
When I add a student to a database from a vuetify form, I want to be able to assign them a course. But the course has to be in a list of available courses (also in the db). I managed to do that and show all the available courses in a dropdown menu.
However, when I add the new student to the database, it sends the name of the course but not the ID of the course, so the database doesn't recognize it. I would like to link the name of the course from the v-select dropdown menu to its object ID and send the ID in the POST request.
My form component:
<v-select
:items="courses"
v-model="Courses"
item-value="name"
item-text="name"
label="Available courses"
prepend-icon="folder"
>
<template v-slot:item="{ item, attrs, on }">
<v-list-item
v-bind="attrs"
v-on="on"
>
<v-list-item-title
:id="attrs['aria-labelledby']"
v-text="item.name"
></v-list-item-title>
</v-list-item>
</template>
</v-select>
Where I store all the available courses:
computed: {
courses() {
return this.$store.state.courses;
},
The axios POST method:
methods: {
async addItem(){
const response = await axios.post("http://localhost:4000/api/student", {
name: this.name,
Courses: this.courses,
});
this.items.push(response.data);
this.name = "";
this.courses ="";
},
},
My mongoDB model:
const Student = mongoose.model(
"Student",
new mongoose.Schema({
name: String ,
Courses:
{
type: mongoose.Schema.Types.ObjectId,
ref:"id"
},
})
);
module.exports = Student;
The Course model:
const Course = mongoose.model(
"Course",
new mongoose.Schema({
name: String ,
available: {type:Boolean , default :true} ,
})
);
module.exports = Course;
Need more information on how each course object looks, and your data, but essentially, set the item-value prop to the item's object ID, and under the addItem function,
async addItem(){
const response = await axios.post("http://localhost:4000/api/student", {
id: this.courseId,
Courses: this.courses,
});
this.items.push(response.data);
this.courseId = "";
this.courses ="";
}
EDIT:
It might be a good idea to name your variables better, e.g.
// in your v-select
v-model="selectedCourse"
// in your addItem function
Course: this.selectedCourse
or
Courses: this.selectedCourses
If you just want to get id of the course in v-model of v-select, You can simply use item-value="id" instead of item-value="name".
Live Demo :
new Vue({
el: '#app',
vuetify: new Vuetify(),
data: () => ({
selectedCourse: null,
courses: [{
id: 1,
name: 'Course 1'
}, {
id: 2,
name: 'Course 2'
}, {
id: 3,
name: 'Course 3'
}, {
id: 4,
name: 'Course 4'
}, {
id: 5,
name: 'Course 5'
}],
}),
methods: {
getSelected() {
console.log(this.selectedCourse) // ID of the selected course
}
}
})
<script src="https://unpkg.com/vue#2.x/dist/vue.js"></script>
<script src="https://unpkg.com/vuetify#2.6.6/dist/vuetify.min.js"></script>
<link rel="stylesheet" href="https://unpkg.com/vuetify#2.6.6/dist/vuetify.min.css"/>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Material+Icons"/>
<div id="app">
<v-app id="inspire">
<v-container fluid>
<v-select
:items="courses"
v-model="selectedCourse"
label="Available courses"
prepend-icon="folder"
item-value="id"
item-text="name"
#change="getSelected"
></v-select>
</v-container>
</v-app>
</div>

Creating a relationship between models with additional info in Mongo, Express, Node

somewhat new to Node and been struggling with this model relationship and not finding an answer here.
I have four models I'm trying to create relationships between:
User
Review
Topics
Courses
When a User leaves a Review on a Course in a certain Topic, I want to track a "topic score" on the User model.
So if a User Reviewed a programming Course, they should get +10 to their programming Topic score. Then I should be able to query User.scores.programming to get their Programming score.
The Reviews are being created fine, it's just the Topic scoring part where I'm running into issues.
Here's how my User schema are set up, just the relevant part:
const userSchema = new mongoose.Schema({
...
scores: [{
topic: {
type: mongoose.Schema.ObjectId,
ref: 'Topic'
},
score: {
type: Number,
default: 0
}
}]
});
And here's the code I have so far right now for trying to increment the score:
const updateUserScores = async (userId, course) => {
const user = await User.findOne({_id: userId}).populate('scores');
const userScores = user.scores;
let topics = await Course.findOne({_id: course}).populate('tags');
topics = topics.tags.map(x => x._id);
// I know it works for here to get the array of topics that they need to be scored on
// Then we need to go through each topic ID, see if they have a score for it...
// If they do, add 10 to that score. If not, add it and set it to 10
for (topic in topics) {
const operator = userScores.includes(topic) ? true : false;
if (!operator) {
// Add it to the set, this is not currently working right
const userScoring = await User
.findByIdAndUpdate(userId,
{ $addToSet: { scores: [topic, 10] }},
{ new: true}
)
} else {
// Get the score value, add 10 to it
}
}
}
I know I probably have a few different things wrong here, and I've been very stuck on making progress. Any pointers or examples I can look at would be extremely helpful!
Alright so after lots of messing around I eventually figured it out.
User model stayed the same:
scores: [{
topic: {
type: mongoose.Schema.ObjectId,
ref: 'Tag'
},
score: {
type: Number,
default: 0
}
}]
Then for the code to increment their score on each topic when they leave a review:
const updateUserScores = async (userId, course) => {
const user = await User.findOne({_id: userId}).populate({
path : 'scores',
populate: [
{ path: 'topic' }
]
});
let userScores = user.scores;
userScores = userScores.map(x => x.topic._id);
let topics = await Course.findOne({_id: course}).populate('tags');
topics = topics.tags.map(x => x._id);
for (t of topics) {
const operator = userScores.includes(t) ? true : false;
if (!operator) {
const userScoring = await User
.findByIdAndUpdate(userId,
{ $addToSet: { scores: {topic: t, score: 10}}},
{ new: true}
);
} else {
const currentScore = await user.scores.find(o => o.topic._id == `${t}`);
const userScoring = await User
.findByIdAndUpdate(userId,
{ $pull: { scores: {_id: currentScore._id}}},
{ new: true }
)
const userReScoring = await User
.findByIdAndUpdate(userId,
{ $addToSet: { scores: {topic: t, score: (currentScore.score + 10)}}},
{ new: true }
)
}
}
}
Ugly and not super elegant, but it gets the job done.

Accessing child methods in parent for mongoose succeeds in array and fails with single child

UPDATE : Solution is at bottom of question
I have an express site using mongoose.
I'll greatly simplify to say that I have adults, kids, and house models. When I create methods on kids, I can call them from within methods on adults and get a result. I can also call them from my .ejs views. However, when I create methods on house, I can only get a result from my .ejs views and get undefined when called from within methods on adults. Example code follows.
adult.js
const mongoose = require('mongoose');
const adultSchema = mongoose.Schema({
name: { type: String },
size: {type: String},
kids: [{type: mongoose.Schema.Types.ObjectId, ref: 'Kid', required: true}]
house:{type: mongoose.Schema.Types.ObjectId, ref: 'House', required: true}
});
adultSchema.method({
getKidsDescription: function() {
if (this.kids.length < 1) {
return 'No kids yet';
} else {
let ev = 'Kids, aged: ';
let kds = this.kids;
kds.forEach(function(k){
ev = ev + 'k.getAge()' // works
})
return ev;
}
},
getHouseDescription: function(){
return 'A fabulous house on '+this.house.getFullStreet(); // does not work
}
})
module.exports = mongoose.model('Adult', adultSchema);
kid.js
const mongoose = require('mongoose');
const kidSchema = mongoose.Schema({
name: { type: String },
size: {type: String},
birthdate: {type:Date}
});
kidSchema.method({
getAge: function() {
return (Math.floor(new Date() - this.birthdate)/(1000*60*60*24*365))
},
})
module.exports = mongoose.model('Kid', kidSchema);
house.js
const mongoose = require('mongoose');
const houseSchema = mongoose.Schema({
name: { type: String },
city: {type: String},
street: {type:String}
});
houseSchema.method({
getFullStreet: function() {
return this.street + ' Road';
},
})
module.exports = mongoose.model('House', houseSchema);
When I make a query for theAdult, it looks like this:
controller.js
exports.main = async (req, res, next) => {
if (req.theAdult) {
try {
const found = await db.fetchAdult(req.theAdult._id)
res.render('/main', {
//theHouse: found.house //below I show this working
});
} catch(e) {
throw new Error(e.message)
}
} else {
res.redirect('/');
}
}
db.js
exports.fetchAdult = (id) => {
return Adult.findById(id)
.populate({ path: 'kids'})
.populate({ path: 'house'})
.exec()
.then(doc => {
return doc;
});
}
Assuming house is passed to view as an object when rendered (commented out above), this works
view.ejs
<p> <%= theHouse.getFullStreet() %></p>
Assuming house populated on the call to load the Adult, this returns undefined.
view.ejs
<p> <%= theAdult.house.getFullStreet() %></p>
At the same time, both of these work
view.ejs
<ul> <% theAdult.kids.forEach(function(k) { %>
<li><%= k.getAge() %> </li>
<% }); %>
</ul>
<p> <% theAdult.getKidsDescription() %> </p>
What I am not understanding is how the method calls work for objects in array and work in the view but do not work for objects on in an array. This is a single child error (for me). If it did not work in the view, I would assume that the method getFullStreet() was the problem, but it works in the view. If the array methods could not be called within the parent, I would assume the issue was with trying to access getFullStreet() in the parent.
What am I missing?
SOLUTION
I was fetching theAdult in my call to show view.ejs, but I was then actually relying on currentAdult which referred to req.adult and did not have the fields populated. My solution was to add a pre hook to the adult schema that always populates house on find.
in adult.js
adultSchema.pre('find', function() {
this.populate('house')
})
Have you tried passing a hydrated theAdult? It might only see the ObjectID, without any other data or methods.

How to do soft delete with mongodb using nodejs

I'm able to delete data from the view , but at the sametime its getting deleted from mongodb which shouldn't happen.
I tried mongoose-soft-delete plugin to perform soft delete, but it isn't working
//schema
var mongoose= require('mongoose');
let softDelete = require('mongoosejs-soft-delete');
var Schema=mongoose.Schema;
var newblogSchema=new Schema({
user_id:Number,
title:String,
description:String,
summary:String,
hashtag:String
})
var newblogs=mongoose.model('NewBlog',newblogSchema);
newblogSchema.plugin(softDelete);
module.exports=newblogs;
//html template
<table>
<tr>
<th>Title</th>
<th>Description</th>
<th>Summary</th>
<th>HashTags</th>
</tr>
<tr *ngFor="let blog of blogs;">
<td >{{blog.title}}</td>
<td [innerHtml]="blog.description| safeHtml">{{blog.description}}</td>
<td>{{blog.summary}}</td>
<td>{{blog.hashtag}}</td>
<td> <a routerLink="/blog"><button type="button"
(click)="editblog(blog._id,blog.title,blog.description,blog.summary,blog.hashtag)">
Edit</button></a>
<td><button type="button" (click)="deleteblog(blog._id)">Delete</button>
</tr>
</table>
//ts file
deleteblog(blogid) {
var result = confirm('Want to delete?');
if (result === true) {
this.blogservice.deleteblog(blogid).subscribe(response => {this.blogs = response; });
}
//service
deleteblog(blogid):Observable<any>{
return Observable.create(observer=>{
this.http.post('http://localhost:4000/api/deleteblog', {_id: blogid}, {headers: new HttpHeaders({'Content-Type':'application/json'})}
)
.subscribe((response:Response)=>{
observer.next(response);
observer.complete();
});
});
}
//api.js
router.post('/deleteblog',(req,res)=>{
var body=req.body;
newblog.findByIdAndRemove({_id:body._id},(error,newblog)=>{if(error){
console.log(error);
}
else{
return res.json({message:'deleted',data:newblog});
}
});
});
Now the data is getting deleted from view as well as mongodb.
Expected result is to delete data only from the view and not from mongodb
we can implement soft delete with plugin, middleware and $isDeleted document method
soft delete plugin code:
import mongoose from 'mongoose';
export type TWithSoftDeleted = {
isDeleted: boolean;
deletedAt: Date | null;
}
type TDocument = TWithSoftDeleted & mongoose.Document;
const softDeletePlugin = (schema: mongoose.Schema) => {
schema.add({
isDeleted: {
type: Boolean,
required: true,
default: false,
},
deletedAt: {
type: Date,
default: null,
},
});
const typesFindQueryMiddleware = [
'count',
'find',
'findOne',
'findOneAndDelete',
'findOneAndRemove',
'findOneAndUpdate',
'update',
'updateOne',
'updateMany',
];
const setDocumentIsDeleted = async (doc: TDocument) => {
doc.isDeleted = true;
doc.deletedAt = new Date();
doc.$isDeleted(true);
await doc.save();
};
const excludeInFindQueriesIsDeleted = async function (
this: mongoose.Query<TDocument>,
next: mongoose.HookNextFunction
) {
this.where({ isDeleted: false });
next();
};
const excludeInDeletedInAggregateMiddleware = async function (
this: mongoose.Aggregate<any>,
next: mongoose.HookNextFunction
) {
this.pipeline().unshift({ $match: { isDeleted: false } });
next();
};
schema.pre('remove', async function (
this: TDocument,
next: mongoose.HookNextFunction
) {
await setDocumentIsDeleted(this);
next();
});
typesFindQueryMiddleware.forEach((type) => {
schema.pre(type, excludeInFindQueriesIsDeleted);
});
schema.pre('aggregate', excludeInDeletedInAggregateMiddleware);
};
export {
softDeletePlugin,
};
you can use as global for all schemas
mongoose.plugin(softDeletePlugin);
or for concrete schema
For Soft delete, you should maintain an active flag column that should only contain values as 0 and 1.
This way, you could analyse whether a record is deleted or not.
While displaying, add another clause for displaying only the records that have flag value 1. And while deleting, just update that flag's value to 0.
This would do the job.
For Example, here user 2 is deleted. with activeFlag as 0.
ID memberID userStatus groupCode activeFlag
1 user1 1 4455 1
2 user2 1 4220 0
3 user3 2 4220 1
Try to use https://www.npmjs.com/package/soft-delete-mongoose-plugin
A simple and friendly soft delete plugin for mongoose.

Resources