Having trouble posting to Mongo DB after external API Call - node.js

I'm attempting to make a fetch call to News API and populate my Articles collection in Mongo DB with the data I fetch. I have two files: articles.js, and article.js. - my schema looks like the following:
// Article.js
const ArticleSchema = new Schema({
title: {
type: String,
required: true
},
author: {
type: String,
required: true
},
description: {
type: String,
required: true
},
url: {
type: String,
required: true
},
datePublished: {
type: String,
required: true
},
source: {
type: String,
required: true
}
})
In articles.js, this is what my route looks like. My logic is that I make my fetch call, wait for the JSON to return, then iterate thru that json and map all the information I need to new story objects. Once I have those objects - I want to create a new instance of an Article for each, then post them to my Articles collection in Mongo DB.
/ #route POST /articles -> adds new instance of an article to the database
router.post('/', async (req, res) => {
try {
const res = await fetch('https://newsapi.org/v2/top-headlines?country=us&category=entertainment&apiKey=63c967f7cbd84c11b263b4e4758f1693');
const data = await res.json();
data.articles.forEach(article => {
const storyData = {
title: article.title,
author: article.author,
description: article.description,
url: article.url,
datePublished: article.publishedAt,
source: article.source.name
}
// console.log(storyData)
new Article(storyData)
.save()
})
} catch (err) {
console.error(err.message)
res.status(400).json({ message: err.message })
}
})
After my forEach loop, I get the objects like so:
{
title: `'Matrix 4' Style "Shifted" From Original Trilogy, Says Neil Patrick Harris - Hollywood Reporter`,
author: 'Ryan Parker',
description: 'The actor, who appears in the fourth installment, called the upcoming film "ambitious."',
url: 'https://www.hollywoodreporter.com/heat-vision/matrix-4-style-shifted-from-original-trilogy-says-neil-patrick-harris',
datePublished: '2020-09-16T17:54:26Z',
source: 'Hollywood Reporter'
}
I'm able to get the objects I'd like with the data I need, however when I attempt to call .save(), I receive validation error similar to the following:
"ValidationError: article validation failed: author: Path author is required."
I'm thinking that it may have to do with the fact that when I make post to the DB, the request should list something along the lines of "title: req.body.title", etc... Any thoughts on what could be causing this? My goal is to simply post these to the DB as I create them. Thanks!!!

Related

Insert same records multiple times to the Mongo database with NodeJs

I want to achive funcionality, where user in the frontend writes how many posts he want to insert in the database.. He can write 1, 5, 10, 15,.. up to 50 same posts.
The posts are then the SAME in the database, just this manually generated _id is different.
At first I thought that it can be done just like that:
exports.addPost = async (req: any, res: any) => {
try {
const newPost = new Post({
title: req.body.title,
description: req.body.description,
author: req.body.author,
});
for (let i: number = 0; i < 5; i++) {
await newPost .save();
}
res.status(201).json(newContainer);
} catch (err) {
res.status(400).json(err);
}
};
Post schema:
const PostSchema = new mongoose.Schema({
title: { type: String, required: true },
description: { type: String, required: true },
author: {
type: Schema.Authors.ObjectId,
ref: "Authors",
required: true,
},
});
module.exports = mongoose.model("Posts", PostSchema);
but I am not sure, if this is really the way to go.. What is some good practice for this (assuming that the number 5 in for loop will come in req.body.. So from user input.
Thanks
You can just use the following code:
try {
await Post.create(
new Array(5).fill(true).map((_) => ({
title: req.body.title,
description: req.body.description,
author: req.body.author,
}))
);
res.status(201).json(newContainer);
} catch (err) {
res.status(400).json(err);
}
model.create does accept passing an array of (new) documents to it. By mapping a new array of size 5 (or depending on user input) to your custom document and passing it to the create function will result in multiple documents created. A huge benefit is that you only have to perform one single database call (and await it).

Axios react node post request is giving a 422 (Unprocessable Entity)

I am trying to save the desired book to my MongoDB database when I press the saved button I get a 422 error I pass in the data as an object but for some reason, the data doesn't appear in the response back from the server The data is being passed to the Axios call but for some reason, the data property always returns an empty object,
The save handler
const handleSave = (event) => {
event.preventDefault();
let save = books.filter((book) => book.id === event.target.id);
// console.log(save);
// const da/ta = ;
// console.log(data);
API.saveBook({
title: save[0].title,
authors: save[0].author,
description: save[0].description,
image: save[0].image,
})
.then((res) => {
alert("book saved");
console.log(res);
// console.log(data);
})
.catch((err) => {
// console.log(data);
console.log("book not saved");
console.log(err.response);
});
};
This is the book model and the heroku link where you can see what is being logged out
const bookSchema = new Schema({
title: { type: String, required: true },
authors: [{ type: String, required: true }],
description: { type: String, required: true },
image: { type: String },
date: { type: Date, default: Date.now },
});
Heroku Link
github
I have console.logs in my inspect so you can check those out to see the response im getting back
I have cloned this repository and tested on both your Heroku link and locally, and cannot recreate the error locally. I suspect something to do with the MongoDB server rather than a code issue. I recommend you test creating a record in the live/Heroku-attached MongoDB server using an alternative method.
Thanks,
Will Walsh
Looks like volumeInfo.description is undefined for some books. The API returns a 422 error since description is required but is not present in the request payload. You could pass a default description if the book doesn't have one.
result = {
// ...
title: result.volumeInfo.title,
description: result.volumeInfo.description || "This book doesn't have a description",
// ...
}
Or you could remove the required validation for the description field if it's not an issue.
I would recommend you rename author to authors in the result object for clarity.
result = {
// ...
authors: result.volumeInfo.authors,
// ...
}

How to count unique views to a blog post (logged in and non logged in user) using Node.Js, MongoDB, Mongoose, Express

I want to display how many times a blog post has been read kind of like what Business Insider has.
The objective is...
View count that doesn't increment with every reload.
Every blog post stores its own view count
Fetch the view count from MongoDB/Mongoose Schema field and display it in HTML.
var express = require('express');
var cookieParser = require('cookie-parser');
var mongoose = require('mongoose');
var express-session = require('express-session');
//Show Blog Post
router.get('/blog/:categorySlug/:slug', function (req, res)
var slug = req.params.slug;
Blog.findOne({'slug' : slug}).populate('comments').exec(function (err, foundBlog) {
if (err) {
console.log(err);
} else {
res.render('show', { main: foundBlog, title: foundBlog.title});
}
});
});
I know that if i use req.session.views, it will increment the view count with every reload across all pages.
router.get('/blog/:categorySlug/:slug', function (req, res) {
req.session.views = (req.session.views || 0) + 1
var slug = req.params.slug;
Blog.findOne({'slug' : slug}).populate('comments').exec(function (err, foundBlog) {
if (err) {
console.log(err);
} else {
res.render('show', { main: foundBlog, title: foundBlog.title, pageViewCount: req.session.views});
}
});
});
So what could i do to save the view count of each blog separately and store that view count value in that blog post's viewCount field in the database and then render it in HTML using <%= main.viewCount %>
// Blog Schema
var mainBlogSchema = new mongoose.Schema({
image: String,
imageDescription: String,
priority: {
type: String,
default: ""
},
title: String,
content: String,
author: {
id: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
username: String,
name: String,
},
slug: {
type: String,
unique: true,
},
status: String,
viewCount: {
type: Number,
default: 0,
},
category: String,
categorySlug: String,
tags: String,
updated: {
type: Boolean,
default: false
},
date: { type: Date, default: Date.now },
comments: [
{
type: mongoose.Schema.Types.ObjectId,
ref: 'Comment',
},
],
},{
timestamps: {
createdAt: 'createdAt',
updatedAt: 'updatedAt'
}
});
One approach can be (might need changes)
Before the UI page loads, a script checks for user-token(separate token or some random string token)in the browser.
If token is not available then send request for new token along with page-id.
If token is available, call API with the token and page-id.
Make sure it gets called for every page load or even page transition.
Backend implementation can be like
Page collection will have page-id along with visits:[] field.
One API creates token and stores it in a separate collection(users) and follows step 3.
Another API will take the token and page-id as input, it will first. check the token is available in our collection(user) and if so it will do a mongo update on Page collection with
db.pages.update(
{ _id: page-id },
{ $addToSet: {visits: [ user_token ] } }
)
This query takes care if the token stored is unique no need to worry about anything. We can maintain a separate field called visitCount if needed, which gets updated with the latest count once query gets executed updating at least one record.
Pros
Page refresh will not affect the count.
The token is also maintained at our end so we can validate.
Cons
We will need huge space to store these token and we have to delete them if a user did not visit the site for a very long time.
API calls overhead.

Skip or Disable validation for mongoose model save() call

I'm looking to create a new Document that is saved to the MongoDB regardless of if it is valid. I just want to temporarily skip mongoose validation upon the model save call.
In my case of CSV import, some required fields are not included in the CSV file, especially the reference fields to the other document. Then, the mongoose validation required check is not passed for the following example:
var product = mongoose.model("Product", Schema({
name: {
type: String,
required: true
},
price: {
type: Number,
required: true,
default: 0
},
supplier: {
type: Schema.Types.ObjectId,
ref: "Supplier",
required: true,
default: {}
}
}));
var data = {
name: 'Test',
price: 99
}; // this may be array of documents either
product(data).save(function(err) {
if (err) throw err;
});
Is it possible to let Mongoose know to not execute validation in the save() call?
[Edit]
I alternatively tried Model.create(), but it invokes the validation process too.
This is supported since v4.4.2:
doc.save({ validateBeforeSave: false });
Though there may be a way to disable validation that I am not aware of one of your options is to use methods that do not use middleware (and hence no validation). One of these is insert which accesses the Mongo driver directly.
Product.collection.insert({
item: "ABC1",
details: {
model: "14Q3",
manufacturer: "XYZ Company"
},
}, function(err, doc) {
console.log(err);
console.log(doc);
});
You can have multiple models that use the same collection, so create a second model without the required field constraints for use with CSV import:
var rawProduct = mongoose.model("RawProduct", Schema({
name: String,
price: Number
}), 'products');
The third parameter to model provides an explicit collection name, allowing you to have this model also use the products collection.
I was able to ignore validation and preserve the middleware behavior by replacing the validate method:
schema.method('saveWithoutValidation', function(next) {
var defaultValidate = this.validate;
this.validate = function(next) {next();};
var self = this;
this.save(function(err, doc, numberAffected) {
self.validate = defaultValidate;
next(err, doc, numberAffected);
});
});
I've tested it only with mongoose 3.8.23
schema config validateBeforeSave=false
use validate methed
// define
var GiftSchema = new mongoose.Schema({
name: {type: String, required: true},
image: {type: String}
},{validateBeforeSave:false});
// use
var it new Gift({...});
it.validate(function(err){
if (err) next(err)
else it.save(function (err, model) {
...
});
})

Nodejs, Mongoose and Jade get no data from Database

i was searching for my Problem but even don't know where is the problem.
I get the title which is set in my route but no data from the database...
My model:
var mongoose = require('mongoose'),
Schema = mongoose.Schema,
ObjectId = Schema.Types.ObjectId;
var blogSchema = new Schema({
title: { type: String, required: true },
author: { type: String, required: true },
body: { type: String, required: true },
date: { type: String, required: true },
hidden: Boolean
});
module.exports = mongoose.model('Blog', blogSchema);
my router:
var express = require('express'),
Blog = require('../models/blog'),
moment = require('moment');
moment.lang('de');
var router = express.Router();
router.get('/articles', function(req, res) {
Blog.find(function(err, docs){
return res.render('blog/articles', {
title: 'Blog_',
articles: docs
});
});
});
app.use('/blog', router);
my jade
extends ../layouts/default
include ../elements/form-elements
block content
h1= title
each article in articles
.col-md-12
div.title= article.title
the only one i get displayed at the Page is
Blog_
So what iam doing wrong?
At the error file it only says:"Cannot read property 'title' of undefined"
So the articles objects are not set...but why?
Thanks so much
edit 1:
change article.title to article doesn't change anything
in the log files is
GET /blog/articles HTTP/1.1 304 - - 3 ms
edit 2:
it seems that node doesnt get any data from the db...
and yes there is one testdata set ;)
console.log() ->
err: null
docs: []
Solution is posted as answer
got the solution...
the model wasn't right...
var blogSchema = new Schema({
title: { type: String, required: true },
author: { type: String, required: true },
body: { type: String, required: true },
date: { type: String, required: true },
hidden: Boolean
}, {collection : 'blog'});
have to name the collection at the end...cause its written in small letters -.-
What a false - never ever do it again ^^
I know this is a very old question and it's marked by OP as answered, but I think the real problem was in "my router", you're not referencing your "docs" (the data coming back from the database) correctly. Keep in mind "docs" is an array so you would need to reference them like this:
router.get('/articles', function(req, res) {
Blog.find(function(err, docs){
return res.render('blog/articles', {
title: docs[0].title, // Get the title for first entry
articles: docs[0].body // Get body for the first entry
});
});
});
I'm hardcoding the array index but you can use a loop to get every item from the array.
I don't think OPs solution fixes the problem because...
By default, when compiling models with:
const someModel = mongoose.model('someModel', SomeSchema);
mongoose creates a collection using 'someModel' name and adding an "s" at the end, so if you check your database, your collection should
appear as 'someModels'. With OP's solution:
{ collection: 'blog' }
as the second parameter when creating the blog schema
var blogSchema = new Schema();
That default behavior is overwritten and the name of your collection will be whatever you set as the value for collection, in this case, "blog".
You can read more about it in Mongoose official docs
or in the Models section in MDN - Node/Express/Mongoose

Resources