Firebase - Filtering data from two arrays - node.js

Currently started learning firebase . I want to compare data from two tables. There is a table 'subscribers' and 'clubMembers'. 'clubMembers' contains the full list of users whereas the 'subscribers' contains the list only those user who have subscribed for the event.
I want to compare emails from both table and extract the ones those are not in 'subscribers' table. So to be make more clear if 'clubMember' have 10 emails, and 'subscribers' have 5 emails, so I want the 5 remaining subscriber email who doesn't have subscribed.
here is the code
db.ref('subscribers/' + today).orderByChild("email").once('value').then(snap => {
db.ref('clubMembers').orderByChild("email").once('value') .then(snapshot => {
const finalNames = [];
const allSubscribers = snap.val();
const allMembers = snapshot.val();
for (const user in allMembers) {
const userObject = allMembers[user];
for(const subUser in allSubscribers){
const subUserObject = allSubscribers[subUser];
if (userObject.email !== subUserObject.email) {
finalNames.push(userObject.email);
}
}
}
the finalNames variable displays the full list of emails, not the filtered ones.

this code get to toy diffrence of array after this code and get data is ok from firebase :
db.ref('subscribers/' + today).orderByChild("email").once('value').then(snap => {
db.ref('clubMembers').orderByChild("email").once('value') .then(snapshot => {
const finalNames = [];
const allSubscribers = snap.val();
const allMembers = snapshot.val();
var res = allMembers.filter(item1 =>
!allSubscribers.some(item2 => (item2.email === item1.emial)))
console.log(res);
var allMembers = [
{ id: 0, email: 'john#a.com' },
{ id: 1, email: 'mary#a.com' },
{ id: 2, email: 'pablo#a.com' },
{ id: 3, email: 'escobar#a.com' }
];
var allSubscribers = [
{ id: 0, email: 'john#a.com' },
{ id: 1, email: 'mary#a.com' }
];
var res = allMembers.filter(item1 =>
!allSubscribers.some(item2 => (item2.email === item1.email)))
console.log(res);

Related

Submitting without files resets the all images in the array when making a PATCH request

I'm trying to make a dynamic field for adding team members using Formik.
In my backend, if I do not choose any file and edit only other field such as "memberName" I'm getting message saying;
"Cast to embedded failed for value "{
_id: '63c5687832a80d5d8f717715',
memberName: 'qqaa',
memberJobTitle: 'qq',
memberDescription: 'qq',
images: [ 'undefined' ]
}" (type Object) at path "team" because of "CastError""
I want to keep the existing images if there is no changes in the input field. I'm having this issue for a week and couldn't figure it out.
This is my controller for making a PATCH request;
const updateSection = async (req, res, next) => {
const files = req.files;
const {
pageId,
welcomeTitle,
welcomeDescription,
aboutUsTitle,
aboutUsDescription,
team,
teamTitle,
} = req.body;
let updates = {};
//update other fields if they are provided in the request body
if (welcomeTitle) {
updates.welcomeTitle = welcomeTitle;
}
if (welcomeDescription) {
updates.welcomeDescription = welcomeDescription;
}
if (aboutUsTitle) {
updates.aboutUsTitle = aboutUsTitle;
}
if (aboutUsDescription) {
updates.aboutUsDescription = aboutUsDescription;
}
if (teamTitle) {
updates.teamTitle = teamTitle;
}
if (team) {
let teamPromises = []; //create an empty array to store promises for updating or creating team members
// updates.team.images = [];
team.forEach((item, i) => {
// item -> current team member being processed, i-> index in the array
let teamMember = {
_id: item._id,
memberName: item.memberName,
memberJobTitle: item.memberJobTitle,
memberDescription: item.memberDescription,
images: item.images,
};
if (files && files[i]) {
let file = files[i];
let img = fs.readFileSync(file.path);
let decode_image = img.toString("base64");
teamMember.images = [
{
filename: file.originalname,
contentType: file.mimetype,
imageBase64: decode_image,
},
];
} else {
teamMember.images = item.images;
}
teamPromises.push(
Section.updateOne(
{ pageId: pageId, "team._id": item._id },
{ $set: { "team.$": teamMember } },
{ new: false }
)
);
});
Promise.all(teamPromises)
.then((result) => {
res.status(200).json({ message: "Section updated successfully!" });
})
.catch((error) => {
res.status(500).json({ error: error.message });
});
} else {
//update other fields if no team member provided
Section.findOneAndUpdate({ pageId: pageId }, updates).then(() => {
res.status(200).json({ message: "Section updated successfully!" });
});
}
};

Mongo DB SORTING data nodejs

i created a code that Backsup/Deletes and Inserts information from the Third Party API every 24hours to update the Third party api changes daily on the database.
how can i sort the information i get from the MongoDB by which score changed the most?
API LOOKS LIKE
{
"_id": "6365e1dbde0dd3639536f4b7",
"position": 1,
"id": "105162",
"score": 2243536903,
"__v": 0
},
MY CODE
app.get('/api/TopFlops', async (req, res) => {
const topflops = await TopFlops.find({}).sort({_id: +1}).limit(5)
res.json(topflops);
})
CODE TO INSERT DATA FROM THE 3RD PARTY API TO DB
cron.schedule('59 23 * * *', async () => {
const postSchema = new mongoose.Schema({
id: {
type: Number,
required: true
},
name: {
type: String,
required: true
},
status: {
type: String,
required: false
},
});
const Post = mongoose.model('players', postSchema);
async function getPosts() {
const getPlayers = await fetch("http://localhost:3008/api/players");
const response = await getPlayers.json();
for( let i = 0;i < response.players.length; i++){
const post = new Post({
id: response.players[i]['id'],
name: response.players[i]['name'],
status: response.players[i]['status'],
});
post.save();
}
}
console.log("Table submitted successfully")
await getPosts();
});
CODE TO FETCH API
const [playerName, setPlayerName] = useState([]);
const [playerRank, setPlayerRank] = useState([]);
const [player, setPlayer] = useState([]);
const [perPage, setPerPage] = useState(10);
const [size, setSize] = useState(perPage);
const [current, setCurrent] = useState(1);
const [players, setPlayers] = useState();
const fetchData = () => {
const playerAPI = 'http://localhost:3001/api/topflops';
const playerRank = 'http://localhost:3001/api/topflops';
const getINFOPlayer = axios.get(playerAPI)
const getPlayerRank = axios.get(playerRank)
axios.all([getINFOPlayer, getPlayerRank]).then(
axios.spread((...allData) => {
const allDataPlayer = allData[0].data
const getINFOPlayerRank = allData[1].data
const newPlayer = allDataPlayer.map(name => {
const pr = getINFOPlayerRank.find(rank => name.id === rank.id)
return {
id: name.id,
name: name.name,
alliance: name.alliance,
position: pr?.position,
score: pr?.score
}
})
setPlayerName(allDataPlayer)
setPlayerRank(getINFOPlayerRank)
console.log(getINFOPlayerRank)
console.log(newPlayer)
setPlayer(newPlayer)
})
)
}
useEffect(() => {
fetchData()
}, [])
const getData = (current, pageSize) => {
// Normally you should get the data from the server
return player?.slice((current - 1) * pageSize, current * pageSize);
};

Serching by role mongoose

I am trying to retrieve users based on role but with every role queried, I want to also retrieve admin too. I am not sure how to go about it. This is how I currently retrieve based on role
public async listAllStaffs(query: ListStaffsRequestQueryDTO) {
var conditions = { }
if (query.role) {
conditions = { role: query.role, 'hub.id': query.merchant };
} else {
conditions = {'hub.id': query.merchant };
}
const data = { page: query.page, limit: query.limit, conditions: conditions};
const all = await this.list(data);
const pagination = {
page: all.page,
limit: all.limit,
rowCount: all.rowCount,
pageCount: all.pageCount
};
const staffs = all.staffs;
return { staffs, pagination };
}
async list(query: PaginationQuery): Promise<any> {
const page = Number(query.page) - 1 || 0;
let limit = Number(query.limit) || 20;
const offset = page * limit;
const sort = query.sort || 'createdAt';
const archived = this.convertArchived(query.archived);
const conditions = {
...query.conditions,
...(!archived
? { deletedAt: undefined }
: { deletedAt: { $ne: undefined } })
};
const staffs = await this.model
.find(conditions)
.select(query.projections)
.skip(offset)
.limit(limit)
.sort(sort);
const totalDocuments = await this.model.countDocuments(conditions);
const result = {
staffs,
page: Number(page) + 1,
limit: Number(limit),
rowCount: Number(totalDocuments),
pageCount: Math.ceil(Number(totalDocuments) / limit)
};
return result;
}
So if role is operation for example, I want to also return users with admin
So you have both operation staff and admin staff returned.
Use $in operator, to look for users in multiple roles, like this:
if (query.role) {
conditions = { role: { "$in": [query.role, "admin"] }, 'hub.id': query.merchant
};
}

Unable to access value of req.body

I am trying to do a PUT request to update my DB in Mongo using Axios so I can update the current week and season. In my route, I can access the req.body, but if I try to set the values, it says the values are undefined. I have tried many different things at this point. I ran into this issue as well: "Cast to Number failed".
Here is what the request looks like:
function currentWeek() {
const currentWeek =
"https://api.sportsdata.io/v3/nfl/scores/json/CurrentWeek?key=...";
axios.get(currentWeek).then((res) => {
const weekCheck = res.data;
const timeframeURL =
"https://api.sportsdata.io/v3/nfl/scores/json/Timeframes/current?key=...";
console.log(weekCheck);
axios.get(timeframeURL).then((res) => {
const timeframeWeek = res.data;
// console.log(timeframeWeek);
const thisWeek = timeframeWeek.filter(
(timeframeWeek) => timeframeWeek.Week === weekCheck
);
console.log(thisWeek);
const config = {
headers: {
"Content-Type": "application/json",
},
};
axios
.put("http://localhost:4000/api/currentweek/5ffce18e78d4742414cf279e", thisWeek, config)
.then((res) => console.log("working"))
.catch((err) => console.error(err));
console.log("Done!");
});
});
}
Here is my route:
router.put("/:_id", async (req, res) => {
const { Season, Week } = req.body;
const { _id } = req.params;
const weekField = {};
// SETING THE VALUES FROM REQ.BODY TO BE IN weekField
if (Season) weekField.Season = Season;
if (Week) weekField.Week = Week;
try {
let weekParam = await CurrentWeek.find({_id});
if (!weekParam) return res.stats(404).json({ msg: "ID in the Params does not exist" });
console.log(_id);
console.log(req.body) // RETURNS THE OBJECT CORRECTLY
console.log(weekField); // RETURNS AS AN EMPTY OBJECT
console.log("From Route ^^");
weekParam = await CurrentWeek.findOneAndUpdate(
_id,
{ $set: weekField },
{ new: true }
);
res.json(weekParam);
} catch (err) {
console.error(err.message);
res.status(500).send("Server Error");
}
});
Here is the model:
const mongoose = require('mongoose');
const CurrentWeekSchema = mongoose.Schema([{
Week: Number,
Season: Number
}]);
const CurrentWeek = module.exports = mongoose.model('currentweek', CurrentWeekSchema);
//^enables require from routes
module.exports.getCurrentWeek = function(callback, limit){
CurrentWeek.find(callback).limit(limit);
}
And here is an example of the object I am trying to take in:
[
{
SeasonType: 3,
Season: 2020,
Week: 2,
Name: 'Divisional Playoffs',
ShortName: 'Divisional',
StartDate: '2021-01-12T00:00:00',
EndDate: '2021-01-18T23:59:59',
FirstGameStart: '2021-01-16T16:35:00',
FirstGameEnd: '2021-01-16T20:35:00',
LastGameEnd: '2021-01-17T22:40:00',
HasGames: true,
HasStarted: true,
HasEnded: false,
HasFirstGameStarted: false,
HasFirstGameEnded: false,
HasLastGameEnded: false,
ApiSeason: '2020POST',
ApiWeek: '2'
}
]
I found out the issue and maybe this will be helpful for someone else in the future!
In my router file I need to change this:
const { Season, Week } = req.body;
const { _id } = req.params;
const weekField = {};
// SETING THE VALUES FROM REQ.BODY TO BE IN weekField
if (Season) weekField.Season = Season;
if (Week) weekField.Week = Week;
To this:
const data = req.body;
const { Season, Week } = data[0];
const { _id } = req.params;
const weekField = {};
// SETING THE VALUES FROM REQ.BODY TO BE IN weekField
if (Season) weekField.Season = Season;
if (Week) weekField.Week = Week;
The reason is because the data that the put request is receiving is coming in an array, so I had to set the req.body to the first object in the array.

Import large amounts of data, but do a .find() for each element

I have a collection of customers of 60.000 items. I need to import a list of people, of 50.000. But for each person, I need to find the ID of the customer and add that to the object that is being inserted.
How should this be done most efficently?
export default async ({ file, user, database }: Request, res: Response) => {
try {
const csv = file.buffer.toString("utf8");
let lines = await CSV({
delimiter: "auto" // delimiter used for separating columns.
}).fromString(csv);
let count = {
noCustomer: 0,
fails: 0
};
let data: any = [];
await Promise.all(
lines.map(async (item, index) => {
try {
// Find customer
const customer = await database
.model("Customer")
.findOne({
$or: [
{ "custom.pipeID": item["Organisasjon - ID"] },
{ name: item["Person - Organisasjon"] }
]
})
.select("_id")
.lean();
// If found a customer
if (!isNil(customer?._id)) {
return data.push({
name: item["Person - Navn"],
email: item["Person - Email"],
phone: item["Person - Telefon"],
customer: customer._id,
updatedAt: item["Person - Oppdater tid"],
createdAt: item["Person - Person opprettet"],
creator: users[item["Person - Eier"]] || "5e4bca71a31da7262c3707c5"
});
}
else {
return count.noCustomer++;
}
} catch (err) {
count.fails++;
return;
}
})
);
const people = await database.model("Person").insertMany(data)
res.send("Thanks!")
} catch (err) {
console.log(err)
throw err;
}
};
My code just never sends an response If I use this as a Express request.

Resources