I'm using FeathersJS and MongoDB to develop an app. I want to prevent some services to create duplicates of some values (or pairs of values).
For example, the FeathersJS "Authenticate" service created with the feathers-cli tool doesn't prevent the app from creating 2 or more users with the same email (at least using MongoDB). Another example would be a service to create some "categories" for each user. I want that the backend prevents a user to create 2 or more categories with the same name, but I need to allow 2 different users to create their own categories although their names are the same (but not the users).
I know I can do this by using indexes in the MongoDB collections, but this would make the app MongoDB dependant.
Is there someone that knows if there's any kind of hook or whatever that is the recommended way to do such things "the FeathersJS way"?
Thank you!
In most cases uniqueness can - and should - be insured at the database or ORM/ODM level since it will give you the best performance (something that in most cases isn't worth sacrificing for portability).
A more Feathers-y way and to accomplish more complex restrictions would be Feathers hooks which are an important part of Feathers and explained in detail in the basics guide.
In this case, a before hook could query the total of items and throw an error if there are any:
const { Conflict } = require('#feathersjs/errors');
app.service('myservice').hooks({
before: {
create: [async context => {
const { fieldA, fieldB } = context.data;
// Request a page with no data and extract `page.total`
const { total } = await context.service.find({
query: {
fieldA,
fieldB,
$limit: 0
}
});
if(total > 0) {
throw new Conflict('Unique fields fieldA and fieldB already exist');
}
return context;
}]
}
})
Related
I am building an API and came across an issue that I have a few ideas of how to solve, but I was wondering what is the most optimal one. The issue is the following:
I have a Product model which has, for the sake of simplicity one field called totalValue.
I have another model called InventoryItems, which, whenever is updated, the totalValue of Product must also be updated.
For example, if the current totalValue of a product is say $1000, when someone purchases 10 screws at a cost of $1 each, a new InventoryItem record will be created:
InventoryItem: {
costPerItem: 1,
quantity: 10,
relatedToProduct: "ProductXYZ"
}
At the same time of creation of that item, totalValue of the respective ProductXYZ must be updated to now $1100.
The question is what is the most efficient and user-friendly way to do this?
Two ways come to my mind (and keep in mind that the code bellow is kinda pseudo, I have intentionally omitted parts of it, that are irrelevant for the problem at hand):
When the new InventoryItem is created, it also queries the database for the product and updates it, so both things happen in the same function that creates the inventory item:
function async createInventoryItem(req, res) {
const item = { ...req.body };
const newInventoryItem = await new InventoryItem({...item}).save();
const foundProduct = await Product.find({ name: item.relatedtoProduct }).exec();
foundProduct.totalValue = foundProduct.totalValue + item.costPerItem * item.quantity;
foundProduct.save();
res.json({ newInventoryItem, newTotalOfProduct: foundProduct.totalValue });
}
That would work, my problem with that is that I will no longer have "a single source of truth" as that approach will make it hard to update the code, as updating a given Product will be scattered all over the project.
The second approach that comes to my mind is that, when I receive the request to create the item, I do create the item, and then I make an internal request to the other endpoint that handles product updates, something like:
function async createInventoryItem(req, res) {
const item = { ...req.body };
const newInventoryItem = await new InventoryItem({...item}).save();
const totalCostOfNewInventoryItem = item.costPerItem * item.quantity;
// THIS is the part that I don't know how to do
const putResponse = putrequest("/api/product/update", {
product: item.relatedtoProduct,
addToTotalValue: totalCostOfNewInventoryItem
});
res.json({ newInventoryItem, newTotalOfProduct: putResponse.totalValue });
}
This second approach solves the problem of the first approach, but I don't know how to implement it, and it is I'm guessing a form of requests chaining or rerouting? Also I am guessing that the second approach will not have a performance penalty, since node will be sending requests to itself, so no time lost in accessing servers across the world or whatever)
I am pretty sure that the second approach is the one that I have to take (or is there another way that I am currently not aware of??? I am open to any suggestions, I am aiming for performance), but I am unsure of exactly how to implement it.
I have a Firestore collection named 'users' and has many documents by the name of each user.
I want to retrieve list of 25 users at a time in alphabetical order and this is what I tried:
const allUsersRef = admin.firestore().collection('users').orderBy('name').offset(0).limit(25)
allUsersRef.get().then((top25Users) => {
let usersList = '``` << Users LIST >>\n'
if (!top25Users.empty) {
top25Users.forEach(eachUser => {
usersList = usersList + `\n${eachUser.data().name} \n${eachUser.data().id}`
})
console.log(usersList)
return
} else {
message.channel.send('Looks like we have no users at the moment!')
return
}
}).catch((error) => {
console.log(error)
return
})
This way I can get the top 25 users easily! But what if I want the next 25? This is a Discord Bot and not an Android Application where I can add a button [view more] and then continue the results query.start() as shown in this firebase video
I can use OFFSET but the number of users is large so using offset(500) won't be affordable :'(
Also I need to fetch users in alphabetical order and when new users register, the order changes.
TL,DR: If I had a list of my users in alphabetical order, how do I get users from 126th position to 150th position on the list which is sort of page 5 for my 25/page query! and without using offset because that just uses more resources!
I had this in firebase realtime database first but then I needed some more advanced querying so I have migrated here :)
Database Struture: Just a single collection named USERS and documents named as username in it.
PS:
const startAtRes = await db.collection('cities')
.orderBy('population')
.startAt(1000000)
.get();
Using something like this ^ from Firebase Documentation is not possible because I won't be knowing from where to start from. As the list changes as new users Register!
Firestore does not support efficient offset based pagination. When you use offset(), you're paying for reads of all the documents up to that point. The only availabe efficient pagination requires that you provide an anchor document, or properties of the anchor document, to navigate between pages, as described in the documentation.
I am using Cloud Function to send a notification to mobile device. I have two collection in Firestore clientDetail and clientPersonalDetail. I have clientID same in both of the collection but the date is stored in clientDetail and name is stored in clientPersonal.
Take a look:
ClientDetail -- startDate
-- clientID
.......
ClientPersonalDetail -- name
-- clientID
.........
Here is My full Code:
exports.sendDailyNotifications = functions.https.onRequest( (request, response) => {
var getApplicants = getApplicantList();
console.log('getApplicants', getApplicants);
cors(request, response, () => {
admin
.firestore()
.collection("clientDetails")
//.where("clientID", "==", "wOqkjYYz3t7qQzHJ1kgu")
.get()
.then(querySnapshot => {
const promises = [];
querySnapshot.forEach(doc => {
let clientObject = {};
clientObject.clientID = doc.data().clientID;
clientObject.monthlyInstallment = doc.data().monthlyInstallment;
promises.push(clientObject);
});
return Promise.all(promises);
}) //below code for notification
.then(results => {
response.send(results);
results.forEach(user => {
//sendNotification(user);
});
return "";
})
.catch(error => {
console.log(error);
response.status(500).send(error);
});
});
}
);
Above function is showing an object like this
{clienId:xxxxxxxxx, startDate:23/1/2019}
But I need ClientID not name to show in notification so I'll have to join to clientPersonal collection in order to get name using clientID.
What should do ?
How can I create another function which solely return name by passing clientID as argument, and waits until it returns the name .
Can Anybody please Help.?
But I need ClientID not name to show in notification so I'll have to join to clientPersonal collection in order to get name using clientID. What should do ?
Unfortunately, there is no JOIN clause in Firestore. Queries in Firestore are shallow. This means that they only get items from the collection that the query is run against. There is no way to get documents from two top-level collection in a single query. Firestore doesn't support queries across different collections in one go. A single query may only use properties of documents in a single collection.
How can I create another function which solely return name by passing clientID as argument, and waits until it returns the name.
So the most simple solution I can think of is to first query the database to get the clientID. Once you have this id, make another database call (inside the callback), so you can get the corresponding name.
Another solution would be to add the name of the user as a new property under ClientDetail so you can query the database only once. This practice is called denormalization and is a common practice when it comes to Firebase. If you are new to NoQSL databases, I recommend you see this video, Denormalization is normal with the Firebase Database for a better understanding. It is for Firebase realtime database but same rules apply to Cloud Firestore.
Also, when you are duplicating data, there is one thing that need to keep in mind. In the same way you are adding data, you need to maintain it. With other words, if you want to update/detele an item, you need to do it in every place that it exists.
The "easier" solution would probably be the duplication of data. This is quite common in NoSQL world.
More precisely you would add in your documents in the ClientDetail collection the value of the client name.
You can use two extra functions in this occasion to have your code clear. One function that will read all the documents form the collection ClientDetail and instead of getting all the fields, will get only the ClientID. Then call the other function, that will be scanning all the documents in collection ClientPersonalDetail and retrieve only the part with the ClientID. Compare if those two match and then do any operations there if they do so.
You can refer to Get started with Cloud Firestore documentation on how to create, add and load documents from Firestore.
Your package,json should look something like this:
{
"name": "sample-http",
"version": "0.0.1",
"dependencies": {
"firebase-admin": "^6.5.1"
}
}
I have did a little bit of coding myself and here is my example code in GitHub. By deploying this Function, will scan all the documents form one Collection and compare the ClientID from the documents in the other collection. When it will find a match it will log a message otherwise it will log a message of not matching IDs. You can use the idea of how this function operates and use it in your code.
I am struggling to find the solution for that.
I want to have users which can belong to many organizations.
Each user can have a different role (I would prefer even roles but it sounds even more complicated...) at a specific organization.
In the table like User_Organization_Role I need to have fields like role (roleId?), isActive. Maybe some more.
I am using Feathers Plus generator but I do not think it matters in this case, however it may be beneficial to add something to the schema file?
I thought having belongsTo with simple organizationId field will be sufficient but I've realized that changing that to manyToMany, later on, would be painful so I think it is much better to implement that now.
I will appreciate any solutions / suggestions / best practices etc.
n:m relations are by far the most difficult to handle, and there's really no one-size-fits-all solution. The biggest thing is to read and understand this page and its sub-pages, and then read them 2 more times for good measure. Try to focus on doing one thing at a time. I outline how I would approach this with feathersjs in this issue:
https://github.com/feathersjs/feathers/issues/852#issuecomment-406413342
The same technique could be applied in any application... the basic flow goes like this:
Create or update your primary objects first (users, organizations, roles, etc.). There are no relations made at this point. You need to have your objects created before you can make any relations.
Create or update the relations. This involves updating a "join" table (aka: "mapping" or "through" table) with data from step #1. The join table can (and should) have its own model. It should contain a foreign key for each of the objects you are associating (userId, organizationId, roleId etc.). You can put other fields in this table too.
Here is some pseudo code for how I would define my models (only showing relevant code for brevity). There is a little more to it than what I describe below, but this should get you started.
const UserOrganizationRole = sequelize.define('User_Organization_Role', {
// Define any custom fields you want
foo: DataTypes.STRING
})
// Let sequelize add the foreign key fields for you.
// Also, save a reference to the relationship - we will use it later
User.Organization = User.belongsToMany(Organization, { through: UserOrganizationRole });
User.Role = User.belongsToMany(Role, { through: UserOrganizationRole });
Organization.User = Organization.belongsToMany(User, { through: UserOrganizationRole });
Role.User = Role.belongsToMany(User, { through: UserOrganizationRole });
... and here is how I would go about handling inserts
const user = await User.create({ ... });
const org = await Organization.create({ ... });
const role = await Role.create({ ... });
await UserOrganizationRole.create({
userId: user.id,
organizationId: org.id,
roleId: role.id,
foo: 'bar'
});
... and finally, load the data like so:
// Now we can reference those relationships we created earlier:
const user = await User.findById(123, {
include: [User.Organization, User.Role]
});
const org = await Organization.findById(456, {
include: [Organization.User]
});
I have an app that stores user uploaded spreadsheets as tables in PostgreSQL. Everytime an user uploads a spreadsheet I create a record in a Dataset table containing the physical table name, its alias and the owner. I can retrieve a certain Dataset information with
GET domain.com/v1/Datasets/{id}
AFAIK, the relation between rows in Dataset and physical tables can't be enforced by a FK, or at least I haven't seen anyones creating FKs on the information_schema of PostgreSQL, and FKs can't drop tables, or can they? So it's common to have orphan tables, or records in Dataset that point to tables that no longer exist. I have managed this with business logic and cleaning tasks.
Now, to access one of those physical tables, for example one called nba_teams I would need to declare an NbaTeams model in loopback and restart the app, then query its records with
GET domain.com/v1/NbaTeams/{id}
But that can't scale, specially if I'm already having like 100 uploads a day. So from where I'm standing, there are two ways to go:
1.- Create one model, then add 4 custom methods that accepts a table name as a string, and perform the next CRUD operation on that table name via raw queries. For example, to list the records:
GET domain.com/v1/Datasets/getTable/NbaTeams
or, to update one team
PUT domain.com/v1/Datasets/getTable/NbaTeams/{teamId}
This sounds unelegant but should work.
2.- Create a custom method that accepts a table name as a string, which in turn creates an ephemeral model and forward the HTTP verb and the rest of the arguments to it
dataSource.discoverAndBuildModels('nba_teams', {
owner: 'uploader'
}, function (err, models) {
console.log(models);
models.NbaTeams.find(function (err, act) {
if (err) {
console.error(err);
} else {
console.log(act);
}
dataSource.disconnect();
});
});
this second one I haven't got to work yet, and I don't know how much overhead it might have, but I'm sure it's doable.
So before I dig in deeper I came to ask: has anybody dealt with this row-to-table relation? What are the good practices in this?
In the end, I did my own hacky workaround and I thought it may help someone, some day.
What I did was put a middleware (with regular express syntax) to listen for /v1/dataset{id_dataset} , create the model on the fly and pass the execution to the next middleware
app.use('/v1/dataset:id_dataset', function(req, res, next) {
var idDataset=req.params.id_dataset;
app.getTheTable(idDataset,function(err,result) {
if(err) {
console.error(err);
res.json({"error":"couldn't retrieve related table"});
} else {
next();
}
});
});
inside the app.getTheTable function, I'm creating a model dynamically and setting it up before callback
app.getTheTable = function (idDataset, callback) {
var Table = app.models.Dataset,
modelName='dataset'+idDataset,
dataSource;
Table.findById(idDataset, function (err, resultados) {
if (err) {
callback(new Error('Unauthorized'));
} else {
if(app.models[modelName]) {
callback(null,modelName); // model already exists
} else {
var theDataset = dataSource.createModel(modelName, properties, options);
theDataset.settings.plural = modelName;
theDataset.setup();
app.model(theDataset);
var restApiRoot = app.get('restApiRoot');
app.use(restApiRoot, app.loopback.rest());
callback(null, modelName);
}
}
});
};
It's hacky, I know, and I believe there must be some kind of performance penalty for overloading restApiRoot middleware, but it's still better tan creating 500 models on startup to cover all possible dataset requests.