I am trying to to update my user and applying Unique validator on email to prevent duplicates.
I need to ignore email uniqueness for provided user_id, and those records which are marked is_deleted to 1.
Only first statement works, if I place is_deleted,1 before id,${data.user_id} it works for deleted. but not for user_id.
get rules() {
const data = this.ctx.request.post()
console.log('current context: ', data.user_id)
return {
// email: `required|email|unique:users,email,id,${data.user_id},is_deleted,1`,
email: `required|email|unique:users,email,id,${data.user_id}|unique:users,email,is_deleted,1`,
phone_number: 'required|max:10',
status_id: 'required'
}
}
However, only first statement for ignore works, second one is not working
I would recommend extending the validation framework and add a custom rule (a good name would be unique_email). You will find it more productive and testable. The code would be similar to this:
const uniqueEmailFn = async (data, field, message, args, get) => {
const email = get(data, 'email')
const userId = get(data, 'user_id')
if (!email) {
/**
* skip validation if value is not defined. `required` rule
* should take care of it.
*/
return
}
const [table, column] = args
const row = await Database.table('users')
.where('id', userId)
.where('email', email)
.where('is_deleted', false)
.first()
if (row) {
throw 'The inputted e-mail is already taken'
}
}
Generally speaking, it's preferable to use the default rules for simple and generic validations, and business-specific ones can be added by extending the framework.
Related
I am new to dynamodb.
I want to increment the Sort Key
If the id=0 the next id=1 and so on,
If the user(Partition key), id(Sort Key) add items the next add items the id increment 1.
The code use on PutItem with dynamodb.
Is possible to do that?
I did not want use the UUID( unique Key)
Most situations don't need an auto-incrementing attribute and DynamoDB doesn't provide this feature out of the box. This is considered to be an anti-pattern in distributed systems.
But, see How to autoincrement in DynamoDB if you really need to.
I understand that you may need this number because it is a legal obligation to have incremental invoice numbers for example.
One way would be to create a table to store your number sequences.
Add fields like:
{
name: "invoices",
prefix: "INV",
numberOfDigits: 5,
leasedValue: 1,
appliedValue: 1,
lastUpdatedTime: '2022-08-05'
},
{
name: "deliveryNotes",
prefix: "DN",
numberOfDigits: 5,
leasedValue: 1,
appliedValue: 1,
lastUpdatedTime: '2022-08-05'
}
You need 2 values (a lease and an applied value), to make sure you never skip a beat, even when things go wrong.
That check-lease-apply-release/rollback logic looks as follows:
async function useSequence(name: string, cb: async (uniqueNumber: string) => void) {
// 1. GET THE SEQUENCE FROM DATABASE
const sequence = await getSequence("invoices");
this.validateSequence(sequence);
// 2. INCREASE THE LEASED VALUE
const oldValue = sequence.appliedValue;
const leasedValue = oldValue + 1;
sequence.leasedValue = leasedValue;
await saveSequence(sequence);
try {
// 3. CREATE AND SAVE YOUR DOCUMENT
await cb(format(leasedValue));
// 4. INCREASE THE APPLIED VALUE
sequence.appliedValue++;
await saveSequence(sequence);
} catch(err) {
// 4B. ROLLBACK WHEN THINGS ARE BROKEN
console.err(err)
try {
const sequence = await getSequence(name);
sequence.leasedValue--;
this.validateSequence(sequence);
await saveSequence(sequence);
} catch (err2) {
console.error(err2);
}
throw err;
}
}
function validateSequence(sequence) {
// A CLEAN STATE, MEANS THAT THE NUMBERS ARE IN SYNC
if (sequence.leasedValue !== sequence.appliedValue) {
throw new Error("sequence is broken.");
}
}
Then, whenever you need a unique number you can use the above function to work in a protected scope, where the number will be rollbacked when something goes wrong.
const details = ...;
await useSequence("invoice", async (uniqueNumber) => {
const invoiceData = {...details, id: uniqueNumber};
const invoice = await this.createInvoice(invoiceData);
await this.saveInvoice(invoice);
})
Can it scale? Can it run on multiple instances? No, it can't. It never will be, because in most countries it's just not legal to do so. You're not allowed to send out invoice 6 before invoice 5 or to cancel invoice 5 after you've send invoice 6.
The only exception being, if you have multiple sequences. e.g. in some cases you're allowed to have a sequence per customer, or a sequence per payment system, ... Hence, you want them in your database.
i have an issue with unique constraint on one of my fields.
I'm adding records to database to be able to check by tests is my code working as expected.
One of table field is unique number that is provided from outside (it's not related to some other table in the same database), i need to generate this unique number for each test, but i met with unique constraint issue.
I have following function:
export const findMinUniqueUserId = async (): Promise<number> => {
const subscriptions = await prisma.$queryRaw<Subscription[]>(`
SELECT "userId"
FROM public."Subscriptions"
ORDER BY "userId" DESC
LIMIT 1
`);
const firstFreeUserId = (subscriptions[0]?.userId || 0) + 1;
return firstFreeUserId;
};
that returns the first minimum free "userId" field.
I have also the following tests:
describe("Test 1", () => {
it("should do something", async () => {
const draftSub = {
userId: await findMinUniqueUserId()
...some other fields
}
await prisma.subscription.create({
data: draftSub
})
...some other test stuff
})
})
And the second one:
describe("Test 2", () => {
it("should do something", async () => {
const draftSub = {
userId: await findMinUniqueUserId()
...some other fields
}
await prisma.subscription.create({
data: draftSub
})
...some other test stuff
})
})
Sometimes i'm getting an error:
Unique constraint failed on the fields: (`userId`)
I've heard that each of test suit (describe block) works on seperate worker thread, i was trying to prepare some kind of singleton class, that can helps me but i think each instance of class is creating in separete worker thread, so generated userId is not unique.
This is what i was trying with singleton class:
export class UserIdManager {
private static instance: UserIdManager
private static userIdShiftBeforeDatabaseCall = 0
private static minFreeUserIdAfterDatabaseCall = 0
private constructor() {
return;
}
private static async init() {
this.minFreeUserIdAfterDatabaseCall = await findMinUniqueUserId();
}
public static async reserveMinFreeUserId() {
let minFreeUserId = UserIdManager.userIdShiftBeforeDatabaseCall;
UserIdManager.userIdShiftBeforeDatabaseCall++;
if (!UserIdManager.instance) {
UserIdManager.instance = new UserIdManager();
await this.init();
}
minFreeUserId += UserIdManager.minFreeUserIdAfterDatabaseCall;
return minFreeUserId;
}
}
But i realize that it doesn't help me with multithreading. I've used this, but with the same result:
....
const draftSub = {
userId: await UserIdManager.reserveMinFreeUserId()
...some other fields
}
....
So, the question is how to generate unique number for each test. When i pass --runInBand option to jest everything is working correctly, but it takes much more time.
What you are using is the typical MAX()+1 method of assigning unique values. Unfortunately this is a virtual guarantee you will get duplicate values for your unique value. This is a result of the Multi-Version Concurrency Control (MVCC) nature of Postgres. In a MVCC database the actions taken by one session cannot be seen by another session until the first session commits. Thus when multiple sessions access max()+1 they each get the same result. The first one to commit succeeds, the second fails. The solution to this is creating a sequence and let Postgres assign the unique value, it will not assign the same value twice regardless how many sessions access the sequence concurrently. The cost however being your values will contain gaps - accept it, get over it, and move on. You can have the sequence generated by defining your userid as a generated identity (Postgres10 or later) or as serial for older versions.
create table subscriptions ( id generated always as identity ...) -- for versions Postgres 10 or later
or
create table subscriptions ( id serial ...) -- for versions prior to Postgers 10
With either of those in place get rid of your findMinUniqueUserId function. You may also want to look into insert...returning... functionality
I am trying to use the "or" operator in Joi ver.17.4.0
As you can see, in the code below, I want either or both of the attributes/properties to be allowed, but not neither.
The problem is that Joi does not allow a string to be empty. So, to have it empty, I need to:
Joi.string().allow('')
This makes it not empty according to the "or" operator. So I can not get the 'name' to be empty in the eyes of "or".
It won't validate properly.
It validates even when I do this (but it shouldn't):
validatePerson(createPerson(''));
Keep in mind that I'm actually validating POST input on a node express API, so this is some simplified code to illustrate the issue:
const Joi = require('Joi');
function createPerson(name, age) {
const person = { name: name, age: age };
console.log(person);
return person;
}
function validatePerson(person) {
const schema = Joi.object({
name: Joi.string().allow(''),
age: Joi.number(),
}).or("name", "age");
console.log(schema.validate(person));
return schema.validate(person);
}
validatePerson(createPerson('')); // This should fail validation but doesn't
validatePerson(createPerson()); // Should fail and does
validatePerson(createPerson('Bob')); // Should pass and does
validatePerson(createPerson('', 7)); // Should pass and does
validatePerson(createPerson('Bob', 7)); // Should pass and does
As far as I understand, you want to allow the name to be empty an string, only if the age exists.
To acheive that, you can use .when:
name: Joi.string().when('age', { is: Joi.exist(), then: Joi.allow('') })
This way, your first example will fail as you expected.
How do I insert a value into a select statement using JavaScript, specifically when using express and postgres?
The createUser, and listAllUsers, is working (included below for reference). The try/catch is working and satisfying the request or throwing the error for those two as well.
Any help would be greatly appreciated!
When using Postman, the output that I receive when I send the get (localhost:4000/user/id with a x-www-formurlencoded key value user_id = 3) is…
{
"name": "error",
"length": 90,
"severity": "ERROR",
"code": "42601",
"position": "37",
"file": "scan.l",
"line": "1134",
"routine": "scanner_yyerror"
}
And in the terminal, it shows the following (trapped from my console.log).
3
QUERY: SELECT * FROM users WHERE user_id = ${user_id}
When I user curl it says the same in the terminal. Here is the curl command and putput…
curl -X GET localhost:4000/user/3
{"name":"error","length":90,"severity":"ERROR","code":"42601","position":"37","file":"scan.l","line":"1134","routine":"scanner_yyerror"}ww10sc2353621:~ james.mcgreggor$ curl -X GET localhost:4000/user/3
Ultimately the 3 that I am passing as the user_id is not being substituted in the select statement. That is my problem. I cannot figure out how to correctly do this. Should I even be taking this approach, or should I try passing it as a parameter in the URL?
This is from my User class file (User.js)
const db = require('../connectors/db.js');
class User {
constructor(id, user_id, first_name, middle_initial, last_name, email, type) {
this.id = id;
this.first_name = first_name;
this.middle_initial = middle_initial;
this.last_name = last_name;
this.email = email;
this.type = type;
this.user_id = user_id;
}
static newUser(user_id, first_name, middle_initial, last_name, email, type) {
return db.one(`
INSERT INTO users ("user_id", "first_name", "middle_initial", "last_name", "email", "type")
VALUES ('${user_id}', '${first_name}', '${middle_initial}', '${last_name}', '${email}', '${type}')
returning id
`)
}
static async allUsers() {
const findAllQuery = 'SELECT * FROM users;';
return db.query(findAllQuery)
}
static async selectUser(user_id) {
console.log(user_id);
const findOneQuery = 'SELECT * FROM users WHERE user_id = ${user_id}';
return db.query(findOneQuery)
}
}
module.exports = User;
This is from my Routes file (Routes.js)
const express = require('express');
const dataFunctions = require('./catalog.js');
const AppRouter = express.Router();
AppRouter.post('/user', dataFunctions.createUser);
AppRouter.get('/users', dataFunctions.listAllUsers);
AppRouter.get('/user/:id', dataFunctions.listUserByUserID);
AppRouter.delete('/user/:id', dataFunctions.deleteUserByUserID);
module.exports = AppRouter;
This is from my Catalog file (Routes.js)
const Users = require('../models/users.js')
// Create
async function createUser(req, res) {
try {
console.log(req.body);
const userId = await Users.newUser(req.body.user_id, req.body.first_name, req.body.middle_initial, req.body.last_name, req.body.email, req.body.type)
res.status(201).send(`User ID: ${userId.id}`);
} catch(error) {
res.status(400).send(error);
}
}
// List all
async function listAllUsers(req, res) {
try {
const userList = await Users.allUsers();
console.log(userList);
res.status(200).send(userList);
} catch(error) {
res.status(400).send(error);
}
}
// List by ID
async function listUserByUserID(req, res) {
try {
const userList = await Users.selectUser(req.body.user_id);
console.log(userList);
res.status(200).send(userList);
} catch(error) {
res.status(400).send(error);
}
}
module.exports = {
createUser,
listAllUsers,
listUserByUserID
}
Never use string concatenation for querying, you already have mechanism called prepared statement, signature like
.query('SELECT * FROM `books` WHERE `author` = ?', ['David'])
It will sanitize input for you and partially prevent sql-injection attacks, also always do validation of input values. And if you are not want to use ORM like typeorm, Sequelize, you can use knex.js which can only create query strings and fully manage db interaction
You should never insert the values directly into your query like that. Consider the example:
db.query(`
INSERT INTO messages (message, username)
VALUES ('${message}', '${username}')
`);
If the username was my authenticated username, but the message was whatever value I typed into the UI, I could pretend to be someone else by sending a message like: I am stupid', 'someone_else') --
The SQL would then look like:
INSERT INTO messages (message, username)
VALUES ('I am stupid', 'someone_else') --', 'my_username')
The --', 'my_username') bit is treated as a comment, so it looks like someone_else said I am stupid. This is one of the most common and easily exploitable vulnerabilities in web applications.
Solution 1
You could parameterise your query:
db.query(`
INSERT INTO messages (message, username)
VALUES (?, ?)
`, [message, username]);
This is secure, but harder to read (in my opinion) and you have to be very careful to always do this consistently.
Solution 2
https://www.atdatabases.org provides database APIs that are relatively easy to use, and totally safe from this kind of attack. You would just do:
import connect, {sql} from '#databases/pg';
const db = connect();
db.query(sql`
INSERT INTO messages (message, username)
VALUES (${message}, ${username})
`);
to safely execute the same query. The sql tag ensures the data is properly handled/escaped and #databases/pg automatically enforces that you always add the sql tag. N.B. there are then no quotes around the parameters.
Instead of using single quotes in select query in static async selectUser use ``.
I need to increment a column with 1 on some occasions, but the default value of that column is null and not zero. How do I handle this case using sequelize? What method could be utilized?
I could do by checking the column for null in one query and updating it accordingly in the second query using sequelize but I am looking for something better. Could I handle this one call?
I'll confess that I'm not terribly experienced with sequelize, but in general you'll want to utilize IFNULL. Here's what the raw query might look like:
UPDATE SomeTable
SET some_column = IFNULL(some_column, 0) + 1
WHERE <some predicate>
Going back to sequelize, I imagine you're trying to use .increment(), but judging from the related source, it doesn't look like it accepts anything that will do the trick for you.
Browsing the docs, it looks like you might be able to get away with something like this:
SomeModel.update({
some_column: sequelize.literal('IFNULL(some_column, 0) + 1')
}, {
where: {...}
});
If that doesn't work, you're probably stuck with a raw query.
First you need to find the model instance and update via itself, or update directly via Sequelize Static Model API.
Then you'll check whether the updated field got nullable value or not ? If fails then do the manual update as JMar propose above
await model.transaction({isolationLevel: ISOLATION_LEVELS.SERIALIZABLE}, async (tx) => {
const user = await model.User.findOne({
where: {
username: 'username',
},
rejectOnEmpty: true,
transaction: tx,
});
const updatedRecord = await user.increment(['field_tag'], {
transaction: tx,
});
if (!updatedRecord.field_tag) {
/** Manual update & Convert nullable value into Integer !*/
await model.User.update({
field_tag: Sequelize.literal('IFNULL(field_tag, 0) + 1')
}, {
where: {
username: 'username',
},
transaction: tx,
});
}
});