Get data by id from firebase - node.js

I use firebase on node.js .
My given structure should look like this:
{
...
batch-1:
id-1(suppose):
name:...
phone:...
id-2:
...
id-3:
...
batch-2:
...
batch-3:
...
...
batch-n:
...
}
How can I get an id-1 object by its identifier in such an architecture?
Does the database have to go around all the batches?
Is there a better solution?
The main task: Create a batch with many objects that will have SHORT and a UNIQUE identifier and optimally receive data by this identifier

To search for a particular ID that is a child of a list of unknown IDs, you need to use orderByChild(). In your use case, you are looking for a particular ID in a list of batch IDs. If you used orderByChild() on this list, you would get back results for each and every batch ID, even if it didn't have the ID you wanted. This is because even null (non-existant) values are included (and sorted at the start) in the results. To get the data of the desired ID, you would get the data for the last result of the query, which if it existed, would be sorted to the end of the list. Note that if the desired ID doesn't exist, the last result (if there are any results) would have a null value. To return only the last result of the query, you would use limitToLast(1).
Putting this all together, gives the following code:
let idToFind = "unique-id-1";
let batchesRef = firebase.database().ref(); // parent key of "batch-1", "batch-2", etc.
// assumed to be the database root here
batchesRef.orderByChild(idToFind).limitToLast(1).once('value')
.then((querySnapshot) => {
if (!querySnapshot.numChildren()) { // handle rare no-results case
throw new Error('expected at least one result');
}
let dataSnapshot;
querySnapshot.forEach((snap) => dataSnapshot = snap); // get the snapshot we want out of the query's results list
if (!dataSnapshot.exists()) { // value may be null, meaning idToFind doesn't exist
throw new Error(`Entry ${idToFind} not found.`);
}
// do what you want with dataSnapshot
console.log(`Entry ${idToFind}'s data is:`, dataSnapshot.val());
})
.catch((error) => {
console.log("Unexpected error:", error);
})
For small data sets, the above code will work just fine. But if the list of batches starts growing quite large, you may wish to build an index that maps a particular ID to the batch ID that contains it.

Here is my method which allows you to search by id or to search by key value such as email uniqueemail
// gets primary key
const getSnapshotValKey = snapshot => (Object.keys(snapshot).length > 0 ? Object.keys(snapshot)[0] : null)
const getUser = async ({ id, key, value }) => {
let user = null
const ref = id ? '/users/' + id : 'users'
const userRef = admin.database().ref(ref)
const valueRef = id ? userRef : await userRef.orderByChild(key).equalTo(value)
const snapshot = await valueRef.once('value')
const val = snapshot.val()
if (val) {
const key = id || getSnapshotValKey(val)
user = {
id: key,
...(id ? val : val[key]),
}
}
return user
}

Related

Paginate firebase realtime database

I want to get the first document in my subscription collection and then use that document as the starting point to retrieve the next 10 documents. The reason why I am not retrieving the first 10 straight away is that I want to run a cron job to paginate the data and change the starting document for every job run.
static start: number = 1;
#Cron(CronExpression.EVERY_5_SECONDS)
async getAllTransaction(): Promise<void> {
this.getAtIndex(TransactionsService.start).then(res => {
const ref = database().ref('subscriptions').limitToFirst(10).startAt(res);
ref.once('value').then(item => {
console.log(item.val());
});
});
}
async getAtIndex(id: number): Promise<any> {
const ref = database().ref('subscriptions').limitToFirst(1);
const snapshot = await ref.once('value');
const value = snapshot.val();
return value;
}
But I get this error
throw new Error('Query: When ordering by priority, the first argument passed to startAt(), ' +
Error: Query: When ordering by priority, the first argument passed to startAt(), startAfter() endAt(), endBefore(), or equalTo() must be a valid priority value (null, a number, or a string).
I have tried using an arbitrary number as the argument for the startAt() but it returns null
Your res is a DataSnapshot, which is not a valid value to pass to startAt. You probably want to pass res.key instead.
const ref = database().ref('subscriptions').limitToFirst(10).startAt(res.key);
In addition: if you don't specify any orderBy... clause for a query, it is ordered by an outdated priority value. Since you most likely want to order by the key, use orderByKey() in all your queries.

Error in returning new value from Mongoose Database (Nodejs)

When I execute a find query in mongodb after executing a findOneAndUpdate query the data that is returned is not the updated one eventhough the data has been correctly updated in the database.
I find the list of all entries which have a particular code/codes,
let codeList:Array<CodeBase> = await codebaseModel.find({codeBaseId: codes});
I loop through each entry and where the fileTree needs to be updated, I get the directory tree and update it
if (codeList.length > 0) {
codeList.forEach( (e): void => {
if (e.fileTree === 'To be updated') {
let tree = directoryTree(src/uploads/${e.codeBaseId}/unzipped/);
let treeData = JSON.stringify(tree);
let value = codebaseModel.findOneAndUpdate(
{ codeBaseId: e.codeBaseId },
{ fileTree: treeData },
{ upsert:true,new:true, }
);
}
});
}
I need get the updated list again (so as to ensure that I have the entries with the updated fileTree values.
let codeListNext:Array<CodeBase> | null | void = [];
await codebaseModel.find({
codeBaseId: codes,
},).then((value:Array<CodeBase>)=>{
codeListNext = value;
});
console.log(Firstlist length is ${codeListNext});
However, the list that is returned is the old list. However, the mongodb database has updated values. I tried using async await but was not successful. Can someone help me understand where am I going wrong ?

Increment the Sort Key dynamoDB node.js

I am new to dynamodb.
I want to increment the Sort Key
If the id=0 the next id=1 and so on,
If the user(Partition key), id(Sort Key) add items the next add items the id increment 1.
The code use on PutItem with dynamodb.
Is possible to do that?
I did not want use the UUID( unique Key)
Most situations don't need an auto-incrementing attribute and DynamoDB doesn't provide this feature out of the box. This is considered to be an anti-pattern in distributed systems.
But, see How to autoincrement in DynamoDB if you really need to.
I understand that you may need this number because it is a legal obligation to have incremental invoice numbers for example.
One way would be to create a table to store your number sequences.
Add fields like:
{
name: "invoices",
prefix: "INV",
numberOfDigits: 5,
leasedValue: 1,
appliedValue: 1,
lastUpdatedTime: '2022-08-05'
},
{
name: "deliveryNotes",
prefix: "DN",
numberOfDigits: 5,
leasedValue: 1,
appliedValue: 1,
lastUpdatedTime: '2022-08-05'
}
You need 2 values (a lease and an applied value), to make sure you never skip a beat, even when things go wrong.
That check-lease-apply-release/rollback logic looks as follows:
async function useSequence(name: string, cb: async (uniqueNumber: string) => void) {
// 1. GET THE SEQUENCE FROM DATABASE
const sequence = await getSequence("invoices");
this.validateSequence(sequence);
// 2. INCREASE THE LEASED VALUE
const oldValue = sequence.appliedValue;
const leasedValue = oldValue + 1;
sequence.leasedValue = leasedValue;
await saveSequence(sequence);
try {
// 3. CREATE AND SAVE YOUR DOCUMENT
await cb(format(leasedValue));
// 4. INCREASE THE APPLIED VALUE
sequence.appliedValue++;
await saveSequence(sequence);
} catch(err) {
// 4B. ROLLBACK WHEN THINGS ARE BROKEN
console.err(err)
try {
const sequence = await getSequence(name);
sequence.leasedValue--;
this.validateSequence(sequence);
await saveSequence(sequence);
} catch (err2) {
console.error(err2);
}
throw err;
}
}
function validateSequence(sequence) {
// A CLEAN STATE, MEANS THAT THE NUMBERS ARE IN SYNC
if (sequence.leasedValue !== sequence.appliedValue) {
throw new Error("sequence is broken.");
}
}
Then, whenever you need a unique number you can use the above function to work in a protected scope, where the number will be rollbacked when something goes wrong.
const details = ...;
await useSequence("invoice", async (uniqueNumber) => {
const invoiceData = {...details, id: uniqueNumber};
const invoice = await this.createInvoice(invoiceData);
await this.saveInvoice(invoice);
})
Can it scale? Can it run on multiple instances? No, it can't. It never will be, because in most countries it's just not legal to do so. You're not allowed to send out invoice 6 before invoice 5 or to cancel invoice 5 after you've send invoice 6.
The only exception being, if you have multiple sequences. e.g. in some cases you're allowed to have a sequence per customer, or a sequence per payment system, ... Hence, you want them in your database.

How to generate unique number getting from postgres sql in jest tests running concurrently

i have an issue with unique constraint on one of my fields.
I'm adding records to database to be able to check by tests is my code working as expected.
One of table field is unique number that is provided from outside (it's not related to some other table in the same database), i need to generate this unique number for each test, but i met with unique constraint issue.
I have following function:
export const findMinUniqueUserId = async (): Promise<number> => {
const subscriptions = await prisma.$queryRaw<Subscription[]>(`
SELECT "userId"
FROM public."Subscriptions"
ORDER BY "userId" DESC
LIMIT 1
`);
const firstFreeUserId = (subscriptions[0]?.userId || 0) + 1;
return firstFreeUserId;
};
that returns the first minimum free "userId" field.
I have also the following tests:
describe("Test 1", () => {
it("should do something", async () => {
const draftSub = {
userId: await findMinUniqueUserId()
...some other fields
}
await prisma.subscription.create({
data: draftSub
})
...some other test stuff
})
})
And the second one:
describe("Test 2", () => {
it("should do something", async () => {
const draftSub = {
userId: await findMinUniqueUserId()
...some other fields
}
await prisma.subscription.create({
data: draftSub
})
...some other test stuff
})
})
Sometimes i'm getting an error:
Unique constraint failed on the fields: (`userId`)
I've heard that each of test suit (describe block) works on seperate worker thread, i was trying to prepare some kind of singleton class, that can helps me but i think each instance of class is creating in separete worker thread, so generated userId is not unique.
This is what i was trying with singleton class:
export class UserIdManager {
private static instance: UserIdManager
private static userIdShiftBeforeDatabaseCall = 0
private static minFreeUserIdAfterDatabaseCall = 0
private constructor() {
return;
}
private static async init() {
this.minFreeUserIdAfterDatabaseCall = await findMinUniqueUserId();
}
public static async reserveMinFreeUserId() {
let minFreeUserId = UserIdManager.userIdShiftBeforeDatabaseCall;
UserIdManager.userIdShiftBeforeDatabaseCall++;
if (!UserIdManager.instance) {
UserIdManager.instance = new UserIdManager();
await this.init();
}
minFreeUserId += UserIdManager.minFreeUserIdAfterDatabaseCall;
return minFreeUserId;
}
}
But i realize that it doesn't help me with multithreading. I've used this, but with the same result:
....
const draftSub = {
userId: await UserIdManager.reserveMinFreeUserId()
...some other fields
}
....
So, the question is how to generate unique number for each test. When i pass --runInBand option to jest everything is working correctly, but it takes much more time.
What you are using is the typical MAX()+1 method of assigning unique values. Unfortunately this is a virtual guarantee you will get duplicate values for your unique value. This is a result of the Multi-Version Concurrency Control (MVCC) nature of Postgres. In a MVCC database the actions taken by one session cannot be seen by another session until the first session commits. Thus when multiple sessions access max()+1 they each get the same result. The first one to commit succeeds, the second fails. The solution to this is creating a sequence and let Postgres assign the unique value, it will not assign the same value twice regardless how many sessions access the sequence concurrently. The cost however being your values will contain gaps - accept it, get over it, and move on. You can have the sequence generated by defining your userid as a generated identity (Postgres10 or later) or as serial for older versions.
create table subscriptions ( id generated always as identity ...) -- for versions Postgres 10 or later
or
create table subscriptions ( id serial ...) -- for versions prior to Postgers 10
With either of those in place get rid of your findMinUniqueUserId function. You may also want to look into insert...returning... functionality

Cloud Functions & Firestore: Iterate over document.data

The database structure looks like this:
User {id}
Settings (Collection)
device_uids(document)
{device_uid_1}: Boolean
(...)
{device_uid_n}: Boolean
I want to get the document and access all of the device_uids within that document.
I tried like this, however the console logs, that forEach is not definded:
const settings_ref = admin.firestore().collection('User').doc(uid).collection('Settings').doc('device_uids');
settings_ref.get()
.then(snap =>{
let uids = snap.data();
uids.array.forEach(element => {
let device = element.key;
if(device != device_uid){
//GO ON
}
});
})
How can I access the values individually?
You don't have a field called array in your document, so uids.array will always be undefined. If you just want to iterate all the properties of the document, it's just like iterating all the properties of a plain old JavaScript object:
const data = snap.data();
for (const key in data) {
const value = data[key];
// now key and value are the property name and value
}

Resources