So my app updates the information inside locally when i call UpdateSync on the table but it doesn't update the online database? Am I doing something wrong?
IMobileServiceSyncTable<Models.About_user> about_user_table;
Update_my_table(Object Item)
{
Models.About_user About_user = (Models.About_user)Item;
await about_user_table.UpdateAsync(About_user);
IMobileServiceSyncTable<Models.About_user> about_user_table;
}
So All I had to is put that on the bottom of my updatesync. Note: this didnt work unless I explicitly had a variable in my about_user model called version like [JsonProperty(PropertyName = "Version")]
public string Version { get; set; }
await about_user_table.UpdateAsync(About_user);
try
{
await Client.SyncContext.PushAsync();
/// await about_user_table.PullAsync("all About_user", about_user_table.CreateQuery());
}
catch (MobileServicePushFailedException ex)
{
if (ex.PushResult != null)
{
foreach (var error in ex.PushResult.Errors)
{
await ResolveConflictAsync(error,"about_user_table");
}
}
}
async Task ResolveConflictAsync(MobileServiceTableOperationError error, string table_name)
{
var serverItem = error.Result.ToObject<About_user>();
var localItem = error.Item.ToObject<About_user>();
// Note that you need to implement the public override Equals(TodoItem item)
// method in the Model for this to work
if (serverItem.Equals(localItem))
{
// Items are the same, so ignore the conflict
await error.CancelAndDiscardItemAsync();
return;
}
// Client Always Wins
localItem.Version = serverItem.Version;
await error.UpdateOperationAsync(JObject.FromObject(localItem));
}
According to your code, you are using the sync table (IMobileServiceSyncTable), for the UpdateAsync operation, it would update your local SQLite database. In order to update your online database, you need to execute the Push operation by using the following code:
await Client.SyncContext.PushAsync();
Note: when executing the push operation, you may need to handle Conflict Resolution. For more details, you could refer to adrian's book about Handling Conflict Resolution and An Offline Client.
Moreover, you could use client.GetTable<Model>() for constructing an online table and do CUD changes against your online table. For more details, you could refer to here. Additionally, you could follow here about offline synchronization.
Related
I was trying to retrieve cached data (redis) from redis using node js which was set using a .Net core 3.1 application
The code snippet which I was used to set cache is as below
nuget package : Microsoft.Extensions.Caching.StackExchangeRedis v 5.0.0 (Eventhough this is deprecated this is the version which is compatible with .Net core 3.1)
public async Task SetCache(dynamic data, string cacheKey)
{
try
{
if (_redisConfig.EnableCache == true)
{
string cachedDataString = JsonSerializer.Serialize(data);
var dataToCache = Encoding.UTF8.GetBytes(cachedDataString);
// Add the data into the cache
cacheKey = _redisConfig.Env + "_" + cacheKey;
await _cache.SetAsync(cacheKey, dataToCache, _options);
}
}
catch (Exception)
{
return;
}
}
Below is the code I used to retrieve cache using node js
npm package: redis: "^4.5.1",
exports.getRedisData = async (key) => {
try {
return await redisClient.get(key);
} catch (error) {
console.log(error);
return null
}
}
after executing the above solution the result I got is,
message:
'WRONGTYPE Operation against a key holding the wrong kind of value'
Snap shot of the above error
Can anyone give me a clue to fix this ?
With the help of #JsonPan's answer, I was able to retrieve the data from Redis with a very simple change in my code. Here I have added my solution for anyone who may get the same issue as me.
By using hGet instead of get I was able to retrieve the Hashed data. (hGet is used to get Hashed data while get is used to retrieve string data)
exports.getRedisData = async (key) => {
try {
return await redisClient.hGet(key,"data");
} catch (error) {
console.log(error);
return null
}
}
I have a client-side form that can create a document upon submission. I want to see if one of the input fields already exists on a Document in the DB though. This would then alert the user and ask them if they want to continue creating the record.
Client-side event
Template.createDoc.events({
'click button[type=submit]'(e, template) {
//This particular example is checking to see if a Doc with its `name` property set to `value` already exists
const value = $('#name');
const fieldName = 'name';
const exists = Meteor.call('checkIfFieldExistsOnDoc', fieldName, value);
if (exists) {
if (confirm(`Doc with ${value} as its ${fieldName} already exists. Are you sure you want to continue creating Doc?`) {
//db.Docs.insert....
}
}
}
});
Server-side Meteor Method
'checkIfFieldExistsOnDoc'(field, val) {
if (this.isServer) {
this.unblock();
check(field, String);
check(val, String);
if (!this.userId) {
throw new Meteor.Error('not-authorized', 'You are not authorized.');
}
const findObj = {};
findObj[field] = val;
const fieldsObj = {};
fieldsObj[fieldsObj] = 1;
const doc = Docs.findOne(findObj, {fields: fieldsObj});
return doc;
}
},
My issue is that the client-side code always gets undefined back when calling the Server method. I now understand why, however, I'm not keen on wrapping all of my subsequent client-code into a callback yet.
So - any other ideas on how I can attempt to do this simple feature?
Also - I was thinking of having the client-side page's onCreated do a 1-time server call to get ALL names for all Docs, storing this in memory, and then doing the check upon form submission using this. Obviously, this is inefficient and not-scalable, although it would work
Meteor.call in the client side is always an async call. Then you need implement a callback.
See docs: https://docs.meteor.com/api/methods.html#Meteor-call
Meteor.call('checkIfFieldExistsOnDoc', fieldName, value, function(error, result) {
if (result) {
if (confirm(`Doc with ${value} as its ${fieldName} already exists. Are you sure you want to continue creating Doc?`) {
//db.Docs.insert....
}
}
});
On the client, you can wrap any Meteor.call with a Promise and then use it with async/await. There are some packages on Atmosphere that do this for you to.
I've used this package for years: https://atmospherejs.com/deanius/promise
On the client I often just use await Meteor.callPromise() which returns a response nicely.
Here are a couple of the best write-ups on the many options available to you:
https://blog.meteor.com/using-promises-on-the-client-in-meteor-fb4f1c155f84
https://forums.meteor.com/t/meteor-methods-return-values-via-promise-async/42060
https://dev.to/jankapunkt/async-meteor-method-calls-24f9
I'm using .Net Core 3.1 and I want to insert bulk data in the background, so I don't need my http request waiting for it "like fire and forget"
So I tried the following code
public object myFunction(){
Task.Factor.StartNew(() => {
_context.BulkInsertAsync(logs);
});
return data;
}
But nothing is happend, no data saved in database
is after my data returned my _context and logs will be null, so the process is filed?
or there is any another method to insert my data and don't wait for it
Note: the background task working if I replace insertion statment with sending mail or any other thing
Solved:
Thanks #Peter , I solved it using
Task.Run(async () => await _context.BulkInsertAsync(logs));
Task.Factory.StartNew or TaskFactory.StartNew cannot accept async delegates (Func<Task>), so you should use Task.Run instead which is indeed has an overload with Func < Task >. You would have to use await await or Unwrap against StartNew to get the same behaviour as with Run. Please read Stephen Toub's excellent blog post.
public object myFunction(){
Task.Run(async () => await _context.BulkInsertAsync(logs));
return data;
}
This is how I would implement it. This assumes the response is not important.
public NotImportantResult ProcessBulkData()
{
myFunctionAsync();
return new NotImportantResult()
}
private static async void myFunctionAsync()
{
await Task.Factory.StartNew(() => new MyBulkProccessor.BulkInsertAsync(logs));
}
I am using Mongoose to access to my database. I need to use transactions to make an atomic insert-update.
95% of the time my transaction works fine, but 5% of the time an error is showing :
"Given transaction number 1 does not match any in-progress transactions"
It's very difficult to reproduce this error, so I really want to understand where it is coming from to get rid of it.
I could not find a very clear explanation about this type of behaviour.
I have tried to use async/await key words on various functions. I don't know if an operation is not done in time or too soon.
Here the code I am using:
export const createMany = async function (req, res, next) {
if (!isIterable(req.body)) {
res.status(400).send('Wrong format of body')
return
}
if (req.body.length === 0) {
res.status(400).send('The body is well formed (an array) but empty')
return
}
const session = await mongoose.startSession()
session.startTransaction()
try {
const packageBundle = await Package.create(req.body, { session })
const options = []
for (const key in packageBundle) {
if (Object.prototype.hasOwnProperty.call(packageBundle, key)) {
options.push({
updateOne: {
filter: { _id: packageBundle[key].id },
update: {
$set: {
custom_id_string: 'CAB' + packageBundle[key].custom_id.toLocaleString('en-US', {
minimumIntegerDigits: 14,
useGrouping: false
})
},
upsert: true
}
}
})
}
}
await Package.bulkWrite(
options,
{ session }
)
for (const key in packageBundle) {
if (Object.prototype.hasOwnProperty.call(packageBundle, key)) {
packageBundle[key].custom_id_string = 'CAB' + packageBundle[key].custom_id.toLocaleString('en-US', {
minimumIntegerDigits: 14,
useGrouping: false
})
}
}
res.status(201).json(packageBundle)
await session.commitTransaction()
} catch (error) {
res.status(500).end()
await session.abortTransaction()
throw error
} finally {
session.endSession()
}
}
I expect my code to add in the database and to update the entry packages in atomic way, that there is no instable database status.
This is working perfectly for the main part, but I need to be sure that this bug is not showing anymore.
You should use the session.withTransaction() helper function to perform the transaction, as pointed in mongoose documentation. This will take care of starting, committing and retrying the transaction in case it fails.
const session = await mongoose.startSession();
await session.withTransaction(async () => {
// Your transaction methods
});
Explanation:
The multi-document transactions in MongoDB are relatively new and might be a bit unstable in some cases, such as described here. And certainly, it has also been reported in Mongoose here. Your error most probably is a TransientTransactionError due to a write-conflict happening when the transaction is committed.
However, this is a known and expected issue from MongoDB and these comments explain their reasoning behind why they decided it to be like this. Moreover, they claim that the user should be handling the cases of write conflicts and retrying the transaction if that happens.
Therefore, looking at your code, the Package.create(...) method seems to be the reason why the error gets triggered, since this method is executing a save() for every document in the array (from mongoose docs).
A quick solution might be using Package.insertMany(...) instead of create(), since the Model.insertMany() "only sends one operation to the server, rather than one for each document" (from mongoose docs).
However, MongoDB provides a helper function session.withTransaction() that will take care of starting and committing the transaction and retry it in case of any error, since release v3.2.1. Hence, this should be your preferred way to work with transactions in a safer way; which is, of course, available in Mongoose through the Node.js API.
The accepted answer is great. In my case, I was running multiple transactions serially within a session. I was still facing this issue every now and then. I wrote a small helper to resolve this.
File 1:
// do some work here
await session.withTransaction(() => {});
// ensure the earlier transaction is completed
await ensureTransactionCompletion(session);
// do some more work here
await session.withTransaction(() => {});
Utils File:
async ensureTransactionCompletion(session: ClientSession, maxRetryCount: number = 50) {
// When we are trying to split our operations into multiple transactions
// Sometimes we are getting an error that the earlier transaction is still in progress
// To avoid that, we ensure the earlier transaction has finished
let count = 0;
while (session.inTransaction()) {
if (count >= maxRetryCount) {
break;
}
// Adding a delay so that the transaction get be committed
await new Promise(r => setTimeout(r, 100));
count++;
}
}
Is it possible to query firestore documents by updateTime. The field that is available from the document snapshot as doc.updateTime and use it in a where query?
I am using the node.js sdk.
As far as I know there is no way to query on the metadata that Firestore automatically maintains. If you need to query the last update date, you will need to add a field with that value to the document's data.
I really need to query Firebase on document _updateTime, so I wrote a function that copies that hidden internal timestamp to a queryable field. This took some work to figure this out, so I am posting the complete solution. (Technically, this is "Cloud Firestore" rather then "Realtime Database".)
This is done using Firebase Functions, which itself took some tries to get working. This tutorial was helpful:
https://firebase.google.com/docs/functions/get-started
However, on Windows 10, the only command line that worked was the new Bash shell, available since about 2017. This was something of a runaround to install, but necessary. The GIT Bash shell, otherwise very useful, was not able to keep track of screen positions during Firebase project setup.
In my example code, I have left in all the 'console.log' statements, to show detail. Not obvious at first was where these logs go. They do not go to the command line, but to the Firebase console:
https://console.firebase.google.com/u/0/
under (yourproject) > Functions > Logs
For testing, I found it useful to, at first, deploy only one function (this is in the CLI):
firebase deploy --only functions:testFn
Below is my working function, heavily commented, and with some redundancy for illustration. Replace 'PlantSpp' with the name of your collection of documents:
// The Cloud Functions for Firebase SDK to create Cloud Functions and setup triggers.
const functions = require('firebase-functions');
// The Firebase Admin SDK to access the Firebase Realtime Database.
const admin = require('firebase-admin');
admin.initializeApp();
// Firestore maintains an interal _updateTime for every document, but this is
// not queryable. This function copies that to a visible field 'Updated'
exports.makeUpdateTimeVisible = functions.firestore
.document('PlantSpp/{sppId}')
.onWrite((sppDoc, context) => {
console.log("Event type: ", context.eventType);
// context.eventType = 'google.firestore.document.write', so cannot use
// to distinguish e.g. create from update
const docName = context.params.sppId // this is how to get the document name
console.log("Before: ", sppDoc.before); // if a create, a 'DocumentSnapshot',
// otherwise a 'QueryDocumentSnapshot'
// if a create, everything about sppDoc.before is undefined
if (typeof sppDoc.before._fieldsProto === "undefined"){
console.log('document "', docName, '" has been created');
// set flags here if desired
}
console.log("After: ", sppDoc.after); // if a delete, a 'DocumentSnapshot',
// otherwise a 'QueryDocumentSnapshot'
// if a delete, everything about sppDoc.after is undefined
if (typeof sppDoc.after._fieldsProto === "undefined"){
console.log('document "', docName, '" has been deleted');
// other fields could be fetched from sppDoc.before
return null; // no need to proceed
}
console.log(sppDoc.after.data()); // the user defined fields:values
// inside curly braces
console.log(sppDoc.after._fieldsProto); // similar to previous except with
// data types, e.g.
// data() has { Code: 'OLDO',...
// _fieldsProto has { Code: { stringValue: 'OLDO' },...
const timeJustUpdated = sppDoc.after._updateTime; // this is how to get the
// internal nonqueryable timestamp
console.log(timeJustUpdated);
// e.g. Timestamp { _seconds: 1581615533, _nanoseconds: 496655000 }
// later: Timestamp { _seconds: 1581617552, _nanoseconds: 566223000 }
// shows this is correctly updating
// see if the doc has the 'Updated' field yet
if (sppDoc.after._fieldsProto.hasOwnProperty('Updated')) {
console.log("doc has the field 'Updated' with the value",
sppDoc.after._fieldsProto.Updated);
console.log("sppDoc:", sppDoc);
const secondsInternal = timeJustUpdated._seconds;
console.log(secondsInternal, "seconds, internal timestamp");
const secondsExternal = sppDoc.after.data().Updated._seconds;
console.log(secondsExternal, "seconds, external timestamp");
// Careful here. If we just update the externally visible time to the
// internal time, we will go into an infinite loop because that update
// will call this function again, and by then the internal time will have
// advanced
// the following exit will not work:
if (secondsInternal === secondsExternal) return null; // will never exit
// instead, allow the external time to lag the internal by a little
const secondsLate = secondsInternal - secondsExternal;
if (secondsLate < 120) { // two minutes sufficient for this purpose
console.log("the field 'Updated' is", secondsLate,
"seconds late, good enough");
return null;
}
console.log("the field 'Updated' is", secondsLate,
"seconds late, updating");
// return a promise of a set operation to update the timestamp
return sppDoc.after.ref.set({
Updated: timeJustUpdated
}, {merge: true}); // 'merge' prevents overwriting whole doc
// this change will call this same function again
} else { // field 'Updated' does not exist in the document yet
// this illustrates how to add a field
console.log("doc does not have the field 'Updated', adding it now.");
// return a promise of a set operation to create the timestamp
return sppDoc.after.ref.set({
Updated: timeJustUpdated
}, {merge: true}); // 'merge' prevents overwriting the whole doc
// this change will call this same function again
}
});
True, there is no query for time created/modified, but when you fetch a document those fields exist in the payload. You have:
payload.doc['_document'].proto.createTime and payload.doc['_document'].proto.updateTime
Sure it's not good practice to rely on private fields, so will prolly need ongoing adjustments as Firestore changes its data model, but for now, for my uses, it gets me this otherwise un-query-able data.