PouchDB Push Replication does not push new documents - node.js

Am using PouchDB version 5.2.1 with nodejs version 0.10.26. Recently I encountered a problem, where out of my two different instances of PouchDB in one instance the push replication was working fine but in other instance the push replication was not happening.
I further dug deeper into the library I found that in the file pouchdb/lib/index.js a function named startNextBatch() is called. Inside the function we are checking for the condition
if (returnValue.cancelled || currentBatch) {
return;
}
In success scenario the currentBatch was undefined and hence the code didnt return back and the push happened, but in failure scenario the currentBatch was defined and return was called.The documents present in currentBatch variable are the previous documents which had got pushed to server but couldn't sync back due to some other issue and also the checkpoints for them couldnt be updated, hence the currentBatch variable was not cleared.
My question is :
1)What impact will it have if i comment the return from the code.
2)Why is push replication stopped because the variable currentBatch is defined.

After some further debugging I found out that the replication has totally stopped with status code 503 unknown database error. This has happened because there is some problem with my sync gateway and some documents takes time to sync back ,hence writing checkpoints throws 503 unknown error (am not exactly sure why checkpoints are getting messed up because some documents got synced and others got synced after some time). The PouchDB library totally aborts the replication and after that no push happens.
1) If I comment the return, the current document will get pushed but since replication is aborted there will be no checkpoints saved.
2) Push replication didnt stop because currentBatch is defined. currentBatch is defined because replication is not happenning.

Related

Multi-threaded PUPL SQL Error code 000099997 occurred in module CIPPUPFN:HA100 - UPDATE TCTL

We are on CC&B 2.6. We run base process PUPL Payment Upload on 8 threads. Occasionally the process fails on a thread. The other threads run successfully. And when we restart the job, that thread recovers and completes successfully.
The error message shown is:
SQL Error code 000099997 occurred in module CIPPUPFN:HA100 - UPDATE TCTL
Has anyone encountered this?
This problem might have been caused by a bug in the product that handles the interaction of
sessions and the ThreadlocalStorage object. Some session-related state, such
as the map of DBLocalCobolSQLProcessBackend instances was stored on
ThreadlocalStorage, when it should be on FrameworkSession.
If another session is temporarily "pushed" onto ThreadlocalStorage (e.g. via
SessionExecutable>>doInReadOnlySession()) the original session is set aside
but the map of DBLocalCobolSQLProcessBackend was improperly destroyed and could not be restored when the original session was restored.
If a subsequent db-related call from COBOL into Java requires access to an instance of DBLocalCobolSQLProcessBackend, the desired instance was missing and a new, empty one was created, which lead to a NPE in downstream code.

Core data and cloudkit sync wwdc 2019 not working for beta 3

I am trying to replicate the result of WWDC talk on syncing core data with cloud kit automatically.
I tried three approaches:
Making a new master slave view app and following the steps at in
wwdc 2019 talk, in this case no syncing happens
Downloading the sample wwdc 2019 app also in this case no symcing happens
I made a small app with a small core data and a cloud kit container in this case syncing happens but I have to restart the app. I suspected it had to do with history management so observed the NSPersistentStoreRemoteChange notification not nothing receives.
Appreciate any help.
I also played around with CoreData and iCloud and it work perfectly. I would like to list some important points that may help you go further:
You have to run the app on a real device with iCloud Acc We can now test iCloud Sync on Simulator, but it will not get notification automatically. We have to trigger manually by select Debug > Trigger iCloud Sync
Make sure you added Push Notification and iCloud capability to your app. Make sure that you don't Dave issue with iCloud container (in this case, you will see red text on iCloud session in Xcode)
In order to refresh the view automatically, you need to add this line into your Core Data Stack: container.viewContext.automaticallyMergesChangesFromParent = true.
Code:
public lazy var persistentContainer: NSPersistentCloudKitContainer = {
/*
The persistent container for the application. This implementation
creates and returns a container, having loaded the store for the
application to it. This property is optional since there are legitimate
error conditions that could cause the creation of the store to fail.
*/
let container = NSPersistentCloudKitContainer(name: self.modelName)
container.viewContext.automaticallyMergesChangesFromParent = true
container.loadPersistentStores(completionHandler: { (storeDescription, error) in
if let error = error as NSError? {
// Replace this implementation with code to handle the error appropriately.
// fatalError() causes the application to generate a crash log and terminate. You should not use this function in a shipping application, although it may be useful during development.
/*
Typical reasons for an error here include:
* The parent directory does not exist, cannot be created, or disallows writing.
* The persistent store is not accessible, due to permissions or data protection when the device is locked.
* The device is out of space.
* The store could not be migrated to the current model version.
Check the error message to determine what the actual problem was.
*/
fatalError("Unresolved error \(error), \(error.userInfo)")
}
})
return container
}()
When you add some data, normally you should see console log begin with CloudKit: CoreData+CloudKit: ..........
Sometimes the data is not synced immediately, in this case, I force close the app and build a new one, then the data get syncing.
There was one time, the data get synced after few hours :(
I found that the NSPersistentStoreRemoteChange notification is posted by the NSPersistentStoreCoordinator and not by the NSPersistentCloudKitContainer, so the following code solves the problem:
// Observe Core Data remote change notifications.
NotificationCenter.default.addObserver(
self, selector: #selector(self.storeRemoteChange(_:)),
name: .NSPersistentStoreRemoteChange, object: container.persistentStoreCoordinator)
Also ran into the issue with .NSPersistentStoreRemoteChange notification not being sent.
Code from Apples example:
// Observe Core Data remote change notifications.
NotificationCenter.default.addObserver(
self, selector: #selector(type(of: self).storeRemoteChange(_:)),
name: .NSPersistentStoreRemoteChange, object: container)
Solution for me was to not set the container as object for the notification, but nil instead. Is it not used anyway and prevents the notification from being received:
// Observe Core Data remote change notifications.
NotificationCenter.default.addObserver(
self, selector: #selector(type(of: self).storeRemoteChange(_:)),
name: .NSPersistentStoreRemoteChange, object: nil)
Update:
As per this answer: https://stackoverflow.com/a/60142380/3187762
The correct way would be to set container.persistentStoreCoordinator as object:
// Observe Core Data remote change notifications.
NotificationCenter.default.addObserver(
self, selector: #selector(type(of: self).storeRemoteChange(_:)),
name: .NSPersistentStoreRemoteChange, object: container.persistentStoreCoordinator)
I had the same problem, reason was that iCloudDrive must be enabled in your devices. Check it in the Settings of every your device
I understand this answer comes late and is not actually specific to the WWDC 19 SynchronizingALocalStoreToTheCloud Apple's sample project to which OP refers to, but I had syncing issues (not upon launch, when it synced fine, but only during the app being active but idle, which seems to be case 3 of the original question) in a project that uses Core Data + CloudKit with NSPersistentCloudKitContainer and I believe the same problems I had - and now apparently I have solved - might affect other Users reading this question in the future.
My app was built using Xcode's 11 Master-Detail template with Core Data + CloudKit from the start, so I had to do very little to have syncing work initially:
Enable Remote Notifications Background Mode in Signing & Capabilities for my target;
Add the iCloud capability for CloudKit;
Select the container iCloud.com.domain.AppName
Add viewContext.automaticallyMergesChangesFromParent = true
Basically, I followed Getting Started With NSPersistentCloudKitContainer by Andrew Bancroft and this was enough to have the MVP sync between devices (Catalina, iOS 13, iPadOS 13) not only upon launch, but also when the app was running and active (thanks to step 4 above) and another device edited/added/deleted an object.Being the Xcode template, it did not have the additional customisations / advanced behaviours of WWDC 2019's sample project, but it actually accomplished the goal pretty well and I was satisfied, so I moved on to other parts of this app's development and stopped thinking about sync.
A few days ago, I noticed that the iOS/iPadOS app was now only syncing upon launch, and not while the app was active and idle on screen; on macOS the behaviour was slightly different, because a simple command-tab triggered sync when reactivating the app, but again, if the Mac app was frontmost, no syncing for changes coming from other devices.
I initially blamed a couple of modifications I did in the meantime:
In order to have the sqlite accessible in a Share Extension, I moved the container in an app group by subclassing NSPersistentCloudKitContainer;
I changed the capitalisation in the name of the app and, since I could not delete the CloudKit database, I created a new container named iCloud.com.domain.AppnameApp (CloudKit is case insensitive, apparently, and yes, I should really start to care less about such things).
While I was pretty sure that I saw syncing work as well as before after each one of these changes, having sync (apparently) suddenly break convinced me, for at least a few hours, that either one of those modification from the default path caused the notifications to stop being received while the app was active, and that then the merge would only happen upon launch as I was seeing because the running app was not made aware of changes.
I should mention, because this could help others in my situation, that I was sure notifications were triggered upon Core Data context saves because CloudKit Dashboard was showing the notifications being sent:
So, I tried a few times clearing Derived Data (one never knows), deleting the apps on all devices and resetting the Development Environment in CloudKit's Dashboard (something I already did periodically during development), but I still had the issue of the notifications not being received.
Finally, I realised that resetting the CloudKit environment and deleting the apps was indeed useful (and I actually rebooted everything just to be safe ;) but I also needed to delete the app data from iCloud (from iCloud's Settings screen on the last iOS device where the app was still installed, after deleting from the others) if I really wanted a clean slate; otherwise, my somewhat messed up database would sync back to the newly installed app.
And indeed, a truly clean slate with a fresh Development Environment, newly installed apps and rebooted devices resumed the notifications being detected from the devices also when the apps are frontmost.So, if you feel your setup is correct and have already read enough times that viewContext.automaticallyMergesChangesFromParent = true is the only thing you need, but still can't see changes come from other devices, don't exclude that something could have been messed up beyond your control (don't get me wrong: I'm 100% sure that it must have been something that I did!) and try to have a fresh start... it might seem obscure, but what isn't with the syncing method we are choosing for our app?

"Server x timed out" during MongoDB aggregation

I have a script that periodically runs aggregation on a mongodb collection. As the dataset has grown, the amount of time it takes to aggregate has also grown. My aggregation script has recently stopped working consistently, and the error logs show:
error: { [MongoError: server <x> timed out]
name: 'MongoError',
message: 'server <x> timed out' }
I've tried debugging this, and the only pattern I can find is that this timeout seems to only occur when the aggregation takes longer than 2 minutes (it times out right around 2m). Does anyone have additional debugging tips for this? The 2-minute thing is giving me the impression that I just need to configure some timeout somewhere but I can't figure out where or if i'm just falling into a red-herring trap.
About the system configuration: This aggregation script is a node.js (v5.9.1) application running in an alpine-based docker (v1.9.1) container. It uses the mongodb node driver (v2.1.19). Single mongodb server (though this is also happening in a separate environment with a replSet) running mongod (v3.2.6)
I got the same problem for logs time aggregation. I think I have the solution for you.
I found that the option socketTimeoutMS is responsible for that.
Check your mongo_client.js default socketTimeoutMS value. For me it was 2min. Mongodb module version 2.1.18.
So just add this option into your url :
mongodb://localhost:27017/test?maxPoolSize=2&socketTimeoutMS=60000
It will set timeout to 10 mins. That does the trick for me.

Random 'ECONNABORTED' error when using sendFile in Express/Node

I have set a node server with Express middleware. I get the ECONNABORTED error randomly on some files when loading an HTML file which triggers about 10 other loads (js, css, etc.). The exact error is:
{ [Error: Request aborted] code: 'ECONNABORTED' }
Generated by this simplified code (after I tried to debug the issue):
res.sendFile(res.locals.physicalUrl,function (err) {
if (err)
console.log(err);
...
}
Many posts talk about this error resulting from not specifying the full path name. That is not the situation here. I do specify the full path and indeed the error is randomly generated. There are times when the page and all its subsequent links load perfectly and there are times when they do not. I tried to flush the cache and did not find any pattern to connect it with this.
This specific error appears to be a a generic term for socket connection getting aborted and is discussed in the context of other applications like FTP.
Having realized that the node worker threads can be increased, I tried to do so using:
process.env.UV_THREADPOOL_SIZE = 20;
However, my understanding is that even absent this, at most the file transfer may have to wait for a worker thread to be free and not get aborted. I am not talking about big files here, all files are less than 1 MB.
I have a gut feeling that this has nothing to do with node directly.
Please point to any other possibilities (node or otherwise) to handle this error. Also, any other indirect solutions? Retrying a few times could be one but that would be clumsy. EDIT: No, I cannot retry. Headers are already sent with the error!
A SIDE NOTE:
Many examples on the use of sendFile skip using the callback thereby giving the impression that it is a synchronous call. It is not. Do use the callback at all times, check for success and only then move on to the "next" middleware or take appropriate steps if the send fails for whatever reason. Not doing so can make it difficult to debug the consequences in an asynchronous environment.
See https://stackoverflow.com/a/36949631/2798152
Could it be possible that in some cases you terminate the connection by calling res.end before the asynchronous call to res.sendFile ends?
If that's not the case - can you pastebin more of your application code?
Uninstalling and Re-installing MongoDB solved this for me.
I was facing the same problem. It started happening when I had to force restart my laptop because it became unresponsive. On restarting, trying to connect to mongo server using nodejs, always threw ECONNABORTED error

Azure PushAsync() error

I encounter a wired error when use PushAsync() function to sync data from local database to Azure. f I take InsertAsync(item) action only once, the sync works wells. But if I did more than two times, the PushAsync() will throw an exception that ask me to check pushasync result.
I followed this tutorial http://azure.microsoft.com/en-us/documentation/articles/mobile-services-windows-store-dotnet-get-started-offline-data/

Resources