Firebase Realtime Database - Intermittently update() works like set() - node.js

Question

Intermittently and unpredictably, the Firebase Realtime Database update() function seems to work like the set() function. Anecdotally, it appears to happen on about 1% of update operations. But we've performed extensive logging, and will see things happen like a particular update is being pushed out to a group of users inside of a loop, we verify that correct information is being sent to all of them in the logs, and update() is being called on each record. However, the outcome we'll see is that sometimes one of the users will wind up with a record that only contains the fields we updated, and all other fields in the record get deleted, while all other users receive the update properly. Running the exact same update() operation subsequently will result in everything updating as expected. Is this a known issue? Are there any workarounds? We are running firebase-admin 6.0.0 on Node 8.14.0
Attempted multiple repeated tests of the update() function. There is no surefire way to cause this issue to reproduce, but it is happening randomly in production.
const contactsRef = admin.database().ref().child('contacts');
...
//targetUID, contactID, contactObj get passed in via PubSub
...
contactsRef.child(targetUID).child(contactID).update(contactObj);
Expected: update() should only update the record fields being passed to it.
Actual: update() seems to work like set() randomly, about 1% of the time. Any fields that are not included in the object being passed to update() are deleted from the target record in the Realtime Database.

Answer 1

It seems very unlikely that the database server behaves differently for 1% of your users. Much more likely it that there is a slight difference in the calls that that 1% of your users makes. It's hard to be certain what that difference is from the code you shared, so below is an educated guess in hopes of unblocking you quickly.
You say you do:
contactsRef.child(targetUID).child(contactID).update(contactObj);
Expected: update() should only update the record fields being passed to it.
It's a bit subtle and unfortunately you don't show how you construct contactObj. So I'll give an example. Say that you start with JSON:
"uid1": {
"name": "unknown",
"id": -1,
"full_name": "unknown",
"metadata": {
"last_seen": "20 minutes ago",
"reputation" 56
}
}
And you run this on that location:
ref.update{
"name": "miles_b",
"id": 2687721
}
In this case only the name and id properties under ref are updated. The other properties are unmodified, so you end up with:
"uid1": {
"name": "miles_b",
"id": 2687721,
"full_name": "unknown",
"metadata": {
"last_seen": "20 minutes ago",
"reputation" 56
}
}
But now say that you also want to update the metadata/reputation. You might think that this works:
ref.update{
"name": "miles_b",
"id": 2687721,
"metadata": {
"reputation": 61
}
}
But here you are telling the database to replace metadata with the object you provided. So the result is:
"uid1": {
"name": "miles_b",
"id": 2687721,
"full_name": "unknown",
"metadata": {
"reputation" 61
}
}
And this means that last_seen is now gone from the database.
To update a nested property, include its full path in the key. So:
ref.update{
"name": "miles_b",
"id": 2687721,
"metadata/reputation": 61
}
And with that, you'll keep metadata/last_seen, while updating metadata/reputation:
"uid1": {
"name": "miles_b",
"id": 2687721,
"full_name": "unknown",
"metadata": {
"last_seen": "20 minutes ago",
"reputation" 61
}
}