I am using node-storage in the following code to store a value in a file, however when I create a new storage object changes from another storage object are not yet saved. I need a way to save the changes before creating the new storage object.
Below is a program called code.js which I am running like so in the console: node code.js. If you run it you will see that the first time it is run the key value pair doesn't yet exist however it does exist the second time.
key = "key"
storage = require('node-storage')
const store1 = new storage("file")
const store2 = new storage("file")
store1.put(key,'val')
console.log(store2.get(key))
My motivation for this is that I want to be able to have a function called "set" which takes a key and a value and sets the key value pair in a dictionary of values that is store in a file. I want to be able to refer to this dictionary later, with for example a 'get' function, and have the changes present.
I am thinking there might be a function called "save" or something similar that applies the changes to the file. Is there such a function or some other solution?
node-storage saves the changes in the dictionary to disk after every call to put or remove. This is not the issue.
Your problem is that the dictionary in store2 has not been updated with the new properties. node-storage only loads the file from disk when the object is first created.
My suggestion would be to only have one instance of storage per file.
However, if this is not possible, then you might want to consider updating store2's cache before you get the property. This can be done using:
store2.store = store2._load();
This may not be the best for performance, as _load loads the entire file from disk synchronously every time it is called, so try to limit its use.
Related
I am creating a push notification server, and I am storing subscriptions in a JSON file, called clients.json. This is working just fine, and I am able to use fs.writeFile('clients.json', data, callback), which is working. The problem I am having is that if there are multiple instances of a subscription in the JSON file, the server sends a push event multiple times, for every instance of the subscription.
What I am trying now is to only write the subscription object to the file if it does not already exist in the file. I have tried the following:
if (!JSON.parse(fs.readFileSync('clients.json')).endpoints.includes(subscription)) {
clients.endpoints.push(subscription);
fs.writeFile('clients.json', JSON.stringify(clients), err=>{if(err){console.log(err)}});
}
Seems correct to me, but it doesn't seem to care about my condition, as the code block runs every time and the subscription is inserted to the file many times.
If it helps, this is clients.json:
{
"endpoints":[
/* Client subscriptions end up in this array */
]
}
Any help would be greatly appreciated.
In js only primitive types are compared by their values. If subscription is an object of any kind, it will return false when compared with another object, even when it contains the same data, because they are compared by object they reference.
I have a simple Core Data entity Story that occasionally I update with the latest data from a network call. This network call sometimes updates many, many stories instances, so I run an NSBatchInsertRequest, shown below. (The other reason I'm using a batch insert is that many stories might need to be added to the persistent store.)
The problem is a user can have already marked a Story as a favorite. When they do that, I set story.isFavorite = true on the main thread and save viewContext.
However, when the batch insert occurs it overwrites story.isFavorite, setting it back to false, even though I'm using NSMergeByPropertyObjectTrumpMergePolicy on both the batch insert and view contexts. I am not touching story.isFavorite in the batch insert handler either so I don't expect that property to be overwritten.
I thought the benefit of a batch insert with this merge policy was to avoid first fetching + then manually updating changed properties + finally saving. What is the right way to avoid changing property values in an NSBatchInsertRequest?
Story
#objc(Story)
public class Story: NSManagedObject {
#NSManaged public var title: String?
#NSManaged public var storyURL: URL?
#NSManaged public var updatedTime: Date?
#NSManaged public var isFavorite: Bool // <- the problem property
}
Batch insert
container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
container.viewContext.automaticallyMergesChangesFromParent = false
let context = NSManagedObjectContext(concurrencyType: .privateQueueConcurrencyType)
context.parent = container.viewContext
context.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
context.perform {
let batchInsert = NSBatchInsertRequest(entity: Story.entity(), managedObjectHandler: { managedObject in
let story = managedObject as! Story
let storyResponse = downloadedStories[I]
// Update story with latest response data BUT don't modify story.isFavorite.
story.title = storyResponse.title
story.storyURL = storyResponse.storyURL
story.updatedTime = storyResponse.updatedTime
// ...
})
let result = try context.execute(batchInsert) as? NSBatchInsertResult
if let insertedIDs = result?.result as? [NSManagedObjectID] {
// Merge changes into parent context. Skip save() because not needed for batch insert.
NSManagedObjectContext.mergeChanges(fromRemoteContextSave: [NSInsertedObjectsKey: insertedIDs], into: [container.viewContext])
}
}
Edit
The Story entity does have a unique value constraint using attribute storyURL.
Update after Michael Tsai's answer
By making the Story entity attribute isFavorite a non-Optional Boolean without a default value (it was marked as Optional before, though I'm not sure it makes a difference here) and keeping the Use Scalar Type box checked, I can confirm that existing objects in the store will not be modified (at all) with this configuration of the batch insert context.
context.persistentStoreCoordinator = container.persistentStoreCoordinator
// HOWEVER, observe that regardless of the merge policy below,
// setting `context.parent = container.viewContext` will also
// overwrite the store data!
context.mergePolicy = NSMergeByPropertyStoreTrumpMergePolicy
// NSMergeByPropertyObjectTrumpMergePolicy ignores objects in the store
// (which have the same unique constraint value, here equal `storyURL`)
// and overwrites all properties.
// To confirm that the batch insert operation does not modify
// existing Story instances (at all), first delete all instances where
// where isFavorite == false. Then load the all story data again and
// execute the NSBatchInsertRequest with this change to managedObjectHandler:
story.title = storyResponse.title + " (modified)"
You will see the missing stories get inserted back, this time with their titles having a suffix " (modified)"; but previously favorited stories
do not get modified (basically, with this setup, the batch insert won't re-insert objects).
So the isFavorite property does not get overwritten BUT neither do any properties that should be changed (because they received a new title, for example).
Therefore, if you don't want your objects to get updated, but you want completely new objects to be inserted, you can use this approach.
However, if you are expecting your objects to require updates here are some alternatives:
you may opt to run a separate update operation, maybe an NSBatchUpdateRequest after you run your batch insert in this way,
or after the batch insert, you can update certain properties in a simple loop in a (possibly background/child) context without a batch operation, which could be fine if there isn't tons of data;
lastly, you might be able to first batch insert new data to a temporary store before somehow manually merging your choice of properties with the new store, then delete the temporary store.
A simpler approach: you could fetch the all properties you want to keep unchanged before you execute the batch insert (storing them in an dictionary keyed by your object's uniqueness constraint value), and then during the batch insert set the property again.
For this approach, you will want to use a different merge policy such as NSMergeByPropertyObjectTrumpMergePolicy so that the updated object gets re-inserted into the store (make sure to fetch all properties that you don't want to lose in advance of the batch insert)
random idea: How to Save Data When Using One ManagedObjectContext and PersistentStoreCoordinator with Two Stores
I don't think it is actually possible to do a partial update with a batch insert request. It's hard to know for sure because I don't think any of this is documented except in WWDC sessions. When I first watched the 2019 session, I was excited because the presenter said:
Attributes that are optional or configured with default values can be omitted from the dictionary as well.
In the case of updating an object with unique constraint, the existing values will not be changed.
I took this to mean that:
You can omit values for new objects, and you'll get the defaults or NULL. That makes sense.
If there's an existing object and you omit a value, that value will not the changed. So you can purposely omit values to do a partial update, i.e. update other values while leaving your isFavorite alone.
But, after writing code to test this and looking at the output from com.apple.CoreData.SQLDebug, what actually seems to happen with NSMergeByPropertyObjectTrumpMergePolicy is:
If you omit a value that's required you get a validation error.
If you omit a value that's optional, it updates the row to NULL. For a Bool property in Swift, this will become false.
If you omit a value with a default value, it updates the row to the default.
This is a shame because it seems like partial updates could be implemented by having the ON CONFLICT clause only specify DO UPDATE SET for the attributes that you actually set. But (as of macOS 11) Core Data seems to always generate SQL to set all of the columns.
In summary, with batch inserts, NSMergeByPropertyObjectTrumpMergePolicy does not actually merge by property based on what's changed (like with a regular Core Data save). Rather, it either inserts a new row (if the object is absent) or overwrites all the columns but preserves the objectID (if the object was present).
NSMergeByPropertyStoreTrumpMergePolicy also doesn't merge by property. It just means to leave the stored object alone if it's already present.
Update (2021-06-24): I heard from DTS that Apple considers the current (iOS 14/macOS 11) behavior described above a bug, and that it should let you batch insert without changing omitted properties. The Radar number is 79747419.
I currently have a Timer triggered Azure Function that checks a data endpoint to determine if any new data has been added. If new data has been added, then I generate an output blob (which I return).
However, returning output appears to be mandatory. Whereas I'd only like to generate an output blob under specific conditions, I must do it all of the time, clogging up my storage.
Is there any way to generate output only under specified conditions?
If you have the blob output binding set to your return value, but you do not want to generate a blob, simply return null to ensure the blob is not created.
You're free to execute whatever logic you want in your functions. You may need to remove the output binding from your function (this is what is making the output required) and construct the connection to blob storage in your function instead. Then you can conditionally create and save the blob.
What is the difference between these Sencha Touch API functions.
Ext.getStore('myStore') and Ext.getStore('myStore').load()
I found at many places including sencha docs but could not find any appropriate answer.
Let's take a look at this:
var myStore = Ext.getStore( 'myStore' );
myStore.load();
Ext.getStore( id ) will search the StoreManager for a store with the provided id. If it finds one it will return it otherwise it will return null.
If you have a store object you can load it via store.load(); That's a function of the store.
Only getting the store via getStore does not mean that the data is up to date. To assure it you have to load the store.
Update:
Let's assume you have a localstore. You have already stored some data in it. Now the user closes the app and restarts it.
When your store is not set to autoLoad: true sencha will create the store object for you which you can access by var store = Ext.getStore( 'myLocalStore' ); This store object will NOT contain any data from the underlying localstorage. You have to load the store manually by store.load();. Now you can add some more data and sync it, so the underlying localstorage will get the new data.
I am trying to port my LWUIT application to Codename one.
I have used RMS in LWUIT and now obviously I have to transform this to Storage.
I don't understand how the Storage class works in Codename one and the documentation for codename one has nothing about either.
1) What is the structure of a storage file?
--> In J2ME RecordStore , you have records bunched together like a table. Every row, corresponds to a record. Each record has a unique record ID and you can access the record with this record id. Every record can have some data stored in it.
How does this map to Storage class?
2)I wish to store some records in my storage, how do i do it?
The documentation says:
static Storage getInstance()
Returns the storage instance or null if the storage wasn't initialized using a call to init(String) first.
--> In LWUIT it was something like Storage.init(storageName). ; However there is no init in codename one!!!. How do I open a Storage in Codename one??
3)If i try to open a storage file which does not exist, what will happen (RMS gives an exception)?
The easiest way to think about Storage is as a flat file system (without directories/folders).
When running on top of RMS this file system abstraction is mapped to the RMS database seamlessly for you.
Notice that init() for Storage in Codename One is no longer necessary, under LWUIT it only performed basic initialization and the name was usually ignored.
The Storage class has several methods:
InputStream createInputStream(String name)
Creates an input stream to the given storage source file
OutputStream createOutputStream(String name)
Creates an output stream to the storage with the given name
boolean exists(String name)
Returns true if the given storage file exists
String[] listEntries()
Lists the names of the storage files
You can use these to just store and check if data exists. However you can also store complex objects in storage without using input/output streams by using these two methods:
Object readObject(String name)
Reads the object from the storage, returns null if the object isn't there
boolean writeObject(String name, Object o)
Writes the given object to storage assuming it is an externalizable type or one of the supported types
So to simulate something like byte[] storage you can do something like this:
Vector p = new Vector();
byte[] myData = ...;
p.addElement(myData);
p.addElement(additionalData);
Storage.getInstance().writeObject("myStore", p);
Then just read it as:
Vector p = (Vector)Storage.getInstance().read("myStore");
// p will be null if nothing was written