I'm fetching a few thousand objects from Core Data and I only want to return those that have at least 1 object related to it.
When I use a predicate similar to the following, it takes a very long time to fetch the objects. Around 5-8 seconds:
NSPredicate(format: "relationName.#count > 0")
Is there a more efficient way of performing this fetch, or should I cache the value in the object for fast lookup (i.e. a hasRelatedObjects attribute).
If caching is the best route, I don't believe it trivial. If I modify my Tag object for example, in the willSave I can grab the relation count and store it in my new attribute. However if a related object adds the tag to itself on it's side of the relationship, the Tag object never changes and therefore the willSave won't get called.
How can I make sure that whether you call myTag.addRelatedObject(obj) (myTag object is updated) or myObj.addRelatedTag(myTag) (myObj is updated), that the value is cached?
First, let's just do a little bit of prototyping to see what that fetch is doing. I assume you are using an SQLite store.
I hacked a quick model, similar to what you described.
I defined an Entity that had a to-many relationship to Subentity, where that Subentity had a to-one inverse relationship.
Now, I was testing in the simulator, so I created a database with 10mm entities. Each time a new entity was created, it had about a 2% chance of having at least one subentity created for it. Each entity so selected randomly got between 1 and 10 subentities.
Thus, I ended up with a database with 10,000,000 Entity objects, and 1,101,223 Subentity objects, with 199,788 Entity objects having at least one Subentity in its relationship.
For the most simple fetch request (the same as the one in your example), we get this code...
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:#"Entity"];
fetchRequest.predicate = [NSPredicate predicateWithFormat:#"subentities.#count != 0"];
NSArray *results = [moc executeFetchRequest:fetchRequest error:NULL];
and the generated SQL, along with how much time it took to do the fetch.
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZNAME, t0.ZSUBCOUNT
FROM ZENTITY t0 WHERE (SELECT COUNT(t1.Z_PK) FROM ZSUBENTITY t1
WHERE (t0.Z_PK = t1.ZENTITY) ) <> ?
CoreData: annotation: sql connection fetch time: 17.9598s
CoreData: annotation: total fetch execution time: 17.9657s for 199788 rows.
If you know much about SQL, you can see that the query is anything but optimal. There is way too much going on underneath on both tables.
If we simply add a cache for the number of relationships, we get this result (note the table is not indexed on count).
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:#"Entity"];
fetchRequest.predicate = [NSPredicate predicateWithFormat:#"subcount != 0"];
NSArray *results = [moc executeFetchRequest:fetchRequest error:NULL];
we then get these results...
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZNAME, t0.ZSUBCOUNT
FROM ZENTITY t0 WHERE t0.ZSUBCOUNT <> ?
CoreData: annotation: sql connection fetch time: 1.5795s
CoreData: annotation: total fetch execution time: 1.5838s for 199788 rows.
Now, let's see what happens if we index the subcount field.
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZNAME, t0.ZSUBCOUNT
FROM ZENTITY t0 WHERE t0.ZSUBCOUNT <> ?
CoreData: annotation: sql connection fetch time: 1.5749s
CoreData: annotation: total fetch execution time: 1.5788s for 199788 rows.
Hmmm. Not much better. What if we change the predicate slightly...
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZNAME, t0.ZSUBCOUNT
FROM ZENTITY t0 WHERE t0.ZSUBCOUNT > ?
CoreData: annotation: sql connection fetch time: 0.7805s
CoreData: annotation: total fetch execution time: 0.7843s for 199788 rows.
Now, that took half the time. I'm not exactly sure why, because even if the slower path did two binary searches, there are no records with a value less than 0.
And, I'd expect a much better improvement, based on the fact that with sorted indexes, it should be able to do a binary search, which should be much better than half the speed of a complete linear scan.
Anyway, it does show that it can get faster than that.
Just to see what our lower bound is, we can do this...
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:#"Test"];
fetchRequest.fetchLimit = 199788;
NSArray *results = [moc executeFetchRequest:fetchRequest error:NULL];
which gives these results, and about the best we can expect to grab that many records since it basically does no search.
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZNAME, t0.ZSUBCOUNT
FROM ZENTITY t0 LIMIT 199788
CoreData: annotation: sql connection fetch time: 0.1284s
CoreData: annotation: total fetch execution time: 0.1364s for 199788 rows.
Now, if we only care about whether they are empty or not, and we don't care about the actual count, we can make our cached count be a boolean instead, which is always either 0 or 1.
By taking this approach, our fetch with predicate
fetchRequest.predicate = [NSPredicate predicateWithFormat:#"subcount > 0"];
yields
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZNAME, t0.ZSUBCOUNT
FROM ZENTITY t0 WHERE t0.ZSUBCOUNT > ?
CoreData: annotation: sql connection fetch time: 0.5312s
CoreData: annotation: total fetch execution time: 0.5351s for 199788 rows.
Changing the predicate back to this
fetchRequest.predicate = [NSPredicate predicateWithFormat:#"subcount != 0"];
yields
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZNAME, t0.ZSUBCOUNT
FROM ZENTITY t0 WHERE t0.ZSUBCOUNT <> ?
CoreData: annotation: sql connection fetch time: 1.5619s
CoreData: annotation: total fetch execution time: 1.5657s for 199788 rows.
And this one
fetchRequest.predicate = [NSPredicate predicateWithFormat:#"subcount == 1"];
yields
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZNAME, t0.ZSUBCOUNT
FROM ZENTITY t0 WHERE t0.ZSUBCOUNT = ?
CoreData: annotation: sql connection fetch time: 0.5332s
CoreData: annotation: total fetch execution time: 0.5366s for 199788 rows.
So, there is still some meat on that bone, but I'll let you have some fun.
OK, so given that we want to cache the changes, how can we accomplish this?
Well, the easiest way is to just provide a custom method that gets used each time a relationship is changed. However, it then requires that all changes go through that, and there's always the chance that some piece of code manipulates the object outside of the special API.
Well, one way to notice that the computed value needs to be update is when the object saves. You can override willSave and make any necessary change there. You could also observe the context-will-save notification and do the work in there.
To me, the main problem with that approach is that the "will save" notifications happen before validation and merging with the persistent store. Either of these processes could change the data, and there are some tricky merge issues that could cause problems.
The only way to really ensure that validations and merging has been taken core of is by hooking into the validation phase.
Unfortunately, this approach is strongly discouraged by Apple documentation. I have had good success with this pattern though.
- (BOOL)validateSubcount:(id*)ioValue error:(NSError**)outError
{
NSUInteger computedValue = [*ioValue unsignedIntegerValue];
NSUInteger actualValue = computedValue;
NSString *key = #"subentities";
if ([self hasFaultForRelationshipNamed:key]) {
if (self.changedValues[#"subcount"]) {
if (has_objectIDsForRelationshipNamed) {
actualValue = [[self objectIDsForRelationshipNamed:key] count];
} else {
actualValue = [[self valueForKey:key] count];
}
}
} else {
actualValue = [[self valueForKey:key] count];
}
if (computedValue != actualValue) {
*ioValue = #(actualValue);
}
return YES;
}
This gets called automatically when saving, and you can call it manually (via validateValue:forKey:error:) from objects-did-change notification (or anyplace else for that matter) if you want more "frequent" consistency not just when saving.
On your question about changing the to-one relationship; core data will handle the inverse relationship properly. Furthermore, all objects involved will have appropriate changes reflected.
Specifically, if you you change a subentity's to-one relationship. You will now have three updated objects: the subentity itself, the entity that used to be at the other end of the relationship, and the entity that is now at the other end of the relationship.
You certainly have defined reverse relationships, right? So the didSet handler on your relationship should be called as well even it is changed from the other side.
Indeed, I think willSave should also be called. Did you verify that it isn't?
I have 2 tables which are linked between themselves with many-to-many relationship.
SQL equivalent doesn't exist because it requires the third table to split a many-to-many relationship to two one-to-many relationships.
For example, I have two core data entities: category with a property (an array of items) and item with a property (an array of categories).
I need to get all the categories except of "empty" categories (when there is no item of this category).
My current temporary solution looks like an incorrect one. I use NSFetchRequest to get all the categories. Then I delete from this array all the categories with an empty item array manually using for-each.
In order to get all categories except the empty ones you could just use a predicate like this:
NSFetchRequest *fetchRequest = [NSFetchRequest fetchRequestWithEntityName:#"Category"];
fetchRequest.predicate = [NSPredicate predicateWithFormat:#"items.#count > 0"];
NSArray *categories = [context executeFetchRequest:fetchRequest error:NULL];
My solution:
[NSPredicate predicateWithFormat:#"ANY items != nil"]
I am trying to write a fetch routine for data from Core Data. I have managed to fetch all the data in the database and display this in a table view but, how do you fetch only some of the data based on a query?
for example, in SQL:
Select * from DB where name = 'bob';
you'll want to use NSPredicate for that. something along these lines
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"(name == %#)", name];
[fetchRequest setPredicate:predicate];
I have an expense tracking iOS Application using Core Data Model:
Scenario:
-> An abstract parent entity named "Money" with attributes "vendor", "date", "amount" and so on. The two sub-entities "Expense" and "Income" inherit from the parent entity "Money". Then there are other entities such as "Category" and "subCategory" with their attributes. Total as of now: 5 entities (Money, Expense, Income, Category and Subcategory) in my data model.
Question: What I want to achieve is to track expenses daily, per week, bi-weekly, monthly and yearly. I am thinking to make an entity say "Months" with 12 attributes ( Jan - Dec - > month names) but isn't that too much complicated?
Thoughts?
I have a Table-view and using NSFetchedResultsController to fill my table-view with the mix od expenses and Incomes.
You have to create an NSFetchRequest with an NSPredicate that filters for the appropriate time period. Let's say you want to list all expenses between two dates your could implement a method that returns an NSFetchRequest that filters for this period:
- (NSFetchRequest *)filteredFetchRequestForEntity:(NSString *)entityName startDate:(NSDate *)startDate endDate:(NSDate *)endDate
{
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"(date >= %#) AND (date <= %#", startDate, endDate];
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
NSEntityDescription *entity = [NSEntityDescription entityForName:entityName inManagedObjectContext:self.managedObjectContext];
[fetchRequest setEntity:entity];
[fetchRequest setPredicate:predicate];
return fetchRequest;
}
You could then use this fetchRequest for example in an NSFetchedResultsController
similar to my answer in your other question:
CoreData and TableViews
Then all you'd have to do is create the NSDate objects for start- and endDate for your desired time period and pass them to this method.
Setup:
I have a Core Data object A that has a to-many relation to B. Call the relation "items". So, a.items returns all B-s associated with A.
Now, I have a manually composed NSSet "itemSet" of B objects.
I want to do the following:
return all A objects whose "items" relation exactly matches itemSet
How do I construct a predicate for that? I’ve tried this:
NSPredicate *predicate = [NSPredicate predicateWithFormat:
#"(ALL items in %#)", itemSet];
But that just gives me Unsupported predicate (null).
This:
NSPredicate *predicate = [NSPredicate predicateWithFormat:
#"(items in %#)", itemSet];
tells me unimplemented SQL generation for predicate. Interesting, but not helpful.
So what’s the right way to filter the relation with a set?
The following predicate could work:
[NSPredicate predicateWithFormat:#"(items.#count == %d) AND (SUBQUERY(items, $x, $x IN %#).#count == %d)",
itemSet.count, itemSet, itemSet.count];
The predicate checks first that the number of items is equal to the size of the given itemSet, and then checks that the number of items which are member of itemSet is also equal to the size of itemSet. If both are true, then items must be equal to itemSet.
Have you tried:
NSPredicate *predicate = [NSPredicate predicateWithFormate:#"items == %#", itemSet];
Alternatively, pull out a subset with a simpler predicate and filter them outside of the fetch request. i.e.
Set a predicate for the number of items in the relationship to the be the same as the number of items in your comparison set.
Fetch the results
Filter these results to show only the ones where the sets contain the same items.