I tried to save NSTimeZone as transformable type, but it doesn't work seems due to it conforms to NSSecureCoding.
Is it any other alternatives?
If the TimeZone has a defined name, you can store that as a string. It may not have a name if you manually constructed the timeZone based off a GMT offset.
... So you could store the GMT offset if you wanted, but then you would lose any of the DST information associated with a named timezone.
Probably the best way is to store the data. That should contain the information necessary to reconstruct the appropriate NSTimeZone instance.
Related
I'm using IBM's Cloudant (CouchDB) data store. I'm planning on storing dates as integers in the format YYYYMMDD instead of JavaScript Dates. Is there any CouchDB functionality that I'd be missing out on by not storing them as JavaScript Dates? Any other reason I shouldn't do this?
I've read this SO Q&A: What's the best way to store datetimes (timestamps) in CouchDB? and from that there appears to be no objections to storing dates in any format. It doesn't answer what built-in functionality might be lost.
You wouldn't be losing any functionality as you would make the date useful by processing it in a Map function as either a Secondary Index/View, Search Index or part of Cloudant Query.
The only downside is that by formatting them as such, you make it more difficult on yourself to use the JavaScript Date functions to modify the date to needs within a Map function.
Storing it as a String is an option. Might be easier to handle this way than as an Integer.
I just want a confirmation from the experts.
I still don't feel confident in what I believe to be the right way of storing and treating dates in such environment.
I'm developing a little app, just for italian users.
Basically, they can create a list of entries, each having a creationDate (I am just interested in the date part, time is not useful in my scenario).
So, the user enters in the "date" form field a date in this format: 22/06/2014 represents the 22th day of June of year 2014. Then, date is parsed like that:
entryData.dateEntry = moment( $(form).find('input[name=dateEntry]').val(), 'DD-MM-YYYY' ).toDate();
Finally, my entry model is added to a backbone.js collection and stored server-side by Node.js + Express in MongoDB.
Querying Mongo for entries, I see:
2014-06-21 22:00:00 +0000
which corresponds to "dateEntry" : Date( 1403388000000 ).
Googling around, I discovered that MongoDB doesn't have the concept of timezone. All dates are stored in UTC and the date object I created before had GMT+2.
But I'm really scared... how will I get back my local timezone's date the easy way?
Next, I'll display entry data in an underscore template, this way:
<%= moment(dateEntry).format('DD/MM/YYYY') %>
And... voilà! I get my local 'italian' date back: 22/06/2014.
Now, my question: is that the right way to go?
The process is: parse dates in local timezone => store in utc => retrieve dates in local timezone. Is it a common practice?
I also thought: can't I simply avoid using timezones and storing my local (italian) time as it was utc time (2014-06-22 00:00:00)? Is that so bad?
Yes, it's a common practise to store all timestamps in UTC, and then convert it to specific timezones in the outer layers. A lot of frameworks automatically do that for you, including Rails. Let's say if going forward you start catering to other timezones as well, you will not face any problems because your DB has UTC entries. You will be saved the overhead of timezone conversions.
In case you want to save only dates, you can do that as well, but I don't see any harm in the way you do it currently - save everything in UTC. I am not sure about node.js but there would be some setting where you can specify the timezone (of Italy), and all your conversions will happen automatically. You may find this thread useful:
How can I set the default timezone in node.js?
I need to save a User model, something like:
{ "nickname": "alan",
"email": ...,
"password":...,
...} // and a couple of other fields
Today, I use a Set: users
In this Set, I have a member like user:alan
In this member I have the hash above
This is working fine but I was just wondering if instead of the above approach that could make sense to use the following one:
Still use users Set (to easily get the users (members) list)
In this set only use a key / value storage like:
key: alan
value : the stringify version of the above user hash
Retrieving a record would then be easier (I will then have to Parse it with JSON).
I'm very new to redis and I am not sure what could be the best. What do you think ?
You can use Redis hashes data structure to store your JSON object fields and values. For example your "users" set can still be used as a list which stores all users and your individual JSON object can be stored into hash like this:
db.hmset("user:id", JSON.stringify(jsonObj));
Now you can get by key all users or only specific one (from which you get/set only specified fields/values). Also these two questions are probably related to your scenario.
EDIT: (sorry I didn't realize that we talked about this earlier)
Retrieving a record would then be easier (I will then have to Parse it with JSON).
This is true, but with hash data structure you can get/set only the field/value which you need to work with. Retrieving entire JSON object can result in decrease of performance (depends on how often you do it) if you only want to change part of the object (other thing is that you will need to stringify/parse the object everytime).
One additional merit for JSON over hashes is maintaining type. 123.3 becomes the string "123.3" and depending on library Null/None can accidentally be casted to "null".
Both are a bit tedious as that will require writing a transformer for extracting the strings and converting them back to their expected types.
For space/memory consumption considerations, I've started leaning towards storing just the values as a JSON list ["my_type_version", 123.5, null , ... ] so I didn't have overhead of N * ( sum(len(concat(JSON key names))) which in my case was +60% of Redis's used memory footprint.
bear in mind: Hashes cannot store nested objects, JSON can do it.
Truthfully, either way works fine. The way you store it is a design decision you will need to make. It depends on how you want to retrieve the user information, etc.
In terms of performance, storing the JSON encoded version of the user object will use less memory and take less time for storage/retrieval. That is, JSON parsing is probably faster than retrieving each field from Redis. And, even if not, it is probably more memory efficient. The difference in performance is probably minimal anyway.
I have been searching for some sample code on how to store an NSArray in Core Data for awhile now, but haven't had any luck. Would anyone mind pointing me to some tutorial or example, or better yet write a simple sample as an answer to this question? I have read this but it doesn't show an example of how to go about implementing a transformable attribute that is an NSArray. Thanks in advance!
If you really need to do it, then encode as data. I simply created a new filed called receive as NSData (Binary data).
Then in the NSManagedObject implementation:
-(void)setReceiveList:(NSArray*)list{
self.receive = [NSKeyedArchiver archivedDataWithRootObject:list];
}
-(NSArray*)getReceiveList{
return [NSKeyedUnarchiver unarchiveObjectWithData:self.receive];
}
Transformable attributes are the correct way to persist otherwise unsupported object values in Core Data (such as NSArray). From Core Data Programming Guide: Non-Standard Persistent Attributes:
The idea behind transformable attributes is that you access an attribute as a non-standard type, but behind the scenes Core Data uses an instance of NSValueTransformer to convert the attribute to and from an instance of NSData. Core Data then stores the data instance to the persistent store.
A transformable attribute uses an NSValueTransformer to store an otherwise unsupported object in the persistent store. This allows Core Data to store just about anything that can be represented as NSData - which can be very useful. Unfortunately, transformable attributes cannot be matched in a predicate or used in sorting results with the NSSQLiteStoreType. This means that transformable attributes are useful only for storage, not discovery of objects.
The default transformer allows any object that supports NSCoding (or NSSecureCoding) to be stored as a transformable attribute. This includes NSArray, UIColor, UIImage, NSURL, CLLocation, and many others. It's not recommended to use this for data that can be arbitrarily large, as that can have a significant performance impact when querying the store. Images, for example, are a poor fit for transformable attributes - they are large bags of bytes that fragment the store. In that case, it's better to use the external records storage capabilities of Core Data, or to store the data separately as a file, and store the URL to the file in Core Data. If you must store a UIImage in Core Data, be sure you know the trade offs involved.
Creating a transformable attribute is easy:
• In the Xcode Core Data Model Editor, select the model attribute you want to modify. In the right side inspector, set the attribute type as "Transformable". You can leave the "Name" field blank to use the default transformer. If you were using a custom transformer, you would enter the class name here and register the class using +[NSValueTransformer setValueTransformer:forName:] somewhere in your code.
• In your NSManagedObject subclass header declare the property that describes the transformable attribute with the correct type. In this case, we're using NSArray:
#property (nonatomic, retain) NSArray *transformedArray;
• In the NSManagedObject subclass implementation file the property should be dynamic:
#dynamic transformedArray;
And you are done. When an NSArray value object is passed to setTransformedArray: that array is retained by the object. When the context is saved Core Data will transform the NSArray into NSData using the NSValueTransformer described in the model. The NSData bytes will be saved in the persistent store.
You don't store an NSArray natively in Core Data. You need to transform the values stored within the array into something Core Data can use, and then save the data in the store so that you can push and pull it to your NSArray as needed.
Philip's answer is right. You don't store arrays in Core Data. It is totally against what Core Data is made for. Most of the time you don't need the information of the array but one and that one can get dynamically loaded by Core Data. In the case of collections, it makes no difference if you iterate through an array of your whatever properties or of an array of fetched results on an NSSet (which is basically just an array too).
Here is the explanation what Philip said. You can't store an array directly, but you can create a property list from it. There is a method in all NS Arraytypes that gives you a nice and clean string and core data love strings. The cool thing about property lists stored as strings is, they can become what they were. There is a method for that in NSString. Tataaa...
There is a price of course.
Arrays as property lists can get gigantic and that doesn't go well with iOS devices where RAM is limited. Trying to save an array to core data indicates a poor entity design especially for large data. A small array is OK for speed reasons.
Another, less space consuming way, is to use binary property lists. Those come close to zip sizes when stored in Core Data or directly in the filesystem. Downside is, you can't simply open and read them like an XML or JSON file. For development I prefer something human readable and for release the binary version. A constant tied to the DEBUG value in the preprocessor takes care of that, so I don't have to change my code.
Core Data stores instances of NSManagedObject or subclasses of same. NSManagedObject itself is very much like a dictionary. To-many relationships between objects are represented as sets. Core Data has no ordered list that would correspond to an array. Instead, when you retrieve objects from a Core Data store, you use a fetch request. That fetch request can specify one or more sort descriptors that are used to sort the objects, and the objects returned by a fetch request are stored in an array.
If preserving the order of objects is important, you'll need to include an attribute in your entity that can be used to sort the objects when you fetch them.
We have a decent sized object-oriented application. Whenever an object in the app is changed, the object changes are saved back to the DB. However, this has become less than ideal.
Currently, transactions are stored as a transaction and a set of transactionLI's.
The transaction table has fields for who, what, when, why, foreignKey, and foreignTable. The first four are self-explanatory. ForeignKey and foreignTable are used to determine which object changed.
TransactionLI has timestamp, key, val, oldVal, and a transactionID. This is basically a key/value/oldValue storage system.
The problem is that these two tables are used for every object in the application, so they're pretty big tables now. Using them for anything is slow. Indexes only help so much.
So we're thinking about other ways to do something like this. Things we've considered so far:
- Sharding these tables by something like the timestamp.
- Denormalizing the two tables and merge them into one.
- A combination of the two above.
- Doing something along the lines of serializing each object after a change and storing it in subversion.
- Probably something else, but I can't think of it right now.
The whole problem is that we'd like to have some mechanism for properly storing and searching through transactional data. Yeah you can force feed that into a relational database, but really, it's transactional data and should be stored accordingly.
What is everyone else doing?
We have taken the following approach:-
All objects are serialised (using the standard XMLSeriliser) but we have decorated our classes with serialisation attributes so that the resultant XML is much smaller (storing elements as attributes and dropping vowels on field names for example). This could be taken a stage further by compressing the XML if necessary.
The object repository is accessed via a SQL view. The view fronts a number of tables that are identical in structure but the table name appended with a GUID. A new table is generated when the previous table has reached critical mass (a pre-determined number of rows)
We run a nightly archiving routine that generates the new tables and modifies the views accordingly so that calling applications do not see any differences.
Finally, as part of the overnight routine we archive any old object instances that are no longer required to disk (and then tape).
I've never found a great end all solution for this type of problem. Some things you can try is if your DB supports partioning (or even if it doesn't you can implement the same concept your self), but partion this log table by object type and then you can further partion by date/time or by your object ID (if your ID is a numeric this works nicely not sure how a guid would partion).
This will help maintain the size of the table and keep all related transactions to a single instance of an object to itself.
One idea you could explore is instead of storing each field in a name value pair table, you could store the data as a blob (either text or binary). For example serialize the object to Xml and store it in a field.
The downside of this is that as your object changes you have to consider how this affects all historical data if your using Xml then there are easy ways to update the historical xml structures, if your using binary there are ways but you have to be more concious of the effort.
I've had awsome success storing a rather complex object model that has tons of interelations as a blob (the xml serializer in .net didn't handle the relationships btw the objects). I could very easily see myself storing the binary data. A huge downside of storing it as binary data is that to access it you have to take it out of the database with Xml if your using a modern database like MSSQL you can access the data.
One last approach is to split the two patterns, you could define a Difference Schema (and I assume more then one property changes at a time) so for example imagine storing this xml:
<objectDiff>
<field name="firstName" newValue="Josh" oldValue="joshua"/>
<field name="lastName" newValue="Box" oldValue="boxer"/>
</objectDiff>
This will help alleviate the number of rows, and if your using MSSQL you can define an XML Schema and get some of the rich querying ability around the object. You can still partition the table.
Josh
Depending on the characteristics of your specific application an alternative approach is to keep revisions of the entities themselves in their respective tables, together with the who, what, why and when per revision. The who, what and when can still be foreign keys.
Although I would be very careful to use this approach, since this is only viable for applications with a relatively small amount of changes per entity/entity type.
If querying the data is important I would use true Partitioning in SQL Server 2005 and above if you have enterprise edition of SQL Server. We have millions of rows partitioned by year down to day for the current month - you can be as granular as your application demands with a maximum number of 1000 partitions.
Alternatively , if you are using SQL 2008 you could look into filtered indexes.
These are solutions that will enable you to retain the simplified structure you have whilst providing the performance you need to query that data.
Splitting/Archiving older changes obviously should be considered.