I'm building an application that needs to be able to store data offline and sync it to an OData server. While reviewing the videos, one of the presenters mentioned a Concurrency Token - can this token be used to help in the sync process? If so, how do I use it and what do I need to watch out for?
For further info, my target platform is Android, so I'm unable to use the Microsoft Sync Framework.
The concurrency token, also called the ETag (as per the HTTP RFC) can be imagined as a version number of the entity instance. The server raises that every time the entity changes. The server sends this along with the entity to the client and the client can send it back to the server (in the If-Match header) when applying changes. The server will then check that the client has the latest version and if so applies the change. Otherwise it fails the update request.
It can help you with the sync process in two ways:
If you get an entity from the server and it has a differente ETag from what your client remembers, the server instance chas changed, so you may either update the client instance or perform some conflict resolution if the client made changes.
The other way would be to send all the changes from the client to the server (With the right ETags) and the server will fail all of those which already changed on the server. The client can then get the new versions from the server, resolve conflicts and try again.
Note though, that unless your ETag is one of the publicly accesible properties on the entity, there is no built-in way to filter on the ETag, so currently you can't ask the server to get you all entities changed since a given version. Although if you do control the service, you might be able to do this (have a global version number and each change raises that and each entity carries the version number of the last change made to it, then you can filter on that version property).
Related
I want to send a cached page back to the user But the problem is that I need to generate a unique VISITOR_ID for every new user and
send it back to the user through headers , so I need to send an API call from varnish proxy server to my backend servers to fetch VISITOR_ID and then append it to the response
We were earlier using Akamai and we were able to implement this using edge workers present there,
I want to know if such a thing is possible to do in varnish or not.
Thanks in Advance
Open source solution for HTTP calls
You can use https://github.com/varnish/libvmod-curl and add this VMOD to perform HTTP calls from within VCL. The API for this module can be found here: https://github.com/varnish/libvmod-curl/blob/master/src/vmod_curl.vcc
Commercial solution for HTTP calls
Despite there being an open source solution, I want to mention that there might be a more stable solution that is actually supported and receives frequent updates. See https://docs.varnish-software.com/varnish-cache-plus/vmods/http/ for the HTTP VMOD that is part of Varnish Enterprise.
Generate the VISITOR_ID in VCL
Your solution implies that an HTTP call is needed for every single response. While that is possible through various VMODs, this will result in a lot of extra HTTP calls. Unless you cache every variation that includes the VISITOR_ID.
You could also consider generating the unique ID yourself in VCL.
See https://github.com/otto-de/libvmod-uuid for a VMOD that generates UUIDs or https://github.com/varnish/libvmod-digest for a VMOD that generates hashes.
Fetch VISITOR_ID from Redis
If you prefer to generate the VISITOR_ID in your origin application, you could use a Key/Value store like Redis to store or generate values.
You can generate the ID in your application and store it in Redis. You could also generate and store it using LUA scripting in Redis.
Varnish can then fetch the key from Redis and inject it in the response.
While this is a similar approach to the HTTP calls, at leasts we know Redis is capable of keeping up with Varnish in terms of high performance.
See https://github.com/carlosabalde/libvmod-redis to learn how to interface with Redis from Varnish.
I'm trying to write a simple node.js program to sync a few address books from a CardDAV server to a local MySQL database. I'm using the node dav client.
I know CardDAV supports only syncing changes since the last sync via sync-token and I see some references to sync tokens when I browse through the source and readme of the dav client. But, I'm very new to DAV, so I'm not 100% sure how to put it all together.
I'm guessing I need to store the sync token (and level?) the server sends back after I run a sync and then include that in my next sync request. Am I on the right track?
Building a CardDAV client is a great resource which describes how all that works, including WebDAV Sync, which is what you are looking for.
Note that a server is not required to provided WebDAV sync (and quite a few don't).
Also note that even if they support WebDAV sync, they can expire the tokens however/whenever they want (e.g. some only store a single token, or only for a limited time).
In short: do not rely on WebDAV-sync. If it is not available, or the token is expired, you need to fallback to a full, regular sync (comparing href's and etag's).
I'm guessing I need to store the sync token (and level?) the server sends back after I run a sync and then include that in my next sync request. Am I on the right track?
Yes you are on the right track. Sync-tokens are usually per collection (Depth:1, I think they can be Depth:infinity, but I'm not sure).
So you need to store it alongside the URL of the collection you are syncing.
Then in the next sync-request, you embed it into the sync-report. If the token is still valid, you get back the new/deleted/changed records. If the token was invalidated, you need to perform a full sync.
Hope that helps :-)
I'm developing a POS Client using Chrome (packaged) Apps. It will run locally on the installed computers and interact with the server via web service. This app should only run on specific computers at the stores.
I know I can go to each store and install the .crx file in which case I don't have to publish the app to Chrome Web Store. However, I want it to be published to Chrome Web Store so that I can take advantage of its auto-updating feature.
What should I do to make sure that the app can only run at the stores' computers? (I can go the the stores and setup anything needed at the first installation).
Options I have thought of:
Create some secret key and enter it to the app at the first time of running.
Build a small tool (winforms application) to generate time-based tokens and install it on the computers. The staff will need to enter the token each time opening the app.
Any better idea how to accomplish this?
You said the app needs to talk to a web service to work. That's the key to a simple approach. (Assume you don't care whether the staff acquires a nonfunctional copy of the client app.)
At startup, app checks for existence of a validation of some kind stored in chrome.storage.local. If it exists, startup continues.
If the validation is missing, the app checks for existence of a GUID stored in chrome.storage.local.
If the GUID is missing, generate and store one using something like window.crypto.getRandomValues().
Ask the server for a validation by sending the GUID and getting a response.
If a validation comes back, save it in chrome.storage.local and go back to the start of this sequence.
Otherwise tell the user to get lost.
A full-strength version of this approach would have some additional features:
Use an HMAC(GUID, secret) for the validation. I'm assuming the staff aren't tech superstars, so something simple like a boolean would probably suffice.
Optionally add a per-launch step that sends up the GUID and validation and confirms it's still valid each time.
When the validation is requested, you might prompt for the secret key you mentioned in your question. In normal cases this would be needed only at provisioning time.
In case you haven't figured it out yet, the server is now acting like a simple licensing server, so it's up to you to decide how to decide whether the validation request succeeds. Maybe it allows only N validations to exist at once, or after you're done provisioning you hardcode future validations to fail. Maybe it limits validation requests to certain IP addresses. You get to choose.
That's the gist. It's a simple DRM system that is easier to manage than the enter-secret-at-installation method, but that won't withstand an attack of more than 30 minutes (since a smart attacker will just inject another machine's GUID and HMAC validation into the duplicate machine's chrome.storage.local).
When creating a document and letting Couch create the ID for you, does it check if the ID already exists, or could I still produce a conflict?
I need to generate UUIDs in my app, and wondered if it would be any different than letting Couch do it.
Use POST /db request for that, but you should be aware the fact that the underlying HTTP POST method is not idempotent, and a client may automatic retry it due to a problem some networking problems, which may create multiple documents in the database.
As Kxepal already mentioned it is generally not recommended to POST a document without providing your own _id.
You could, however, use GET /_uuids to retrieve a list of UUIDs from the server and use that for storing your documents. The UUIDs returned will depend on the algorithm that is used, but the chance of a duplicate are (for most purposes) insignificantly small.
You can and should give a document id, even when using the bulk document interface. Skipping that step makes the problem of resubmitted requests creating duplicate documents even worse. On the other hand, if you do assign ID's, and part of the request reaches couchdb twice (as in the case of a reconnecting proxy), then your response will include some conflicts, which you can safely ignore, you know the conflict was from you, in the same request
I am using CouchDB for my Data Layer in a Rails 3 application using CouchRest::Model hosted on Heroku.
I am requesting a List of Documents and returning them as JSON to my Browser and using jQuery Templates to represent that data.
Is there a way I could build the request on the server side, and return the request that would need to be called from the browser WITHOUT opening a huge security hole i.e. giving the browser access to the whole database?
Ideally it would be a one off token access to a specific query, Where the token would be generated on the server side, and CouchDB would take the token, and make sure it matches what the query should be, and give access to the results.
One way that comes to mind would be to generate a token Document and use a show function (http://guide.couchdb.org/draft/show.html) to return the results for that token Document's view results. Though I am not sure if that is possible.
Though another is to put a token on the Document itself and use a list function (http://guide.couchdb.org/draft/transforming.html)
Save that, any other ideas?
Thanks in Advance
Is there a way I could build the
request on the server side, and return
the request that would need to be
called from the browser WITHOUT
opening a huge security hole i.e.
giving the browser access to the whole
database?
Yes. One method is to create a rack app and mount it inside your rails app. You can have it receive requests from users' browsers at "/couch" and forward that request to your "real" couchdb url, returning couch's JSON response as-is or modifying it however you need.
You may also be able to use Couch's rewrite and virtual host features to control what Couch URLs the general public is able to reach. This probably will necessitate the use of list or show functions. http://blog.couchone.com/post/1602827844/of-rewrites-and-virtual-hosting-an-introduction
Ideally it would be a one off token access to a specific query, Where the token would be generated on the server side, and CouchDB would take the token, and make sure it matches what the query should be, and give access to the results.
You might use cookies for this since list and show functions can set and get cookie values on requests.
But you could also include a hash value as part of each request. Heroku's add-on API has a good example of how this works. https://addons.heroku.com/provider/resources/technical/build/sso
Notice that the API calls are invalid outside of a certain window of time, which may be exactly what you need.
I'm not sure I precisely understand your needs, but I hope I have been able to give you some helpful ideas.