I'm using OnDemandGrid with JsonRest store to retrieve data from RESTful API and to show it on the table. The table is rather complex and all JsonRest CRUD methods are used.
Here is the basic structure I'm using:
JsonRest:
...
var restStore = Observable(Cache(JsonRest({
target:"source",
idProperty: "id"
}), Memory()));
...
OnDemandGrid:
...
var grid = new (declare([OnDemandGrid, Selection, Keyboard]))({
sort: "name",
store: restStore,
columns: [
{field: "name", label: "Name"},
{field: "state", label: "State"},
{field: "city", label: "city"}
],
loadingMessage: "Loading data...",
noDataMessage: "No data"
}, "grid");
grid.startup();
...
I want to filter the data on client side without sending HTTP requests. Can you give me some ideas to solve this issue?
Own resarch:
Dgrid tutorial stands on the fact that all depends on the dojo-store.
When dgrid interacts with a store, all paging, filtering, and sorting responsibilities fall upon the store, not the grid. ... When encountering data rendering issues, always check that the store implementation (and backend service, if applicable) are performing as expected.
So this means I have to resolve this issue on store side. I suppose, I have to extend QueryResults of JsonRest store but I am hitting the wall all the time.
I have also thought to query against Cache - but I loose the JsonRest then...
If you're basically interested in initially retrieving a data payload from your service all at once up-front but then do all sorting/filtering/paging client-side, have a look at dojo-smore/RequestMemory - you pass it a url and it basically acts like a Memory store once it fetches data from the URL, except its methods return promises rather than immediate values.
Related
I am new to pouchdb and I am reading below source code:
db.query('product_index', {
startkey: ["01234"],
endkey: ["01234", {}],
include_docs: false
});
this code executes for a long time. After read some pouchdb document it looks like it builds index on the database when it run the first time. But I don't understand which fields are indexed based on above code.
Below code I can see it builds index on field foo. But how can I understand query API for building index? What is the different between using query and createIndex from index perceptive?
db.createIndex({
index: {
fields: ['foo']
}
})
Have you seen the PouchDB Guide Bulk operations section Please use 'allDocs()'. Seriously.?
Far too many developers overlook this valuable API, because they
misunderstand it. When a developer says "my PouchDB app is slow!", it
is usually because they are using the slow query() API when they
should be using the fast allDocs() API.
When designing your data structures it's very important to bear that in mind. You should define your record id fields to optimize data accessibility through allDocs().
I'm using aws-appsync in a Node.js client to keep a cached list of data items. This cache must be available at all times, including when not connected to the internet.
When my Node app starts, it calls a query which returns the entire list of items from the AppSync data source. This is cached by Apollo's cache storage, which allows future queries (using the same GraphQL query) to be made using only the cache.
The app also makes a subscription to the mutations which are able to modify the list on other clients. When an item in the list is changed, the new data is sent to the app. This can trigger the original query for the entire list to be re-fetched, thus keeping the cache up to date.
Fetching the entire list when only one item has changed is not efficient. How can I keep the cache up to date, while minimising the amount of data that has to be fetched on each change?
The solution must provide a single point to access cached data. This can either be a GraphQL query or access to the cache store directly. However, using results from multiple queries is not an option.
The Apollo documentation hints that this should be possible:
In some cases, just using [automatic store updates] is not enough for your application ... to update correctly. For example, if you want to add something to a list of objects without refetching the entire list ... Apollo Client cannot update existing queries for you.
The alternatives it suggests are refetching (essentially what I described above) and using an update callback to manually update the cached query results in the store.
Using update gives you full control over the cache, allowing you to make changes to your data model in response to a mutation in any way you like. update is the recommended way of updating the cache after a query.
However, here it is referring to mutations made by the same client, rather than syncing using between clients using subscriptions. The update callback option doesn't appear to be available to a subscription (which provides the updated item data) or a query (which could fetch the updated item data).
As long as your subscription includes the full resource that was added, it should be possible by reading from and writing to the cache directly. Let's assume we have a subscription like this one from the docs:
const COMMENTS_SUBSCRIPTION = gql`
subscription onCommentAdded {
commentAdded {
id
content
}
}
`;
The Subscription component includes a onSubscriptionData prop, so we should be able to do something along these lines:
<Subscription
subscription={COMMENTS_SUBSCRIPTION}
onSubscriptionData={({ client, subscriptionData: { data, error } }) => {
if (!data) return
const current = client.readQuery({ query: COMMENTS_QUERY })
client.writeQuery({
query: COMMENTS_QUERY,
data: {
comments: [...current.comments, data.commentAdded],
},
})
}}
/>
Or, if you're using plain JavaScript instead of React:
const observable = client.subscribe({ query: COMMENTS_SUBSCRIPTION })
observable.subscribe({
next: (data) => {
if (!data) return
const current = client.readQuery({ query: COMMENTS_QUERY })
client.writeQuery({
query: COMMENTS_QUERY,
data: {
comments: [...current.comments, data.commentAdded],
},
})
},
complete: console.log,
error: console.error
})
I’m trying to understand what the advantages are of GraphQL. I’ve read about reducing the number of endpoints and the complexity of server responses, but it seems that the same results can be achieved with JS alone.
Here’s an example of a data object that could be sent as JSON to a node server with MongoDB. This would be an example of a game app where the client is retrieving user info:
let data = {
db: "users",
params: {_id: "xxxxx"},
fields: ["username", "level"],
games:
{
db: "games",
params: {userID: "xxxxx"},
fields: ["opponent”]
}
}
In this example, db, params, and fields would be standard keys, and games would be like a special key for the specific purpose of retrieving the user’s games, however, the syntax of the games object would follow the same standard format as the overall data object.
Then on the server, the Mongo query would look something like this:
db.collection(data.db).find(params)
You’d then filter out the extraneous Mongo fields in some standardized way and respond to the client.
I’m a relative beginner with JS, but I think you could also chain promises based on whether certain special keys (e.g., “games” from above) are included in the data object.
This seems like it achieves the same benefits as GraphQL with less complexity. What other benefits does GraphQL have that a plain JS equivalent does not?
Alright, so I need to implement a fairly large local database for the iOS and Android mobile browsers (~30 MB). I am researching the options and it looks like WebSQL (the option I wanted to use) is being actively abandoned. Also, it looks like IndexedSQL is not fully supported.
What do you recommend for a local browser database? Thanks!
IndexedDB with use of IndexedDBShim (a polyfill for WebSQL), for sure looks fulfills your requirement. But do notice, that IOS devices allows maximum of 50 MB storage.
I have worked on the similar requirement as of yours and this combination worked across all modern browsers.
I don't think you have an other choice than using the indexeddb. WebSQL is deprecated and localstorage is to small and not performante to serve the needs.
I wrote a library that implements a linq like interface. By using methods you can easily query the database. Example:
linq2indexeddb.from("store").where("field").equals("value").select()
Because the indexededdb is async you will get back a promise.
You can find my library at codeplex
I'm replying to this in 2016 (2 years after you asked this question) and everything concerning the deprecation of WebSQL still stands. IndexedDB on the other hand, enjoys the support of all of the major browser vendors.
Now would be a good time to state that "IndexedSQL" is neither an alternative name for IndexedDB, nor a name of any other existing client-side database :) . Pointing this out may seem a bit pedantic, but it's not: IndexedDB is a non-relational document-store, and as such does not natively support SQL.
Regardless of what you call it, IndexedDB is currently the only database on the W3C standards track, and as such is the only future-proof option for anyone tasked with choosing a client-side database.
As implied by GemK, however, such a decision isn't one that necessarily has to be made; one can simply choose (or make) a library which utilizes whichever database is available on a client machine.
BakedGoods differs from such libraries already suggested here in several ways; most pertinently, it allows the storage type(s) that are to be utilized to be explicitly specified, in turn allowing the developer to introduce other factors (such as performance characteristics) in to the decision-making process.
With it, conducting storage operations in whichever of the database types is supported is a matter of...
... specifying the appropriate operation options and equivalent configs for both database types:
//If the operation is a set(), and the referenced structures
//don't exist, they will be created automatically.
var webSQLOptionsObj = {
databaseName: "Example_DB",
databaseDisplayName: "Example DB",
databaseVersion: "",
estimatedDatabaseSize: 1024 * 1024,
tableData: {
name: "Main",
keyColumnName: "lastName",
columnDefinitions: "(lastName TEXT PRIMARY KEY, firstName TEXT)"
},
tableIndexDataArray: [name: "First_Name_Index", columnNames: "(firstName)"]
};
var indexedDBOptionsObj = {
databaseName: "Example_DB",
databaseVersion: 1,
objectStoreData: {
name: "Main",
keyPath: lastName,
autoIncrement: false
},
objectStoreIndexDataArray: [
{name: "First_Name_Index", keyPath: "firstName", unique: false, multiEntry: false}
],
};
var optionsObj = {
conductDisjointly: false,
webSQL: webSQLOptionsObj,
indexedDB: indexedDBOptionsObj
};
... and conducting the operation:
bakedGoods.set({
data: [
{value: {lastName: "Obama", firstName: "Barack"}},
{value: {lastName: "Biden", firstName: "Joe"}}
],
storageTypes: ["indexedDB", "webSQL"],
options: optionsObj,
complete: function(byStorageTypeStoredItemRangeDataObj, byStorageTypeErrorObj){}
});
Its simple interface and unmatched storage facility support comes at the cost of lack of support for some storage facility-specific configurations. For instance, it does not support the conduction of storage operations in WebSQL tables with multi-column primary keys.
So if you make heavy use of those types of features, you may want to look elsewhere.
Oh, and for the sake of complete transparency, BakedGoods is maintained by yours truly :) .
I'm trying to get notifications in a CouchDB change poll as soon as pre-defined field is set or changed. I've already had a look at filters that can be used for filtering change events(db/_changes?filter=myfilter). However, I've not yet found a way to include this temporal information, because you can only get the current version of the document in this filter functions.
Is there any possibility to create such a filter?
If it does not work, I could export my field to a separate database and the only poll for changes in that db, but I'd prefer to keep together my data for obvious reasons.
Thanks in advance!
You are correct: filters and _changes feeds can only see snapshots of a document. What you need is a function which can see the old document and the new document and act correctly. But that is unavailable in _filters and _changes.
Obviously your client code knows if it updates that field. You might update your client code however there is a better solution.
Update functions can access both documents. I suggest you make an _update
function which notices the field change and flags that in the document. Next you
have a simple filter checking for that flag. The best part is, you can use a
rewrite function to make the HTTP API exactly the same as before.
1. Create an update function to flag interesting updates
Your _design/myapp would be {"updates", "smart_updater": "(see below)"}.
Update functions are very flexible (see my recent update handlers
walkthrough). However we only want to mimic the normal HTTP/JSON API.
Your updates.smart_updater field would look like this:
function (doc, req) {
var INTERESTING = 'dollars'; // Set me to the interesting field.
var newDoc = JSON.parse(req.body);
if(newDoc.hasOwnProperty(INTERESTING)) {
// dollars was set (which includes 0, false, null, undefined
// values. You might test for newDoc[INTERESTING] if those
// values should not trigger this code.
if((doc === null) || (doc[INTERESTING] !== newDoc[INTERESTING])) {
// The field changed or created!
newDoc.i_was_changed = true;
}
}
if(!newDoc._id) {
// A UUID generator would be better here.
newDoc._id = req.id || Math.random().toString();
}
// Return the same JSON the vanilla Couch API does.
return [newDoc, {json: {'id': newDoc._id}}];
}
Now you can PUT or POST to /db/_design/myapp/_update/[doc_id] and it will feel
just like the normal API except if you update the dollars field, it will add
an additional flag, i_was_changed. That is how you will find this change
later.
2. Filter for documents with the changed field
This is very straightforward:
function(doc, req) {
return doc.i_was_changed;
}
Now you can query the _changes feed with a ?filter= parameter. (Replication
also supports this filter, so you could pull to your local system all documents
which most recently changed/created the field.
That is the basic idea. The remaining steps will make your life easier if you
already have lots of client code and do not want to change the URLs.
3. Use rewriting to keep the HTTP API the same
This is available in CouchDB 0.11, and the best resource is Jan's blog post,
nice URLs in CouchDB.
Briefly, you want a vhost which sends all traffic to your rewriter (which itself
is a flexible "bouncer" to all design doc functionality based on the URL).
curl -X PUT http://example.com:5984/_config/vhosts/example.com \
-d '"/db/_design/myapp/_rewrite"'
Then you want a rewrites field in your design doc, something like (not
tested)
[
{
"comment": "Updates should go through the update function",
"method": "PUT",
"from": "db/*",
"to" : "db/_design/myapp/_update/*"
},
{
"comment": "Creates should go through the update function",
"method": "POST",
"from": "db/*",
"to" : "db/_design/myapp/_update/*"
},
{
"comment": "Everything else is just like normal",
"from": "*",
"to" : "../../../*"
}
]
(Once again, I got this code from examples and existing code I have laying
around but it's not 100% debugged. However I think it makes the idea very clear.
Also remember this step is optional however the advantage is, you never have to
change your client code.)