I make an ajax request to couchdbwith POST method, giving a list of keys of the docs I want to retrieve.
Everything seems to work fine except the fact that I get 0 rows because the offset is set on the last line.
So it means that:
I'm communicating well with couchdb server (cloudant)
the POST method works
it seems to retrieve the list but just giving it with the offest of last element, i.e. an empty list
Also, trying to order the results differently had no success.
riList var is something like this (from google chrome dev tools):
keys: Array[194]
0: "Wire line diamond core drilling rig"
1: "VUA - isotope geochemistry laboratory"
2: "Volcanologic and Seismological Observatories"
3: "VESOG"
4: "Utrecht University - TecLab, Tectonic Laboratory"
5: "Utrecht University - Experimental and Analytical Laboratories"
.....
which basically is the same of
var riList=["Wire line diamond core drilling rig", "VUA - isotope geochemistry laboratory","Volcanologic and Seismological Observatories","VESOG","Utrecht University - TecLab, Tectonic Laboratory","Utrecht University - Experimental and Analytical Laboratories"];
Here is the code
var riList= ListOfRU.pluck('ri_name');
var queryParams={"keys":riList};
var riResponseList=[];
var ajaxURL= ('_view/'+ self.parentMcDropDownValue);
console.log(ajaxURL, queryParams);
$.ajax({ //retrieve and show on map LABORATORY coordinates
async: true,
url: ajaxURL,
type:"POST",
data:JSON.stringify(queryParams),
dataType: 'json',
timeout:5000,
success:function(response){
console.log("response",response);
riResponseList=response.rows;
},
error:function(){
alert('fetching error');
}
});
chrome developer tools output
response
Object
offset: 194
rows: Array[0]
total_rows: 194
__proto__: Object
as you can see in the output of chrome devtools the offset is 194 so that I have an array with 0 rows because it starts from the last key.
Any idea?
I was tempted to delete this question when I found the bug.
But finally I decided to let it and to explain what was wrong.
If you feel it should be deleted, please motivate this and I'll delete it.
So the answer to my question is very simple: the keys I was retrieving WERE NOT the keys I was selecting with the "keys" parameter.
The error was indeed in the map function where I wrote:
//WRONG ONE!!
function(doc){
if(doc.doctype=='ri'){
emit(doc.ri, doc);
}
}
INSTEAD OF
//RIGHT ONE!!
function(doc){
if(doc.doctype=='ri'){
emit(doc.ri_name, doc);// RI_NAME INSTEAD OF RI!!!!!
}
}
So the lesson is:
When you get an offset which equals the lenght of the array, check if the results are really the one you want to obtain with the map function.
Related
Take a windowed virtual list with the capability of loading an arbitrary range of rows at any point in the list, such as in this following example.
The virtual list provides a callback that is called anytime the user scrolls to some rows that have not been fetched from the backend yet, and provides the start and stop indexes, so that, in an offset based pagination endpoint, I can fetch the required items without fetching any unnecessary data.
const loadMoreItems = (startIndex, stopIndex) => {
fetch(`/items?offset=${startIndex}&limit=${stopIndex - startIndex}`);
}
I'd like to replace my offset based pagination with a cursor based one, but I can't figure out how to reproduce the above logic with it.
The main issue is that I feel like I will need to download all the items before startIndex in order to receive the cursor needed to fetch the items between startIndex and stopIndex.
What's the correct way to approach this?
After some investigation I found what seems to be the way MongoDB approaches the problem:
https://docs.mongodb.com/manual/reference/method/cursor.skip/#mongodb-method-cursor.skip
Obviously he same approach can be adopted by any other backend implementation.
They provide a skip method that allows to skip an arbitrary amount of items after the provided cursor.
This means my sample endpoint would look like the following:
/items?cursor=${cursor}&skip=${skip}&limit=${stopIndex - startIndex}
I then need to figure out the cursor and the skip values.
The following code could work to find the closest available cursor, given I store them together with the items:
// Limit our search only to items before startIndex
const fragment = items.slice(0, startIndex);
// Find the closest cursor index
const cursorIndex = fragment.length - 1 - fragment.reverse().findIndex(item => item.cursor != null);
// Get the cursor
const cursor = items[cursorIndex];
And of course, I also have a way to know the skip value:
const skip = items.length - 1 - cursorIndex;
I am new to SAP UI5 development. Currently the table is using "growing" and "growingThreshhold", then users can click more to see data of next page. Since we have thousands of data in that table, it takes user time to click more and more again to load next page data. we try to implement a function, that user can enter the page number then click a button and go to the specific page.
<Table id="genTable" growing="true" growingThreshold="60" fixedLayout="false" selectionChange="onHandleSelectChange"
backgroundDesign="Solid" updateFinished="onHandleGeneratorQueueUpdateFinished">
Expected UI:
I added a bar then UI display is good.
<Bar design="SubHeader">
<contentMiddle>
<Input type="Number" id="pageNumber" width="50px"></Input>
<Button id="goToButton" text="Go to" type="Emphasized" press="onHandleGoTo"></Button>
</contentMiddle>
</Bar>
For the backend logic, I refer to below articles, but still doesn't work.
https://blogs.sap.com/2016/12/14/sapui5-pagination-in-sap.m-table-on-button-click-using-odata-service/
https://sapyard.com/advance-sapui5-19-pagination-in-table-control-with-top-and-skip-query-options/
I tried to use read, the it can get the data back from odata service, but the data can't be refreshed in the table.
oModel.read("/ViewQueueSet", {
urlParameters: {
"$top": top,
"$skip": count
},
filters: [new Filter("RoleCode", FilterOperator.EQ, "G")],
useBatch: true,
success: function (tdata) { //successful Read in the server
var json = new JSONModel();
json.setData(tdata);
that.getView().setModel(json,"sapmodel");
sap.ui.core.BusyIndicator.hide();
},
error: function () {
sap.ui.core.BusyIndicator.hide();
}
});
}
also tried to call bindItems
//that.getView().setModel(json,"sapmodel");
//oTable.setModel(json); //JSON is preferred data format
//oTable.bindItems("/results",that.oGenQueueTemplate);
that.getView().byId("genTable").setModel(json);
that.getView().byId("genTable").bindItems("/results",that.oGenQueueTemplate);
Another approach I tried is to use bindItems, it call send the request to odata service, but it doesn't add the parameter top and skip parameter.
oTable.bindItems({
path: "/ViewQueueSet",
model: "sapmodel",
filters: [new Filter("RoleCode", FilterOperator.EQ, "G")],
template: this.oGenQueueTemplate,
// urlParameters: {
// "$top": top,
// "$skip": count
// },
parameters: {
"$top": top,
"$skip": count
}
});
Anyone has any idea about how to implement this functionality?
before I go into detail, please consider using other controls and/or ux patterns. imagine having thousands or millions of elements in backend and user equests to scroll to page 9292929 => for a responsive table (sap.m.Table) you would need to load all elements up to that page. Maybe filtering or some completely different approach could be tha right one.
The correct way to do this is by getting the listbinding and ask it to load more elements. how to ask the binding, may depend on the type of binding as well.
oTable = ... // get a reference on table
oItemsBinding = oTable.getBinding("items");
oItemsBinding.getLength() // will give you total number of elements
oItemsBinding.isLengthFinal() // will tell you if the length is final
oItemsBinding.getCurrentContexts() // will give you array of all loaded contexts.
now a few words to length and the length being final. If you have a binding implementation that knows the total number of objects (e.g. json - since it loads all elements to client, or OData, if cont is implemented in backend) then getLength will tell you the total number of objects.
if the backend doesnt have the count feature implemented, the length becomes final once you reach the end of the list (backend gives you less elements than you require - e.g. top=10,skip=90 returns 10 elements => length 100, not final; top=10,skip=100 returns 4 elements => length=104 becomes final)
Now, you can have a look at various binding implementations. But be aware that there is a lot to consider (direction of growing - upwards/downwards), at least you dont need to think about filtering/sorting - as this is part of the binding.
There is a nice (private) feature in sap.m.Table (or in sap.m.ListBase, to be more precise), which is called GrowingEnablement. you can use it like this:
// dont forget if _oGrowingDelagate is not undefined or similar
oTable._oGrowingDelegate.requestNewPage()
this will load one more page => you could start from reading the implementation of this method if you want to load several pages in one go.
you could also do a simple trick:
// assume you have 20 elements per page (default)
// and want to get to 7th page (elements 121 - 140)
// ckecks for 7th page exists and 7th page not yet loaded are omitted
oTable.setGrowingThreshold(70) // half of 140, so following load will load second page => 71 to 140
oTable._oGrowingDelegate.requestNewPage() // this will load the second page 71 - 140
// once loading is finished (take care of asynchronity)
oItemsBinding.attachEventOnce("dataReceived", function(oEvent){
// reset the growing threshold to 20
oTable.setGrowingThreshold(20)
// scroll to first element of 7th page (index 120, since count starts from 0)
oTable.scrollToInex(120)
})
I am looking for a poor perfomance solution to sort my couchdb view by value (because large data). I am using on my NodeJS Application the "nano" package to get the database/view connection.
I created a CouchDB View Map Function configure a key value pair and the Reduce to _count:
function (doc) {
emit(doc.msg, 1);
}
To get my View i am using:
alice.view('VIEWNAME', 'INDEXNAME', {'group': true).then((body) => {
body.rows.forEach((doc) => {
console.log(doc.key + " " + doc.value);
}
}
The View returns for example "Hello" as doc.key and "155" as doc.value. So i have 155 Documents with the Key Hello.
Now i want to sort my View DESC from Value:
Hello 155
Foo 140
Bar 100
But the only Sorted Version of my View is by Key i get my View sorted by Key ASC.
I tried serval solution on NodeJS side but i dont want to lose much perfomance.
You cannot do this. Views, by nature, are sorted by key only. The solution is to create a second view that sorts by a different key.
I build a work around on NodeJS Side:
body.rows.sort(function (a, b) {
return parseInt(b.value,10) - parseInt(a.value,10);
});
Im not 100 percent satisfied, but the solution helped me. I still cannot estimate how much perfomanace the sorting takes on the NodeJS side
I'm using a MongoDB mapReduce to code a ranking feed algorithm, it almost works but the latest thing to implement is the pagination. The map reduce supports the results limitation but how could I implement the offset (skipping) based e.g. on the latest viewed _id of the results, knowing that I'm using mongoose?
This is the procedure I wrote:
o = {};
o.map = function() {
//log10(likes+comments) / elapsed hours from the post creation
emit(Math.log(this.likes + this.comments + 1) / Math.LN10 / Math.abs((now - this.createdAt) / 6e7 + 1), this);
};
o.reduce = function(key, values) {
//sort the values, when they have the same score
values.sort(function(a, b) {
a.createdAt - b.createdAt;
});
//serialize the values, because mongoose does not support multiple returned values
return JSON.stringify(values);
};
o.scope = {now: new Date()};
o.limit = 15;
Posts.mapReduce(o, function(err, results) {
if (err) return console.log(err);
console.log(results);
});
Also, if the mapReduce it's not the way to go, do you suggest other on how to implement something like this?
What you need is a page delimiter which is not the id of the latest viewed as you say, but your sorting property. In this case, it seems to be the formula Math.log(this.likes + this.comments + 1) / Math.LN10 / Math.abs((now - this.createdAt) / 6e7 + 1).
So, in your mapReduce query needs to hold a where value of that formula above. Or specifically, 'formula >= . And also it needs to hold the value of createdAt at the last page, since you don't sort by that. (Assuming createdAt is unique). So yourqueryof mapReduce would saywhere: theFormulaExpression, createdAt: { $lt: lastCreatedAt }`
If you do allow multiple identical createdAt values, you have to play a little outside of the database itself.
So you just search by formula.
Ideally, that gives you one element with exactly that value, and the next ones sorted after that. So in reply to the module caller, remove this first element off the array (and make sure you actually ask for more results then you need because of this).
Now, since you allow for multiple similar values, you need another identifying prop, say, object id or created_at. Your consumer (caller of this module) will have to provide both (last value of the score, createdAt of the last object). Say you have a page split exactly in the middle - one or more objects is on the previous page, another set on the next
. You'd have to not simply remove the top value (because that same score is already served on the previous page), but possibly several of them from the top.
Then it goes really crazy, because potentially your whole page was already served - compare the _ids, look for the first one after the one your module caller has provided you with. Or look into the data and determine how many matching values like that are there, try to get at least as many more values from mapReduce then you have on your actual page size.
Aside from that, I would do this with aggregation instead, it should be much more preformant.
I have fixed row numbers to 10 for my subgrids, but if reccount is less than 10 I would want to adjust height subgrid to "auto" or "100%".
So here is my code for this subgrid :
// SUBGRID FOURTH LEVEL
var subgrid_table_id = subgrid_id+"_d",
pager_id = "p_"+subgrid_table_id;
$("#"+subgrid_id).append("<table id='"+subgrid_table_id+"' class='scroll'></table><div id='"+pager_id+"' class='scroll'></div>");
$("#"+subgrid_table_id).jqGrid({
url:"sg31b.php?id="+row_id+"&clt="+clt,
datatype: "json",
idPrefix:"sgd_",
colNames: ['Id','Article','Désignation','Marque','Equivalence'],
colModel: [
{name:'e.id',index:'e.id',hidden:true},
{name:'a.code',index:'a.code', width:100},
{name:'a.descr',index:'a.descr', width:450},
{name:'k.code',index:'k.code', width:80},
{name:'e.equiv',index:'e.equiv',width:100}
],
pager: pager_id,
sortname: 'a.code',
hiddengrid:true,
scroll:true,
height:230,
rowNum:10,
autowidth:true,
caption:'4 - EQUIVALENCE ARTICLES',
gridComplete:function(){
sortDataCol(this);
if($("#"+subgrid_id+"_d").jqGrid('getGridParam','records') < $("#"+subgrid_id+"_d").jqGrid('getGridParam','rowNum')){
$("#"+subgrid_id+"_d").jqGrid('setGridHeight','100%');
}else{
$("#"+subgrid_id+"_d").jqGrid('setGridParam',[{npage:1}]).jqGrid('setGridHeight',230);
}
}
});
$("#"+subgrid_table_id).jqGrid('navGrid',"#"+pager_id,{search:false,add:false,edit:false,del:false});
$("#"+subgrid_table_id).jqGrid('filterToolbar',{stringResult: true,searchOnEnter : false});
fullInputCss();
and the snapshot of result for less than 10 filtered rows :
Now if I press Backspace in search field to obtain more rows, it seems that search doesn't fire because Firebug doesn't show any trace of request :
If I delete added 'setGridHeight' lines in gridcomplete, all runs fine !
I think that one more time I'm wrong in my coding and understanding how jqGrid runs.
Please could someone give me some way to solve this trouble ?
Many thanks in advance. Have a nice day. JiheL
I suppose that the origin of the problem could be id duplicates on your page. Just now I wrote the answer on another your question where I described the problem detailed.
Current implementation of jqGrid (version 4.4.5) has problem in the code of filterToolbar which constructs id for input fields of the filter toolbar based on the following rule:
id="gs_" + cm.name
(see the line of code). It means that the id of the input field for the column a.code will be gs_a.code for every subgrid which you use. So you can have id duplicates.
So I recommend you redesign the naming concept in your code. You can use for example
name: row_id + "a_code", index: "a.code"
In the way the value like "a.code" will be still send during sorting of the grid, but you will have no id duplicates. In some scenarios (is you use repeatitems: false in jsonReader) you could need to use additional jsonmap attribute, but you don't need it in you current code.