Suppose I want to search for bugs reported in recent 2 years. The initial result page says "This result was limited to 500 bugs"
Apparently there are more than 500 bugs, so I click See all search results for this query. This time, it shows 10000 bugs, but with a message saying "This list is too long for Bugzilla's little mind; the Next/Prev/First/Last buttons won't appear on individual bugs"
So my question is:
How do I know the exact number of bugs returned by my query (it's unlikely to be exactly 10000)
How do I view the entire search results? Currently it seems like if the search results exceed 10000, the results are truncated. And I didn't find any prev/next page button to navigate the search results page.
You may not see all the bugs because of the configuration your administrator set on your bugzilla instance.
However, using the search function from the bugzilla webservice you can retrieve the list of bugs. If the number of bugs returned by the query is capped, then iterate on the search query using a higher offset and limit. Here is some pseudocode
offset = 0
limit = 5000
currentcount = ws.search(criterias, offset, limit).count
while currentcount == limit
{
offset += limit
currentcount = ws.search(criterias, offset, limit).count
}
totalbugs = currentcount + offset
The same algorithm would work if you also wanted to get the whole list of bugs instead of just the count.
If the idea of sending multiple queries to the webservice don't feel right you may have to talk to the admin to know what hard limits are set on your install of bugzilla and see how you can tweak them to get the results you need
Related
Running into an issue where say I have a paginated result set always delivering 10 items per page (or less if there's no more left in DB), I've noticed that certain pages (eg: page 2 of 20 or page 5 of 74, etc.) sometimes contains less than 10. After diving into the code, yes we do skip/takes but there's also a bit of post processing afterwards where it hits 3rd party APIs and depending on the result, will filter the items out. This could lead to certain pages having 3 items instead of 10. How does one take into account post filtering during pagination? Is that even a thing?
I am using NearBySearch from Microsoft Azure. In the official documentation it says that when you make a query the totalResults that the API can return is X. However, you can also read that there is a limit on the number of items returned which is at most 100.
In the case that the totalResults >= limit == 100, the API will only display the first 100 results, thus not showing the remaining ones.
Question: Would you be able to suggest a way to retrieve the additional results using the NearBySearch function?
Note: On the Google API NearBySearch there is a parameter called next_page_token, which allows to view all the possible results. Is there something similar in Azure?
You have a limit of 100 results each query. If you have 150 totalResults. You can execute the query with ofs= 0 and limit= 100 to get the first 100 entries. After that you execute the second query with the ofs=100 (because it is like an index). Your limit is 100. After that you will get the next 100 results. Because there are only 50 results left, your numResults will be 50.
I hope it is understandable
Would you be able to suggest a way to retrieve the additional results
using the NearBySearch function?
Looking at the documentation, I noticed that there is an offset parameter (ofs) which by default is zero. You should be able to use that to get the next set of results if the total results are more than the limit specified by you.
I want to use pagination on a large dataset.
Until now, the following has been working well.
await myRepository.findAndCount({
where: whereClauses,
skip: currentPage * pageSize,
take: pageSize
)};
My problem is in the following situation:
Say I have pageSize=1000 and the result set is 300,000 results. The findAndCount method is still taking a long time to complete. I would like to set something like a maxResults where the highest possible number of pages would be maxResults/pageSize in order to reduce the processing time.
In the case above, with a pageSize of 1,000 and total results of 300k, the max results would prevent me from ever counting the full 300k dataset, instead cutting it off at 100k.
I would still like to use the findAndCount functionality so that on my UI, I can keep showing users how many pages of data exist (i.e. "You are on Page 3 of 100") but throw up a warning if they hit the max number of results.
Is this possible in typeORM?
TableStorage & Nodejs
Using the function "queryEntities" sometimes result.entries.length is 0, even when I am pretty sure there are a lot of entries in the database. The "where" parameters are ok, but sometimes (maybe one every 100) it returns 0 entries. Not error returned. Just 0 entries.
And in my function that's causing troubles.
My theory is that the database sometimes is saturated because this function executes every 10 seconds and maybe sometimes before one finish another one starts and both operate over the same table, and instead of error it returns a length 0 , what is something awful.
There is any way to resolve this? Shouldn't it return error?
This is expected behavior. In this particular scenario, please check for the presence of continuation tokens in the response. Presence of these tokens in the response indicate that there may be entities available matching the query and you should execute the same query again with the continuation token you received.
Please read this document for explanation: https://learn.microsoft.com/en-us/rest/api/storageservices/query-timeout-and-pagination.
From this link:
A query against the Table service may return a maximum of 1,000 items
at one time and may execute for a maximum of five seconds. If the
result set contains more than 1,000 items, if the query did not
complete within five seconds, or if the query crosses the partition
boundary, the response includes headers which provide the developer
with continuation tokens to use in order to resume the query at the
next item in the result set.
I've created a few reasonably complex M queries and have started running into some severe performance issues. I'm wondering if has to do with how I sometime organize my code.
The issues I've been having are:
1) Power Query constantly uses all of several CPU cores, calculating something, even if I'm not waiting for a result.
2) In task manager I can sometimes see that the Power Query threads ("Microsoft.mashup.Container.NetFX40.exe") are nearly idle, while Excel.exe is using 100% of one core for tens of minutes - even though at most I'm looking values in a few parameter tables that don't contain more than a couple dozen cells.
3) Some steps take extremely long to calculate, even though the operations involved are trivial. For example, I have a list of 10 text values taken from an Excel table. This list appears as one of my query steps when I 'preview' it. Then I want to remove a single value, so the next step = List.RemoveItems(myList, {"val"}). It didn't compute after 30 minutes, even though I could see the list was correctly loaded in a previous step.
4) UI sometimes becomes unresponsive for several minutes after changing code. Can still right-click on Queries at left hand side to enter advanced editor, and click the red X at top right and choose to keep changes, but all the rest is unresponsive. Not greyed out, just unresponsive.
Anyway, I just wanted to ask if anyone's had similar trouble, and if anyone knows what triggers particularly bad performance in PQ.
I'll often use something like the following pattern to keep the total number of queries down while still being able to easily inspect individual steps:
let
ThisWB = Excel.CurrentWorkbook(),
CfgTbl = ThisWB{[Name="myCfgTbl"]}[Content],
x = aFn(CfgTbl),
y = bFn(CfgTbl),
output = [ThisWB=ThisWB, CfgTbl=CfgTbl, x=x, y=y]
in
output
Is this likely to lead to any issues? Just thought it might because at one point after waiting a very long time for a simple function result, I created a new query = Excel.CurrentWorkbook(){[Name="myCfgTbl"]}[Content], referenced it from the other query, and my result calculated immediately. No idea why.
It calculates previews. Turn off auto preview generation.
I messed with something like this in cases with formula-heavy tables.
The rest probably requires code examples, especially your last case.
BTW, is your version of power query (or Excel 2016) up-to-date?