I'm really engaged with subsonic but I'm not sure how make it work with paging
I mean how can I get "the page" in a list or how is the best way to managing
the total table in my base, page by page
You'll see I tried three things:
m02colegio is an class generated from activerecord
IList<m02colegio> loscolegios;
loscolegios = m02colegio.GetPaged(0, 80).ToList();
----------- and:
SubSonic.Schema.PagedList<m02colegio> loscolegios;
loscolegios = m02colegio.GetPaged(0, 80);
----------- and:
var paged = m02colegio.GetPaged(0,80).All<m02colegio>(x=>x.m02ccolnom.Contains(" "));
// 'cause i dont know how to tell it to consider all records
loscolegios = m02colegio.All().ToList();
but after every try I don't get any exception and loscolegios always is NULL
I need to access the records in this manner
so, what is the best way?
how can I get the first page and then how advance among pages??
public ActionResult Index(int? page)
{
if (!validateInt(page.ToString()))
page = 0;
else
page = page - 1;
if (page < 0) page = 0;
const int pagesize = 9;
IQueryable<m02colegio> Mym02colegio = m02colegio.All().Where(x => x.category == "test").OrderBy(x => x.id);
ViewData["numpages"] = m02colegio.All().Where(x => x.category == "test").OrderBy(x => x.id).Count() / pagesize;
ViewData["curpage"] = page;
return View(new PagedList<material>(Mym02colegio, page ?? 0, pagesize));
}
that is in a MVC sense however it gives you the idea, Index accepts a null or a page number
you get all the records then return a pagelist of the records you got.
I'm not sure if this is a bug that's been fixed in the current github source or if it's by design but I've found that GetPaged only works with a 1 based index for the first argument. So if you do the following you should find it works as you'd expect:
IList<m02colegio> loscolegios = m02colegio.GetPaged(1, 80);
Related
Now we have the code that scrapes the links in an article. We need also the number of clicks on a link. Can some one help?
Sow far we have this code:
String[] articles = {"Abdominal_pain"};
void setup() {
for (int i = 0; i < articles.length; i++) {
String article = articles[i];
String start = "20160101"; // YYYYMMDD
String end = "20170101"; // YYYYMMDD
// documentation: https://wikimedia.org/api/rest_v1/?doc#!/Pageviews_data/get_metrics_pageviews_per_article_project_access_agent_article_granularity_start_end
// >> https://en.wikipedia.org/w/api.php?action=query&format=json&prop=links&meta=&titles=Albert+Einstein&pllimit=500
String query = "https://en.wikipedia.org/w/api.php?action=query&format=json&prop=links&meta=&titles="+article+"&pllimit=500";
String[] lines = loadStrings(query);
for (int j = 0; j < lines.length; j++) {
String line = lines[j];
if (line.contains("\"title\":")) {
println(line);
// java string split
}
}
}
}
The query you're using apparently gives you a bunch of articles that your main article "Abdominal_pain" links to.
You need to go a step further and loop through all of those links. You can make your life a lot easier by using JSONObjects instead of parsing Strings like you're currently doing. Check out the loadJSONArray() function for more info, but basically you'd do this:
JSONArray links = loadJSONArray(query);
for (int i = 0; i < values.size(); i++) {
JSONObject link = values.getJSONObject(i);
String title = link.getString("title");
//fetch the info for that title
}
Once you have the title, you can then fetch the information for that page. An example query url is https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/all-agents/Abdominal_pain/daily/20151010/20151012 which returns this JSON:
{"items":[{"project":"en.wikipedia","article":"Abdominal_pain","granularity":"daily","timestamp":"2015101000","access":"all-access","agent":"all-agents","views":1134},{"project":"en.wikipedia","article":"Abdominal_pain","granularity":"daily","timestamp":"2015101100","access":"all-access","agent":"all-agents","views":1160},{"project":"en.wikipedia","article":"Abdominal_pain","granularity":"daily","timestamp":"2015101200","access":"all-access","agent":"all-agents","views":1313}]}
You'll have to do some aggregating to get the totals, or maybe the total is somewhere else in the API.
You're going to have to do a little bit of research on exactly what the API can return. Reading through the documentation is a big part of programming. Luckily the Wikipedia API has great documentation, and that's where you should be looking.
I'd recommend trying something out and posting another question, along with an MCVE, if you get stuck. Good luck.
See also: How to use Wikipedia API to get the page view statistics of a particular page in wikipedia?
I am offloading my search feature on a relational database to Azure Search. My Products tables contains columns like serialNumber, PartNumber etc.. (there can be multiple serialNumbers with the same partNumber).
I want to create a suggestor that can autocomplete partNumbers. But in my scenario I am getting a lot of duplicates in the suggestions because the partNumber match was found in multiple entries.
How can I solve this problem ?
The Suggest API suggests documents, not queries. If you repeat the partNumber information for each serialNumber in your index and then suggest based on partNumber, you will get a result for each matching document. You can see this more clearly by including the key field in the $select parameter. Azure Search will eliminate duplicates within the same document, but not across documents. You will have to do that on the client side, or build a secondary index of partNumbers just for suggestions.
See this forum thread for a more in-depth discussion.
Also, feel free to vote on this UserVoice item to help us prioritize improvements to Suggestions.
I'm facing this problem myself. My solution does not involve a new index (this will only get messy and cost us money).
My take on this is a while-loop adding 'UserIdentity' (in your case, 'partNumber') to a filter, and re-search until my take/top-limit is met or no more suggestions exists:
public async Task<List<MachineSuggestionDTO>> SuggestMachineUser(string searchText, int take, string[] searchFields)
{
var indexClientMachine = _searchServiceClient.Indexes.GetClient(INDEX_MACHINE);
var suggestions = new List<MachineSuggestionDTO>();
var sp = new SuggestParameters
{
UseFuzzyMatching = true,
Top = 100 // Get maximum result for a chance to reduce search calls.
};
// Add searchfields if set
if (searchFields != null && searchFields.Count() != 0)
{
sp.SearchFields = searchFields;
}
// Loop until you get the desired ammount of suggestions, or if under desired ammount, the maximum.
while (suggestions.Count < take)
{
if (!await DistinctSuggestMachineUser(searchText, take, searchFields, suggestions, indexClientMachine, sp))
{
// If no more suggestions is found, we break the while-loop
break;
}
}
// Since the list might me bigger then the take, we return a narrowed list
return suggestions.Take(take).ToList();
}
private async Task<bool> DistinctSuggestMachineUser(string searchText, int take, string[] searchFields, List<MachineSuggestionDTO> suggestions, ISearchIndexClient indexClientMachine, SuggestParameters sp)
{
var response = await indexClientMachine.Documents.SuggestAsync<MachineSearchDocument>(searchText, SUGGESTION_MACHINE, sp);
if(response.Results.Count > 0){
// Fix filter if search is triggered once more
if (!string.IsNullOrEmpty(sp.Filter))
{
sp.Filter += " and ";
}
foreach (var result in response.Results.DistinctBy(r => new { r.Document.UserIdentity, r.Document.UserName, r.Document.UserCode}).Take(take))
{
var d = result.Document;
suggestions.Add(new MachineSuggestionDTO { Id = d.UserIdentity, Namn = d.UserNamn, Hkod = d.UserHkod, Intnr = d.UserIntnr });
// Add found UserIdentity to filter
sp.Filter += $"UserIdentity ne '{d.UserIdentity}' and ";
}
// Remove end of filter if it is run once more
if (sp.Filter.EndsWith(" and "))
{
sp.Filter = sp.Filter.Substring(0, sp.Filter.LastIndexOf(" and ", StringComparison.Ordinal));
}
}
// Returns false if no more suggestions is found
return response.Results.Count > 0;
}
public async Task<List<string>> SuggestionsAsync(bool highlights, bool fuzzy, string term)
{
SuggestParameters sp = new SuggestParameters()
{
UseFuzzyMatching = fuzzy,
Top = 100
};
if (highlights)
{
sp.HighlightPreTag = "<em>";
sp.HighlightPostTag = "</em>";
}
var suggestResult = await searchConfig.IndexClient.Documents.SuggestAsync(term, "mysuggestion", sp);
// Convert the suggest query results to a list that can be displayed in the client.
return suggestResult.Results.Select(x => x.Text).Distinct().Take(10).ToList();
}
After getting top 100 and using distinct it works for me.
You can use the Autocomplete API for that where does the grouping by default. However, if you need more fields together with the result, like, the partNo plus description it doesn't support it. The partNo will be distinct though.
I have an item bucket with more then 30 000 items inside. What I need is to quickly search items that have particular field set to particular value, or even better is to make something like SELECT WHERE fieldValue IN (1,2,3,4) statement. Are there any ready solutions?
I searched the web and the only thing I found is "Developer's Guide to Item
Buckets and Search" but there is no code examples.
You need something like this. The Bucket item is an IIndexable so it can be searched using Sitecore 7 search API.
This code snippet below can easily be adapted to meet your needs and it's just a question of modifying the where clause.if you need any further help with the sitecore 7 syntax just write a comment on the QuickStart blog post below and I'll get back to you.
var bucketItem = Sitecore.Context.Database.GetItem(bucketPath);
if (bucketItem != null && BucketManager.IsBucket(bucketItem))
{
using (var searchContext = ContentSearchManager.GetIndex(bucketItem as IIndexable).CreateSearchContext())
{
var result = searchContext.GetQueryable<SearchResultItem().Where(x => x.Name == itemName).FirstOrDefault();
if(result != null)
Context.Item = result.GetItem();
}
}
Further reading on my blog post here:
http://coreblimey.azurewebsites.net/sitecore-7-search-quick-start-guide/
Using Sitecore Content Editor:
Go to the bucket item then In search tab, start typing the following (replace fieldname and value with actual field name and value):
custom:fieldname|value
Then hit enter, you see the result of the query, you can multiple queries at once if you want.
Using Sitecore Content Search API:
using Sitecore.ContentSearch;
using Sitecore.ContentSearch.Linq;
using Sitecore.ContentSearch.SearchTypes;
using Sitecore.ContentSearch.Linq.Utilities
ID bucketItemID = "GUID of your bucket item";
ID templateID = "Guid of your item's template under bucket";
string values = "1,2,3,4,5";
using (var context = ContentSearchManager.GetIndex("sitecore_web_index").CreateSearchContext())
{
var predicate = PredicateBuilder.True<SearchResultItem>();
predicate = PredicateBuilder.And(item => item.TemplateId == new ID(templateID)
&& item.Paths.Contains(bucketItemID));
var innerPredicate = PredicateBuilder.False<SearchResultItem>();
foreach(string val in values.Split(','))
{
innerPredicate = PredicateBuilder.False<SearchResultItem>();
innerPredicate = innerPredicate.Or(item => item["FIELDNAME"] == val);
}
predicate = predicate.And(innerPredicate);
var result = predicate.GetResults();
List<Item> ResultsItems = new List<Item>();
foreach (var hit in result.Hits)
{
Item item = hit.Document.GetItem();
if(item !=null)
{
ResultsItems .Add(item);
}
}
}
The following links can give good start with the Search API:
http://www.fusionworkshop.co.uk/news-and-insight/tech-lab/sitecore-7-search-a-quickstart-guide#.VPw8AC4kWnI
https://www.sitecore.net/learn/blogs/technical-blogs/sitecore-7-development-team/posts/2013/06/sitecore-7-poco-explained.aspx
https://www.sitecore.net/learn/blogs/technical-blogs/sitecore-7-development-team/posts/2013/05/sitecore-7-predicate-builder.aspx
Hope this helps!
Given that the CRM 2011 linq provider performs paging automatically behind the scenes.
Is there a way to set an upper limit on the number of records fetched when a linq
query is executed (similar to setting a PagingInfo.Count on a QueryExpression for paging)
I have a scenario where I need approx 20K+ records to be pulled for an update(no I cannot and do not need to filter down the record set further). Ideally I'd prefer to use the Skip & Take operators but since Count is not supported how would you know how many records to skip and when
to stop fetching more records.
Ideally I'd like to use TPL and processes batches of say 3K or 5K records in parallel so that I can get more throughput and don't have to block. The OrganizationserviceContext is not thread safe from what I know. Are there any good examples that illustrate how to partition the dataset in this case say using Parallel.For or Parallel.ForEach.
How would you partition and would you need to use a different context object for each parition?
Thanks.
UPDATE:
Here is what I came up with:
The idea is to get the total count of records to process and use PLINQ to farm out the processing of each subset of data across tasks using a new OrganizationServiceContext object per task.
static void Main(string[] args)
{
int pagesize = 2000;
// use FetchXML aggregate functions to get total count
// Reference: http://msdn.microsoft.com/en-us/library/gg309565.aspx
int totalcount = GetTotalCount();
int totalPages = (int)Math.Ceiling((double)totalcount / (double)pagesize);
try
{
Parallel.For(0, totalPages, () => new MyOrgserviceContext(),
(pageIndex, state, ctx) =>
{
var items = ctx.myEntitySet.Skip((pageIndex - 1) * pagesize).Take(pagesize);
var itemsArray = items.ToArray();
Console.WriteLine("Page:{0} - Fetched:{1}", pageIndex, itemsArray.Length);
return ctx;
},
ctx => ctx.Dispose()
);
}
catch (AggregateException ex)
{
//handle as needed
}
}
So the way I would do this would be to keep querying the records using skip and take until I run out of records.
Check out my example below, it uses int's for simplicity, but the approach should still apply to Linq-to-Crm.
So just keep performing your query, skipping previous records, taking the ones you want for that page, then counting at the end to see if you recieved a full page - if you didnt then you have run out of records.
Code
List<int> ints = new List<int>()
{
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
};
int pageNumber = 0;
int recordsPerPage = 3;
while(true)
{
IEnumerable<int> page = ints.Where(i => i < 11).Skip(recordsPerPage * pageNumber).Take(recordsPerPage);
foreach(int i in page)
{
Console.WriteLine(i);
}
Console.WriteLine("end of page");
pageNumber++;
if (page.Count() < recordsPerPage)
{
break;
}
}
Output:
1
2
3
end of page
4
5
6
end of page
7
8
9
end of page
10
end of page
Given a collection of items { url: 'http://blah' }. How can I tell if a record exists where the url is "http://stackoverflow.com"?
P.s. I am communicating with the c# driver
For any of the previous suggestions to be efficient you should be sure that there is an index on the url element. Otherwise a full collection scan will be required.
If you only expect the answer to be 0 or 1, count is probably the most efficient approach. If you think the count will be very large and all you really care about is whether there is one or more, FindOne is the most efficient approach.
It probably doesn't matter that FindOne returns the whole document unless the document is actually rather large. In that case you could tell the server to only return one field (the _id seems the most likely candidate):
var query = Query.EQ("url", "http://stackoverflow.com");
var fields = Fields.Include("_id");
var res = collection.Find(query).SetFields(fields).SetLimit(1).FirstOrDefault();
if (res == null) {
// no match found
}
you simply need check count of items returned by the query:
int count = collection.FindAs<Item>(Query.EQ("url", "http://stackoverflow.com")).Count();
if(count > 0)
{
//do some stuff
}
IMongoQuery query = Query.EQ("url", "http://stackoverflow.com");
var res = collection.FindOne(query);
if(res == null)//don't exist
{
}
Existence of Key in MongoDB can check by using Exists and second parameter as true or false
var filter = builder.Exists("style", false);
var RetrievedData = collection.Find(filter).ToList()
Refference Link