Getting AutoQuery pagination to work with left join - servicestack

In my AutoQuery request I have a left join specified so I can query on properties in the joined table.
public class ProductSearchRequest : QueryDb<Book>
, ILeftJoin<Book, BookAuthor>, ILeftJoin<BookAuthor, Author>
{}
If I use standard way of autoquery like so:
var q = AutoQuery.CreateQuery(request, base.Request);
var results = AutoQuery.Execute(request, q);
And 100 are being requested, then often less than 100 will be retuned as the Take() is based on results with a left join.
To remedy this I am doing this instead:
var q = AutoQuery.CreateQuery(request, base.Request);
q.OrderByExpression = null //throws error if orderby exists
var total = Db.Scalar<int>(q.Select(x => Sql.CountDistinct(x.Id))); //returns 0
var q1 = AutoQuery.CreateQuery(request, base.Request).GroupBy(x => x);
var results = Db.Select<Book>(q1);
return new QueryResponse<Book>
{
Offset = q1.Offset.GetValueOrDefault(0),
Total = total
Results = results
};
The group by appears to return correct number of results so paging works but the Total returns 0.
I also tried:
var total2 = (int)Db.Count(q1);
But even though q1 has a GroupBy() it returns the number of results including the left join and not the actual query
How can I get the true total of the query?
(Getting some official docs on how to do paging and totals with autoquery & left join would be very helpful as right now it's a bit confusing)

Your primary issue stems from trying to return a different total then the actual query AutoQuery executes. If you have multiple left joins, the total is the total results of the query it executes not the number of rows in your source table.
So you're not looking for the "True total", rather you're looking to execute a different query to get a different total than the query that's executed, but still deriving from the original query as its basis. First consider using normal INNER JOINS (IJoin<>) instead of LEFT JOINS so it only returns results for related rows in joined tables which the total will reflect accordingly.
Your total query that returns 0 is likely returning no results, so I'd look at looking at the query in an SQL Profiler so you can see the query that's executed. You can also enable logging of OrmLite queries with Debug logging enabled and in your AppHost:
OrmLiteUtils.PrintSql();
Also note that GroupBy() of the entire table is unusual, you would normally group by a single or multiple explicit selected columns, e.g:
.GroupBy(x => x.Id);
.GroupBy(x => new { x.Id, x.Name });

Related

Count distinct doesn't work when using OrderBy & join

I have the following query trying to get count of a query:
var testQuery = Db
.From<Blog>()
.LeftJoin<BlogToBlogCategory>()
.Where(x => x.IsDeleted == false)
.OrderBy(x => x.ConvertedPrice);
var testCount = Db.Scalar<int>(testQuery.Select<Blog>(x => Sql.CountDistinct(x.Id)));
var results = Db.LoadSelect(testQuery.SelectDistinct());
It gives error:
42803: column "blog.converted_price" must appear in the GROUP BY clause or be used in an aggregate function
Issue seems to be the orderby statement. If I remove it then the error goes away. Why does this stop count distinct working?
I am having to clear orderby on all queries I do like this. Is it supposed to work this way?
Also I just realised count is wrong. Results is 501 unique records and testCount is 538.
What am I doing wrong?
Whenever in doubt with what an OrmLite query is generating, you can use the BeforeExecFilter to inspect the DB command before its executed or to just output the query to the Console you can use:
OrmLiteUtils.PrintSql();
You shouldn't be using OrderBy with aggregate scalar functions like COUNT which is meaningless and will fail in your case because it needs to included the GROUP BY clause for joined table queries.
Your specifically querying for COUNT(DISTINCT Id) if you wanted the row count for the query you can instead use:
var testCount = Db.RowCount(testQuery);
If you wanted to use COUNT(*) instead, you can use:
var testCount = Db.Count(testQuery);

AutoQuery/OrmLite incorrect total value when using joins

I have this autoquery implementation
var q = AutoQuery.CreateQuery(request, base.Request).SelectDistinct();
var results = Db.Select<ProductDto>(q);
return new QueryResponse<ProductDto>
{
Offset = q.Offset.GetValueOrDefault(0),
Total = (int)Db.Count(q),
Results = results
};
The request has some joins:
public class ProductSearchRequest : QueryDb<GardnerRecord, ProductDto>
, ILeftJoin<GardnerRecord, RecordToBicCode>, ILeftJoin<RecordToBicCode, GardnerBicCode>
{
}
The records gets returned correctly but the total is wrong. I can see 40,000 records in database but it tells me there is 90,000. There is multiple RecordToBicCode for each GardnerRecord so it's giving me the number of records multiplied by the number of RecordToBicCode.
How do I match the total to the number of GardnerRecord matching the query?
I am using PostgreSQL so need the count statement to be like
select count(distinct r.id) from gardner_record r etc...
Dores OrmLite have a way to do this?
I tried:
var q2 = q;
q2.SelectExpression = "select count(distinct \"gardner_record\".\"id\")";
q2.OrderByExpression = null;
var count = Db.Select<int>(q2);
But I get object reference not set error.
AutoQuery is returning the correct total count for your query of which has left joins so will naturally return more results then the original source table.
You can perform a Distinct count with:
Total = Db.Scalar<long>(q.Select(x => Sql.CountDistinct(x.Id));

How to get a SELECT DISTINCT on a SelectMulti query in ServiceStack OrmLite?

I'm trying to get a distinct result set of tuples, but the Distinct never gets added to query.
Example
List<Tuple<Alpha, Beta>> results;
var q = dbConn.From<Alpha>()
.Join<Alpha, Beta>((a, b) => a.Id == b.AlphaId)
...
... more joins and Wheres
...
.SelectDistinct();
results = dbConn.SelectMulti<Alpha, Beta>(q);
Adding the SelectDistinct, or not, make no difference to the outputted SQL and hence results.
How do I get SelectMulti to work with Distinct?
Thanks.
I've just added support for this in this commit where if .SelectDistinct() is used in the SqlExpression<T> then it will execute the SQL query using SELECT DISTINCT, e.g:
var results = dbConn.SelectMulti<Alpha, Beta>(q.SelectDistinct());
This change is available from v5.4.1 that's now available on MyGet.

Return joined column with kentico get documents

I have the following query to get a set of documents with their page views
var activityQuery = ActivityInfoProvider.GetActivities()
.Columns($"COUNT(*) AS [{ColumnNames.PageViews}]", ColumnNames.ActivityValue)
.WhereEquals(ColumnNames.ActivityType, ActivityTypes.BlogPostView)
.GroupBy(ColumnNames.ActivityValue)
.ToString(true);
var query = DocumentHelper.GetDocuments(PageTypes.BlogPost)
.Columns(_blogCardColumns)
.Source(src => src.LeftJoin(new QuerySourceTable($"({activityQuery})", "A"), "V.DocumentGUID", "A.ActivityValue"))
.OrderByDescending(ColumnNames.BlogPostDate)
.Published(selectOnlyPublished)
.NestingLevel(1)
.Path(path);
This query works well, but it returns me a set of treenodes and as far as I can see, it doesn't return me the PageViews column.
Is there a way to do a query and the result set include the page views column (or is there a way to get it from the treenode element)?
I have tried the following with the results, but it just returns null:
var nodes = query.ToList()
nodes.First().GetValue("PageViews");
but this returns null, even though if I get the query sql and run it in sql management studio, it returns 28 for PageViews
Ok as it turns out .ToList() returns a treenode collection. If I change the last line to
var dataset = query.Result;
It returns a dataset with all the columns I requested in _blogCardsColumns and then I can access my pageviews by using
foreach (DataRow row in dataSet.Tables[0].Rows)
{
int pageviews = int.Parse(dataRow["PageViews"].ToString())
}

Error in Linq: The text data type cannot be selected as DISTINCT because it is not comparable

I've a problem with LINQ. Basically a third party database that I need to connect to is using the now depreciated text field (I can't change this) and I need to execute a distinct clause in my linq on results that contain this field.
I don't want to do a ToList() before executing the Distinct() as that will result in thousands of records coming back from the database that I don't require and will annoy the client as they get charged for bandwidth usage. I only need the first 15 distinct records.
Anyway query is below:
var query = (from s in db.tSearches
join sc in db.tSearchIndexes on s.GUID equals sc.CPSGUID
join a in db.tAttributes on sc.AttributeGUID equals a.GUID
where s.Notes != null && a.Attribute == "Featured"
select new FeaturedVacancy
{
Id = s.GUID,
DateOpened = s.DateOpened,
Notes = s.Notes
});
return query.Distinct().OrderByDescending(x => x.DateOpened);
I know I can do a subquery to do the same thing as above (tSearches contains unique records) but I'd rather a more straightfoward solution if available as I need to change a number of similar queries throughout the code to get this working.
No answers on how to do this so I went with my first suggestion and retrieved the unique records first from tSearch then constructed a subquery with the non unique records and filtered the search results by this subquery. Answer below:
var query = (from s in db.tSearches
where s.DateClosed == null && s.ConfidentialNotes != null
orderby s.DateOpened descending
select new FeaturedVacancy
{
Id = s.GUID,
Notes = s.ConfidentialNotes
});
/* Now filter by our 'Featured' attribute */
var subQuery = from sc in db.tSearchIndexes
join a in db.tAttributes on sc.AttributeGUID equals a.GUID
where a.Attribute == "Featured"
select sc.CPSGUID;
query = query.Where(x => subQuery.Contains(x.Id));
return query;

Resources