pymongo: 3.12.0
mongoengine: 0.23.1
I have a document:
class Logs(Document):
reference_id = StringField(default=None)
data = DictField(default=None)
In data field, i have a list failed_stories. This can have hundreds of elements and I want to perform pagination on it. So, i write this query as:
start_idx = 0
page_size = 10
reference_id = 'asdfg345678'
Logs.objects(reference_id=reference_id).fields(slice__data__failed_stories=[start_idx, page_size])
With this, i get one document in which all field are None except the dociment id (_id).
The following query results in document with correct data in document fields.
Logs.objects(reference_id=reference_id).get()
Is there any issue with the way I am writing this?
Note: I would like to do this with mongoengine only, if possible.
like the title mentions I want a Query that gets a random row of the existing database. After that I want to save the data which is in a specific column of that row in a variable for further purposes.
The query I have at the moment is as follows:
#Query("SELECT * FROM data_table ORDER BY RANDOM() LIMIT 1")
fun getRandomRow()
For now I am not sure if this query even works, but how would I go about writing my function to pass a specific column of that randomly selected row to a variable?
Ty for your advice, tips and/or solutions!
Your query is almost correct; however, you should specify a return type in the function signature. For example, if the records in the data_table table are mapped using a data class called DataEntry, then the query could read as shown below (note I've also added the suspend modifier so the query must be run using a coroutine):
#Query("SELECT * FROM data_table ORDER BY RANDOM() LIMIT 1")
suspend fun getRandomRow(): DataEntry?
If your application interacts with the database via a repository and view model (as described here: https://developer.android.com/topic/libraries/architecture/livedata) then the relevant methods would be along the lines of:
DataRepository
suspend fun findRandomEntry(): DataEntry? = dataEntryDao.getRandomRow()
DataViewModel
fun getRandomRecord() = viewModelScope.launch(Dispatchers.IO) {
val entry: DataEntry? = dataRepository.findRandomEntry()
entry?.let {
// You could assign a field of the DataEntry record to a variable here
// e.g. val name = entry.name
}
}
The above code uses the view model's coroutine scope to query the database via the repository and retrieve a random DataEntry record. Providing the returning DataEntry record is not null (i.e. your database contains data) then you could assign the fields of the DataEntry object to variables in the let block of the getRandomRecord() method.
As a final point, if it's only one field that you need, you could specify this in the database query. For example, imagine the DataEntry data class has a String field called name. You could retrieve this bit of information only and ignore the other fields by restructuring your query as follows:
#Query("SELECT name FROM data_table ORDER BY RANDOM() LIMIT 1")
suspend fun getRandomRow(): String?
If you go for the above option, remember to refactor your repository and view model to expect a String instead of a DataEntry object.
I am using query builder to return number of search result from the database table. Now I would like to display the result in the UI by only first three rows. How can I achieve this?
QueryAPI is lazy, when .toList(), .toTypedArray(), .toCollection(), .where(), etc occurs all resultset is retrieved (eager).
I recommend you to use this:
var limit = 3
var rs = Query.make(entity.XXX)...select()
rs.setPageSize(limit)
var paginatedRS = com.google.common.collect.Iterables.limit(rs,limit)
setPageSize method specifies how many rows will be fetch "by page"
limit method make a new iterator that have only the first (limit) rows
using (IDbConnection db = dbFactory.OpenDbConnection()) {
List<long> x = db.SelectLazy<long>(
"SELECT Id FROM MyTable").ToList();
}
Why is x null?
It works when I use Select instead of SelectLazy, or when I use SelectLazy on the entire row and not just the Id.
In OrmLite you use different API's to match the results you're after, e.g:
Select* API's for returning a List<MyTable>
Column* API's for returning a column of field values, e.g List<long>
Single* API's for returning a Single Row, e.g Table
Scalar* API's for returning a Single field value, e.g long
So to select a column as a List of fields you use db.Column, e.g:
var results = db.Column<long>(db.From<MyTable>().Select(x => x.Id));
These also have Raw Sql* equivalents, e.g:
var results = db.SqlColumn<long>("SELECT Id FROM MyTable");
I have a list that looks like:
Movie Year
----- ----
Fight Club 1999
The Matrix 1999
Pulp Fiction 1994
Using CAML and the SPQuery object I need to get a distinct list of items from the Year column which will populate a drop down control.
Searching around there doesn't appear to be a way of doing this within the CAML query. I'm wondering how people have gone about achieving this?
Another way to do this is to use DataView.ToTable-Method - its first parameter is the one that makes the list distinct.
SPList movies = SPContext.Current.Web.Lists["Movies"];
SPQuery query = new SPQuery();
query.Query = "<OrderBy><FieldRef Name='Year' /></OrderBy>";
DataTable tempTbl = movies.GetItems(query).GetDataTable();
DataView v = new DataView(tempTbl);
String[] columns = {"Year"};
DataTable tbl = v.ToTable(true, columns);
You can then proceed using the DataTable tbl.
If you want to bind the distinct results to a DataSource of for example a Repeater and retain the actual item via the ItemDataBound events' e.Item.DataItem method, the DataTable way is not going to work. Instead, and besides also when not wanting to bind it to a DataSource, you could also use Linq to define the distinct values.
// Retrieve the list. NEVER use the Web.Lists["Movies"] option as in the other examples as this will enumerate every list in your SPWeb and may cause serious performance issues
var list = SPContext.Current.Web.Lists.TryGetList("Movies");
// Make sure the list was successfully retrieved
if(list == null) return;
// Retrieve all items in the list
var items = list.GetItems();
// Filter the items in the results to only retain distinct items in an 2D array
var distinctItems = (from SPListItem item in items select item["Year"]).Distinct().ToArray()
// Bind results to the repeater
Repeater.DataSource = distinctItems;
Repeater.DataBind();
Remember that since there is no CAML support for distinct queries, each sample provided on this page will retrieve ALL items from the SPList. This may be fine for smaller lists, but for lists with thousands of listitems, this will seriously be a performance killer. Unfortunately there is no more optimized way of achieving the same.
There is no DISTINCT in CAML to populate your dropdown try using something like:
foreach (SPListItem listItem in listItems)
{
if ( null == ddlYear.Items.FindByText(listItem["Year"].ToString()) )
{
ListItem ThisItem = new ListItem();
ThisItem.Text = listItem["Year"].ToString();
ThisItem.Value = listItem["Year"].ToString();
ddlYear.Items.Add(ThisItem);
}
}
Assumes your dropdown is called ddlYear.
Can you switch from SPQuery to SPSiteDataQuery? You should be able to, without any problems.
After that, you can use standard ado.net behaviour:
SPSiteDataQuery query = new SPSiteDataQuery();
/// ... populate your query here. Make sure you add Year to the ViewFields.
DataTable table = SPContext.Current.Web.GetSiteData(query);
//create a new dataview for our table
DataView view = new DataView(table);
//and finally create a new datatable with unique values on the columns specified
DataTable tableUnique = view.ToTable(true, "Year");
After coming across post after post about how this was impossible, I've finally found a way. This has been tested in SharePoint Online. Here's a function that will get you all unique values for a column. It just requires you to pass in the list Id, View Id, internal list name, and a callback function.
function getUniqueColumnValues(listid, viewid, column, _callback){
var uniqueVals = [];
$.ajax({
url: _spPageContextInfo.webAbsoluteUrl + "/_layouts/15/filter.aspx?ListId={" + listid + "}&FieldInternalName=" + column + "&ViewId={" + viewid + "}&FilterOnly=1&Filter=1",
method: "GET",
headers: { "Accept": "application/json; odata=verbose" }
}).then(function(response) {
$(response).find('OPTION').each(function(a,b){
if ($(b)[0].value) {
uniqueVals.push($(b)[0].value);
}
});
_callback(true,uniqueVals);
},function(){
_callback(false,"Error retrieving unique column values");
});
}
I was considering this problem earlier today, and the best solution I could think of uses the following algorithm (sorry, no code at the moment):
L is a list of known values (starts populated with the static Choice options when querying fill-in options, for example)
X is approximately the number of possible options
1. Create a query that excludes the items in L
1. Use the query to fetch X items from list (ordered as randomly as possible)
2. Add unique items to L
3. Repeat 1 - 3 until number of fetched items < X
This would reduce the total number of items returned significantly, at the cost of making more queries.
It doesn't much matter if X is entirely accurate, but the randomness is quite important. Essentially the first query is likely to include the most common options, so the second query will exclude these and is likely to include the next most common options and so on through the iterations.
In the best case, the first query includes all the options, then the second query will be empty. (X items retrieved in total, over 2 queries)
In the worst case (e.g. the query is ordered by the options we're looking for, and there are more than X items with each option) we'll make as many queries as there are options. Returning approximately X * X items in total.