Using Django 3, python 3.6, django-datatable-view 1.19.1
Trying to do a datatable with columns from my model and computed before output ones.
I draw all values that I needed but after trying to sort custom column getting an error:
Cannot resolve keyword 'XXXX' into field. Choices are: ...
I've found a way to register a column as virtual column, without db source, but it was in django-datatable-view 0.5.4 docs but those ways don't work anymore. In last version documentation links with info that I need are unavailable.
Please, help me to figure out, how can I deal with custom computed columns from my model's fields( sort, render )
This is a little tricky now.
To define computed fields:
class ListJson(BaseDatatableView):
columns = ["id", "status_code", "computed_field"]
order_columns = ["id", "status_code"]
# Override render_column method
def render_column(self, row, column):
if column == "computed_field":
return row.computed_field()
else:
return super(ListJson, self).render_column(row, column)
This allows you to return the computed_field method value().
The situation becomes complicated when we want to sort on calculated fields. In this case, it is best to disable serverSide operations in JavaScript
$.extend($.fn.dataTable.defaults, {
serverSide: false,
});
However, you will then have to return all the lines at once, which can kill the server.
If you want to sort on the backend side, you need to arrange and return the appropriate queryset with a virtual field.
class ListJson(BaseDatatableView):
def get_initial_queryset(self):
return qs
Just build your query like THIS.
Related
I'm new to Flask and have started designing a front end for an inventory management database using Flask-AppBuilder.
I have created several models and have have managed to display my sqlite data in tables using Flask-AppBuilder's views.
However, I don't seem to be able to find the equivalent of SQLite WHERE clause to filter or "restrict" column data. I've been reading a lot about sqlalchemy, filters, queries but this has left me more confused that anything else and the explanations seem to be extremely elaborate and complicated to do something which is extremely simple.
Assuming we reproduce the following SQLite query in Flask-AppBuilder:
SELECT Field_A
FROM Table_A
WHERE Field_A = 'some text'
with:
result = session.query(Table_A).filter_by(Field_A = 'some text').all()
Where does the above line of code go in my app?
Considering I have the following Class:
class Table_A(Model):
id = Column(Integer, primary_key=True)
Field_A = Column(String)
def __repr__(self):
return self
and View:
class Table_AView(ModelView):
datamodel = SQLAInterface(Table_AView)
label_columns = {'Field_A':'A'}
list_columns = ['Field_A']
After much digging flask-appbuilder uses it's own filterclass in order to enable you to filter your views.
All the classes are referenced here on GitHub:
Flask Filter Clases List
Also not the difference between FilterEqual and FilterEqualFunction here:
What is the difference between : FilterEqual and FilterEqualFunction?
For other customisation and first port of call of Flask-appbuilder go straight to the API Reference where you'll find a couple of examples of the filterclass in action.
In essence it is extremely simple. In your views.py code within the ModelView class you want to filter simply add base_filters = [['field_A', FilterEqual, 'abc']] like so:
`class Table_AView(ModelView):
datamodel = SQLAInterface(Table_AView)
label_columns = {'Field_A':'A'}
list_columns = ['Field_A']
base_filters = [['field_A', FilterEqual, 'abc']]`
This will only show the lines where the field_A variable is equal to abc.
Hope this helps someone as it took me nearly (sigh) two weeks to figure it out...
SQLALchemy is an ORM (Object-Relational Mapping), it mean that you dont have to deal with raw SQL, you will call a function that you "build" (by adding filters in your case). It will transparently generate an SQL query, execute it, and return the result as python objects.
I would suggest you to read closely at sqlalchemy documentation about filters again, especially filter_by :
http://docs.sqlalchemy.org/en/latest/orm/query.html#sqlalchemy.orm.query.Query.filter_by
It is the easiest way to apply a WHERE with sqlalchemy.
If you have declared correctly the model for Table_A, you should be able to use it so:
result = session.query(Table_A).filter_by(Field_A = 'some text').all()
Here session.query(Table_A).filter_by(Field_A = 'some text') will generate the SQL, and .all() will execute it.
I'm using Python Pandas Dataframe for Data Analyse of some logs.
I have a csv with something like:
number_items event_type ... ... ... session_id ... ... ...
My problem is that in my session there are different types of events, and only one of them has something for number_items. Or, numbers_items is what interests me.
So what I want to see is how each parameter of each event influences the number_items.
So, what I want to do is:
Copy the number_items of the event that has it (always the last one in the session) to all the other events of the session. Separate each event_type in a different Dataframe (to avoid a lot of nulls that exist only because the attribute doesn't correspond to the event) and analyse it.
I'm blocked at the first part
I tried something like this:
currentSession = '0'
currentItems = 0
for index, row in reversed(df.iterrows()) :
if row['session_id'] == currentSession :
row['number_items'] = currentItems
else :
currentSession = row['session_id']
currentItems = row['number_items']
Obviously, it's not working, I just wanted to show the idea.
I'm kind of new in Python, so I would appreciate some help.
Thanks
edit: data sample here
For security reasons, I let only the relevant information
The rows you get back from iterrows are copies so they dont overwrite your original dataframe. Use another form of iterator that references the original dataframe.
see here Updating value in iterrow for pandas
(also im note entirely sure what it is you are trying to do but instinctively it seems very inefficient - i suspect there are natural pandas methods which might do what you trying to achieve in one or two lines, look up the where() method)
Is it possible to do a "Lookup" with Kiba.
Since it's quite a normal process in a etl.
Could you show a demo if yes, thanks.
Yes, a lookup can be done with Kiba!
For a tutorial, see this live coding session I recorded, I create a lookup transform to lookup extra fields using a given fields by tapping in the MovieDB database.
Leveraging this example, you could for instance implement a simple ActiveRecord lookup using a block transform:
# assuming you have a 'country_iso_2' field in the row above
transform do |row|
country = Country.where(iso_2: row['country_iso_2']).first
row['country_name'] = country.try(:name) || 'Unknown'
row
end
or you could extract a more reusable class transform that you would call like this:
transform ActiveRecordLookup, model: Country,
lookup_on: 'country_iso_2',
fetch_fields: { 'name' => 'country_name' }
transform DefaultValue, 'name' => 'Unknown'
Obviously, if you have the need for large volumes, you will have to implement some improvements (e.g. caching, bulk reading).
Hope this helps!
With Django/Haystack/SOLR, I'd like to be able to restrict the result of a search to those records within a particular range of django_ids. Getting these IDs is not a problem, but trying to filter by them produces some unexpected effects. The code looks like this (extraneous code trimmed for clarity):
def view_results(request,arg):
# django_ids list is first calculated using arg...
sqs = SearchQuerySet().facet('example_facet') # STEP_1
sqs = sqs.filter(django_id__in=django_ids) # STEP_2
view = search_view_factory(
view_class=SearchView,
template='search/search-results.html',
searchqueryset=sqs,
form_class=FacetedSearchForm
)
return view(request)
At the point marked STEP_1 I get all the database records. At STEP_2 the records are successfully narrowed down to the number I'd expect for that list of django_ids. The problem comes when the search results are displayed in cases where the user has specified a search term in the form. Rather than returning all records from STEP_2 which match the term, I get all records from STEP_2 plus all from STEP_1 which match the term.
Presumably, therefore, I need to override one/some of the methods in for SearchView in haystack/views.py, but what? Can anyone suggest a means of achieving what is required here?
After a bit more thought, I found a way around this. In the code above, the problem was occurring in the view = search_view_factory... line, so I needed to create my own SearchView class and override the get_results(self) method in order to apply the filtering after the search has been run with the user's search terms. The result is code along these lines:
class MySearchView(SearchView):
def get_results(self):
search = self.form.search()
# The ID I need for the database search is at the end of the URL,
# but this may have some search parameters on and need cleaning up.
view_id = self.request.path.split("/")[-1]
view_query = MyView.objects.filter(id=view_id.split("&")[0])
# At this point the django_ids of the required objects can be found.
if len(view_query) > 0:
view_item = view_query.__getitem__(0)
django_ids = []
for thing in view_item.things.all():
django_ids.append(thing.id)
search = search.filter_and(django_id__in=django_ids)
return search
Using search.filter_and rather than search.filter at the end was another thing which turned out to be essential, but which didn't do what I needed when the filtering was being performed before getting to the SearchView.
I want to facet the results based on the different model_names (classes) returned. Is there an easy way to do this?
Have you tried adding a SearchIndex field with this information? E.g.
class NoteIndex(SearchIndex, indexes.Indexable):
title = CharField(model_attr='title')
facet_model_name = CharField(faceted=True)
def get_model(self):
return Note
def prepare_facet_model_name(self, obj):
return "note"
class MemoIndex(SearchIndex, indexes.Indexable):
title = CharField(model_attr='title')
facet_model_name = CharField(faceted=True)
def get_model(self):
return Memo
def prepare_facet_model_name(self, obj):
return "memo"
And so on, simply returning a different string for each search index. You could also create a mixin and return the name of the model returned by get_model too.
Presuming you've added this field to each of your SearchIndex definitions, just chain the facet method to your results.
results = form.search().facet('facet_model_name')
Now the facet_counts method will return a dictionary with the faceted fields and count of results for each facet value, in this case, the model names.
Note that the field here is labeled verbosely to avoid a possible conflict with model_name, a field added by Haystack. It's not faceted, and I'm not sure if duplicating it will cause a conflict.
If you just want to filter on the model type, you can use the ModelSearchForm
The Docs have a really good walk-through for this.
The minimum you'll need:
is to add faceted=True to the params of your model_names field.
Rebuild your schema and indices.
add .facet('model_names') to whatever SearchQuerySet you're wanting to facet.
More explanation on the question would enable a more complete answer.