I'm trying to setup CursorPagination with DRF for a list of transaction records (ordered by creation date).
I can't figure out how to do an initial request because I don't know the cursor yet at that stage. Surprisingly, I can't find an example of this.
Also, is there a way to set the page size per request with CursorPagination, the PageNumberPagination have page_size_query_param and max_page_size and they aren't there for CursorPagination.
Here's what I have so far:
class RecordPagination(pagination.CursorPagination):
page_size = 10
class RecordsOverview(generics.ListAPIView):
serializer_class = RecordSerializer
logging_methods = ['GET']
queryset = Record.objects.all()
pagination_class = RecordPagination
# Note: this is my way to dynamically set the page size,
# it is totally hacky, so I'm open to suggestions
# is_number method is left out for brevity
def get(self, request, *args, **kwargs):
page_size = request.GET.get('page_size', '')
if self.is_number(page_size) and int(page_size) > 0:
self.paginator.page_size = int(page_size)
return self.list(request, *args, **kwargs)
Then in my test I do a GET request:
response = self.client.get(GET_RECORDS_URL, data={'page_size': 2})
I get back a 'next' url that looks something like this:
http://testserver/api/v1/records/?cursor=cj0xJnA9MjAxNy0wOS0yMCsxNCUzQTM0JTNBNDkuNjUxMDU4JTJCMDAlM0EwMA%3D%3D&page_size=2
If I do get(next_url) I will get the next records OK, and this time I don't have to pass data={'page_size': 2}.
Please, let me know if I can do all of this in a cleaner and more consistent way.
This is how I use CursorPagination without any complications:
from rest_framework.pagination import CursorPagination
class CursorSetPagination(CursorPagination):
page_size = 5
page_size_query_param = 'page_size'
ordering = '-timestamp' # '-created' is default
class MyListAPIView(generics.ListAPIView):
queryset = MyObject.objects.all()
serializer_class = MySerializer
pagination_class = CursorSetPagination
Related
i have the following code for a function based view:
fav = bool
if post.favourites.filter(id=request.user.id).exists():
fav=True
but i want to put that in the below class based view:
class PostDetailView(DetailView):
model = Post
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
context['productobj'] = Post.objects.get(id = self.kwargs['pk'])
return context
i am fairly new to Django, so i don't know all that much about functions inside class based views and how to pass parameters from them. can anyone help to add the code to the above class based view and pass "fav" to a context dictionary.
In the function get_context_data you can use the self. before the request. So...
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
context['productobj'] = Post.objects.get(id=self.request.user.id)
return context
I have a class-based view (lets say, DetailView) which renders a page with a list of objects based on slug in the URL. But if an object with the given slug does not exist, it gives me an error. What I want is to redirect to the main page instead of raising an error. It should be easy, but I can't understand how to do this, so I wanna ask for help here.
Simply, I wanna find something like "success_url" but for errors.
Example:
views.py
class ShowExerciseRecords(ListView):
def get_queryset(self):
exercise = Exercise.objects.get(slug=self.kwargs['slug'])
return exercise.record__set.all()
urls.py
urlpatterns = [
path('/exercise/<slug:slug>/', ShowExerciseRecords.as_view())
path('', index, name='home')
]
please try this code,
class ShowExerciseRecords(ListView):
def get_queryset(self):
records = Record.objects.none() # you should replace the exact model name
exercise = Exercise.objects.filter(slug=self.kwargs['slug']).first()
if exercise:
records = exercise.record__set.all()
return records
def get(self, request, *args, **kwargs):
self.object_list = self.get_queryset()
if self.object_list:
context = self.get_context_data()
return self.render_to_response(context)
return redirect("main-page-url") # you should change the url to your case
you can use try except for example:
try:
Exercise.objects.get(slug=self.kwargs['slug'])
except Exercise.DoesNotExist:
redirect("main-page-url")
I try to run Scrapy spider and some SDK call to another resource inside Django. The main idea collect results from both of them in one list once it will be ready and output it to view. SDK is working in sync way, so there are no issues. But I could not get results from a spider. Anyone could point me to the correct solution?
My code to run parses looks like this:
class scrapyParser(Parser):
def __init__(self, keywords=None, n_items=None):
super().__init__(keywords, n_items)
def parse(self):
result = []
if not super().parse():
return False
crawler = UrlCrawlerScript(Parser1, result, [BASE_PATH + self.keywords])
crawler.start()
crawler.join()
print(crawler.outputResponse)
return result[:self.n_items]
class UrlCrawlerScript(Process):
def __init__(self, result, urls):
Process.__init__(self)
settings = get_project_settings()
self.crawler = Crawler(spider, settings=settings)
self.crawler.signals.connect(reactor.stop, signal=signals.spider_closed)
self.spider = spider
self.urls = urls
self.outputResponse = result
#inlineCallbacks
def cycle_run(self):
yield self.crawler.crawl(Parser1, outputResponse=self.outputResponse, start_urls=self.urls)
returnValue(self.outputResponse)
def run(self):
result = self.cycle_run()
result.addCallback(print)
reactor.run()
Parse code is very simple and it has such a template:
import scrapy
class Parser1(scrapy.Spider):
name = 'items'
allowed_domains = ['domain.com']
def parse(self, response):
...
# parsing page
for item in row_data:
scraped_info = {
...
}
self.outputResponse.append(scraped_info)
So I could not get anything in the output of parse. It returns an empty list. However, I'm at the very beginning of my way with async calls in Python and Twisted framework. It's highly possible I just messed something.
After doing a lot of different code snippets and looking for SO answers I finally found an easy and elegant solution. Using scrapyscript.
class scrapyParser(Parser):
def __init__(self, keywords=None, n_items=None):
super().__init__(keywords, n_items)
def parse(self):
result = []
if not super().parse():
return False
processor = Processor(settings=None)
job1 = Job(Parser1, url=URL1 + self.keywords)
job2 = Job(Parser2, url=URL2 + self.keywords)
return processor.run([job1, job2])
Source: https://stackoverflow.com/a/62902603/1345788
I'm attempting to learn flask, so decided to follow this tutorial:
https://www.blog.pythonlibrary.org/2017/12/14/flask-101-adding-editing-and-displaying-data/
I just updated my main function with the below:
#app.route('/results')
def search_results(search):
results = []
search_string = search.data['search']
if search.data['search'] == '':
qry = db_session.query(Album)
results = qry.all()
if not results:
flash('No results found!')
return redirect('/')
else:
# display results
table = Results(results)
table.border = True
return render_template('results.html', table=table)
but when I add an album to the DB and try to query it back using search option it says no results. The DB file was created correctly and I have exactly the same code as in the tutorial up to this point.
The only change I made was adding from tables import Results. Full main.py below. Can you please give me some guidance about where to look for the culprit? Like I said, just learning, so any suggestions re resources in a friendly laid out way would be much appreciated (beginner programmer).
from app import app
from db_setup import init_db, db_session
from forms import MusicSearchForm, AlbumForm
from flask import flash, render_template, request, redirect
from models import Album, Artist
from tables import Results
init_db()
def save_changes(album, form, new=False):
"""
Save the changes to the database
"""
# Get data from form and assign it to the correct attributes
# of the SQLAlchemy table object
artist = Artist()
artist.name = form.artist.data
album.artist = artist
album.title = form.title.data
album.release_date = form.release_date.data
album.publisher = form.publisher.data
album.media_type = form.media_type.data
if new:
# Add the new album to the database
db_session.add(album)
# commit the data to the database
db_session.commit()
#app.route('/', methods=['GET', 'POST'])
def index():
search = MusicSearchForm(request.form)
if request.method == 'POST':
return search_results(search)
return render_template('index.html', form=search)
#app.route('/results')
def search_results(search):
results = []
search_string = search.data['search']
if search.data['search'] == '':
qry = db_session.query(Album)
results = qry.all()
if not results:
flash('No results found!')
return redirect('/')
else:
# display results
table = Results(results)
table.border = True
return render_template('results.html', table=table)
#app.route('/new_album', methods=['GET', 'POST'])
def new_album():
"""
Add a new album
"""
form = AlbumForm(request.form)
if request.method == 'POST' and form.validate():
# save the album
album = Album()
save_changes(album, form, new=True)
flash('Album created successfully!')
return redirect('/')
return render_template('new_album.html', form=form)
if __name__ == '__main__':
app.run()
No doubt you have already peppered your source code with print() statements and found nothing illuminating. Cached rows in the DB model might be the aspect that is hard to understand, here, and logging sqlite calls would shed light on that.
Use this:
import logging
logging.getLogger('sqlalchemy.engine').setLevel(logging.INFO)
It's noisy, but it will show when rows hit the backend DB and when they are retrieved.
Get in the habit of repeatedly issuing debug queries like this, so you know for sure what has been persisted:
$ echo 'select * from album;' | sqlite3 music.db
For repeatable testing, it can be convenient to copy the database file to a backup location, and then cp that frozen snapshot on top of the active file before each test run. It's important that the running flask app be restarted after such copying. Setting FLASK_DEBUG=1 can help with that.
Also, jeverling suggests using SQLAlchemyDebugPanel.
I'm trying to replace the standard queryset:
queryset: MyModel.objects.all()
on my:
def get_queryset(self, username=None):
if username is not None:
user = UserModel.objects.get(username=username)
queryset = MyModel.filter(author=user)
return queryset
else:
queryset = MyModel.objects.all()
return queryset
when I remove the "queryset", and leave only "get_queryset", an error appears:
AssertionError: base_name argument not specified, and could not automatically determine the name from the viewset, as it does not have a .queryset attribute.
All together looks so:
class MyModelView(viewsets.ModelViewSet):
permissions_classes = (permissions.IsAuthenticated,)
serializer_class = MyModelleSerializer
def get_queryset(self, username=None):
if username is not None:
user = UserModel.objects.get(username=username)
queryset = MyModel.filter(author=user)
return queryset
else:
queryset = MyModel.objects.all()
return queryset
lookup_field = 'username'
lookup_value_regex = '[a-zA-Z0-9$&(._)\-]+'
So how to override method correctly?
In the latest DRF, you need to explicitly set base_name in your viewset url if you don't have queryset defined.
So, something like this should do good:
router.register(r'my-model/', MyModelView, basename='MyModel')
See this: docs
You must add an argument called basename for the register method in the url.py file, Like the following code in url.py :
"In url.py"
...
from rest_framework import routers
router = routers.DefaultRouter()
router.register(r'my-model/' , MyModelView , basename='MyModel')
urlpattern=[...]
You need to set basename attribute in your url conf. Docs here
in my case i'm using rest framework Default router and changing view name to exact match with model name solved the problem.
View:
class DailyQuote(ModelViewSet):
queryset = DailyQuote.objects.all()
serializer_class = DailyQuoteSerializer
Model:
class DailyQuote(models.Model):
quote = models.TextField()
text = models.TextField()
so just change MyModelView to Model.