I have three view-functions in views.py in django project that using a same three arguments in them:
paginator = Paginator(post_list, settings.POSTS_LIMIT)
page_number = request.GET.get('page')
page_obj = paginator.get_page(page_number)
How can I put em in a single function (make an utility) to use one string of code in my view-functions, instead of repeat using three?
Thats my first question here, thank you :)
As you note, you can create a single function to handle this, taking the info it needs as arguments. You can include this as a helper function in your views.py or separate it out into a utils.py and then import it. Assuming the latter, for tidiness and future-proofing
utils.py
from django.core.paginator import Paginator
from django.conf.settings import POSTS_LIMITS #you may have another place for your settings
from .utils import make_pagination #utils.py file in same directory
def make_pagination(request, thing_to_paginate, num_per_page=POSTS_LIMITS)
paginator = Paginator(thing_to_paginate, num_per_page)
page_number = request.GET.get('page')
page_obj = paginator.get_page(page_number)
return page_obj
views.py
from .utils import make_pagination
def records(request):
...
#we'll use the provided settings-based default for num_per_page
page_obj = make_pagination(request, post_list)
return render(request, 'template.html', {'page_obj': page_obj})
If you wanted more than just the page_obj for some reason, you can return more than the one value, eg,
utils.py
...
return page_obj, paginator
views py
...
page_obj, paginator = make_pagination(request, post_list)
I've gotten the page number in the function itself, but you can also do that either in the view itself, or even in the function call in the view eg,
make_pagination(request.GET.get('page') or 1, post_list)
(If you go this path, don't forget to change the function to accommodate the different argument)
Related
I have over a dozen spiders in a scrapy project with variety of items being extracted from different sources, including others elements mostly i have to copy same regex code over and over again in each spider for example
item['element'] = re.findall('my_regex', response.text)
I use this regex to get same element which is defined in scrapy items, is there a way to avoid copying? where do i put this in project so that i don't have to copy this in each spider and only add those that are different.
my project structure is default
any help is appreciated thanks in advance
So if I understand your question correctly, you want use the same regular expression across multiple spiders.
You can do this:
create a python module called something like regex_to_use
inside that module place your regular expression.
example:
# regex_to_use.py
regex_one = 'test'
You can access this express this one in your spiders.
# spider.py
import regex_to_use
import re as regex
find_string = regex.search(regex_to_use.regex_one, ' this is a test')
print(find_string)
# output
<re.Match object; span=(11, 15), match='test'>
You could also do something like this in your regex_to_use module
# regex_to_use.py
import re as regex
class CustomRegularExpressions(object):
def __init__(self, text):
"""
:param text: string containing the variable to search for
"""
self._text = text
def search_text(self):
find_xyx = regex.search('test', self._text)
return find_xyx
and you would call it this way in your spiders:
# spider.py
from regex_to_use import CustomRegularExpressions
find_word = CustomRegularExpressions('this is a test').search_text()
print(find_word)
# output
<re.Match object; span=(10, 14), match='test'>
If you have multiple regular expressions you could do something like this:
# regex_to_use.py
import re as regex
class CustomRegularExpressions(object):
def __init__(self, text):
"""
:param text: string containing the variable to search for
"""
self._text = text
def search_text(self, regex_to_use):
regular_expressions = {"regex_one": 'test_1', "regex_two": 'test_2'}
expression = ''.join([v for k, v in regular_expressions.items() if k == regex_to_use])
find_xyx = regex.search(expression, self._text)
return find_xyx
# spider.py
from regex_to_use import CustomRegularExpressions
find_word = CustomRegularExpressions('this is a test').search_text('regex_one')
print(find_word)
# output
<re.Match object; span=(10, 14), match='test'>
You can also use a staticmethod in the class CustomRegularExpressions
# regex_to_use.py
import re as regex
class CustomRegularExpressions:
#staticmethod
def search_text(regex_to_use, text_to_search):
regular_expressions = {"regex_one": 'test_1', "regex_two": 'test_2'}
expression = ''.join([v for k, v in regular_expressions.items() if k == regex_to_use])
find_xyx = regex.search(expression, text_to_search)
return find_xyx
# spider.py
from regex_to_use import CustomRegularExpressions
# find_word would be replaced with item['element']
# this is a test would be replaced with response.text
find_word = CustomRegularExpressions.search_text('regex_one', 'this is a test')
print(find_word)
# output
<re.Match object; span=(10, 14), match='test'>
If you use docstrings in the function search_text() you can see the regular expressions in the Python dictionary.
Showing how all this works...
This is a python project that I wrote and published. Take a look at the folder utilities. In this folder I have functions that I can use throughout my code without having to copy and paste the same code over and over.
There is a lot of common data that is usual to use across multiple spiders, like regex or even XPath.
It's a good idea to isolate them.
You can use something like this:
/project
/site_data
handle_responses.py
...
/spiders
your_spider.py
...
Isolate functionalities with a common purpose.
# handle_responses.py
# imports ...
from re import search
def get_specific_commom_data(text: str):
# probably is a good idea handle predictable errors here (`try except`)
return search('your_regex', text)
And just use where is needed that functionality.
# your_spider.py
# imports ...
import scrapy
from site_data.handle_responses import get_specific_commom_data
class YourSpider(scrapy.Spider):
# ... previous code
def your_method(self, response):
# ... previous code
item['element'] = get_specific_commom_data(response.text)
Try to keep it simple and do what you need to solve your problem.
I can copy regex in multiple spiders instead of importing object from other .py files, i understand they have the use case but here i don't want to add anything to any of the spiders but still want the element in result
There are some good answers to this but don't really solve the problem so after searching for days i have come to this solution i hope its useful for others looking for similar answer.
#middlewares.py
import yourproject.items import youritem()
#find the function and add your element
def process_spider_output(self, response, result, spider):
item = YourItem()
item['element'] = re.findall('my_regex', response.text)
now uncomment middleware from
#settings.py
SPIDER_MIDDLEWARES = {
'yourproject.middlewares.YoursprojectMiddleware': 543,
}
For each spider you will get element in result data, i am still searching for better solution and i will update the answer because it slows the spider,
from django.contrib import admin
from .models import Shop
#admin.register(Shop)
class ShopAdmin(admin.ModelAdmin):
#admin.display(description='Name')
def upper_case_name(self,obj):
return("%s" % (obj.name)).upper()
Using the answer from Jermaine, you can create your own decorator and monkey patch it to the admin module. This would allow you to use the #admin.display(description="...") decorator.
# Monkey patch admin to include the display decorator (available in future Django versions)
if not hasattr(admin, "display"):
def display(description):
def decorator(fn):
fn.short_description = description
return fn
return decorator
setattr(admin, "display", display)
Complete example:
from django.contrib import admin
from .models import Shop
# Monkey patch admin to include the display decorator (available in future Django versions)
if not hasattr(admin, "display"):
def display(description):
def decorator(fn):
fn.short_description = description
return fn
return decorator
setattr(admin, "display", display)
#admin.register(Shop)
class ShopAdmin(admin.ModelAdmin):
#admin.display(description='Name')
def upper_case_name(self,obj):
return("%s" % (obj.name)).upper()
This is not a super clean approach, but if you would like to use the decorator you can do it this way.
In my case you should activate virtualenv , by typing "pipenv shell" command
I think it helps you too
In my case, for some reason, django was downgraded. I ran "pipenv update django" and it worked.
You can still achieve the same goal without using the display decorator.
list_display = ('upper_case_name',)
def upper_case_name(self, obj):
return "%s" % (obj.name.upper())
upper_case_name.short_description = "Name"
Django==3.1
I was fighting with this for a while, I didn't understand why the #admin.action() decorator didn't appear, I started to read the Django code and I found that when the actions for the Django Admin start to load, it calls a method called _get_base_actions, and this same method gets the action and description in this way:
# django.contrib.admin.ModelAdmin._get_base_actions
...
description = getattr(func, 'short_description', name.replace('_', ' '))
...
I had already tried several ways to load the action and to correctly take the description of the action, if it was not defined, what Django did was to take the name of the same function and replace the underscores with spaces, resulting in a somewhat frightening result.
The first thing I thought of as a quick and valid solution was to create a class and modify the __new__ method, which is called before the __init__ method.
This is an example of what I needed, to be able to generate a PDF from Django actions:
class GeneratePDF:
short_description = "WRITE THE DESCRIPTION OF ACTION HERE"
def __new__(cls, modeladmin, request, queryset):
result = cls.generate_pdf_resume(modeladmin, request, queryset)
return result
#classmethod
def generate_pdf_resume(cls, modeladmin, request, queryset):
...
Finally, I added it to the list of ModelAdmin actions.
from project.own_actions import GeneratePDF
#admin.register(Patient)
class PatientAdmin(admin.ModelAdmin):
...
actions = [GeneratePDF]
Then it appeared with the description that I had placed in the class as a class attribute.
I could also reuse this class to create a URL and successfully generate the PDF.
# project/urls.py
from project.own_actions import GeneratePDF
urlpatterns = [
path('admin/', admin.site.urls),
...
path('export/', GeneratePDF, name="export-pdf")
]
as the title states, I want to be able to iterate through my Confluence System and find all spaces, in which a certain user group is in.
I am able to find a user group in a single space with the code below, but I can not seem to find an answer how to do this with ALL spaces.
import com.atlassian.confluence.spaces.SpaceManager
import com.atlassian.sal.api.component.ComponentLocator
import com.atlassian.confluence.security.SpacePermissionManager
import com.atlassian.confluence.security.SpacePermission
import com.atlassian.user.GroupManager
import com.atlassian.confluence.core.ContentPermissionManager
import com.atlassian.confluence.internal.security.SpacePermissionContext
def spaceManager = ComponentLocator.getComponent(SpaceManager)
def spacePermissionManager = ComponentLocator.getComponent(SpacePermissionManager)
def groupManager = ComponentLocator.getComponent(GroupManager)
def targetSpace = spaceManager.getSpace("NameOfSpace")
def targetGroup = groupManager.getGroup("UserGroup")
if (spacePermissionManager.getGroupsWithPermissions(targetSpace).contains(targetGroup)) {
//do something (in my case, remove User Group)
}
I tried it with "def allSpaces = spaceManager.getAllSpaces()" and substituted it into the getGroupsWithPermissions() method with no success.
Thanks!
Have you tried SpacePermissionManager#getAllPermissionsForGroup? I haven't tested this with Groovy, but at least in Java, it returns a list of the space permissions attached with the user group.
Not all SpacePermissions will be attached to actual spaces so you will most likely need to loop through the list and filter results where getSpace() is not null.
I would like to import all methods from a module with altered names.
For instance, instead of
from module import repetitive_methodA as methodA, \
repetitive_Class1 as Class1, \
repetitive_instance4 as instance4
I'd prefer something along the lines of
from module import * as *-without-"repetitive_"
this is a rephrasing of this clumsy unanswered question, I have not been able to find a solution or similar questions yet.
You can do it this way:
import module
import inspect
for (k,v) in inspect.getmembers(module):
if k.startswith('repetitive_'):
globals()[k.partition("_")[2]] = v
Edit in response to the comment "how is this answer intended to be used?"
Suppose module looks like this:
# module
def repetitive_A():
print ("This is repetitive_A")
def repetitive_B():
print ("This is repetitive_B")
Then after running the rename loop, this code:
A()
B()
produces this output:
This is repetitive_A
This is repetitive_B
What I would do, creating a work-around...
Including you have a file named some_file.py in the current directory, which is composed of...
# some_file.py
def rep_a():
return 1
def rep_b():
return 2
def rep_c():
return 3
When you import something, you create an object on which you call methods. These methods are the classes, variables, functions of your file.
In order to get what you want, I thought It'd be a great idea to just add a new object, containing the original functions you wanted to rename. The function redirect_function() takes an object as first parameter, and will iterate through the methods (in short, which are the functions of your file) of this object : it will, then, create another object which will contain the pointer of the function you wanted to rename at first.
tl;dr : this function will create another object which contains the original function, but the original name of the function will also remain.
See example below. :)
def redirect_function(file_import, suffixe = 'rep_'):
# Lists your functions and method of your file import.
objects = dir(file_import)
for index in range(len(objects)):
# If it begins with the suffixe, create another object that contains our original function.
if objects[index][0:len(suffixe)] == suffixe:
func = eval("file_import.{}".format(objects[index]))
setattr(file_import, objects[index][len(suffixe):], func)
if __name__ == '__main__':
import some_file
redirect_function(some_file)
print some_file.rep_a(), some_file.rep_b(), some_file.rep_c()
print some_file.a(), some_file.b(), some_file.c()
This outputs...
1 2 3
1 2 3
I have Python 3.2 set up with Apache via mod_wsgi. I have CherryPy 3.2 serving a simple "Hello World" web page. I'd like to start templating using Jinja2 as I build out the site. I'm new to Python and therefore don't know much about Python, CherryPy, or Jinja.
Using the code below, I can load the site root (/) and the products page (/products) with their basic text. That at least lets me know I've got Python, mod_wsgi, and CherryPy set up somewhat properly.
Because the site will have many pages, I'd like to implement the Jinja template in a way that prevents me from having to declare and render the template in each page handler class. As far as I can tell, the best way to do that is by wrapping the PageHandler, similar to these examples:
http://docs.cherrypy.org/dev/concepts/dispatching.html#replacing-page-handlers
http://docs.cherrypy.org/stable/refman/_cptools.html#cherrypy._cptools.HandlerWrapperTool
I've implemented the code in the second example, but it doesn't change anything.
[more details after code]
wsgi_handler.py - A mash-up of a few tutorials and examples
import sys, os
abspath = os.path.dirname(__file__)
sys.path.append(abspath)
sys.path.append(abspath + '/libs')
sys.path.append(abspath + '/app')
sys.stdout = sys.stderr
import atexit
import threading
import cherrypy
from cherrypy._cptools import HandlerWrapperTool
from libs.jinja2 import Environment, PackageLoader
# Import from custom module
from core import Page, Products
cherrypy.config.update({'environment': 'embedded'})
env = Environment(loader=PackageLoader('app', 'templates'))
# This should wrap the PageHandler
def interpolator(next_handler, *args, **kwargs):
template = env.get_template('base.html')
response_dict = next_handler(*args, **kwargs)
return template.render(**response_dict)
# Put the wrapper in place(?)
cherrypy.tools.jinja = HandlerWrapperTool(interpolator)
# Configure site routing
root = Page()
root.products = Products()
# Load the application
application = cherrypy.Application(root, '', abspath + '/app/config')
/app/config
[/]
request.dispatch: cherrypy.dispatch.MethodDispatcher()
core module classes
class Page:
exposed = True
def GET(self):
return "got Page"
def POST(self, name, password):
return "created"
class Products:
exposed = True
def GET(self):
return "got Products"
def POST(self, name, password):
return "created"
Based on what I read on a Google Group I figured I might need to "turn on" the Jinja tool, so I updated my config to this:
/app/config
[/]
tools.jinja.on = True
request.dispatch: cherrypy.dispatch.MethodDispatcher()
After updating the config, the site root and products pages display an CherryPy generated error page "500 Internal Server Error". No detailed error messages are found in the logs (at least not in the logs I'm aware of).
Unless it came pre-installed, I know I probably need the Jinja Tool that's out there, but I don't know where to put it or how to enable it. How do I do that?
Am I going about this the right way, or is there some better way?
Edit (21-May-2012):
Here is the Jinja2 template I'm working with:
<!DOCTYPE html>
<html>
<head>
<title>{{ title }}</title>
</head>
<body>
<h1>Hello World</h1>
</body>
</html>
I figured it out.
In the interpolator function, the next_handler function call the original PageHandler (Page.GET or Products.GET in this case). Those original PageHandlers return strings while the interpolator function is treating the response like a python dict (dictionary), hence the double asterisk when it's passed to template.render as **response_dict.
The Jinja template has a placeholder for title, so it needs a title to be defined. Passing a plain string to the render function doesn't define what title should be. We need to pass an actual dict to the render function (or nothing at all, but what good is that?).
Note: For either of these fixes, the jinja tool does need to be enabled, as shown in the question by setting tools.jinja.on to True in the config.
Quick Fix
Define the title as the render function is called. To do this I need to change this line:
return template.render(**response_dict) # passing the string as dict - bad
to this:
return template.render(title=response_dict) # pass as string and assign to title
Like this, the template renders with my PageHandler text as the page title.
Better Fix
Because the template will grow to be more complex, one render function probably won't always be able to correctly assign the necessary placeholders. It's probably a good idea to let the original page handler return an actual dict with the template's many placeholders assigned.
Leave the interpolator function as it was:
def interpolator(next_handler, *args, **kwargs):
template = env.get_template('base.html')
response_dict = next_handler(*args, **kwargs)
return template.render(**response_dict)
Update Page and Products to return actual dicts that define the value's for the template's placeholders:
class Page:
exposed = True
def GET(self):
dict = {'title' : "got Page"}
return dict
def POST(self, name, password):
# This should be updated too
# I just haven't used it yet
return "created"
class Products:
exposed = True
def GET(self):
dict = {'title' : "got Products"}
return dict
def POST(self, name, password):
# This should be updated too
# I just haven't used it yet
return "created"
Like this, the template renders with my PageHandler title text as the page title.
There's also an updated Jinja2 tool found on a repository of various recipes the community has contributed to.