Trying to make a form with two tabs (in imperative style) for deform 2, colander 1.0. The idea of the form is to choose between adding webpage and it's title manually, or alternatively a feed URL:
#property
def webpage_form(self):
schema = colander.SchemaNode(colander.Mapping(unknown='preserve'),
name="webpage_schema")
schema_page = colander.SchemaNode(colander.Mapping(unknown='preserve'),
title=u"Webpage", missing={})
webpage_name = colander.SchemaNode(
colander.String(),
name='webpage_name',
default=u'Webpage',
missing=u'',
)
webpage_url = colander.SchemaNode(
colander.String(),
name='url',
default=u'-',
missing=u'',
)
schema_page.add(webpage_name)
schema_page.add(webpage_url)
schema_feed = colander.SchemaNode(colander.Mapping(unknown='preserve'),
title=u"Feed", missing={})
feed = colander.SchemaNode(
colander.String(),
name='feed_url',
default=u'-',
missing=u'',
)
schema_feed.add(feed)
schema.add(schema_page)
schema.add(schema_feed)
return deform.Form(schema, buttons=('submit',))
In the form controller (Pyramid):
controls = request.POST.items()
...
appstruct = self.webpage_form.validate(controls)
While the controls seems to have data:
[('_charset_', u'UTF-8'),
('__formid__', u'deform'),
('__start__', u':mapping'),
('webpage_name', u'Webpage'),
('url', u'http://slashdot.org'),
('__end__', u':mapping'),
('__start__', u':mapping'),
('feed_url', u'-'),
('__end__', u':mapping'),
('submit', u'submit')]
The appstruct is empty:
{'': {}}
Experimented with title, missing, default, unknown parameters, without result. What is wrong with the form?
Also, instead of two tabs there are two consequent fieldsets, but it may be another story.
Solved the problem. The name parameter must be used with colander.SchemaNode, not title. The resulting appstruct then looks like this:
{'feed': {'feed_url': u''},
'webpage': {'url': u'http://slashdot.org', 'webpage_name': u'Slashdot'}}
which is what was expected.
It helped to read the docs again:
Each schema node object has a required type, an optional preparer for
adjusting data after deserialization, an optional validator for
deserialized prepared data, an optional default, an optional missing,
an optional title, an optional description, and a slightly less
optional name.
Related
Here is an example of an url.
url = 'https://rapaxray.com'
# logo
html_content = requests.get(url, headers=headers).text
soup = BeautifulSoup(html_content, "lxml")
images_found = soup.findAll('img', {'src' : re.compile(r'(jpe?g)|(png)|(svg)$')})
images_found
First I'm narrowing down the list of elements to the ones containing jpg, png or svg in a tag. In this case I only get 3 elements. Then I would like to filter those elements to show me only the ones that have a key word 'logo' in ANY attribute.
The element I'm looking for in this example looks like this:
'img alt="Radiology Associates, P.A." class="attachment-full size-full astra-logo-svg" loading="lazy" src="https://rapaxray.com/wp-content/uploads/2019/09/RAPA100.svg"/'
I want to filter out this element out of all elements based on condition that it has a key word 'logo' in ANY of its attributes
The challenge is that:
I have thousands of urls, and key word logo could be in a different attribute for different url
logic: if 'logo' in ANY(attribute for attribute in list_of_possible_attributes_that_this_element_has) doesn't work the same way as list comprehensions because I couldn't find the way of how to access any possible attribute without using its specific name
Checking all specific names is also problematic because particular attribute could exist in one element but not the other which throws error
Case above is also extra challenging because attribute value is a list, so we would need to flatten it to be able to check if the key word is in it.
For most of the urls the element I'm looking for is not returned as the top one like in this example so choosing top first is not an option.
Is there a way of filtering out elements based on a key word in ANY of its attributes? (without prior knowledge of what the name of the attribute is?).
If I understood you correctly, you could use a filter function similar to this answer to search for all tags such that any tag attribute's value contains val:
def my_filter(tag, val):
types = ['.jpg','.jpeg','.svg','.png']
if tag is not None and tag.name == "img" and tag.has_attr("src"):
if all(y not in tag['src'] for y in types):
return False
for key in tag.attrs.keys():
if isinstance(tag[key], list):
if any(val in entry for entry in tag[key]):
return True
else:
if val in tag[key]:
return True
return False
res = soup.find_all(lambda tag: my_filter(tag, "logo"))
I have a serializer with a PrimaryKeyRelatedField:
field_name = serializers.PrimaryKeyRelatedField(queryset=ModelClass.objects.all(), read_only=False)
With this setup, the Schema properly identifies the parameter as an integer (the PK). But, when I change to:
field_name = serializers.PrimaryKeyRelatedField(read_only=True)
(it will not let you specify queryset and read_only at the same time) then the parameter is identified as a string in the Schema.
Why would this be? Is this correct/expected behavior or perhaps a bug?
Well, I had banged my head on this one long enough to post on SO, but as I've continued to dig into it I think I am making some progress.
At rest_framework/schemas/openapi.py:380 inside of map_field() you find the following:
if isinstance(field, serializers.PrimaryKeyRelatedField):
model = getattr(field.queryset, 'model', None)
if model is not None:
model_field = model._meta.pk
if isinstance(model_field, models.AutoField):
return {'type': 'integer'}
RelatedField won't let you specify a queryset when read_only=True though, so then the "model" in the above code snippet is always None when read_only=True, and you end up getting type="string" at the bottom of the map_field function.
So this explains why it is currently happening, and seems like a bug in DRF, but now I guess I just need to look into what the appropriate fix would be.
I've opened up a corresponding issue against DRF: https://github.com/encode/django-rest-framework/issues/7427
Maybe I'm missing something absurd, I'm not seeing, but this is my first app to study web2py.
I am unable to enter the data in Table Movies, which has fields related to other tables.
The list is loaded, but it is not registered in Movies registration.
Under the codes and the results.
db.py
Movie = db.define_table('movies',
Field('title','string', label = 'Title'),
Field('date_release','integer', label = 'Date Release'),
Field('duraction','integer', label = 'Duraction'),
Field('category','string','list:reference categories', label = 'Category'),
Field('actor','list:reference actors', label = 'Actor'),
Field('director','list:reference directors', label = 'Diretor'),
)
Category = db.define_table('categories',
Field('title','string', label = 'Title'),
)
validators.py
Movie.title.requires = [IS_NOT_EMPTY(), IS_NOT_IN_DB(db, 'movies.title')]
Movie.category.requires = IS_IN_DB(db, 'categories.title')
Movie.director.requires = IS_IN_DB(db, 'directors.name')
Movie.actor.requires = IS_IN_DB(db, 'actors.name')
Movie.duraction.requires = IS_INT_IN_RANGE(0, 1000)
Category.title.requires = IS_NOT_EMPTY()
movie.py
def add():
form = SQLFORM(Movie)
if form.process().accepted:
response.flash = "Successful! New movie added!"
redirect(URL('add'))
elif form.errors:
response.flash = 'Error'
else:
response.flash = 'Form, set data'
return dict(form = form)
List Load another tables - ok:
The items of list not record in DB:
The widgets displayed in the form are based on the IS_IN_DB field validators you have specified, and there are three problems with the way you have coded them.
First, list:reference fields, like standard reference type fields, store the record IDs of the records they reference -- they do not store values of other fields within the referenced records. So, the second argument to the IS_IN_DB validator should always be the ID field (e.g., categories.id).
Second, although the field will store record IDs, you want the form widget to show some other more descriptive representation of each record, so you should specify the "label" argument of the IS_IN_DB validator (e.g., label='%(title)s').
Third, list:reference fields allow for multiple selections, so you must set the "multiple" argument of the IS_IN_DB validator to True. This will result in a multi-select widget in the form.
So, the resulting validator should look like this:
Movie.category.requires = IS_IN_DB(db, 'categories.id', label='%(title)s', multiple=True)
The above will allow multiple db.categories IDs to be selected, though the form widget will display category titles rather than the actual IDs.
Now, all of the above can be made much easier if you instead define the referenced tables before the db.movies table and specify a format argument for each table:
Category = db.define_table('categories',
Field('title','string', label = 'Title'),
format='%(title)s')
Movie = db.define_table('movies',
...,
Field('category', 'list:reference categories', label = 'Category'),
...)
With the above code, there is no need to explicitly specify the IS_IN_DB validator at all, as the db.movies.category field will automatically get a default validator exactly like the one specified above (the format attribute of the db.categories table is used as the label argument).
You might want to read the documentation on list:reference fields and the IS_IN_DB validator.
As an aside, you might consider specifying your field validators within the table definitions (via the requires argument to Field()), as this is more concise, keeps all schema-related details in one place, and eliminates the need to read and execute an additional model file on every request.
Research: Rails 4.0 beta, fields_for not accepting pluralized model_name in one-to-many association, Rails 4 Nested Attributes with fields_for Don't Save to Database
First, let's get the most common problem out of the way: incorrectly named attributes parameters for strong parameters. Mine is correctly plural.
class AdultsController < ApplicationController
...
def update
authorize #user
respond_to do |format|
if #user.update_attributes(user_params)
format.html { redirect_to unit_adult_path(#unit, #user), notice: "#{#user.full_name} was successfully updated." }
else
format.html { render action: 'edit' }
end
end
end
def user_params
params.require(:adult).permit(:first_name, :last_name, phones_attributes: [])
end
end
And my models are setup correctly
class User < ActiveRecord::Base
has_many :phones, dependent: :destroy
accepts_nested_attributes_for :phones, allow_destroy: true, reject_if: proc { |a| a["number"].blank? }
end
class Phone < ActiveRecord::Base
belongs_to :user, touch: true
end
And the view
# adult/_form.html.haml
= bootstrap_form_for [#unit, #user] do |f|
= f.text_field :first_name, control_col: 'col-md-4'
= f.text_field :last_name, control_col: 'col-md-4'
= f.fields_for :phones do |f_phone|
= f_phone.form_group do
= f_phone.select :kind, options_for_phones, hide_label: true, layout: :default
= f_phone.phone_field :number, hide_label: true, layout: :default
= f_phone.check_box :_destroy, label: 'remove'
But, when I submit the User form to save
Started PATCH "/units/2/adults/1" for 127.0.0.1 at 2014-07-11 15:20:17 -0700
Processing by AdultsController#update as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"pDjDVSiEs5qqHLqnbxQMeGWDOUGvhXPPvgyRGmitmps=", "adult"=>{"first_name"=>"Karl", "last_name"=>"Smith", "phones_attributes"=>{"0"=>{"kind"=>"other", "number"=>"888.1212", "_destroy"=>"0", "id"=>"173"}, "1"=>{"kind"=>"mobile", "number"=>"888.1212", "_destroy"=>"0", "id"=>"174"}} }, "commit"=>"Update Adult", "unit_id"=>"2", "id"=>"1"}
Unpermitted parameters: phones_attributes
I don't understand why the nested data is being rejected by the strong parameter evaluation. It looks correct to me.
The one thing I do notice is that the params data for "phones_attributes" value is a HASH not an ARRAY. In the user_params, phones_attributes: [] looks like it expecting an ARRAY. So I changed it to a HASH.
def user_params
params.require(:adult).permit(:first_name, :last_name, phones_attributes: {})
end
But now I get the following error.
Unpermitted parameters: 0, 1
So I tried specifying the field names in the "phones_attributes" array.
def user_params
params.require(:adult).permit(:first_name, :last_name, phones_attributes: [:id, :kind, :number])
end
And I still get.
Unpermitted parameters: phones_attributes
I know I must be missing something small, but I can't find my error.
EDIT: all my nested attributes forms do not work. Not 100% sure when they stopped, but ones that worked previously no longer work and have not been modified.
Figured this out. I was using javascript to copy the phone fields (kind, number) to make a new set of inputs available for entry. The script was adding non numeric characters to part of the field id, and this was causing rails to ignore all the submitted phone_attributes.
For the next person that comes along...
When fields_for renders out the fields, it will index each input name for uniqueness when the submitted post data is converted to params. In the example below, this number field
<input id="adult_phones_attributes_0_number" name="adult[phones_attributes][0][number]" type="tel" value="7773331111">
will look something like this when converted to params
"phones_attributes"=>{"0"=>{"number"=>"7773331111"}}
The hash key of "0" comes from the index created by fields_for. It's the "[0]" portion of the name.
In versions of rails past, if that nested attributes params hash key was not a number, the k/v pair was just ignored. Well now with strong parameters (I'm guessing the culprit), it will reject the entire "phones_attributes" hash.
My script was copying the input field, doing a regex on the html to change the "[0]" index to a random number. But sometimes it would replace it will non-digit characters. And this was causing the problem.
I want to facet the results based on the different model_names (classes) returned. Is there an easy way to do this?
Have you tried adding a SearchIndex field with this information? E.g.
class NoteIndex(SearchIndex, indexes.Indexable):
title = CharField(model_attr='title')
facet_model_name = CharField(faceted=True)
def get_model(self):
return Note
def prepare_facet_model_name(self, obj):
return "note"
class MemoIndex(SearchIndex, indexes.Indexable):
title = CharField(model_attr='title')
facet_model_name = CharField(faceted=True)
def get_model(self):
return Memo
def prepare_facet_model_name(self, obj):
return "memo"
And so on, simply returning a different string for each search index. You could also create a mixin and return the name of the model returned by get_model too.
Presuming you've added this field to each of your SearchIndex definitions, just chain the facet method to your results.
results = form.search().facet('facet_model_name')
Now the facet_counts method will return a dictionary with the faceted fields and count of results for each facet value, in this case, the model names.
Note that the field here is labeled verbosely to avoid a possible conflict with model_name, a field added by Haystack. It's not faceted, and I'm not sure if duplicating it will cause a conflict.
If you just want to filter on the model type, you can use the ModelSearchForm
The Docs have a really good walk-through for this.
The minimum you'll need:
is to add faceted=True to the params of your model_names field.
Rebuild your schema and indices.
add .facet('model_names') to whatever SearchQuerySet you're wanting to facet.
More explanation on the question would enable a more complete answer.