Running different query base on resource parent - pyramid

I'm working on a pyramid traversal project and i want to run different query for retrieving the same resource base on its parent or so.
Say i have 3 resources: Customer, Store, Order. now both Customer and Store could have a list of Orders. So i define an OrderListResource as a resource and my view is like:
#view_config(context=resources.OrderListResource, renderer='json')
def view_order_list(context, request):
return {orders: context.retrieve_method()}
Now i wonder what is the best practice to retrieve data base on resource parent?
For example how OrderListResource should decide to retrieve orders by customer id or by store id. the logics should lie on the view or the resource itself?
Thanks.

Well, since you're using traversal, getting to the parent of your resource is very easy:
parent = context.__parent__
query = session.query(OrderList)
if isinstance(parent, CustomerResource):
query = query.filter(OrderList.customer_id == parent.id)
elif isinstance(parent, StoreResource):
query = query.filter(OrderList.store_id == parent.id)
If the parent you want to check may not be an immediate parent of your resource you can use pyramid.traversal.find_interface method.
Alternatively, if the logic differs a lot depending on whether the Order is contained within a Customer or Store, you may have two separate views using the containment view predicate.
#view_config(context=resources.OrderListResource, containment=CustomerResource, renderer='json')
def view_order_list_for_customer(context, request):
return ...
#view_config(context=resources.OrderListResource, containment=StoreResource, renderer='json')
def view_order_list_for_store(context, request):
return ...

Given that every resource in a resource tree has an identity then what is the purpose and the identity of a resource called OrderListResource?
To match a resource as a context during traversal it has to be created previously. Do you want to update these resources while doing CRUD operations on your real resources??? That is why I understand your idea behind OrderListResource as a collection of Order resources as a result of a dynamic query at application runtime.
You want to store customers, stores and orders. Every customer, store and order has an identity that enables you to find the item in a collection. You could create such a resource tree.
root (/) (Root Resource)
|- customers (Container supporting traversal)
|- stores (Container supporting traversal)
|- orders (Container supporting traversal)
URL patterns to traverse the resource tree
/customers/{uid}
/stores/{uid}
/orders/{uid}
Given that you save the relation to store and customer in an order resource, how to query the relationship between your resources with a natural URL pattern that fits well into the existing URL patterns?
/customers/{uid}/orders
/stores/{uid}/orders
Find orders from a single customer
#view_config(name='orders', context=resources.Customer, renderer='json')
def view_order_list_for_customer(context, request):
customer = context.__name__
orders = request.root['orders']
orders_by_customer = [order for order in orders if order.customer == customer]
return dict(orders = orders_by_customer)
Find orders from a single store
#view_config(name='orders', context=resources.Store, renderer='json')
def view_order_list_for_store(context, request):
store = context.__name__
orders = request.root['orders']
orders_by_store = [order for order in orders if order.store == store]
return dict(orders = orders_by_store)
These views are location-aware and enable you to create the query with information from the given context.
Please notice that the view name matches the name of the resource containing all Order resources.
To explore the relationship between two resources your URLs would look usually like
/resource/identifier/resource

Related

Can I iterate through other resources in main.tf in readResource() when developing a TF provider?

Context: I'm developing a TF provider (here's the official guide from HashiCorp).
I run into the following situation:
# main.tf
resource "foo" "example" {
id = "foo-123"
name = "foo-name"
lastname = "foo-lastname"
}
resource "bar" "example" {
id = "bar-123"
parent_id = foo.example.id
parent_name = foo.example.name
parent_lastname = foo.example.lastname
}
where I have to declare parent_name and parent_lastname (effectively duplicate them) explicitly to be able to read these values that are necessary for read / create request for Bar resource.
Is it possible to use a fancy trick with d *schema.ResourceData in
func resourceBarRead(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
to avoid duplicated in my TF config, i.e. have just:
resource "foo" "example" {
id = "foo-123"
name = "foo-name"
lastname = "foo-lastname"
}
resource "bar" "example" {
id = "bar-123"
parent_id = foo.example.id
}
infer foo.example.name and foo.example.lastname just based on foo.example.id in resourceBarRead() somehow so I won't have to duplicate those fields in both resources?
Obviously, this is minimal example and let's assume I need both foo.example.name and foo.example.lastname to send a read / create request for Bar resource? In other words, can I iterate through other resource in TF state / main.tf file based on target ID to find its other attributes? It seems to be a useful feauture howerever it's not mentioned in HashiCorp's guide so I guess it's undesirable and I have to duplicate those fields.
In Terraform's provider/resource model, each resource block is independent of all others unless the user explicitly connects them using references like you showed.
However, in most providers it's sufficient to pass only the id (or similar unique identifier) attribute downstream to create a relationship like this, because the other information about that object is already known to the remote system.
Without knowledge about the particular remote system you are interacting with, I would expect that you'd be able to use the value given as parent_id either directly in an API call (and thus have the remote system connect it with the existing object), or to make an additional read request to the remote API to look up the object using that ID and obtain the name and lastname values that were saved earlier.
If those values only exist in the provider's context and not in the remote API then I don't think there will be any alternative but to have the user pass them in again, since that is the only way that local values (as opposed to values persisted in the remote API) can travel between resources.

django remove m2m instance when there are no more relations

In case we had the model:
class Publication(models.Model):
title = models.CharField(max_length=30)
class Article(models.Model):
publications = models.ManyToManyField(Publication)
According to: https://docs.djangoproject.com/en/4.0/topics/db/examples/many_to_many/, to create an object we must have both objects saved before we can create the relation:
p1 = Publication(title='The Python Journal')
p1.save()
a1 = Article(headline='Django lets you build web apps easily')
a1.save()
a1.publications.add(p1)
Now, if we called delete in either of those objects the object would be removed from the DB along with the relation between both objects. Up until this point I understand.
But is there any way of doing that, if an Article is removed, then, all the Publications that are not related to any Article will be deleted from the DB too? Or the only way to achieve that is to query first all the Articles and then iterate through them like:
to_delete = []
qset = a1.publications.all()
for publication in qset:
if publication.article_set.count() == 1:
to_delete(publication.id)
a1.delete()
Publications.filter(id__in=to_delete).delete()
But this has lots of problems, specially a concurrency one, since it might be that a publication gets used by another article between the call to .count() and publication.delete().
Is there any way of doing this automatically, like doing a "conditional" on_delete=models.CASCADE when creating the model or something?
Thanks!
I tried with #Ersain answer:
a1.publications.annotate(article_count=Count('article_set')).filter(article_count=1).delete()
Couldn't make it work. First of all, I couldn't find the article_set variable in the relationship.
django.core.exceptions.FieldError: Cannot resolve keyword 'article_set' into field. Choices are: article, id, title
And then, running the count filter on the QuerySet after filtering by article returned ALL the tags from the article, instead of just the ones with article_count=1. So finally this is the code that I managed to make it work with:
Publication.objects.annotate(article_count=Count('article')).filter(article_count=1).filter(article=a1).delete()
Definetly I'm not an expert, not sure if this is the best approach nor if it is really time expensive, so I'm open to suggestions. But as of now it's the only solution I found to perform this operation atomically.
You can remove the related objects using this query:
a1.publications.annotate(article_count=Count('article_set')).filter(article_count=1).delete()
annotate creates a temporary field for the queryset (alias field) which aggregates a number of related Article objects for each instance in the queryset of Publication objects, using Count function. Count is a built-in aggregation function in any SQL, which returns the number of rows from a query (a number of related instances in this case). Then, we filter out those results where article_count equals 1 and remove them.

Terraform providers - how would you represent a resource that doesn't have clearly defined CRUD operations?

For work I'm learning Go and Terraform. I read in their tutorial how the different contexts are defined but I'm not clear on exactly when these different contexts are called and what triggers them.
From looking at the Hashicups example it looks like when you put this:
resource "hashicups_order" "new" {
items {
coffee {
id = 3
}
quantity = 2
}
items {
coffee {
id = 2
}
quantity = 2
}
}
in your Terraform file that is going to go look at hashicups_order remove the hashicups prefix and look for a resource called order. The order resource provides the following contexts:
func resourceOrder() *schema.Resource {
return &schema.Resource{
CreateContext: resourceOrderCreate,
ReadContext: resourceOrderRead,
UpdateContext: resourceOrderUpdate,
DeleteContext: resourceOrderDelete,
What isn't clear to me is what triggers each context . From that example it seems like since you are increasing the value of quantity it will trigger the update context. If this were the first run and no previous state existed it would trigger create etc.
However it my case the resource is a server and one API resource I want to present to the user is server power control. However you would never "create/destroy" this resource... or would you? You could read the current power state and you could update the power state but, at least intuitively, you wouldn't create or destroy it. I'm having trouble wrapping my head around how this would be modeled in Terraform/Go. I conceptually understand the coffee resource in the example but I'm having trouble making the leap to imagining what that looks like as something like a server power capability or other things without a clear matching to the different CRUD operations.

Navigating from Location to Workorder

I need to :
1. Create a single page location application
2. Display all the asset present in the selected location in a table
3. Provide a button from which user can navigate to WOTRACK to view all the workorder(s) created on selected location and its asset(s).
I am facing difficulty in the 3rd one. I have tried Launch in Context and it is working fine except am not able to pass sql query like 'location={location} and assetnum in ({asset.assetnum})'. I need to filter workorders with particular location and all its assets.
I tried to save all the assets in the location to a Non-persistant attribute and passing the values of the attribute in the Launch in context url, Its working as expected but to do so I have written a script on 'Initialize value' which is causing performance issues.
script goes like this:
from psdi.server import MXServer;
from psdi.mbo import MboConstants;
if app == "LOCATION1" :
if mbo.getString("LOCATION") is not None:
Locsite = mbo.getString("SITEID")
desc = mbo.getString("DESCRIPTION")
MaxuserSet = MXServer.getMXServer().getMboSet("MAXUSER", mbo.getUserInfo())
MaxuserSet.setWhere(" userid='"+user+"' ")
MaxuserSet.reset()
UserSite = MaxuserSet.getMbo(0).getString("DEFSITE")
if Locsite == UserSite:
AssetSet = mbo.getMboSet("ASSET")
AssetSet.setFlag(MboConstants.DISCARDABLE, True);
if not AssetSet.isEmpty():
AssetList = ""
AssetMbo = AssetSet.moveFirst()
while AssetMbo is not None:
AssetList = AssetList + str(AssetMbo.getString("ASSETNUM")) + "%2C"
AssetMbo = AssetSet.moveNext()
mbo.setValue("non-persitant",str(AssetList),11L)
and in the LIC url i have given : 'http://xx.x.x.xx/maximo/ui/?event=loadapp&value=wotrack&tabid=List&additionalevent=useqbe&additionaleventvalue=location={LOCATION}|assetnum={non-persistant}'
Is there any other feasible solution to the requirement?
Thanks in Advance
Launch In Context is better used for sending the user to an outside-of-Maximo application and passing along some data from inside-Maximo to provide context in that external app.
What you are doing sounds like a good place to use a workflow process with an Interaction node. The developer tells the Interaction node which app to take the user to and which Relationship to use to find the data the user should work with there.
Why don't you add a table control inside the table details (expanded table row) and show a list of the work orders there. From the WONUM in that table, you could have an app link to take them to WOTRACK, if they want more details about a particular work order. No customization (automation scripting) needed. No workflow needed. Nice and simple.

Indexing Sitecore Item security and restricting returned search results

I have several roles defined, each with different restrictions to content and media items and I would like to restrict the search results that are returned based on the access rights of the currently logged in user, rather than displaying the result and the user then presented with an "Access Denied" page. Some content will obviously be accessible to extranet\anonymous so they should be returned for all users regardless.
The security follows the standard Sitecore practices so Role inheritance (roles within roles) will be used, so it will need to take this into account also.
I couldn't see anything in the Advanced Database Crawler module that would help and I've looked through the Sitecore Search and Indexing Guide (version 6.6 and version 7) but couldn't find any information about indexing the security applied to items. The following articles have some suggestions:
How can I set up a Lucene index in Sitecore that handles security correctly?
This feels "dirty" and has the potential for performance issues, particularly when there are a large number of items returned. Also, (see in the comments) the issue with paging results.
Security (aka Permissions) and Lucene - How ? Should it be done?
The above looks more realistic, and would filter out the results based on indexed security roles, there would obviously be a need to expand the roles to handle roles within roles. My concern here would be that we would need to handle denied permissions, when we specifically need to deny/restrict access for certain roles to content items (I know this is not recommended practice, but there is a very specific need to always deny).
I'm at the planning stage at the moment so with the release of Sitecore 7 today there is also the possibility to use the updated Lucene libraries and/or SOLR if that makes life easier - assuming of course that some of the modules like WebForms for Marketers and Email Campaign Manager are updated before too long.
What are the solutions that people are using for returning search results taking into account security? Any alternatives than the linked questions above? Maybe something in Sitecore 7 that I can leverage, the updated Lucene libraries or SOLR?
I'd prefer to keep this all "out of the box" Sitecore and not use other third party search products if at all possible.
A slight alternative to the suggestion from Klaus:
In Sitecore.ContentSeach.config you'll find a pipeline called contentSearch.getGlobalSearchFilters
Processors added to this pipeline will be applied to any query, so if we drop in one that applies a filter based on roles we're good.
ComputedField
To start, we want a computed field added to our index configuration:
<fields hint="raw:AddComputedIndexField">
<field fieldName="read_roles" returnType="stringCollection">Sitecore.ContentSearch.ComputedFields.ReadItemRoles,Sitecore.ContentSearch</field>
</fields>
NOTE the stored type is a collection of strings. We'll use it to index all the names of roles that can read an item.
Implementation
We have a base abstract class to handle the extraction of item security details
public abstract class ItemPermissions: IComputedIndexField
{
public string FieldName { get; set; }
public string ReturnType { get; set; }
public object ComputeFieldValue(IIndexable indexable)
{
var indexableItem = indexable as SitecoreIndexableItem;
if (indexableItem == null) return null;
var security = indexableItem.Item.Security;
return GetPermissibles(security);
}
protected abstract object GetPermissibles(ItemSecurity security);
}
We implement the above with the abstracted method
public class ReadItemRoles : ItemPermissions
{
protected override object GetPermissibles(ItemSecurity security)
{
var roles = RolesInRolesManager.GetAllRoles();
return roles.Where(security.CanRead).Select(r => r.Name);
}
}
NOTE There's obviously a performance impact here, this will reduce your indexing speed.
To reduce the impact, only add the the computed field to the index configuration for the index that contains secured content. E.g. If your web content is only accessed by the anonymous user it will add no benefit.
Pipeline
Add the entry in to the config
<contentSearch.getGlobalSearchFilters>
<processor type="Sitecore.ContentSearch.Pipelines.GetGlobalFilters.ApplyGlobalReadRolesFilter, Sitecore.ContentSearch" />
</contentSearch.getGlobalSearchFilters>
Implementation
Implement the pipeline filter to check the roles of the context user
public class ApplyGlobalReadRolesFilter : GetGlobalFiltersProcessor
{
public override void Process(GetGlobalFiltersArgs args)
{
var query = (IQueryable<SitecoreUISearchResultItem>)args.Query;
var userRoles = Context.User.Roles.Select(r => r.Name.Replace(#"\", #"\\"));
var predicate = PredicateBuilder.True<SitecoreUISearchResultItem>();
predicate = userRoles.Aggregate(predicate, (current, role) => current.Or(i => i["read_roles"].Contains(role)));
if(predicate.Body.NodeType != ExpressionType.Constant)
args.Query = query.Filter(predicate);
}
}
Summary
Create a ComputedField that returns a list of all valid roles for a given access right
Apply a pipeline processor to contentSearch.getGlobalSearchFilters to add a query filter to each search request.
Use the PredicateBuilder class to ensure the role names are OR'ed together
The big benefit here is that you take the hit at index time and the handling of item restriction is handled through a search query as normal. No need to worry about the facet numbers or search counts being incorrect.
You can restrict the roles you are checking to compute the field and you can vary the application of the pipeline filter. You can even take out the pipeline filter and just update your queries to filter when you require it.
NOTE The biggest problem with this set up is the requirement to re-index your content when security restrictions change. Should you be applying security restrictions to users themselves, you'll have to include additional computed fields.
Edit 02/06/2013
I was just tinkering with this in a project and realised that it was AND'ing the roles in the query. If a user had multiple roles assigned then both roles would have to have declared rights to the item. I've updated the pipeline processor to use the PredicateBuilder class to OR the roles. A check is also added to ensure the predicate is not a constant, this ensures the query is updated only if we have a filter to apply.
After some more searching around, the Linq to Sitecore article pointed me to the following lines of code:
var index = SearchManager.GetIndex("sitecore_master_index");
var context = index.CreateSearchContext(SearchSecurityOptions.EnableSecurityCheck))
Digging through Sitecore.ContentSearch.dll and Sitecore.ContentSearch.LuceneProvider.dll in dotPeek decompiler and the mention of the indexing.filterIndex.outbound pipeline in the Sitecore 7 Search document I found the following code:
Sitecore.ContentSearch.LuceneProvider.LuceneSearchReults
public IEnumerable<SearchHit<TElement>> GetSearchHits()
{
for (int idx = this.startIndex; idx <= this.endIndex; ++idx)
{
Document doc = this.context.Searcher.IndexReader.Document(this.searchHits.ScoreDocs[idx].Doc, (FieldSelector) this.fieldSelector);
if (!this.context.SecurityOptions.HasFlag((Enum) SearchSecurityOptions.DisableSecurityCheck))
{
string secToken = doc.GetField("_uniqueid").StringValue;
string dataSource = doc.GetField("_datasource").StringValue;
if (!string.IsNullOrEmpty(secToken))
{
bool isExcluded = OutboundIndexFilterPipeline.CheckItemSecurity(new OutboundIndexFilterArgs(secToken, dataSource));
if (!isExcluded)
yield return new SearchHit<TElement>(this.searchHits.ScoreDocs[idx].Score, this.configuration.IndexDocumentPropertyMapper.MapToType<TElement>(doc, this.selectMethod, this.virtualFieldProcessors, this.context.SecurityOptions));
}
}
else
yield return new SearchHit<TElement>(this.searchHits.ScoreDocs[idx].Score, this.configuration.IndexDocumentPropertyMapper.MapToType<TElement>(doc, this.selectMethod, this.virtualFieldProcessors, this.context.SecurityOptions));
}
}
Sitecore.ContentSearch.Pipelines.IndexingFilters
public class ApplyOutboundSecurityFilter : OutboundIndexFilterProcessor
{
public override void Process(OutboundIndexFilterArgs args)
{
if (args.IndexableUniqueId == null || !(args.IndexableDataSource == "Sitecore"))
return;
ItemUri uri = new ItemUri(args.IndexableUniqueId);
if (args.AccessRight != AccessRight.ItemRead || Database.GetItem(uri) != null)
return;
args.IsExcluded = true;
}
}
So it looks like Sitecore 7 gives us the ability to filter the the search results using the security rights of the context user straight out of the box, albeit using a very similar method of checking the item read permissions that Mark Cassidy suggested. Which is good news, since if this is a requirement for Sitecore 6 implementations then we can easily update the Advanced Database Crawler to do the same thing.
I'm still not convinced on the performance of this though, given that Sitecore 7 comes with Item Buckets and the possibility of storing millions of items. It should be possible to create several indexes though, and only EnableSecurityCheck on indexes with security enabled content, but then we need to think about combining the results from several indexes for "global search" results and also take into acccount Boosting which will mean re-ordering the combined results.
well - your considerations seems quite on target.
The easy implementation is to check the item through the database lookup - but paging, facetting and other statistics will fail in that.
The approach we do is to index security tokens for roles - having an inclusion field for allow and an exclusion field for deny rights. You then need to build a query for this - and expand all roles in roles for the query.
There are two issues you might be facing.
One is very complex OR queries for all roles in roles memberships. Might be less then well-performing.
The other is indexing congestion - as you will need to index major parts of the contentitems when security and permissions change as then are inherited.
The alternative is to use join queries - but normally that would be bad performing.
Sitecore developers made a silly mistake, it will never work, because of that statement:
if ((args.IndexableUniqueId != null) && (args.IndexableDataSource == "Sitecore"))
as args.IndexableDataSource will always be equal to "sitecore" not "Sitecore".
I'm currently upgrading big project to latest 7.2 Update and found out that silly mistake, oh Sitecore Devs usual mistakes :)

Resources