Scrapy: passing instance variables between pipelines - python-3.x

Does passing spider instance variables between pipelines work?
Unfortunately I do not have the code but I'll try to explain as short and clear as possible.
Order is the following:
Pipeline_1: high priority (#700)
Pipeline_2: low priority (#900)
In Pipeline_1 I'm defining a spider instance variable with spider.variable=[] in init method and filling it in close_spider method of the same pipeline.
In Pipeline_2 I'm accessing it in spider_closed method (tried close_spider too) but it is empty.
In pipeline_1 I'm processing some items and I need to pass them all at once in pipeline_2 and this is the only solution I've been thinking about.

So after reading a bit more I found that defining an instance variable in the pipelines is not that great of an idea (also what Alexander mentioned) - I've solved the issue by implementing the pipeline_2 methods into pipeline_1 and do everything in close_spider method.

Related

In Bitbucket Pipelines, can manual pipelines present descriptions in the web interface?

In Bitbucket Pipelines, for manually run pipelines (i.e. "custom pipelines") where you use the web UI to set variables, is it possible to insert any documentation? For example, so that the UI presents a description above or alongside the input form for a variable? (Or are you limited only to being able to name the pipeline and optionally give the variables each a default value and a set of allowed values?)
I don't want other users of the same pipeline (from the web UI) to misinterpret what a keyword expects, or indeed what the pipeline will do, and doubt they will always refer the source code itself to find comments.
Not that I know.
Documentation only describes a name, default and allowed-values attributes https://support.atlassian.com/bitbucket-cloud/docs/configure-bitbucket-pipelinesyml/#variables
But I think the culprit of your issue boils down to a general programming problem: adequately naming variables. Comments making up for bad variable names is among the top-10 programming bad practices.
Variable names should always be unambiguous and self-explanatory for anyone reading it.

Dimensioning Family Instances.

I would like to Dimension the family instances. What is the reference I should be provided while creating dimension.
I have tried with several approaches but the dimensions are not getting deleted when I delete the family instances. This means that, the dimensions are not getting attached to the family instances.
Please help.
Have you found a way to address this manually through the user interface? That is mostly the best place to start when tackling a Revit API task. If you can solve it through the UI, the chances are good it can also be automated. If no UI solution is found, automation is mostly impossible as well.
I would analyse the exact differences caused in the Revit database on the elements involved and their parameters by executing the manual modification. Once you have discovered exactly what is changed by the manual UI interaction, you can probably replicate the same changes programmatically through the API. Here is a more exhaustive description of how to address a Revit API programming task:
http://thebuildingcoder.typepad.com/blog/2017/01/virtues-of-reproduction-research-mep-settings-ontology.html#3

How to destroy a cytoscape.js instance

We are using cytoscape.js to render graphs in an Angular JS app and it seems to be leaking DOM nodes. A snapshot comparison in Chrome Dev Tools shows Detached DOM Trees being retained by the "instances" array in the global cytoscape object.
The cytoscape instance is created in the link function of the directive and I would like to clear these objects on the scope $destroy event. Even if I manually nullify the references to these instances, there are other global objects like the CanvasRenderer.data.container or CanvasRenderer.bindings[].target which still hold on to these elements which prevents them from being garbage collected.
My questions is: does cytoscape have a destroy() method that would free up references to these DOM elements that I could call on the angular $destroy event? OR what is the right way to get rid of these global references?
Screenshots from the Chrome Dev Tools profiler are here: https://drive.google.com/folderview?id=0B6OGkJMuELQHeC01U1FBYkd4NVU&usp=drive_web
(Lack sufficient reputation for attaching images here)
Options:
(1) You can reuse your Cytoscape.js instances and associated DOM elements instead of destroying them. You'll have better performance that way (esp. w.r.t. the DOM), and this is probably the approach you should be using anyway. Though perhaps your choice of frameworks has limited your options in this regard.
(2) You can call the private cytoscape.removeRegistrationForInstance(cy).
This issue stems from some old code that uses registrations for some compatibility as a jQuery plugin. I'll remove it, but you can work around this for now.
Finally, please create issues on Github for feature requests or bug reports. Stackoverflow should only be used for help questions (e.g. "I don't understand this function in the API" etc.).

Parameters in method

I am having a method in which there are more then 7 parameters ,type of all the parameters are different.
Well my question is it fine or i should replace all parameters with the single class(which will contaion all parameters as a instance variable).
Well my question is it fine or i should replace all parameters with the single class
7 is way too much. Replace with a class. With my VS custom theme and fonts settings Intellisense wouldn't fit on the screen when there is a method with so many parameters :-) I find it more readable and easier to understand when working with classes.
Of course those are just my 2 cents and it's subjective. I've seen people writing methods with many many parameters.
Well, there are places where I'd consider that okay - but they're few and far between. I would generally consider using a "parameter class" instead.
Note that it doesn't have to be an "all or nothing" approach - would it make more sense to encapsulate, say, 4 of the parameters together? Would that allow the new class to be used in other methods?
Other thing to consider is whether the method might be doing too much - does the functionality of the method definitely feel right as a single cohesive unit?
Thare are Microsoft APIs with even more parameters; anyway I agree with #Darin, a class could be a good solution, clear and efficient that can avoid passing parameters in the right order... so for example if you change order for oen without using refactoring you're in a mess...
Consider also if that class can be used in other parts of the program...
I somewhat agree with all given answers. However it might be a good idea to break down the method into smaller parts as Jon is suggesting. Usually when you have a situation like this means that the method is doing too much. Typically methods should have 1 or 2 parameters at most. While you would achieve the same by replacing all your parameters with a class parameter you might just be hiding the bigger issue. Is there any chance you can post the method or describe what it's doing?
As a rule of thumb, if you can't fit the entire method, including declaration onto 1 screen, it's too big. Programmers will normally recommend to break down methods to make them reusable. But, it also makes perfect sense to break them down to increase readability.

C# 4.0 Named Parameters - should they always be used when calling non-Framework methods?

I really this is a hugely subjective topic but here is my current take:
When calling methods which do not form part of the .NET BCL named parameters should always be used as the method signatures may well change, especially during the development cycle of my own applications.
Although they might appear more verbose they are also far clearer.
Is the above a reasonable approach to calling methods or have I overlooked something fundamental?
I think it's over the top, personally. (I also believe it's more correct and clearer to call them named arguments - the parameters always have names, as that's part of the declaration.)
In many cases the meaning is crystal clear from the method name - particularly if there are only a couple of arguments. Why bother to add the name? Why does the fact that the name may change affect this?
I would suggest using named arguments:
When optional parameters are also being used
When there are multiple parameters of the same type, so they can be confused (e.g. a dialog box's title and content text)
When you're using null for an argument, and it's not clear what it's doing
To put it another way, how often did not having named arguments cause you confusion before C# 4? In my case it's certainly a non-zero number of times - the situations above do happen - but it's not something that regularly got in my way. Optional parameters make it more reasonable for a method (or particularly a constructor) to have potentially many parameters, and that leads to named arguments being more useful (and indeed necessary if you want to specify "late" optional parameters having skipped earlier ones).

Resources