My question will be hard to form, but to start:
I have an MFC SDI app that I have worked on for an embarrassingly long time, that never seemed to fit the Doc/View architecture. I.e. there isn't anything useful in the Doc. It is multi-threaded and I need to do more with threading, etc.
I dream about also porting it to Linux X Windows, but I know nothing about that programming environment as yet. Maybe Mac also.
My question is where to go from here?
I think I would like to convert from MFC Doc/View to straight Win API stuff with message loops and window procedures, etc. But the task seems to be huge.
Does the Linux X Windows environment use a similar kind of message loop, window procedure architecture?
Can I go part way? Like convert a little at a time without rendering my program unusable for long periods of work?
Added later:
My program is a file compare program (sounds simple enough.) So, stating my confusion in a simple way, normally a document can have multiple views, but in this app, I have one view with multiple (two) documents (files). I have a "compare engine" that I first wrote back in the DOS days, that is the heart of the program and the view is just looking at the output of that routine. Sometimes I think that some of my "view" code could make sense in a "document" class but I hardly know where to begin to separate it into more classes. I have recently started reading "Programming Windows" 5th Ed. by Charles Petzold, (I know that is quite out of date (C) 1998) hoping to get a better understanding of direct Windows programming.
I get overwhelmed with the proliferation of options like C#, NET, MFC, MVC, Qt, wxWidgets, etc.
I find I am often stuck trying to understand something going on in the MFC framework because something in my code doesn't work as it seems it should, but the problem is that I don't really understand how MFC is handling things in the background. That is why I am trying to learn "straight Windows programming" where my program has all the message passing code that I write. I hope this helps give enough insight into my question so someone can guide me on my way.
X works enough differently that a raw Windows program and a raw X program probably wouldn't be able to share much UI code at all.
If you want portability between the two, chances are pretty good that you want to use something like Qt or wxWidgets. Of the two, wxWidgets is more similar to MFC, so it would probably require less rewriting, but would maintain (more or less) the same "disconnect" you're seeing between what you want and what it provides.
Without knowing more about your application, and why it doesn't fit well with MFC, it's impossible to guess whether Qt would be a better fit or not. An immediate guess would be "probably not".
MFC uses a "document/view" architecture, where Qt uses the original Model-View-Controller architecture. For the most part, MFC's Document class is equivalent basically a Model and a Controller rolled into one -- so if your Document contains nothing useful, in Qt you'd apparently have both a Model and a Controller, neither of which did much that was useful.
That said, I have to raise a question about why your Document currently doesn't do much. The MVC pattern has proven applicable to a wide variety of problems, so while it's possible it can't work well for your problem, it's also possible that it could work well, and you're simply not using it. Without knowing more about what you're doing, it's impossible to even guess at that though.
Edit: Okay, the clarification helps quite a bit. The first thing to realize is that a Document does not necessarily equate to a file. Quite the contrary, a document can perfectly reasonably relate to an arbitrary number of files.
Just for example, consider a web browser. All the data needed to compose the page its currently displaying would reasonably be part of the same document. Depending on your viewpoint, that's either zero files, or a whole bunch of them (it will start as an arbitrary number of files coming from the server(s), but won't necessarily be stored as files locally at all). Storing any of it as a file locally will be a (more or less) accidental by-product of caching, and mostly unrelated to browsing per se.
In your case, you're presumably reading the two (or three?) files into memory and storing them along with some sort of data structure to hold the result of the comparison. After the comparison is complete, you might or might not discard the contents of the files themselves. I think it's safe to say that the "normal" separation of responsibilities would be for that data and the code that produces that data to be in the Document.
The View should contain only the code to take that result from that data structure, and display it on screen. Nearly the only data you normally want to store in the View would be things related to how the data is presented (e.g., things like a zoom level or current scroll position). Likewise, the code in the view should relate only to displaying the result and reacting to user input, NOT to "creating" the data in the first place.
As such, I think your program could be rewritten to use the Document/View pattern more effectively, or could be rewritten to use MVC. That, in turn, means a port to Qt could/would probably work just fine -- provided you're willing to put some time and effort into understanding how it's intended to work and then make what may be fairly substantial changes to your code to work the way it's designed to.
As I commented previously, wxWidgets is more like MFC in this respect -- it uses a Document and View, not a Model, View, and Controller. It's also going to work best if you do some rewriting to separate responsibilities the way it's designed for. The good point is that it's probably a bit easier to do that one step at a time: rewrite the code in MFC, which which you're already familiar, and then port it to wxWidgets -- but given the similarity between the two, that "Port" will probably be little more than minor editing -- often just changing some names from C* to wx* is just about enough. To my recollection, the only place I've run into much work was in creating menus -- with MFC they're normally handled via resources, but (at least a few years ago when I used it) wxWidgets normally directly exposed the code that created the menu entries.
Porting to Qt would probably be more work -- you pretty much have to learn a new framework, and substantially reorganize your code at the same time. The good point is that when you're done, the result will probably be somewhat cleaner, though given what you're doing, the difference may be pretty minor. In a Document/View, the View displays data, and reacts to user input. In a Model/View/Controller, the View only displays data, but user input (that modifies the underlying data) goes through the Controller. Since you (presumably) don't expect to modify the underlying data, the only user input involved probably belongs in the view in any case (e.g., things like scrolling). It's barely possible you might have a few things you could put in the Document/Model that would be open to change (e.g., things like the current font or colors the user has selected).
Related
On one hand the advice to always close objects is so common that I would feel foolish to ignore it (e.g. VBScript Out Of Memory Error).
However it would be equally foolish to ignore the wisdom of Eric Lippert, who appears to disagree: http://blogs.msdn.com/b/ericlippert/archive/2004/04/28/when-are-you-required-to-set-objects-to-nothing.aspx
I've worked to fix a number of web apps with OOM errors in classic asp. My first (time consuming) task is always to search the code for unclosed objects, and objects not set to nothing.
But I've never been 100% convinced that this has helped. (That said, I have found it hard to pinpoint exactly what DOES help...)
This post by Eric is talking about standalone VBScript files, not classic ASP written in VBScript. See the comments, then Eric's own comment:
Re: ASP -- excellent point, and one that I had not considered. In ASP it is sometimes very difficult to know where you are and what scope you're in.
So from this I can say that everything he wrote isn't relevant for classic ASP i.e. you should always Set everything to Nothing.
As for memory issues, I think that assigning objects (or arrays) to global scope like Session or Application is the main reason for such problems. That's the first thing I would look for and rewrite to hold only single identifider in Session then use database to manage the data.
Basically by setting a COM object to Nothing, you are forcing its terminator to run deterministically, which gives you the opportunity to handle any errors it may raise.
If you don't do it, you can get into a situation like the following:
Your code raises an error
The error isn't handled in your code and therefore ...
other objects instantiated in your code go out of scope, and their terminators run
one of the terminators raises an error
and the error that is propagated is the one from the terminator going out of scope, masking the original error.
I do remember from the dark and distant past that it was specifically recommended to close ADO objects. I'm not sure if this was because of a bug in ADO objects, or simply for the above reason (which applies more generally to any objects that can raise errors in their terminators).
And this recommendation is often repeated, though often without any credible reason. ("While ASP should automatically close and free up all object instantiations, it is always a good idea to explicitly close and free up object references yourself").
It's worth noting that in the article, he's not saying you should never worry about setting objects to nothing - just that it should not be the default behaviour for every object in every script.
Though I do suspect he's a little too quick to dismiss the "I saw this elsewhere" method of coding behaviour, I'm willing to bet that there is a reason Eric didn't consider that has caused this to be passed along as a hard 'n' fast rule - dealing with junior programmers.
When you start looking more closely at the Dreyfus model of skill acquisition, you see that at the beginning levels of acquiring a new skill, learners need simple to follow recipes. They do not yet have the knowledge or ability to make the judgement calls Eric qualifies the recommendation with later on.
Think back to when you first started programming. Could you readily judge if you were "set[tting an] expensive objects to Nothing when you are done with them if you are done with them well before they go out of scope"? Did you really know which objects were expensive or when they truly went out of scope?
Thus, most entry level programmers are simply told "always set every object to Nothing when you are done with it" because it is within their grasp to understand and follow. Unfortunately, not many programmers take the time to self-educate, learn, and grow into the higher-level Dreyfus stages where you can use the more nuanced situational approach.
And then we come back to my earlier statement - even the best of us started out at that earlier stage, where we reflexively closed all objects because that was the best we were capable of. We left large bodies of code that people look at now, and project our current competence backwards to the earlier work and assume we did that for reasons we don't understand.
I've got to get going, but I hope to expand this a little futher...
I understand the appeal of using the data-driven Entity-Component System for game development. Naturally I am trying to find other areas to apply this paradigm. As I am about to embark on developing a small business application, I've been wondering how well Entity-Component would fit in with it. However I cannot find any examples or discussions on using Entity-Component in anything besides games. Is there a reason? Would there be any advantages in using Entity-Component in software besides games?
I ended up taking a risk and trying to use ECS outside of the gaming domain (now as an indie, formerly company employee) and with results that astounded me. I wouldn't do things any other way now and have an easier-to-maintain system than ever before (not perfect, but so much better than the COM-style architectures we used to use in my industry). I took the plunge mainly because it seemed to provide answers for all the things me and my team in the past were struggling with using a COM architecture, though I imagined with such a risky move that I might just end up exchanging one set of problems for another (was willing to take the risk now that I was on my own). Turned out I didn't exchange one can of worms for another. ECS solved practically all those problems while barely introducing any new ones.
That said, I'm in the VFX domain and it's not that different from games. We still have to animate things like characters, emit particles, interact with meshes, textures, play sound clips, render the result, allow people to write plugins, scripts, etc.
To try to apply ECS in a business domain is far more ballsy. That said, I imagine it could really help create a maintainable system if you have relatively few systems processing a huge number of entity combinations.
Maintainability
What I found that made ECS so much easier for me to maintain compared to previous object-oriented approaches, and even within my personal projects, was that the previous approaches often transferred the maintenance overhead away from the clients using the classes to the classes themselves. However, there would be dozens of interfaces, hundreds of subclasses, all inheriting different things and implementing different interfaces to maintain individually. Testing also becomes difficult with so many granular classes and the need to do mock testing.
My brain can only handle so much and hundreds of subclasses interacting with each other was far beyond the limit. Very quickly I found myself no longer able to reason about what was going on, let alone when or where, overwhelmed by complex interactions leading to complex side effects, and never so confident that I could sandwich new code somewhere in there without causing unwanted side effects.
The computing scientist’s main challenge is not to get confused by the
complexities of his own making. -- E. W. Dijkstra
This applied even for projects I exclusively authored myself. There came a breaking point, typically after a few hundred thousand LOC or so, where I could no longer even comprehend my own creation. I'd refactor here and there, pick up a little momentum, only to take a vacation, come back, and be lost all over again.
ECS removed that challenge, and I don't mean to the degree that I can take a 2-week vacation, come back to the codebase, look at some code, and get the vision of crystal clarity that I had when I was writing it in the first place. ECS doesn't improve things that much in this regard and it still takes some time for me to reacquaint myself with code I haven't looked at in a good while. The reason ECS helped so much is that I didn't need to recall everything I wrote in order to extend and change the software. The systems are so decoupled from each other that it's not a huge deal if I forgot how one works exactly. I can just concentrate on what I need to do and not have to worry about complex interactions of side effects being triggered through complex interactions of control flow. I can just focus on what I need to do and not have to think much about anything else.
This applies even when introducing brand new core-level features integrated into the product. These days when I introduce a new central feature to the product, like a brand new audio system central to the product, the only thing I have to think much about is how to integrate it into the user interface. Integrating it into the architecture is relatively effortless compared to previous architectures I worked in.
Meanwhile with the ECS, I only have to maintain a couple dozen systems to provide no less functionality than the above. They do have some complex logic inside, but I don't have to maintain the hundreds of different entity combinations there are, since they just store components, and I don't have to maintain component types since they just store raw data and I rarely ever find the need to go back and change them (very close to never).
Extensibility
Being able to extend an ECS architecture in hindsight with central concepts is about the easiest thing I've encountered so far and requires the minimum amount of knowledge of how the existing codebase works.
As a very fresh example, I recently encountered a strong desire for scripters using my software to be able to access entities in the scene using a simple, global name. Before they had to specify a full scene path like, Scene.Lights.World.Sunlight as opposed to simply, Sunlight.
Normally in the previous architectures I worked in, that would have ranged from a highly intrusive to moderately intrusive change. A COM-style system revolving around pure interfaces might require introducing a new interface or, worse, changing an existing one, and updating a few hundred subtypes to implement the new functions. If we had a central abstract base class that everything already inherited, we might be able to modify that one centrally to implement this new interface (or the new parts of an existing interface), but it would likely be monstrous if there was a central base class for everything that might want such a name, and require wading through a lot of delicate code.
With the ECS, all I had to do was introduce a new component, GlobalName, with a system that processes GlobalName components and can find an entity quickly through a specified name. It also handles making sure that no two GlobalName components have a matching name. Due to the nature of the ECS, it's also very easy to pick up when this GlobalName component is destroyed as a result of an entity being destroyed or the component being removed from it to keep the data structure used to accelerate searches by name (a trie) in sync.
After that I was just able to attach this GlobalName component to anything that scripters wanted to refer to by a global name. They can also attach it themselves and then refer to a given entity later through that name. Components also serialize themselves in ways that preserves backwards compatibility for the most part (ex: previous versions of the software which did not know what GlobalName was will simply ignore it upon loading scene data referring to it).
It was about as painless and as non-intrusive of a change as I could change imagine considering that this was added very late in hindsight to a 4-year old software which did not anticipate the need for this whatsoever. And I managed to get it working just fine on the very first try. As a bonus, all the non-trivial code newly added to make this work lives isolated in its own space; it's not jumbled up with anything else and doesn't contribute to the complexity of anything else as would inevitably have to be the case if I used abstract interfaces or base classes. I did not have to modify anything central to make this work except a few lines of trivial script and some trivial GUI code to display these global names when available.
"Inherit Anywhere"
Have you ever wished you could extend a class's functionality from anywhere in your code without actually modifying its code? For example:
// In some part of the system exists a complex beast of a class
// which is tricky modify:
class Foo {...};
// In some other part of the system is a simple class that offers
// new behavior we'd like to have in 'Foo', with abstract functionality
// (virtual functions, i.e.) open to substitution:
class Bar {...};
// In some totally different part of the system, maybe even a script,
// make Foo inherit Bar's behavior on the fly, including its default
// constructor, copy constructor, and destructor behavior for Bar's state.
Foo.inherit(Bar);
The above leaves the question: where will the abstract functionality of Bar be implemented, since Foo doesn't provide such an
implementation? That's where systems analogically kick in for ECS.
I think the temptation will be there for most of us who had to wade through some existing class's complex code to just make it do a few new things while risking causing unwanted side effects/glitches/toe-stepping, or we might have even faced a temptation for a third party library outside of our control to just offer a little bit more functionality that we'd find very useful throughout the code using this third party library if it just provided "this one thing", or we might just hate the idea of having to change our colleagues' existing code (don't want to step on toes) even though we're tasked to provide new central behavior.
ECS offers you that kind of flexibility although in a very different way from the above example (but gives you the analogical benefits). It allows you to extend anything's behavior/functionality/state from anywhere. As in the above example of extensibility, I did not have to modify anything that exists to provide that global name searching functionality and state. I can extend the behavior of these entities from the outside, even from script, by just adding a new type of component to any entity I want at which point any systems I write interested in such components will then be able to pick up and process using a duck typing approach ("If it has a GlobalName component, it can be provided a global name which can be utilized to find a matching component very quickly").
Associating Data
Similar to the above, have you ever faced a temptation to associate data to existing objects in the code? In such cases we might have to maintain parallel arrays or associative containers like dictionaries/maps, and such code can be tricky to write correctly given that it has to stay in sync as new objects are added and removed.
ECS solves that problem at a central level, since now you can just attach components and remove components to/from any entity you want very efficiently. That becomes your means of associating new data on the fly. You no longer have to manually synchronize associative data structures.
Testing
Another issue for me just personally, and it may be because I never mastered the art of unit testing (though I did work with a colleague who really studied up on the subject), is that it never made me confident that a system was relatively bug-free. Integration tests gave me greater confidence in that regard. The problem for me was this: even if the unit test passes, how do you know the client will not misuse the interface? What if they use it at the wrong time? What if they try to use it from multiple threads when it's deliberately not designed to be thread-safe?
I get no huge sense of relief to see unit tests passing, since most of the bugs encountered had to do with what was going on between the interfaces being tested, and we had many incoming in spite of all of the hundreds of unit tests we wrote passing. I love test-driven development, and I did find value in the unit test of telling me that this one unit was doing what it was supposed to do which allowed me to use it more confidently throughout the codebase, but the unit testing never gave me a huge sense of relief about the correctness of the codebase as a whole.
ECS solved that problem for me and made unit testing much more valuable even to someone like me who never mastered the art of testing since there are a handful of systems, they each do their hefty share of work (not granular little objects), and they're concrete. If we have to do anything resembling mock testing, it's simply to insert the components/entities necessary to run them and test them. It starts to feel like testing a system is closer to integration testing than unit testing, even though the system is the smallest testable unit.
Homogeneous Processing
To apply ECS requires embracing a more loopy kind of logic with more homogeneous loops doing one thing at a time. A lot of OOP tends to encourage non-homogeneous control flows and complex interactions causing many things to happen in any given phase/state of the system. This was the most difficult part I found initially since I wanted to apply disparate tasks at one time to a given entity/set of components, and my temptation couldn't be satisfied so directly given decoupled systems which only perform one task at a time. So I had to learn how to defer processing, storing some state for the next system to use, and I also use (to a minimum) an event queue so that systems can trigger events which get processed by others.
Nevertheless, I found ways to program the equivalent of a complex interaction as a result of a series of simple loops doing one thing at a time. It never turned out to be as difficult as I imagined to force myself to work this way, applying one uniform task over one set of entities at one time. And after being forced to do this for a while and maintaining the results -- wow! I should have been doing that all along. It's actually kind of depressing reflecting back on a decade of maintaining architectures that were so much harder to maintain than they needed to be after getting the breath of fresh air that was the ECS architecture.
Interactions
This is a simplified "interaction" diagram (not necessarily indicative of direct coupling, as the coupling version would be from concrete objects to abstract interfaces) comparing the differences before and after I adopted ECS. Here's before:
Except that's just between a small number of types (I was too lazy to draw hundreds). And this was why I always struggled to maintain these things and felt tangled up in the code. It's because the interactions between the code were actually a tangled mess, leading you to all sorts of remote functions in the system causing side effects along the way. After (and now the components are just raw data, they contain no functionality of their own):
And the second version was so, so much easier to comprehend, so much easier to extend, so much easier to maintain, so much easier to reason about in terms of correctness, so much easier to test, etc. If your business architecture can effectively fit into the second type of model, I can't overstate how much it can simplify everything.
Invariants
One of the scariest parts to me when I started developing the ECS engine was the lack of information hiding. When components are just raw data, they're dangling what I thought should be their privates in the air for anyone to touch. This could be doubly scary in a business domain that might be more mission-critical in nature.
Yet I found invariants just as easy to maintain, if not more, due to the limited number of systems that access any given component (and typically if the data is modified, it only makes sense for one system in the entire codebase to do it), the extremely simple control flows, and the extremely predictable side effects that result. And it's pretty easy to test the codebase for correctness when you just have a handful of systems to worry about as far as functionality.
Conclusion
So if you are willing to take the risk, I think it could potentially be applied very effectively in certain business domains. The main thing I think is worth thinking about upfront first is if you can model the entirety of your software's needs as a handful of systems processing data stored in components, with each system still performing a bulky but singular responsibility (the analogical equivalents of a RenderingSystem, GuiSystem, PhysicsSystem, InputSystem, etc). Naturally the benefits of ECS diminish if you find you need hundreds of disparate systems to capture the business logic.
If you're interested, I can extend my answer in some later iterations and try to go over some of the minor struggles I faced with the ECS when I was completely wet behind the ears about it.
(Apologies for the necromancy)
Coming from an enterprise background, I have recently been considering this question. Entity-component systems are comparatively new, and represent a completely different design paradigm to what most business developers will have experience with.
Considering my own company's example, I have seen a few scenarios where an entity-component system would offer benefits.
For example, in our primary application, addresses are associated with contacts and organisations. (There are ContactAddress and OrganisationAddress joining tables in our database.) One client wishes to associate projects with addresses as well. There are many ways of achieving this, but an entity-component based approach would seem quite elegant to me - simply add an Addressable component to the Project entity, and the GUI should sort itself out.
Instead, we will likely be adding a new joining table and new data-input pages (albeit re-using common controls).
The primary disadvantage, I would think, would be (initial) lack of developer awareness of the best ways of applying this paradigm to business software, precisely because it doesn't appear to have been done before. Once you start with such an approach, you are committed to it - if it proves frustrating once your project reaches a certain complexity, there's no way out without a significant rewrite.
I'm a Linux user, and I have been very hesitant to use Glade to design GUIs, since the xml files it produces can easily be modified. I know it doesn't sound like a major issue, but what if it's a commercial app that you just don't want people changing?
I use Mac OS X every once in a while, and I figured out that they use files called ".nib"s for GUIs. I think they're essentially the same type used in Nextstep and Openstep (there's even a Linux app which lets you edit these files). Anyway, these files are included in the application bundle, and according to some people, are completely editable. This person claims he even successfully edited Keynote's interface.
Now, why would that be possible? Is it completely okay for the end user to change the interface? Or is it better to have the GUI directly in the compiled application code, like traditional GTK apps?
OS X nib files are one option; the other option is to do things programmatically. For android, XML files can define the GUI or program code can do it. In Windows WPF, the UI is made in XML. Firefox/Mozilla? XUL, another XML-based UI language.
Most modern GUI toolkits have either both of these options or even just defining UIs in files.
But even binaries are modifiable. With a good binary reverse engineering tool, it's wide open. The only way to be really certain is to do what Apple did with iOS, and run signed code; the entire bundle is signed by a key and can't be run if modified.
This isn't a problem for most everyone. Why do you care if the UI is modified? The underlying code isn't, so functionality can't be added or modified.
As a corollary (and a little off-topic) something that you might have a valid concern about is stuff a little more like this.
I don't really see a problem with it. If a user messes up his UI, then it's his problem. Think of it like moddable games. Users always loved them, and in the end, most games benefit from it. There is usually nothing secret about an application's user interface. If there is, you could always do some sort of encryption.
As others have said, you can also add checksums if you just want to disallow editing.
The xml specifies little more than what the interface looks like. Without the compiled-in event handling code, it's pretty much useless. My opinion is customers change it at their own risk, and you might actually get some free useful improvements out of their hacks.
If you're really paranoid about people changing it, you could always add an MD5 digest verification step or something when you load the xml, or compile the xml string into a header file, but that defeats many of the benefits.
The theming engine can make substantial-looking changes to your GUI, as can tools like Parasite. Updating the Glade layout — at their own risk — is much safer than either of those.
What's wrong with users customizing the UI anyway?
What are some ways in which I can write code that is easily modified?
The one I have learned from experience is that I almost always need to write one to throw away. That way I have developed a sense of the domain knowledge and program structure required before coding the actual application.
The general guidelines are offcourse
High cohesion, low coupling
Dont repeat yourself
Recognize design patterns and implement them
Dont recognize design patterns where they are not existing or necassary
Use a coding standard, stick to it
Comment everyting that should be commented, when in doubt : comment
Use unit tests
Write comments and tests before implementation, that way you know exactly what you want to do
And when it goes wrong : refactor, refactor, refactor. With good tests you can be sure nothing breaks
And oh yeah:
read this : http://www.pragprog.com/the-pragmatic-programmer
Everything (i think) above and more is in it
I think your emphasis on modifiability is more important than readability. It is not hard to make something easy to read, but the real test of how well it is understood comes when someone else (or you) has to modify it in repsonse to changing requirements.
What I try to do is assume that modifications will be necessary, and if it is not really clear how to do them, leave explicit directions in the code for how to do them.
I assume that I may have to do some educating of the reader of the code to get him or her to know how to modify the code properly. This requires energy on my part, and it requires energy on the part of the person reading the code.
So while I admire the idea of literate programming, that can be easily read and understood, sometimes it is more like math, where the only way to do it is for the reader to buckle down, pay close attention, re-read it a few times, and make sure they understand.
Readability helps a lot: If you do something non-obvious, or you are taking a shortcut, comment. Comments are places where you can go back and refactor if you have time later. Use sensible names for everything, makes it easier to understand what is going on.
Continuous revision will let you move from that first draft to a better one without throwing away (too much) work. Any time you rewrite from scratch you may lose lessons learned. As you code, use refactoring tools to eliminate code representing areas of exploration that are no longer needed, and to make obvious things that were obscure. The first one reduces the amount that you need to maintain; the second reduces the effort per square foot. (Sqft makes about as much sense as lines of code, really.)
Modularize appropriately and enforce encapsulation and separation of logic between your modules. You don't want too many dependencies on any one part of the code or that part becomes inherently harder to understand.
Considering using tried and true methods over cutting edge ones. You give up some functionality for predictability.
Finally, if this is code that people will be using before and after modification, you need(ed) to have an appropriate API insulating your code from theirs. Having a strong API lets you change things behind the scenes without needing to alert all your consumers. I think there's a decent article on Coding Horror about this.
Hang Your Code Out to D.R.Y.
I learned this early when assigned the task of changing the appearance of a web-interface. The code was in C, which I hated, and was compiled to a CGI executable. And, worse, it was built on a library that was abandoned—no updates, no support, and too many man-hours put into its use to change it. On top of the framework was a disorderly web of code, consisting of various form and element builders, custom string implementations, and various other arcane things (for a non-C programmer to commit suicide with).
For each change I made there were several, sometimes many, exceptions to the output HTML. Each one of these exceptions required a small change or improvement in the form builder, thanks to the language there's no inheritance and therefore only functions and structs, and instead of putting the hours in the team instead wrote these exceptions frequently.
In my inexperience I was forced to change the output of each exception, rather than consolidate the changes in an improved form builder. But, trawling through 15,000 lines of code for several hours after ineffective changes would induce code-burn, and a fogginess that took a night's sleep to cure.
Always run your code through the DRY-er.
The easiest way to modify a code is NOT to write code. Write pseudo code not just for algo but how your code should be structured if you are unsure.
Designing while writing code never works...for me :-)
Here is my current experience: I'm working (Java) with a kind of database schema that might often change (fields added/removed, data types modified). My strategy is to parse this schema and to generate the code with apache velocity. The BaseClass generated is never modified by the programmer. Else, a MyClass extends BaseClass is created and the logical components of this class (e.g. toString() ! )are implemented using the 'getters' and the 'setters' of the super class.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
A couple of years ago the media was rife with all sorts of articles on
how the idea of code reuse was a simple way to improve productivity
and code quality.
From the blogs and sites I check on a regular basis it seems as though
the idea of "code reuse" has gone out of fashion. Perhaps the 'code
reuse' advocates have all joined the SOA crowd instead? :-)
Interestingly enough, when you search for 'code reuse' in Google the
second result is titled:
"Internal Code Reuse Considered Dangerous"!
To me the idea of code reuse is just common sense, after all look at
the success of the apache commons project!
What I want to know is:
Do you or your company try and reuse code?
If so how and at what level, i.e. low level api, components or
shared business logic? How do you or your company reuse code?
Does it work?
Discuss?
I am fully aware that there are many open source libs available and that anyone who has used .NET or the Java has reused code in some form. That is common sense!
I was referring more to code reuse within an organizations rather than across a community via a shared lib etc.
I originally asked;
Do you or your company try and reuse code?
If so how and at what level, i.e. low level api, components or shared business logic? How do you or your company reuse code?
From where I sit I see very few example of companies trying to reuse code internally?
If you have a piece of code which could potentially be shared across a medium size organization how would you go about informing other members of the company that this lib/api/etc existed and could be of benefit?
The title of the article you are referring to is misleading, and is actually a very good read. Code reuse is very beneficial, but there are downsides with everything. Basically, if I remember correctly, the gist of the article is that you are sealing the code in a black box and not revisiting it, so as the original developers leave you lose the knowledge. While I see the point, I don't necessarily agree with it - at least not to a "sky is falling" regard.
We actually group code reuse into more than just reusable classes, we look at the entire enterprise. Things that are more like framework enhancement or address cross-cutting concerns are put into a development framework that all of our applications use (think things like pre- and post-validation, logging, etc.). We also have business logic that is applicable to more than one application, so those sort of things get moved to a BAL core that is accessible anywhere.
I think that the important thing is not to promote things for reuse if they are not going to really be reused. They should be well documented, so that new developers can have a resource to help them come up to speed, as well. Chances are, if the knowledge isn't shared, the code will eventually be reinvented somewhere else and will lead to duplication if you are not rigorous in documentation and knowledge sharing.
We reuse code - in fact, our developers specifically write code that can be reused in other projects. This has paid off quite nicely - we're able to start new projects quickly, and we iteratively harden our core libraries.
But one can't just write code and expect it to be re-used; code reuse requires communication among team members and other users so people know what code is available, and how to use it.
The following things are needed for code reuse to work effectively:
The code or library itself
Demand for the code across multiple projects or efforts
Communication of the code's features/capabilities
Instructions on how to use the code
A commitment to maintaining and improving the code over time
Code reuse is essential. I find that it also forces me to generalize as much as possible, also making code more adaptable to varying situations. Ideally, almost every lower level library you write should be able to adapt to a new set of requirements for a different application.
I think code reuse is being done through open source projects for the most part. Anything that can be reused or extended is being done via libraries. Java has an amazing number of open source libraries available for doing a large number of things. Compare that to C++, and how early on everything would have to be implemented from scratch using MFC or the Win32 API.
We reuse code.
On a small scale we try to avoid code duplication as much as posible. And we have a complete library with a lot of frequently used code.
Normally code is developed for one application. And if it is generic enough, it is promoted to the library. This works excelent.
The idea of code reuse is no longer a novel idea...hence the apparent lack of interest. But it is still very much a good idea. The entire .NET framework and the Java API are good examples of code reuse in action.
We have grown accustomed to developing OO libraries of code for our projects and reusing them in other projects. Its a part of the natural life cycle of an idea. It is hotly debated for a while and then everyone accepts and there is no reason for further discussion.
Of course we reuse code.
There are a near infinite amount of packages, libraries and shared objects available for all languages, with whole communities of developers behing them supporting and updating.
I think the lack of "media attention" is due to the fact that everyone is doing it, so it's no longer worth writing about. I don't hear as many people raising awareness of Object-Oriented Programming and Unit Testing as I used to either. Everyone is already aware of these concepts (whether they use them or not).
Level of media attention to an issue has little to do with its importance, whether we're talking software development or politics! It's important to avoid wasting development effort by reinventing (or re-maintaining!) the wheel, but this is so well-known by now that an editor probably isn't going to get excited by another article on the subject.
Rather than looking at the number of current articles and blog posts as a measure of importance (or urgency) look at the concepts and buzz-phrases that have become classics or entered the jargon (another form of reuse!) For example, Google for uses of the DRY acronym for good discussion on the many forms of redundancy that can be eliminated in software and development processes.
There's also a role for mature judgment regarding costs of reuse vs. where the benefits are achieved. Some writers advocate waiting to worry about reuse until a second or third use actually emerges, rather than spending effort to generalize bit of code the first time it is written.
My personal view, based on the practise in my company:
Do you or your company try and reuse code?
Obviously, if we have another piece of code that already fits our needs we will reuse it. We don't go out of our way to use square pegs in round holes though.
If so how and at what level, i.e. low level api, components or shared business logic? How do you or your company reuse code?
At every level. It is written into our coding standards that developers should always assume their code will be reused - even if in reality that is highly unlikely. See below
If your OO model is good, your API probably reflects your business domain, so reusable classes probably equates to reusable business logic without additional effort.
For actual reuse, one key point is knowing what code is already available. We resolve this by having everything documented in a central location. We just need a little discipline to ensure that the documentation is up-to-date and searchable in a meaningful way.
Does it work?
Yes, but not because of the potential or actual reuse! In reality, beyond a few core libraries and UI components, there isn't a large amount of reuse.
In my personal opinion, the real value is in making the code reusable. In doing so, aside from a hopefully cleaner API, the code will (a) be documented sufficiently for another developer to use it without trawling the source code, and (b) it will also be replaceable. These points are a great benefit to on-going software maintenance.
Do you or your company try and reuse code? If so how and at what
level, i.e. low level api, components or shared business logic? How do
you or your company reuse code?
I used to work in a codebase with uber code reuse, but it was difficult to maintain because the reused code was unstable. It was prone to design changes and deprecation in ways that cascaded to everything using it. Before that I worked in a codebase with no code reuse where the seniors actually encouraged copying and pasting as a way to reuse even application-specific code, so I got to see the two extremities and I have to say that one isn't necessarily much better than the other when taken to the extremes.
And I used to be an uber bottom-up kind of programmer. You ask me to build something specific and I end up building generalized tools. Then using those tools, I build more complex generalized tools, then start building DIP abstractions to express the design requirements for the lower-level tools, then I build even more complex tools and repeat, and at some point I start writing code that actually does what you want me to do. And as counter-productive as that sounded, I was pretty fast at it and could ship complex products in ways that really surprised people.
Problem was the maintenance over the months, years! After I built layers and layers of these generalized libraries and reused the hell out of them, each one wanted to serve a much greater purpose than what you asked me to do. Each layer wanted to solve the world's hunger needs. So each one was very ambitious: a math library that wants to be amazing and solve the world's hunger needs. Then something built on top of the math library like a geometry library that wants to be amazing and solve the world's hunger needs. You know something's wrong when you're trying to ship a product but your mind is mulling over how well your uber-generalized geometry library works for rendering and modeling when you're supposed to be working on animation because the animation code you're working on needs a few new geometry functions.
Balancing Everyone's Needs
I found in designing these uber-generalized libraries that I had to become obsessed with the needs of every single team member, and I had to learn how raytracing worked, how fluids dynamics worked, how the mesh engine worked, how inverse kinematics worked, how character animation worked, etc. etc. etc. I had to learn how to do pretty much everyone's job on the team because I was balancing all of their specific needs in the design of these uber generalized libraries I left behind while walking a tightrope balancing act of design compromises from all the code reuse (trying to make things better for Bob working on raytracing who is using one of the libraries but without hurting John too much who is working on physics who is also using it but without complicating the design of the library too much to make them both happy).
It got to a point where I was trying to parametrize bounding boxes with policy classes so that they could be stored either as center and half-size as one person wanted or min/max extents as someone else wanted, and the implementation was getting convoluted really fast trying to frantically keep up with everyone's needs.
Design By Committee
And because each layer was trying to serve such a wide range of needs (much wider than we actually needed), they found many reasons to require design changes, sometimes by committee-requested designs (which are usually kind of gross). And then those design changes would cascade upwards and affect all the higher-level code using it, and maintenance of such code started to become a real PITA.
I think you can potentially share more code in a like-minded team. Ours wasn't like-minded at all. These are not real names but I'd have Bill here who is a high-level GUI programmer and scripter who creates nice user-end designs but questionable code with lots of hacks, but it tends to be okay for that type of code. I got Bob here who is an old timer who has been programming since the punch card era who likes to write 10,000 line functions with gotos in them and still doesn't get the point of object-oriented programming. I got Joe here who is like a mathematical wizard but writes code no one else can understand and always make suggestions which are mathematically aligned but not necessarily so efficient from a computational standpoint. Then I got Mike here who is in outer space who wants us to port the software to iPhones and thinks we should all follow Apple's conventions and engineering standards.
Trying to satisfy everyone's needs here while coming up with a decent design was, probably in retrospect, impossible. And in everyone trying to share each other's code, I think we became counter-productive. Each person was competent in an area but trying to come up with designs and standards which everyone is happy with just lead to all kinds of instability and slowed everyone down.
Trade-Offs
So these days I've found the balance is to avoid code reuse for the lowest-level things. I use a top-down approach from the mid-level, perhaps (something not too far divorced from what you asked me to do), and build some independent library there which I can still do in a short amount of time, but the library doesn't intend to produce mini-libs that try to solve the world's hunger needs. Usually such libraries are a little more narrow in purpose than the lower-level ones (ex: a physics library as opposed to a generalized geometry-intersection library).
YMMV, but if there's anything I've learned over the years in the hardest ways possible, it's that there might be a balancing act and a point where we might want to deliberately avoid code reuse in a team setting at some granular level, abandoning some generality for the lowest-level code in favor of decoupling, having malleable code we can better shape to serve more specific rather than generalized needs, and so forth -- maybe even just letting everyone have a little more freedom to do things their own way. But of course all of this is with the aim of still producing a very reusable, generalized library, but the difference is that the library might not decompose into the teeniest generalized libraries, because I found that crossing a certain threshold and trying to make too many teeny, generalized libraries starts to actually become an extremely counter-productive endeavor in the long term -- not in the short term, but in the long run and broad scheme of things.
If you have a piece of code which could potentially be shared across a
medium size organization how would you go about informing other
members of the company that this lib/api/etc existed and could be of
benefit?
I actually am more reluctant these days and find it more forgivable if colleagues do some redundant work because I would want to make sure that code does something fairly useful and non-trivial and is also really well-tested and designed before I try to share it with people and accumulate a bunch of dependencies to it. The design should have very, very few reasons to require any changes from that point onwards if I share it with the rest of the team.
Otherwise it could cause more grief than it actually saves.
I used to be so intolerant of redundancy (in code or efforts) because it appeared to translate to a product that was very buggy and explosive in memory use. But I zoomed in too much on redundancy as the key problem, when really the real problem was poor quality, hastily-written code, and a lack of solid testing. Well-tested, reliable, efficient code wouldn't suffer that problem to nearly as great of a degree even if some people duplicate, say, some math functions here and there.
One of the common sense things to look at and remember that I didn't at the time is how we don't mind some redundancy when we use a very solid third party library. Chances are that you guys use a third party library or two that has some redundant work with what your team is doing. But we don't mind in those cases because the third party library is great and well-tested. I recommend applying that same mindset to your own internal code. The goal should be to create something awesome and well-tested, not to fuss over a little bit of redundancy here and there as I mistakenly did long ago.
So these days I've shifted my intolerance towards a lack of testing instead. Instead of getting upset over redundant efforts, I find it much more productive to get upset over other people's lack of unit and integration testing! :-D
While I think code reuse is valuable, I can see where this sentiment is rooted. I've worked on a lot of projects where much extra care was taken to create re-usable code that was then never reused. Of course reuse is much preferable to duplicate code, but I have seen a lot of very extenisve object models created with the goal of using the objects across the enterprise in multiple projects (kind of the way the same service in SOA can be used in different apps) but have never seen the objects actually used more than once. Maybe I just haven't been part of organizations taking good advantage of the principle of reuse.
The two software projects I've worked on have both been long term development. One is about 10 years old, the other has been around for over 30 years, rewritten in a couple versions of Fortran along the way. Both make extensive reuse of code, but both rely very little on external tools or code libraries. DRY is a big mantra on the newer project, which is in C++ and lends itself more easily to doing that in practice.
Maybe the better question is when do we NOT reuse code these days? We are either in a state on building using someone elses observed "best practices" or prediscovered "design patterns" or just actually building on legacy code, libraries, or copying.
It seems the degree to which code A is reused to make code B is often based around how much the ideas in code A taken to code B are abstracted into design patterns/idioms/books/fleeting thoughts/actual code/libraries. The hard part is in applying all those good ideas to your actual code.
Non-technical types get overzealous about the reuse thing. They don't understand why everything can't be copy-pasted. They don't understand why the greemelfarm needs a special adapter to communicate the same information that it used to to the old system to the new system, and that, unfortunately we can't change either due to a bazillion other reasons.
I think techies have been reusing from day 1 in the same way musicians have been reusing from day 1. Its an ongoing organic evolution and sythesis that will keep ongoing.
Code reuse is an extremely important issue - where code is not reused, projects take longer and are harder for new team members to get into.
However, writing reusable code takes longer.
Personally, I try to write all my code in a reusable way, this takes longer, but it results in the fact that most of my code has become official infrastructures in my organization and that new projects based on these infrastructures take significantly less time.
The danger in reusing code, is if the reused code is not written as an infrastructure - in a general and encapsulated manner with as few as possible assumptions and as much as possible documentation and unit testing, that the code can end up doing unexpected things.
Also, if bugs are found and fixed, or features added, these changes are rarely returned to the source code, resulting in different versions of the reused code, that no one knows of or understands.
The solution is:
1. To design and write the code with not only one project in mind, but to think of future requirements and try to make the design flexible enough to cover them with minimal code change.
2. To enclose the code within libraries that are to be used as-is and not modified within using projects.
3. To allow users to view and modify the code of of the library withing its solution (not within the using project's solution).
4. To design future projects to be based on the existing infrastructures, making changes to the infrastructures as necessary.
5. To charge maintaining the infrastructure to all projects, thus keeping the infrastructure funded.
Maven has solved code reuse. I'm completely serious.