In Martin Fowler's book Patterns of Enterprise Application Architecture he writes on page 2: "In some ways enterprise applications are much easier than telecoms software -- we don't have very hard multithreading problems ...".
Is anybody aware of a summary of those "very hard multithreading problems" and solutions, in the form of design patterns, like the famous GoF Design Patterns book?
There is the POSA book. But those books might be too general and fundamental. More domain focused examples would be what this question is after.
Check out Joe Armstrong's thesis from 2003:
Making reliable distributed systems in the presence of software errors.
Armstrong designed the software for a high-speed network switching device during his time at Ericsson. It was implemented in Erlang, which was specifically designed to provide highly reliable and highly concurrent applications.
In his thesis, he presents the underlying design decisions for the Erlang language itself and the OTP (Open Telecom Platform) library. He also makes some suggestions about how to design application modules for such applications -- this part comes closest to your 'design pattern', although not in the detail you're accustomed to after reading GoF's Design Patterns.
It's not a recipe book, but he nevertheless draws a few interesting conclusions about how applications should be designed.
Actors
The actor model an architectural pattern where a system is made up of a set of loosely-coupled actors that interact through message passing.
An actor is a computational entity that, in response to a message it receives, can concurrently:
send a finite number of messages to other actors;
create a finite number of new actors;
designate the behavior to be used for the next message it receives.
One of the properties of such a system is that failure propagation is reduced, and as a consequence individual actors become more robust.
The programming language Erlang was specifically designed for telephony switches and supports the actor model.
State Machines
A pattern that is common in real-time & embedded software engineering are state machines. State machines can be implemented on top of actors, but also provide a mechanism to represent complex states and associated behavior.
Finite state machines (FSM) are a possibility, but they quickly start to get large and difficult to maintain because of State Explosion. A more expressive formalism that solves this problem are Hierarchical State Machines (HSM) as originally developed by David Harel.
A more recent implementation of the same semantics that fits object-oriented design is the UML State Machine, see Section 15 of the UML specification. This defines a model for state machines complete with their execution semantics.
Related
In my community, recently we actively use the term "falsification" of a formal specification. The term appears in, for instance:
https://www.cs.huji.ac.il/~ornak/publications/cav05.pdf
I wonder whether Alloy Analyzer does falsification. It seems true for me, but I'm not sure. Is it correct? If not, what is the difference?
Yes, Alloy is a falsifier. Alloy's primary novelty when it was introduced 20 years ago was to argue that falsification was often more important than verification, since most designs are not correct, so the role of an analyzer should be to find the errors, not to show that they are not present. For a discussion of this issue, see Section 1.4, Verification vs. Refutation in Software analysis: A roadmap (Jackson and Rinard, 2000); Section 5.1.1, Instance Finding and Undecidability Compromises in Software Abstractions (Jackson 2006).
In Alloy's case though, there's another aspect, which is the argument that scope-complete analysis is actually quite effective from a verification standpoint. This claim is what we called the "small scope hypothesis" -- that most bugs can be found in small scopes (that is analyses that are bounded by a small fixed number of elements in each basic type).
BTW, Alloy was one of the earliest tools to suggest using SAT for bounded verification. See, for example, Boolean Compilation of Relational Specifications (Daniel Jackson, 1998), a tech report that was known to the authors of the first bounded model checking paper, which discusses Alloy's predecessor, Nitpick, in the following terms:
The hypothesis underlying Nitpick is a controversial one. It is that,
in practice, small scopes suffice. In other words, most errors can be
demonstrated by counterexamples within a small scope. This is a purely
empirical hypothesis, since the relevant distribution of errors cannot
be described mathematically: it is determined by the specifications
people write.
Our hope is that successful use of the Nitpick tool will justify the
hypothesis. There is some evidence already for its plausibility. In
our experience with Nitpick to date, we have not gained further
information by increasing the scope beyond 6.
A similar notion of scope is implicit in the context of model checking
of hardware. Although the individual state machines are usually
finite, the design is frequently parameterized by the number of
machines executing in parallel. This metric is analogous to scope; as
the number of machines increases, the state space increases
exponentially, and it is rarely possible to analyze a system involving
more than a handful of machines. Fortunately, however, it seems that
only small configurations are required to find errors. The celebrated
analysis of the Futurebus+ cache protocol [C+95], which perhaps marked
the turning point in model checking’s industrial reputation, was
performed for up to 8 processors and 3 buses. The reported flaws,
however, could be demonstrated with counterexamples involving at most
3 processors and 2 buses.
From my understanding of what is meant by falsification, yes, Alloy does it.
It becomes quite apparent when you look at the motivation behind the creation of Alloy, as forumalted in the Software Abstraction book:
This book is the result of a 10-year effort to bridge this gap, to develop a language (Alloy) that captures the essence of software abstractions simply and succinctly, with an analysis that is fully automatic, and can expose the subtlest of flaws.
I'm a student of software engineering. My lecturer of "Software Architecture and Design" has told us that we can generate source code from all the UML diagrams (or most).
I already can / have generated code from class diagram. I'm unable to generate code from other diagrams.
Do I have to someway connect those diagrams with class diagrams to do that?
This is simply nonsense. You can not generate code from any diagram at all. You can however generate code from a UML model. This can (but not must) have a couple of diagrams to help visualization for humans.
Now, code is related to classes. That means you need at least some classes defined in your model. A use case helps understanding why classes will do things they are supposed to do. But in no case can you create code from a use case.
There are other model elements which help support creating more detailed code. Those are e.g. state machines which can translate into equivalent code sections.
Activity and sequence diagram also help visualizing how certain code sections run during execution. But you will not (seriously) use them to create code.
Yes, you can, but it's not as simple as what you're describing. Model-Driven Architecture is an active area of research right now, but it hasn't really "caught on" yet. Its proponents argue that it allows for a higher level of abstraction in much the same way that C offered a higher level of abstraction than assembly language and Java offered a higher level of abstraction than C. I think that this could be very useful in the future if they can get the tooling right.
Actually, this isn't even an entirely new idea - the idea of graphical programming in general (which, if you think about it, is basically a generalization of UML-derived programming) has been around at least since the 1980s that I know of (and probably a lot earlier). In fact, Frederick Brooks Jr. talks about it in No Silver Bullet – Essence and Accident in Software Engineering (which was originally published in 1986 and appears in current editions of The Mythical Man-Month):
A favorite subject for Ph.D. dissertations in software engineering is graphical, or visual, programming, the application of computer graphics to software design. Sometimes the promise of such an approach is postulated from the analogy with VLSI chip design, where computer graphics plays so fruitful a role. Sometimes this approach is justfied by considering flowcharts as the ideal program design medium, and providing powerful facilities for constructing them.
Nothing even convincing, much less exciting, has yet emerged from such efforts. I am persuaded that nothing will...
His argument was that, at the time it was written, the tooling just wasn't "there" yet; for example, screen sizes were notoriously small. Also, the flow chart is actually a really bad design mechanism. Also,
More fundamentally, as I have argued above, software is very difficult to visualize. Whether we diagram control flow, variable scope nesting, variable cross-references, data flow, hierarchical data structures, or whatever, we feel only one dimension of the intricately interlocked software elephant. If we superimpose all the diagrams generated by the many relevant views, it is difficult to extract any global overview. The VLSI analogy is fundamentally misleading - a chip design is a layered two-dimensional object whose geometry reflects its essence. A software system is not.
I'll leave it to you to judge whether or not you agree with him or whether this still applies.
So, to summarize: yes, it's at least theoretically possible, and there have been considerable efforts to generate code from UML diagrams, but you'll need multiple diagrams to generate much more than basic class structures and method stubs. It's not like you can write a use case diagram, press a button, and magically have a complete software system.
I think I have found the answer. We can generate code. Say I have a "use case". I right-click on it. Go to "advance" and select "instance classifier". Over there I can actually make my "use cases" , "sequence diagram objects" etc the instances of an already created class or I can even create a class right over there.
I've read a lot of articles about distributed Haskell. Much work has been done but seems to be in the area of distributing computations. I saw the remote package which seems to implement Erlang-style messaging passing but it is 0.1 and early stage.
I'd like to implement a system where there are many separate processes that provide distinct services, and are tied together by several main processes. This seems to be a natural fit for Erlang, but not so for Haskell. But I like Haskell's type safety.
Has there been any recent adoption of Erlang-style process management in Haskell?
If you want to learn more about the remote package, a.k.a CloudHaskell, see the paper as well as Jeff Epstein's thesis. It aims to provide precisely the actor abstraction you want, but as you say it is in the early stages. There is active discussion regarding improvements on the parallel-haskell mailing list, so if you have specific needs that remote doesn't provide, we'd be happy for you to jump in and help us decide its future directions.
More mature but lower-level than remote is the haskell-mpi package. If you stick to the Simple interface, messages can be sent containing arbitrary Serialize instances, but the abstraction is still way lower than remote.
There are some experimental systems, such as described in Implementing a High-level Distributed-Memory Parallel Haskell in Haskell (Patrick Maier and Phil Trinder, IFL 2011, can't find a pdf online). It blends a monad-par approach of deterministic dataflow parallelism with a limited ability to make the I-structures serializable over the network. These sorts of abstraction have promise for doing distributed computation, but since the focus is on computing purely-functional values rather than providing Erlang-style processes, they probably wouldn't be a good fit for your application.
Also, for completeness, I should point out the Haskell wiki page on cloud and HPC Haskell, which covers what I describe here, as well as the subsection on distributed Haskell, which seems in need of a refresh.
I frequently get the feeling that IPC and actors are an oversold feature. There are plenty of attractive messaging systems out there that have Haskell bindings e.g. MessagePack, 0MQ or Thrift. IMHO the only thing you have to add is proper addressing of processes and decide who/what is managing this addressing capability.
By the way: a number of coders adopt e.g. 0MQ into their Erlang environments, simply because it offers the possibility to structure messaging via message brokers rather then relying on pure process to process messaging in super scale.
In a "massively multicore world" I personally assume that shared memory approaches will eventually be outperforming messaging. Someone can then always come and argue with asynchrony of course. But already when you write that you want to "tie together" your processes by "several main processes" you in fact speak about synchronization. Also, you can of course challenge whether a single function, process or thread is the right level of parallelization.
In short: I would probably see whether MessagePack or 0MQ could fit my needs in Haskell and care for the rest in my code.
I might have a silly question about DDD:
Are there any disadvatages of DDD generraly? I mean, besides of using it when it is not necessary, or needed. (e.g. small/not complex projets)
Thanks
It is very easy to do it wrong.
Eric Evans has said in a JavaOne presentation that DDD is best applicable when there is a lot of business domain complexity. Moreover, he explicitly states that DDD is not suitable for problems where the is substantial technical complexity but that has little business domain complexity. An example of the later is an embedded system with very simple inputs (possibly independent of the number of states it possesses) while simultaneously presenting a lot of technical complexity (in terms of getting the hardware to work.)
How we go about quantifying a lot or a little, that's an open subject. But the narrative should work as a guideline of when and where DDD is best applicable.
-- edit --
I got the video presentation through Emule, and I've never been able to find that same lecture on youtube or similar video venues.
I found this discussion of DDD in the Microsoft Application Architecture Guide to be helpful in understanding the challenges of that particular style:
As the core of the software is the
domain model, which is a direct
projection of this shared language, it
allows the team to quickly find gaps
in the software by analyzing the
language around it. The creation of a
common language is not merely an
exercise in accepting information from
the domain experts and applying it.
Quite often, communication problems
within development teams are due not
only to misunderstanding the language
of the domain, but also due to the
fact that the domain's language is
itself ambiguous. The Domain Driven
Design process holds the goal not only
of implementing the language being
used, but also improving and refining
the language of the domain. This in
turn benefits the software being
built, since the model is a direct
projection of the domain language.
In order to help maintain the model as
a pure and helpful language construct,
you must typically implement a great
deal of isolation and encapsulation
within the domain model. Consequently,
a system based on Domain Driven Design
can come at a relatively high cost.
While Domain Driven Design provides
many technical benefits, such as
maintainability, it should be applied
only to complex domains where the
model and the linguistic processes
provide clear benefits in the
communication of complex information,
and in the formulation of a common
understanding of the domain.
At an italian conference, I talked about the topic (see these slides).
DDD projects require domain experts that are often expensive to hire, since they hold valuable knowledge (read, if you don't need a domain expert, you don't need DDD).
Moreover DDD requires strong skills on the modeler side. In particular they have to be:
Humble, since they have to accept their ignorance and learn from an expert
Really skilled in OOP
Diligent, since they have to track decisions
Comunicative, since they have to explain the domain to the other devs (even through documentation)
Finding such developers can be much more expensive than expected, since you can really know that they are good at the job after some months of trials.
Martin Fowler in his "PoEA" book has a great discussion of when and what domain logic patterns to use.
For instance, simplest pattern, Transaction Script, involves no domain model.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I was having a conversation last week with a coworker about architecture (real architecture, as in designing buildings). During our talk it came up that architectural blueprints give an architect, civil engineer, and contractor all the detail they need to build something. It got both of us thinking about the state of software engineering and that there is no universally adopted approach for describing the design of software.
We have UML, but I find that it is often hard to convey enough detail without the diagrams being overly complex. Are there good examples of large software that was designed out using elaborate UML diagrams?
Then again, is having a large set of software blueprints even useful? After all refactoring and rebuilding software is much cheaper than rebuilding a skyscraper. Are architectural blueprints the wrong analogy for software design? Is there a better analogy that you can think of?
I think you can't compare software architecture with real architecture. When you build a house you have to have everything planned in advance and what's more important you also can plan almost everything in advance.
Recently I read that software engineering is more similar to gardening than it is to real architecture. I think this comparison comes closer to reality: you can't know what will work out and what won't; you have to rework things that seemed good in theory but prove to be impractical and you can constantly improve your plan while your garden/software is getting more complete.
In summary: Software blueprints shouldn't have the same level of detail than blueprints for building houses because more often than not you find that you simply cannot stick to your original plan.
Architectural blueprints are a nearly-precise representation of the actual house. They are not - usually - an abstraction conforming to a model of how houses should look, they are a representation of how the house will be.
Contrast that with UML/Flowcharts/Rational Rose/Methodology-of-the-month - those are models. They abstract away implementation details, and presume that a given model(Say, OO) is how software should be, while in reality, software is always breaking those abstractions, because the models are not a good representation.
In a sense, this ties into a question of explanatory power and computability: a house blueprint is a fixed representation with a fixed expression, and a fixed input; whereas a software blueprint must account for variable input, possibly even of potentially unbounded length. Software that permits plugins or other "computing" tie-ins now has what amounts to a Turing machine operator embedded into it, which gives rise to a host of unpredictability. So the input space of software vis-a-vis a house is mathematically larger, meaning the representational techniques must be correspondingly more computationally powerful. And this is where UML et al. falls down - they are not homomorphic with real software.
I'd say that designing software is closer to Mad-Libs than blueprints
One of the arguments made in Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools is that UML is not adequate. Even with the addition of constraints, it is still unclear. Among other things, it does not express the authors intent sufficiently that good code could be reliably generated.
UML is fine, but photographs of whiteboard diagrams drawn roughly are just as good or better in practice (in a time/cost sense of things)
So it's more like drawing a strategy in the sand before lanching an attack, that attitude seems to work better in most cases.
Besides half the time UML gets drawn by some guy with lots of imagination and no investment in the actual implementation.
For large, computationally dense, long-lived, safety-critical, software systems like DoD and FAA weapons and sensor systems, blueprints are essential to long term success. (phew, that was a mouthful :)) Without a set of blueprints for these behemoths, maintainers, and even the original developers, will experience distress and frustration when they try to locate/fix bugs or add major features. Without blueprints, incorporating changes, even small ones, will become a high risk game and failure could mean the loss of lives downstream.
Having said that, UML and it's offspring SysML, are (right now) the only game in town. Modeling and abstraction are important tools in the battle against ambiguity and complexity and they'll become more important in the future. The sooner they are embraced by people who want to grow, the better.
Thanx for listening.
I have just completed a successful C#/Sql Server project where I used a UML diagram to flesh out the application design. That UML diagram avoided any misunderstandings about what the application was designed to do and not do. All class relationships along with the class deletion rules (composite, aggregate, none) were spelled out. Along with a couple of easy to understand State diagrams and some OCL (Object Constraint Language), it was a breeze to discuss with the stakeholders how the application was supposed to work. UML and OCL abstract out a tremendous amount of mundane and low level programming that I was able to avoid. UML and OCL are simple enough that users can understand what is going on under the hood. When my users ask how calculations were arrived at, I simply refer them to the UML and OCL. What could be easier? So, yes, IMHO UML is very appropriate in making software blueprints. There is something to said about employing domain driven development.
The combination of Text + Diagrams is usually the best way to explain how your architecture works. Rational Rose can only get you so far.
I think any metaphor is only going to stretch so far. You will get value comparing some aspects of programming to building houses and also from comparing different aspects to gardening / playing chess/ reading the dictionary whilst standing on your head...
I think it is easier in building to specify what level of detail is required for a particular project as there are generally accepted practices, that have been around for some time, for managing a building project.
Maybe in 50 years time, if everyone settles on a methodology, something similar will happen in our industry.
In my experience, uml is garbage.
You can achieve much, more by using TDD and have 10000x more fun.. by jumping in and writing test cases and seeing how your objects interact.
UML designs just suck. I am a coder, not a data entry type person.
Before TDD I used random pieces of paper to sketch out the basic entities and relationships and then jumped right into coding.
I don't see these tools being used commonplace and the popularity of them is whaning.
I'd say that UML is limited. Yes, you can represent basic relationships, but you still don't get much when you think about interactions and constraints (even with OCL)
If you want to give a software team "all the detail they need to build something" then put your efforts into requirements analysis and creating a nailed-down functional specification. This will contain descriptions of every feature that the customer wants. If those descriptions include UML diagrams then all well and good - in many cases UML is a better language than English/French/German/whatever for describing software - but don't get hung up on creating UML diagrams for the sake of it. Joel on Software has a series of feature articles on how to write functional specs and they are well worth reading - start here: http://www.joelonsoftware.com/articles/fog0000000036.html.