Interesting Thoughts About Architectural Evolution July 9, 2007
Posted by Jiri Lundak in Agile, Architecture.add a comment
Tom Ayerst, whom I met when becoming a Certified ScrumMaster in Vienna in 2004, writes here some interesting notes about architecture evolution, like:
“What does this mean? The ideal architecture is as described above simple, and focused, it should also be balanced and not restrict you from taking it in any a particular direction. This last desirable trait is invisible, you don’t know you cannot go in a particular direction until you try to go there.”
and
“Over time your architecture will become less balanced and amenable to change. Getting the balance back will probably require an architectural review of some kind and some kind of more disruptive change than another simple refactoring.”
I think he is quite right. I can observe this with our own application framework we use at work. No simple refactoring will do. After multiple refactoring sessions done often as a pair, we are still trying to bring the architecture back on track. It all depends on how far away you already are from the ideal line.
Scrum Talk at Event for Swiss FDJP Project Managers June 26, 2007
Posted by Jiri Lundak in Agile, Scrum.2 comments
I will be giving a talk about our experience with Scrum in a large project at an event of the Swiss Federal Department of Justice and Police on June, 28th.
Here the slides (in German) in Powerpoint format: Scrum in der Praxis (Powerpoint).
And as PDF: Scrum in der Praxis (PDF).
The 3 Axes of Agile Architecture April 30, 2007
Posted by Jiri Lundak in Agile, Architecture, Programming.4 comments
When thinking about my last post today, I was not quite satisfied with it. The values seamed quite unfocused to me in hindsight.
Actually I think we need to condense – especially the values – some more, so that we get at the essence of the agile spirit we need to be able to measure a good architecture against it.
In the end I think it boils down to the following three axes, along which an architecture (maybe embodied by a framework) can be judged (see the figure below).
I would contend, that an architecture satisfying all three of the axes is THE agile architectural basis for software development that is able to deliver on the agile promise.
Let us have a deeper look at the individual axes:
Change-Encouraging
This axis focuses on embracing change. For an architecture to support this, it needs to be easily modifiable, extendable, refactorable. If it is hard to change, it will lead – over the long run – to clumsy code, horrible work-arounds, lots of glue code and very extensive documentation, that needs to explain all of its quirks and potholes.
If it is easy to change and the developer is free to change his application at will, without too much effort, without breaking existing functionality of his application, then the overall design of the architecture (and the framework eventually embodying it), will remain simple, clean and easy to grasp over quite a long period of time (good tending practice implied).
Quality-Enforcing
Agility has one of its foundations in the delivery of production quality code on from iteration 1 to whatever number of iterations a project takes to complete. As such an application developed in agile manner needs a foundation, that has quality build right into it. This kind of architecture makes it hard for the average developer to make stupid mistakes.
It simple does not allow to circumvent good coding practices. It allows freedom of interpretation only in dimensions, that are in the application domain. It helps the developer to test his code in an easy, dare I say fun fashion, so that he wonders, how he could ever live without the safety net he is constantly weaving.
Value-Producing
Though I mention this axis at the end of this text, it is the focal point of agile software development. When we do not produce the right thing, it is of no use to produce it as fast as possible with the highest possible quality.
That said, it is crucial to produce value for the customer. The architecture should not stand in the developer’s way to produce some useful application. It should encourage him to produce value out of the box, to be able to show his customer a thin, but fully working, slice of the final product he will get.
At the same time it should prove to be a tool in the developers hands, that allows to try out easily alternatives, that can be quickly transformed into a visible artifact to put in the hands of the customer, so that fast feedback can be harvested, which eventually flows back into the development cycle.
Complementary products like acceptance testing support and generic self-documentation should help to produce value faster.
Conclusion
To produce an architecture (embodied in a application framework) that supports these essential values, may not be easy, but at least this is a starting point to think more deeply about principles that support this value-system.
As a nice side-effect I now have a system, that allows me to have a look at existing application development framework and weight them against each other. But I will spare this for another post.
Agile Software Architecture Principles April 30, 2007
Posted by Jiri Lundak in Agile, Architecture, Programming.6 comments
It is not a secret that I am fond of Agile software developement. I have published multiple articles over the last few month on pitfalls with Agile implementation and I am even writing a book about it.
But in my chest there still beats a heart of an architect and developer. As such I would like to see still better tools, frameworks and development practices. Last friday I was the last standing man at our company, with a colleague of mine and we were fantasising about our personal take on a lean, mean application development framework. We found quite a common ground, when it came to our dislikes with the existing way to develop software in the Java world.
It is always easier to articulate your dislikes, instead of reflecting on the values and principles, that you would like to see flourish.
So what we dislike about existing frameworks boils down to the following short list:
- Configuration in monstruous XML configuration files
- The jungle of XML WebServices as the SOA silver bullet for all application integration
- Way too much code generation in the past
- Although we are fond of metadata to be leveraged in applications, we dislike too much annotations (obscuring further how things work)
- Writing still applications for the single VM, not using the power of many distributed ressources
- Frameworks, that grow to such giant dimensions, that you have to install a 60 MB package, of which you use only 1% in your application
- Frameworks, that want to be everybody’s darling, by including hunderts of alternatives (we integrate anything) for each module and for this sacrifices ease of use and efficiency in programming
- Generation of boiler-plate code seen as THE holy grail in increasing productivity
When going through these points in retrospect, I would like to synthesize some values and principles, that I would like to see in a framework, that really helps me concentrate on giving a customer what he needs, without much of the grieveance included, when using today’s breed of tools.
Here are the values I would like to see embodied by an ideal framework:
- Efficiency
- Purity
- Easy Modifiability
- Expressiveness
- Openness
These should be support by principles the framework should enforce and follow:
- Ease of use
- Transparency
- Scalability
- Maintainability
- Testability
There is some tendency towards easing the pain of development with the advent some of the newer frameworks for web development, maybe started by the hype surrounding Ruby on Rail. Other new frameworks, that fit in this category are SEAM and Grails.
While these frameworks tend to point into the direction I would like to imagine, they are still not there, yet. Too much boiler-plate code, sometimes too much XML. And if all else fails: I could still argue, that they are not Java frameworks! 
I just would like to take the Agile thought to the extreme and make it as easy as possible (in Java) to implement and maintain an enterprise-ready application. “I know I’m a dreamer…but I’m not the only one”, to speak with Lennons words.
But who knows, out of idealism, sometimes something new and revolutionary can emerge. Let’s get surprised.
Low Scrum Adoption Rate in German-speaking world? April 30, 2007
Posted by Jiri Lundak in Agile, Scrum.1 comment so far
A week ago I was presenting a session at the JAX conference’s Agile Day, hosted by Jutta Eckstein. Exceptional was the interest people have in Agile software development. First it was thought that only about 100 people would attend the Agile track, instead 280 showed up, showing the increased awareness.
Quite contrary to this only a few people (about 10) raised their hand, when asked who was already using Scrum. But even more surprising to me was the fact, that only about another 10 or so signaled their interest to use Scrum in the near future, when I asked them.
I think, that there is still a lot of potential to educate people to use Agile in their work. But how can we lower the entry barrier? During lunch I was talking to a group of people, that had their reservation about what to expect from Agile development. They mentioned the thought, that what was needed to provide would be some week long immersion workshop, to experience the benefits, for one’s own work.
Maybe this is missing, especially in the German-speaking part of the world.
Poor People … Being So Platform-Dependent April 1, 2007
Posted by Jiri Lundak in Programming, Technology.add a comment
In the April 1st 2007 issue of SD Times, in an article called “At Last, Visual Studio 2005 Works on Vista” David Worthington wrote:
“The Visual Studio product team had a decision to make last fall as Windows Vista progressed toward release: It could either ship a first service pack for Visual Studio 2005 that worked on the then-current versions of Windows, or wait for Vista. If it waited, there would be no compatible version of Visual Studio ready when Vista launched. Since a delay wasn’t acceptable to the team, the service pack was made available in December 2006. Vista was unveiled one month later, but Visual Studio 2005 was not certified to work with the operating system.
Since that time, Microsoft has caught up with itself. Visual Studio 2005 Service Pack 1 for Windows Vista arrived in March. Earlier versions of Visual Studio cannot run on Vista.”
This is just one consequence for making the software development environment dependant on the operating system. It actually shows the problems Microsoft is going to have in the future, when it continues with these dependencies.
Sure this makes developers dependent on the operating system release and forces them to buy a new operating system version, to get new features and to buy a new IDE version to be able to develop on top of the new operating system release. A downward spiral.
This does not lend itself to an agile software development approach and only make developers slaves of the platform.
This also shows how plattform abstraction in the virtual machine of a runtime environment is so important. And Microsoft is not able to deliver something similar. So I remain still grateful, that I never went down the track of .NET too far. Else I would have been crippled in my choices, something I prefer not to be.
QCon Day 5 March 17, 2007
Posted by Jiri Lundak in Agile, Team.2 comments
As I had to fly back home on Friday afternoon, the report on Friday’s sessions is quite short. Although I wanted attend the planned Open Space on “Reflecting on your Agile journey – How do we reach Mastery?” led by Deborah Hardman and Diana Larsen, I just did not make it. This mainly, because in parallel there was still an Agile track going, with too much of good content. So I had to sacrifice something. 
I decided to hear what my friend Joseph Pelrine had to say about “When Agile Hits The Wall – Dealing with the organizational challanges of Agile adoption”. Joseph’s talk was video-taped, so I hope it will be available as Video-Stream somewhere from the conference website soon. At least they promised to make the material available.
In his entertaining and metaphorically rich manner Joseph showed why there are fundamental clashes in an organization, when some part of it (like a team) tries to adopt Agile values and practices. Sometimes all goes well within one team, but as soon as the team tries to reach outside its zone of influence things seldom work out as expected. Actually even resistence and suppression can be the response.
Joseph compared going to fast with Agile adoption with running downhill – if you let go, you accelerate and accelerate until you go to fast and stumble over your own feet – a nice metaphor. Your organization might well feel fear and threat and build “antibodies”.
So the first step for resolving anything is to know the problem space and to capture the force that are at work within the organisation. Usually there is a clash in the unterstanding of “cause and effect”. Often management and an Agile team live in two different worlds. Management often remains stuck in a Newtonian world view, while the Agile team embraces a Complexity Science based approach. These are not compatible with each other.
The Newtonian point of view is that cause and effect are directly linked and can be traced and linked together in advance and with some effort can lead to the predication how a system will behave in the future.
The Agile team instead operates in the Complex realm. In this quadrant there is no direct link between an effect and its cause(s) observable, let it be predictable. Instead things can be only judged in retrospect, knowing how things evolved (this is called “retrospective coherence”).
In this realm we can not know in advance what will influence our system and how the agents within the system will react. Such a system we usually find, where people are involved, because people do not act deterministically. To many factors to influence us.
This is also, as Joseph stated, why a simple Inspect -> Adapt loop does not work. Joseph pointed out, that we need first to act, before in a complex environment something happens and any inspection is possible. So he extended the loop to: Apply -> Inspect -> Adapt.
QCon Day 4 March 15, 2007
Posted by Jiri Lundak in Conferences, Programming.add a comment
We have arrived at the forth day of the QCon conference. Although today there is an Agile track, I will attend some of the more technical session, too.
Deborah Hartmann introduced the sessions and off we go. I will start with the session on “Grails” by Guillaume Laforge, the Groovy project manager and co-initiator of the framework, that tries to port ideas from Ruby on Rails to the world of the Groovy scripting language. A colleague of mine that is writing a book on Grails has recommended the session, so I will go and see what I get for my money.
Grails = Spring + Hibernate re-invented
Here are the main messages of the presentation:
- Even simple things are quite painful to accomplish with Java frameworks (ORM persistence hard, numerous layers and config files lead to chaos, UI integration messy)
- Grails is an MVC action-based web framework
- Principles: Convention over Configuration (CoC), Don’t repeat yourself (DRY)
- Grails builds on proven components: Spring, Hibernate, Groovy, SiteMesh, AJAX
How Grails addresses pain points:
Pain Point 1: ORM persistence (many XML config files)
- Domain model based on POGO (Plain Old Groovy Objects)
- Relationships between POGO’s based on convention
- Grails generates (called scaffolding) view and controllers for you for a quick start
- By adding constraints to fields an easy way of validation for correct values is provided
- Write only your domain model objects and generate the rest
- Modify some GSP (like JSP) page (view) or some text and reload the page and you are done
Pain Point 2: Numerous config files and layers
- Most things work over conventions (dynamic finder-method construction, based on known fields)
- Reading over relations
- URL’s have easy, consisten mapping like ‘controller/action/id’
- Parameter passing thru easys maps
- Easy job scheduling using Quartz
Pain Point 3: Messy JSP-like pages
- Predefined Taglibs and Components
- Define own Taglibs
Sweet Spot: Enterprise-ready
- Existing Java libraries
- Employee skills & knowledge
- Spring configured beans
- Hibernate mappings for legacy schemas (but still benefit from dynamic finders)
- EJB3 annotated mapped beans
- JSPs, taglibs for the view
- Deploy to your pricey Java app-server & database
- Grails will fit in your J2EE enterprise architecture
Positive impressions about Grails (maybe to incorporate somewere else)
- Conventions over Configuration
- Extend that with metadata where needed
- Dynamic language, so short code-build-run cycle
Questions that remain
- How much logic goes into the GSP-pages?
- Taglibs, huhhh?
- Still generation needed? Why?
All in all an informative talk of Guillaume. Maybe I should try some first steps with Groovy (although its syntax does not appeal very much to me).
The afternoon starts with another scripting language: JRuby. I first wanted to attend the Test-Driven Development session of Steve Freeman, but during his introduction I noted it was too elementary for my knowledge, so I voted with my feet and went for the session about how to integrate Java and JRuby.
In contrast to the Groovy session, the JRuby session seamed quite low-level. No so much the content, but the form of presentation. The presenter (Rob Harrop) just started with the jirb (JRuby interpreter) in commandline manner. He type away quite fast, but after the code moved over the top of the screen, I think, nobody new anymore, what exactly he was doing (actually creating a small Swing application) directly in the interpreter.
So how does JRuby compare to Groovy/Grails. I am basing my lay judgement on few things I saw on the two sessions, so see this not as an expert talking.
Ruby integration into Java is not seamless. Instead you have to use special integration classes to bind it in. Groovy instead makes the use of Java objects a breeze. Java objects and Groovy objects can be used as if they were the same.
Ruby showed a strength in creating new domain specific languages (DSLs). From the point of view of a poor Java programmer JRuby feels very awkward and strange. Many things are not obvious, this is one of my main critique points. Explicitness ist often missing, like when you overload operators, etc.
A issue I find critical with JRuby and Groovy is the dynamic additon of an API (say methods, functionality) to ONE specific instance of a class. While this might be useful in some case, it can actually be nightmarish when it comes to read and understand code. It actually decouples classes from logic making it more difficult track down, where something happens at which point in time (without recurring to debugging).
The next session was “SEAM” presented by Gavin King. He started with the usual rant against Agile and Test-Driven Development (TDD), by saying you need nothing of this “crap”, just use Seam and you are up to a quick start with your web application.
Seam generates a skeleton for a web application inclusive some simple test code. You can modify the code of your application much like in Grails.
“We need no f*cking unit tests. Any one programmer can write those simple POJOs without error. The problems come, when you test a class together with its collaborators.”. So much about a professional attitude to testing.
Seam uses much XML configuration and is in that regard not better than Hibernate or Spring. Gavin admits, that he would prefer doing it in plain Java.
What Seam brings to the table:
- Single component model for JSF, ESB, EJB, etc.
- No actions
- Persistence: No DAOs, instead binds components that access database through JPA/Hibernate directly to the view
- After changing the DB structure, you need to restart the web server.
Seam incorporates the concepts of conversation based transaction. This a very valid approach to transaction handling in web applications. In fact we have incorporated something quite similar in our own framework.
The next session I attended swang back to Agile software development, when Jeff Sutherland, the co-inventor of Scrum, talked about “Agile Project Management: Lessons Learned at Google”. The room was crowded full, which nicely reflected the interest people had in Scrum. The voices I overheard at the conference showed clearly, that many people would like to try Agile, but feel very unsecure, what agile would bring them and how to start with it. Many doubt that it is worth the effort.
At Google the Adwords project is using Scrum, they have begun about a year ago in early 2006 to introduce individual Scrum practices, like a release backlog, into the company. Originally in 2001 Google decided, that they did not need any Managers, because they were not adding value to the company. So they eliminated all managment positions. So from that time on, projects evolved organically. When Google was growing bigger and the software (in case of Adword) increased in size (500′000 LOC and still growing fast), the project manager of that project saw the need for a little bit more structured approach, which would not destroy the organic nature of the current way to do things, as this was seen as a good thing.
QCon Day 3 March 15, 2007
Posted by Jiri Lundak in Architecture, Technology.2 comments
This is the first real conference day, as the last two were mere tutorial days. So here small report one the day’s session from my perspective:
I opted for concentrating on architecture in the morning, after the keynote of Amazon’s CTO Werner Vogels, on how Amazon’s single web application (yes, the famous online book shop) evolved into a application development platform that Amazon now is opening and making available to the software development community.
What stood out of his talk? Four things:
- To scale: No direct access to the database anymore. Instead data access is encapsulated in services (code and data together), with a stable, public interface.
- To decouple: Services are aggregateable from other services or very “thin” web application, allowing to mesh different services together and leave out, what you do not need, so supporting many different applications and uses.
- One small team own a service in all its aspects. This has the advantage to make the team responsible for the functioning of the service and at the same time giving it the freedom to do whatever it takes to implement it and make it work.
- Scale later. It is soooo difficult to make it right, that sometimes the effort to do it up front is not justified. Or leave it to somebody, that has the knowhow and has done it already…like Amazon (remember S3 – Virtual Disk, etc.).
Then Kevlin Henney introduced as host the architecture track, talking about architecture quality attributes, something I need currently also look at.
The session of Martin Fowler talking about “Modifiability”. His approach to hold the session, made him even more sympathetic for me. He downplayed his role as Agile fore-thinker by taking a backseat during the session and let instead five of Thoughworks’ senior software designers (or architects, if you like) illustrate different ideas around evolutionary architecture.
More detailed notes on this will be published on these pages as soon as I have transcribed them.
In the afternoon I attended the “Performance & Scalability” talk of Cameron Purdy of Tangorsol. Besides that he put up just too many slide for a talk of one hour (more than 70) and his talking stile is somewhat monotonous, his talk was very information-rich. Especially I liked his list of architectural “Don’t’s”. Recognized several of them in our own architecture on the current project we are doing. We really have to throw out the legacy code, that is lurking in all corners of the current framework we are using.
In one point Cameron made somewhat contradictory statement in contrast to the keynote of Werner Vogels: Architect for performance and scalability up front, because it will cost you way too much to put it in at a later point in a project. So who is right? Well, when I try to consolidate those two talks overall message I come to this conclusion: Include thoughts on performance and scalability in your initial draft of your architecture, but be ready to revide it and to optimize it at a later point, when you know a.) more about the real requirement for your system, and b.) when you are able to see with your own eyes (or mesure), where the effective bottlenecks hide.
The next talk was again by Werner Vogels, about “Availability & Consistency”. A very interesting talk, that needs further digging, because there a lot of ideas essential to high end web applications. More on that in a later entry.
The last talk was by Peter Sommerlad on “Security Patterns”, based on his new volume of the famous POSA (Pattern-Oriented Software Architecture) books. Unfortunately he was not able to present the theme in an involving manner, that would spark more interest in the auditorium. Partly this can be traced to him not being a native English speaker, so he was sometimes having difficulty finding words and he often lost the flow that is so important to a talk. At the same time he bothered to explain far an long what patterns are and instead of analyzing thoroughly only two pattern (an easy one and a complicated one), he wanted to give an overview over the whole book. An impossible task to accomplish in one hour. Too bad. It seamed more like a sales pitch, than a session, people can actually take something away. I know he can do better, especially in German.
QCon Day 2 March 14, 2007
Posted by Jiri Lundak in Agile, Architecture, Programming, Technology.1 comment so far
On the second day I participated in Kevlin Henney’s “Hands-On Agile Development Workshop”. You may ask yourself, why as a proponent of Agile development and coming from a company, that tries to employ Agile practices across all its projects I attend such a newbie-event.
Well this is mainly for one reason: Part of my role at Löwenfels Partner is that of a Agile advocat or evangelist. The other part is to coach our teams in Agile practices and values, by leading through giving an example.
So my main interest in this session was to see how Kevlin presented the session. Some things I observed about his session:
What I liked:
- Organized Scrum-like in four Sprints
- Complex-enough problem to solve
- Sprint 0 used to introduce Agile concepts in a very fun and lightweight fashion.
- Sprint planning/commitment included
- Strong test-first approach thaught effectively
- Team collaboration stressed through pair-programming, changing pair, focus on clean interfaces for easier integration of team results
- Practical stories from real world projects mixed in
- No dogmatic propagation of Agile practices, but pushing common sense conclusions
- Real struggling with personality types (people you do not know) when pair-programming and cope with that diversity
- Introduction of kind of a metric for a good unit test: nearly non conditional logic in the test itself. This leads to more readable and understandable tests, that communicate well the intention of application unter test, a lot like a good a specification.
- Nearly as much test code as application code to be tested.
- “By omitting tests we will not be able to write the double quantity of code”.
What could be improved:
- Kevlin talked very much himself. I at least would have prefered more interaction
And the reaction of the attendants? Most of them were convinced to need to apply the trained practices in their organizations, some even against the current paradigma in their companies.
Would I recommend the workshop? Definitely! We should do something similar in our organisation (at least for new employees, if not all). I will try in the coming months to build on that.
