Showing posts with label agile. Show all posts
Showing posts with label agile. Show all posts

The grails diaries #5: Do you need a software architect?

Wednesday, February 20, 2008

Previous: Grails, is it really worth it?

I think there is a general misunderstanding of what kind of value can give Grails to a development shop. It seems that there are people out there that believe Grails, RoR and other breeds mean the end of Software Architectures in the web layer. The reasoning is: if the workload to develop a web application is shorter probably it means we need less skilled developers to code our applications.

Wrong.

This is a bizarre way of thinking from my point of view. I have heard of this before. Lowering the workload of development does not mean poorer skills in your development team. The workload is lower because these frameworks follow the Don't Repeat Yourself (DRY) principle. This means that all developers (seniors and juniors) can deliver faster because they don't have to perform repetitive tasks. That's all. But developers still have to face problems, and have to take decisions about their design to fix them. And experienced developers can identify better solutions.

Back to the original question: Do you need a software architect in a project with Grails? Yes, you do. But may be you don't need a full time architect. I have identified several scenarios:

  • Cowboy developer: Grails is a great framework if you are cowboy developer. The framework can enhance your productivity if you are a freelance developer working alone. If you don't need to work with colleagues it means you probably don't need to integrate with third parties (services, legacy databases, existing code). In this case, the cowboy developer can play the role of the architect, which makes all the sense. But, I'm not sure if Grails can be successful in this scenario: Ruby on Rails can fit better from my point of view (cheaper hosting, more accepted...).
  • Development shop: Your customers accept Grails as a valid option (this is a prerequisite, don't' forget to confirm first if they are happy with Groovy on Grails. Running inside a JVM and an application server is not enough!). You have a trained team of Grails developers. You probably have requirements regarding integration with third parties. In this scenario you need an architect (or a senior developer that can take the role) to take decisions about how to do the things and draft a design. Due to the fact that Grails is a very constrained development environment, you can save time of your architects. They will focus on giving the best solution, but they don't have to care about small details that Grails takes care of (remember, convention over configuration).
  • IT Departments: Congratulations, you have persuaded your management about the benefits of Grails. You have now a team in your staff of Grails developers plus some contractors. Normally, development teams inside the companies focus on the core business.The projects are evolution of others, there is a lot of service integration and maintenance. So your architect(s) will focus on smooth integrations with existing legacy code and services. In this scenario a strong architect is a must because you need somebody with good skills to design the bridge between your existing legacy systems and your shiny new Grails applications. This architect has to be a Grails believer: otherwise he will lose the faith when the firsts unexpected problems arrive, something common when starting with new frameworks and technologies (do you remember your first experiences with Struts, Maven or Hibernate ?)
An architect can save time with Grails because it's a constrained development environment and web developers can be very productive, but forget about saving time in the service layer if you have to integrate with a legacy middleware system, a complex database or web services. The best scenario for the service layer can be compared to a classic development using Spring and Hibernate. Still, I have some doubts about how is the development process and the architecture of a Grails application using a rich middle layer with EJBs and Web Services. I feel like a void under my feet when there is no database and the domain classes are not mapped to this database.

Would you fire a developer if he/she does not follow the coding conventions of the team?

Thursday, February 14, 2008

There is a very interesting discussion about how good or bad are code reviews in Martín Pérez's Blog (sorry, in Spanish!). Most of us think that the biggest problem of coding convention is the ego of the developers. But here comes something very difficult to manage, specially when adopting code reviews for the first time: what if some developers reject to follow the coding conventions? How can we enforce developers to follow coding conventions? And finally: is the replacement of the rebel developer by a new developer an option?

The Grails Diaries #4: Grails, is it really worth it?

Wednesday, February 06, 2008

Previous: How Grails will change my project plan

The results

I wanted to post my impressions and thoughts about Grails before, but you don't know how the events in your life can twist your plans: a delay in the delivery of the HTML designer, a flu followed by a nephritic colic, a laptop dying with all your development environment inside, the wrap up in Amplía, the Christmas holidays and my new start in The Server Labs. All these things made impossible for me to write down my conclusions about Grails applied to a small size project. Well, finally here it is, it's not a broad post-mortem analysis, but probably it will help you to take some decisions about this new technology.

The development environment

You can read in my previous post my development environment. I'm a Eclipse fan, but Eclipse still lacks of a good Grails IDE. So IntelliJ is an excellent option. I used the 30 days trial to develop the Proof of Concept, and I recommend it if you are going to use it for Grails development professionally. Sadly, I will not have the chance to use it in the near future unless in The Server Labs we have the chance to work in a Grails project soon. But if you are one of the lucky ones to work with Grails in an Enterprise environment, go for it. The learning curve of IntelliJ is not flat coming from Eclipse, but it's not hard. I think IntelliJ is more developer-oriented than Eclipse, honestly. The integration of Groovy and Grails is excellent, you can create specific artifacts because it acts as a Visual facade to the Grails commands. It took me just a couple of hours to understand the naming conventions, what artifacts were created from the menus and how to setup a Grails project. A tip: don't add the suffix of the type of the artifact to create; the commands will add the suffix automatically (convention over configuration, remember this my Java brain...). I changed the version of Grails several times, from 0.5.6 to 1.0RC3, with no major issue.
The configuration of a Grails application is simple because it's centralized under the /conf directory. I modified the Datasource.groovy and resources.xml under the /spring directory. I kept unmodified Bootstrap.groovy, Config.groovy and UrlMappings.groovy. Again, convention over configuration shows it pays off.
I started my project in debug mode, and then I modified all the artifacts (Java and Groovy classes, static files, Groovy Server Pages, properties files) without restarting the project. The environment detected the change and then reloaded the modified artifact. In some cases it did not relaunch the application, but in some others did. A change in a groovy or a java class under /src was followed by a reload of the application. The behaviour is similar to Sysdeo Tomcat, for example.

Groovy for an (experienced?) Java developer

I think this is the main concern of all Java developers when starting with Grails: how long will it take to learn Groovy? I cannot say learning Groovy is hard, but it takes time to change your mind to the new syntax, and of course it takes much more time to get full advantage of the features of the language. There are good resources out there to help, but I recommend to buy a book about the language. I started to read Groovy in Action as an ordinary book, but soon I realized it was better to have it by my side while coding as a reference guide.
First thing you realize is how you end all the lines of your code with a semicolon when it's not necessary :-) Not a big deal. Groovy can do more things with less lines of code once you use all the stuff the language provides. But it's not so easy to use them at the beginning. Your first lines of code with look like Java. As rule of thumb I coded first a la Java, and then I refactored the code with the help of the book Groovy in Action. My code was groovier and groovier day after day. Still, I think I did not get full advantage of some features of the language, specially dynamically extending the language and dynamic types. I did not use dynamic types, and my experience with loosely typed languages like Visual Basic made me a supporter of strongly-typed languages.
The excellent integration of Grails with Spring does not help either. Instead of trying to implement something more complex in Groovy, you implement it in Java and use it just adding the variable to inject the dependency in your Controller or Service classes. Obviously this is an advantage, but you have to disciplined if you want to learn Groovy using Grails and follow the approach 'Groovy first'.

Using GORM

GORM (Grails Object Relational Mapping) is the solution to map the Domain model to your database to hold the state of your business and blah, blah, blah... GORM sometimes made me feel like using a Domain Specific Language. The navigation through the object hierarchy of your domain classes is simpler because you avoid the getters and setters. Code is shorter and more expressive. I used dynamic finders for all the queries, I did not need to use Hibernate criterias -I didn't need them-. I don't know if I was doing something wrong, but I tried a dynamic finder with more than two parameters and it didn't work. I should check now with the new 1.0 final. I used constraints to check the input fields and get better scaffolding. It worked except for the email field (again my fault?).
My first version of the domain classes tried to reuse an existing database schema. But this database schema followed a naming convention very common in Entity-Relationship modelling, but an absolute nightmare in a Domain model. The properties and class names killed all the expressiveness of GORM as DSL language. In a real-life project, you should probably map the domain model to the ER model with .hbm.xml files -the old Hibernate way, killing the convention over configuration approach- if you cannot modify your database schema, but I decided to change the naming conventions of the ER model to something that GORM could digest with convention over configuration.
I think this is going to be one of the main problems to implement Grails in Enterprise environments where it's mandatory to follow some strict guidelines regarding naming at database level. It will work with the .hbm.xml Hibernate files workaround, but killing the convention over configuration.

Update: It seems that Grails 1.0 Final allows the direct mapping in the domain classes.

The scaffolding

It does not matter if you are going to use CRUD only operations or not, the scaffolding generates the basic skeleton for the typical cases in a web application. It means you will have code ready to use. I followed an agile approach, refactoring the generated code again and again until I got the functionality I wanted. After several iterations, part of the code was removed because it was unnecessary while the rest of the code was improved until it covered my initial specifications.
I tried to develop something from the scratch, without the scaffolding, but I found myself cutting and copying the code from other parts of the application. Then I decided to generate the CRUD operations with the scaffolding and modify the automatically generated code.

Web development basics

I have used several MVC implementations -basically Struts-, and some Web frameworks like Sitemesh and Tiles. Struts and Tiles configuration is hell: I tried Xdoclet to relief this pain, but I configuration was still and issue. With Grails you don't have to configure anything. Only plain convention over configuration. And I can't say it took me time to understand how it worked, the learning curve was almost non existent. I had already worked before with Sitemesh, and the concepts behind this framework applies 100% in Grails. I think I really save a lot of time with Grails configuring the layouts of the views.
GSP are almost identical to JSP. Just change the scriptlets code from Java to Groovy and use the Grails tag libraries. The number of tag libraries is big, but honestly I have just coded with the basic ones: conditionals, loops, access to form values, errors...). My HTML designer gave me the pages, and I just modified them to insert the dynamic stuff. I did not use any scriptlet, only the tags libraries. The localization of the pages was like in classic web development: modify the messages_*.properties files and use the g:message tab library.
There is a new scope concept called 'Flash' scope. With this scope, the information only is available for the current request and the next request. It's great to render errors text or warnings.
By the way, I tried to implement my test application avoiding the usage of HTTP sessions.

The controllers

All the controllers were created in the beginning with the scaffolding, but I refactored them iteration after iteration until I get functionality I wanted. Again convention over configuration for the name of the actions helps you to save precious time (compared to Struts this is really an advantage). Closures are key in Groovy and Controllers. You can feel intimidated with the code generated if you don't understand closures and don't know very well the Groovy syntax. I needed some time to understand the structure of a Controller and to double check how render, redirect and direct return of model objects worked. Probably this was my hardest time with Grails: fully understand how to extend Controllers and writing good Groovy code.

Spring to rescue

In my test application I wanted to send an email rendering a Velocity template with some basic information (username, password, url, etc...). I have developed this functionality for several projects in the past with Java, and I have already integrated it with Spring. So I took the Java code and pasted in the project. I updated the resources.xml file of Spring and modified a Controller to reference the bean. The integration of Groovy code and Java code using Spring was perfect. Spring wired the bean automatically, injecting the bean into the Controller. I could recycle my Java code like breeze.

Still not very clear...

I think there are several things to polish:
  • If you modify a domain class, the application will enter in an endless loop. Every time I modified a domain class I had to restart my project in debug mode. This was really annoying.
  • I could not force the change of the language adding a parameter in the url, as the documentation said.
  • The application created a session for every single element of the web application -static and dynamic-. It was a reported bug that should have been already fixed.
  • The problem with the session made me wonder if Grails was really ready for production. I tracked down the issue in the Jira of the project and it seems they knew and was scheduled to be fixed in RC4. Still, I think it was a serious bug that impacts directly on the scalability.

Conclusions

I took me about 40 hours of development to implement my test application. I sized the project with the classic Java web stack of Struts+Tiles+XDoclet+Spring+Hibernate in a range of 60-80 hours. So I was quite surprised with the results. I met a Ruby on Rails developer that told me it took him only 25% of the time to develop a simple web application in RoR compared to the classic Java web stack. I think he was exaggerating, but probably he could improve more than 50%. Now that I have more experience with Groovy and Grails I could improve my performance in future Grails projects reducing from 40 to 30 hours. That's one third of the work needed for a classic Java web stack. Of course, stand alone development cannot be compared with team development. A developer can build in 800 hours more than a team of four people in 200, because of code integration issues, knowledge transfer, meetings, etc. But you can really feel that Grails improves your performance. A lot.
Regarding the adoption of Grails in Enterprise environments, I would like to explain why I think it will be adopted broadly instead of Ruby on Rails. But that's a new story for a new post more focused on management and architectural topics.

Next: Do you need a Software Architect?

The Grails Diaries #3: How Grails will change my project plan

Tuesday, October 09, 2007

Previous: Setting up the development environment

The strategy

One of the most important reasons why I have chosen Grails and not JRuby/Ruby on Rails is that I'm lazy and I do not want to throw away all my Java knowledge. But now I'm wondering if Grails will change too much the way I plan my projects. And what do I mean with 'the way I plan my projects'? I mean the strategic decisions I usually take when planning my project. OK, and what 'strategic decisions'? A strategy is a long term plan of action designed to achieve a particular goal. Examples of strategic decisions planning a project sorted by (my) relevance are:
  1. What development technology I will use: A very common mistake when planning a project is the assumption that the development technology does not affect the development processes and workflows. I will show you later how Grails can modify the development workflow compared to Struts.
  2. How I will assign the resources: Though I'm a music man playing all the instruments in this project, in bigger projects you should normally find the following resources playing single roles. I will show you later why with Grails I think some of these roles can be redundant or even irrelevant:
    • Project Manager
    • Software Architect / Technical Lead
    • HTML Designers / Developers
    • Web layer developers
    • Business layer developers
    • Testers
  1. What development workflow I will use: Each member of the development team creates small deliverables to be consumed by the next member in the chain. Again, the standard development workflow (analyze-design-implement-test roundtrips) can change.
  2. How I will design the solution: Some of the design decisions, blue prints and patterns used in other web frameworks like Struts or others MVC implementations do not apply with Grails and it's convention over configuration approach. I will talk about it in future posts in this blog.

So yes, I think Grails will change how you plan your project. Here goes the details:

The development technology

I have seen some really big projects failing because of this naive assumption: a development technology can work just like another 'similar' technology. So, even before starting to analyze the project and with my short knowledge of Grails, I can forecast that Grails will constrain my planning in a way or another. Due to my lack of knowledge of Grails I cannot budget how much time and resources I will need as a buffer, so I should be conservative and reserve a good development buffer to manage the uncertain.

The resources assignment

In a classical Web project in Java it's very common to have a HTML Designer/Developer that builds the static pages of the site. Then he/she gives the pages to the Java Web developer. He/she converts them to JSP pages and creates the Action and Form Classes (if he/she works with Struts) and implements the logic regarding HTTP Sessions, cookies and all the web stuff. Finally, the business layer developer(s) build(s) the components that interacts with the database or other third party systems and expose(s) the interfaces to the web developers. This horizontal layered distribution of the developers helps the isolation of roles in teams and encourages the use of clear interfaces and pushes the implementation of automatic unit tests specially in the business layer.
Grails forces a different approach: The Vertical Slice. The Vertical Slice developer works in all the layers crossing the boundaries of the systems. It works changing HTML, GSP, Controllers and Domain classes at once. I guess this is one of the reasons of the productivity of Grails and RoR. Removing the public interfaces between layers avoids repeating unnecessary code. I have always liked the Vertical Slice approach because it fits perfectly with UML Use Cases. A Use Case is normally described crossing different layers, just like the Vertical Slice.
So, the roles of Web and Business layer developers can merge. Hence you need developers with a broader range of knowledge (HTML, Javascript, GSP, Groovy and or Java and HQL) and may be more experienced, but if Grails is more productive you can save in headcounts.
The Software Architect role is not clear in a Grails project. Due to the fact that the configuration is almost null and the architecture is so rigid I think that a full-time architect is an unnecessary resource. I would only consider a full-time architect if I need to integrate Grails with external components or existing Java projects (like OpenID in Sprint 2). If you only need to use the database, you don't need a full-time architect.

The development workflow

As I have said above, the development of a web interface with the classical approach is:
  1. A designer creates the look and feel, and then draft the CSS.
  2. The HTML Developer builds the pages and the navigation. Javascript code is also added here.
  3. The Web developer converts the HTML pages to JSP pages, adding the server side code and creating the Action and Form classes
  4. (Optional) The Web developer uses the business logic layer or develops the business logic by herself.

One of the most popular features of Ruby on Rails and Grails is the creation of automatic web controllers for basic CRUD actions, the scaffolding. The dynamic generation of code to manipulate classes of the Domain reduce the repetitive tasks of building Create-Retrieve-Update-Delete logic. And this is one of my doubts about the development workflow. Should I modify the HTML pages and convert them to GSPs and then create the controllers and their CRUD actions manually? Or should I use the automatically created scaffolding first, and then merge the HTML page and the automatically generated GSP? I can see pros and cons for both, and I will probably take the decision based on the complexity of the User Story to implement. Help and advice from experience Rails and Grails developers will be welcomed!
There is another workflow that can change: classic Web development takes the bottom-up approach to build the applications. Grails encourages the top-down approach because of its use of Hibernate and Domain classes. With the bottom-up approach, the database entity-relationship model should be implemented before starting with the business logic. With the top-down, the databases ER model arises as result of the Domain classes. This is not a trivial change in the workflow. In classic web development the Data architect tries to build a very stable version of the ER model before starting with the business logic, because the roundtrips 'change ER model, change business logic,change ER model, etc.' are 'expensive'. Developing with Grails (and with Hibernate too) the top-down approach and the features of Hibernate synchronizing the changes of the Domain classes with the ER model automatically set the developer free of the manual tracking of the changes and can build very fast the Domain classes.
There is an exception to the top-down approach with Grails: if you are using a legacy database you cannot take this approach. You must map the database to the domain classes, and probably the naming convention won't match, making necessary the use of hbm.xml Hibernate configuration files or annotations.

These are my thoughts about how to plan the development.In my next post I will talk about the analysis and design phases of the project.

Next: Grails, is it really worth it?

The Grails Diaries #2: Setting up the development environment

Thursday, October 04, 2007

Previous: JRuby on Rails or Grails?

Once I have explained why I will use Grails I will go deeper in the development environment.

Methodology: Scrum

One of the things I would like to test is how Grails works with Agile development methodologies. I'm not a ScrumMaster but I like Scrum and I would like to go deeper in this methodology. So I will divide the development of the project in two Scrum Sprints:
  1. Basic functionality of the web site and
  2. Some extra functionality integrating Spring an OpenID java library for authentication.

Tools

Mingle

Since I cannot guarantee how much of my time I will dedicate to this project every day, the Scrum approach of relative estimations is perfect.
I have been testing a tool from Thoughtworks called Mingle. This tool is not a supertool, but the interface is very simple and it's very configurable. The funny thing about this tool is it has been developed with JRuby on Rails!? The only bad thing about this tool is the slow response of some of the actions. And I guess it's not a problem of JRuby (Charles Nutter commented something about it ), but a problem of the application in itself (a problem with the ActiveRecord pattern?). All the user stories and tasks will be created with this tool, and I will publish them in my blog.

JetBrains IDEA 7.0M2

I was not a Jetbrains IntelliJDEA guy, but I have been testing a little bit the tool with Grails and it's the right tool for me. Sorry Emacs and Vi talibans, I need auto-complete and a lot of windows in my desktop. I will use a 30 days trial license. If the tool convince me,I will probably change from Eclipse.

EMS SQL Manager

My favourite SQL Administration Tool for MySQL. I can do almost everything from it.

CVS

I have a CVS server at home (and who doesn't?) running in my little and beautiful Buffalo Kurobox.

Books

I have these books to help me with this task:

Groovy in Action is probably the best guide to Groovy language. It's an excellent book, with lots of examples and I think covers all the things that matters in Groovy. If you are familiar with Java reading this book will make the transition very smooth. If you are not familiar with Java then you will need a Java language book by your side for sure.
Getting Started with Grails is like a very big tutorial about how to build Grails applications. My first impression was poor, because the book is very short: 120 pages. I read it a Sunday afternoon. The book is short, but it's good. It does not go into the details of Grails because when something deeper and detailed is needed then it refers to the content in the website of Grails. It just focuses in a step by step example of how to build an application. So I think it will help in my tests.

Software Architecture: The basics

And the software products I'm going to use.

Operating System
Windows XP SP2
JDK
6 Update 2
Database
MySQL Server 5.0 Community Edition
Development Application Server
Jetty 6
Production Application Server
Tomcat 6.0
JDBC Driver
ConnectorJ 5.0.7
IDE
IntelliJ IDEA 7.0M2

Installing the IDE and the Grails development environment

If you are not familiar with IntelliJ like me then you will probably waste a lot of time trying and discovering all the features of the tool. The first thing to do is to download the JetGroovy plugin. Go to 'IDE Settings > Plugins > Available' and find 'JetGroovy'. Click on the right button and download and install the plug-in. You will have to restart the IDE. Now we have to download the Groovy 1.0 libraries and Grails 0.5.6. You can install them anywhere, it seems there is not any directory restrictions. Now go to the Groovy/Grails node in IDE Settings and select the existing directories of the tools.
The current version of IntelliJ IDEA 7.0M2 does not support version 0.6. It seems that the groovy-starter.jar has been re-factored in another jar file and now you cannot run the grails applications. Hopefully it will be fixed in future versions of the IDE.
Update: Read the comments at the bottom to fix this issue. It's possible to use 0.6 with IDEA 7.0M2. Thanks Dierk and Graeme.

Now we have to create a Grails project from the scratch. No problem, go to 'New Project' > Select 'Create Project from scratch' > Choose your favourite name and 'Grails Application > Create a source directory and click 'Finish'. The tool creates all the Grails stuff, just like running the create-app command from the command line. Be patient, it can take some time.
The setup of the server running the code is easy: go to 'Run > Run/Debug Configurations' and 'Add a new Configuration', 'Grails Application'. I have named it 'website'. I have used the default parameters (Run Application, No VM parameters, Make before launch). Click on the 'Run' button and voila! if you connect to http://localhost:8080/website you will see the Grails Welcome Page.
The environment is ready. In future posts I will show how I'm going to plan the development of the project.

Next: How Grails will change my project plan

The Grails Diaries: JRuby on Rails or Grails?

Monday, October 01, 2007

I must admit it: I'm weak. During last two years I have been trying to avoid the dark attraction of Ruby on Rails. And it was hard. I have been searching for almost seven years something that alleviates my feeling of waste of time when developing a Web user interface. And I have been promiscuous, believe me. I have tried everything in the Java world: Tapestry, Velocity, Cocoon,Altio, GWT, Flex, Laszlo... I even tried ASP.NET!!! But finally I gave up. It took me a lot of time to realize that it's not about the framework, it's about the level of details in the development of Web applications.

So I stopped looking for my Golden Hammer and assumed that Web development is hard. But I still listened the mermaids in the blogosphere claiming for the Holy Grail of Web Development: Ruby on Rails. The approach was different: 'OK, Web development is hard, but may be we can try to avoid the repetitive tasks using a language full of expressiveness -Ruby- and a framework based on conventions and not on configuration. Coming from the Java World this convention over configuration thing was like a heresy, but an attractive heresy.
In my daily job -the one I do for living- I just code after 5PM and/or because there is an emergency. But I still enjoy coding and it's something I do in my spare time. It's like blogging, only if I have time to do it. So I have decided to perform a project to testJRuby on Rails or Grails just for fun. A very simple project, because I still think this frameworks only work for small projects.

But why not pure Ruby on Rails? Here goes my reasons:

  • I'm going to do this test just for fun, but I feel it's possible to use these frameworks inside the Enterprise. And the only way to convince IT departments to run these frameworks is reusing all their Java infrastructure. If it runs on top of a JVM and inside the application server X, then it's OK.
  • There are TONS of excellent open source software for Java. And we have already built some nice components running with Spring orEJB containers. It makes a lot of sense to reuse it. So, Integration of existing components is an added value for Grails and JRoR.
Now the thing is, should I use JRuby on Rails or Grails? I have decided to use Grails because of the following reasons:
  • The main reason is laziness: I have a little notion of Groovy, but very few of Ruby.
  • Spring integration. I need to integrate this website with some business logic already developed. Grails have a good Spring integration.
  • ActiveRecord pattern. I don't like this pattern. I feel it cannot work in complex models. I prefer the Hibernate approach of Grails.
  • The development IDE. I'm fed up of Eclipse. I want to try Jetbrains IDEA 7.0 with Grails support.
  • I have the feeling that JRuby is not still mature enough. Charles Nutter is making an incredible job, but Groovy works solid as a rock.

So, the first step is done: I will go for Grails. The next is step is to define the project scope and the development methodology. So stay tuned!

Next: Setting up the development environment

Ruby on Rails hype: the party is over

Monday, September 24, 2007

Finally it seems that Ruby on Rails is starting to show its real face. Two years of RoR development put into the trash. Wow! Being seven times more productive than Java that means fourteen years of development! Seriously, it's not common to hear of failures of a new and cool software stuff -you can look stupid- so I guess this is the peak of the iceberg of Ruby On Rails projects. In my opinion, Ruby on Rails failed in this project because when dealing with the very small details of a project (that 5% of the project that takes 90% of the time) RoR convention over configuration approach fails. This, and of course integration with legacy systems (considering a database a legacy system). It's clear that any other general purpose language and framework (JEE, .NET) can solve the integration problem better and faster.

Why developers hate bug-fixing

Monday, September 10, 2007

Probably the only common thing among all developers I have worked with is the hate to bug-fixing. No matter if bugs belong to them or not. As a developer and as a manager I have quite contradictory feelings.

As a developer there was nothing more irritating than the hateful Bugzilla emails. When I was working as Technical Lead of a development team in Madrid with a QA team in the West Coast of USA they usually started to work at our 18:00 and we usually left at 19:00 (Madrid is GMT+2 summer time). So we counted the number of bugs per hour submitted in Bugzilla and extrapolated to their 8 working hours to get the number of new bugs in the system. We left the office betting on the number of new bugs for the next day. We felt the QA team was our enemy and the relationship between both teams was difficult. We felt like being in 'Groundhog day' movie! We were listening constantly how bug-fixing is important for the customer, how new features were delayed because stupid bugs and so on...

As a manager I knew that bug-fixing was the most irritating part of development for developers, but you cannot avoid it. Bug fixing is a must. I'm very happy when I found new bugs in our products or projects and I tried to make developers to understand that it's great to find bugs before our customers find them -wrong!-. This is true for managers, if you find bugs in advance then the management team does not need to loose the face with the client due to problems with the quality of the product. But it is not true for developers, and I will try to explain why.

  • Good developers are creators. They feel their work as part of them. Most of the programmers are people very motivated because they like what they do. As a creator their work is his/her creation and telling them 'your creation has a defect' it's a bit like saying 'your kid is ugly'. We react like feeling attacked by the management, the customer or QA team.
  • Programmers don't work like Managers do. Programmers need to find 'the flow'. We need time to get the focus on the problem, and 'the flow' must be durable. Managers work in 'constantly interrupted' mode. So emails reporting defects every X minutes, phone calls or 'mini-meetings' really disturb. Developers mentally link bugs to interruptions.
  • Bug-fixing is normally underestimated in the project plans.They normally include the raw time to fix the defects, but not the time taken due to the 'context change' of the developer: from new feature mode to bug-fixing mode and back to new feature mode. These means that developers link again bug fixing to overtime, because it delays the development of the new features with due date.
  • Programmers hate to hear from Management: 'It's great to find bugs before the customer finds them'. Why? Because we are not stupid. We know it. It's great to finds bug before the customer finds them and report to you, Bastard Manager. I will have to fix the bug anyway, no matter if the QA team or the Customer finds them. I have no tangible gain for fixing the bug before the delivery. But the Evil Manager does, he won't have to justify bugs to the customer, he will live better.
The summary is: Developers hate to fix bugs because they feel like being insulted (your kid is ugly), annoyed (emails, phone calls, minimeetings), poorly managed (overtime), cheated (finding bugs in advance is great!).

What can we do -as a Manager- to minimize the impact of bug-fixing in your team?:
  • Developers will feel attacked every time a bug is found. Remove all adjectives when referring to bugs. Example: This is a stupid bug, this is a trivial bug... Just use the adjectives to categorize them. Use the terms used by your tools.
  • Never ever tell a developer how to fix a bug, except if he/she asks for it. Example: This is just an 'if...then...else'... 'Use a try...catch...'.
  • Do not allow customers to call or send emails directly to the developers. Customer does not know how to communicate the defects. The defects reported by the customer must be handled by the Managers.
  • Set up your bug-fixing reporting tools to send the bugs once everyday or better, once per iteration. Only categorized and approved bugs must reach the developers. Avoid unnecessary stress.
  • Don't forget to size your projects with enough time for bug-fixing.
  • Dedicate one or two days per iteration to 'bug-fixing days'. Stop the development of new features and get the commitment of the team to work hard that couple of days to 'clean the bug list'. If the environment of your organization allows it, bring drinks, food and have some beers afterwards. Give the same relevance to the bug-fixing days than delivery days.
  • It's great to find bugs before customer finds them, that's true. But don't use the excuse of the customer anymore, or at least not so often. For programmers customers are irrelevant, they/we are not customer oriented. Change to 'We will find bugs for sure, and every time we find a bug we should congratulate because the QA team is doing an excellent job. If we don't find bugs then we are not doing good QA'.

What do you think? What do you recommend to help developers to fix bugs?