Welcome to the Jungle (Redux)

Previous: How Grails will change my project plan

The results

I wanted to post my impressions and thoughts about Grails before, but you don't know how the events in your life can twist your plans: a delay in the delivery of the HTML designer, a flu followed by a nephritic colic, a laptop dying with all your development environment inside, the wrap up in Amplía, the Christmas holidays and my new start in The Server Labs. All these things made impossible for me to write down my conclusions about Grails applied to a small size project. Well, finally here it is, it's not a broad post-mortem analysis, but probably it will help you to take some decisions about this new technology.

The development environment

You can read in my previous post my development environment. I'm a Eclipse fan, but Eclipse still lacks of a good Grails IDE. So IntelliJ is an excellent option. I used the 30 days trial to develop the Proof of Concept, and I recommend it if you are going to use it for Grails development professionally. Sadly, I will not have the chance to use it in the near future unless in The Server Labs we have the chance to work in a Grails project soon. But if you are one of the lucky ones to work with Grails in an Enterprise environment, go for it. The learning curve of IntelliJ is not flat coming from Eclipse, but it's not hard. I think IntelliJ is more developer-oriented than Eclipse, honestly. The integration of Groovy and Grails is excellent, you can create specific artifacts because it acts as a Visual facade to the Grails commands. It took me just a couple of hours to understand the naming conventions, what artifacts were created from the menus and how to setup a Grails project. A tip: don't add the suffix of the type of the artifact to create; the commands will add the suffix automatically (convention over configuration, remember this my Java brain...). I changed the version of Grails several times, from 0.5.6 to 1.0RC3, with no major issue.
The configuration of a Grails application is simple because it's centralized under the /conf directory. I modified the Datasource.groovy and resources.xml under the /spring directory. I kept unmodified Bootstrap.groovy, Config.groovy and UrlMappings.groovy. Again, convention over configuration shows it pays off.
I started my project in debug mode, and then I modified all the artifacts (Java and Groovy classes, static files, Groovy Server Pages, properties files) without restarting the project. The environment detected the change and then reloaded the modified artifact. In some cases it did not relaunch the application, but in some others did. A change in a groovy or a java class under /src was followed by a reload of the application. The behaviour is similar to Sysdeo Tomcat, for example.

Groovy for an (experienced?) Java developer

I think this is the main concern of all Java developers when starting with Grails: how long will it take to learn Groovy? I cannot say learning Groovy is hard, but it takes time to change your mind to the new syntax, and of course it takes much more time to get full advantage of the features of the language. There are good resources out there to help, but I recommend to buy a book about the language. I started to read Groovy in Action as an ordinary book, but soon I realized it was better to have it by my side while coding as a reference guide.
First thing you realize is how you end all the lines of your code with a semicolon when it's not necessary :-) Not a big deal. Groovy can do more things with less lines of code once you use all the stuff the language provides. But it's not so easy to use them at the beginning. Your first lines of code with look like Java. As rule of thumb I coded first a la Java, and then I refactored the code with the help of the book Groovy in Action. My code was groovier and groovier day after day. Still, I think I did not get full advantage of some features of the language, specially dynamically extending the language and dynamic types. I did not use dynamic types, and my experience with loosely typed languages like Visual Basic made me a supporter of strongly-typed languages.
The excellent integration of Grails with Spring does not help either. Instead of trying to implement something more complex in Groovy, you implement it in Java and use it just adding the variable to inject the dependency in your Controller or Service classes. Obviously this is an advantage, but you have to disciplined if you want to learn Groovy using Grails and follow the approach 'Groovy first'.

Using GORM

GORM (Grails Object Relational Mapping) is the solution to map the Domain model to your database to hold the state of your business and blah, blah, blah... GORM sometimes made me feel like using a Domain Specific Language. The navigation through the object hierarchy of your domain classes is simpler because you avoid the getters and setters. Code is shorter and more expressive. I used dynamic finders for all the queries, I did not need to use Hibernate criterias -I didn't need them-. I don't know if I was doing something wrong, but I tried a dynamic finder with more than two parameters and it didn't work. I should check now with the new 1.0 final. I used constraints to check the input fields and get better scaffolding. It worked except for the email field (again my fault?).
My first version of the domain classes tried to reuse an existing database schema. But this database schema followed a naming convention very common in Entity-Relationship modelling, but an absolute nightmare in a Domain model. The properties and class names killed all the expressiveness of GORM as DSL language. In a real-life project, you should probably map the domain model to the ER model with .hbm.xml files -the old Hibernate way, killing the convention over configuration approach- if you cannot modify your database schema, but I decided to change the naming conventions of the ER model to something that GORM could digest with convention over configuration.
I think this is going to be one of the main problems to implement Grails in Enterprise environments where it's mandatory to follow some strict guidelines regarding naming at database level. It will work with the .hbm.xml Hibernate files workaround, but killing the convention over configuration.

Update: It seems that Grails 1.0 Final allows the direct mapping in the domain classes.

The scaffolding

It does not matter if you are going to use CRUD only operations or not, the scaffolding generates the basic skeleton for the typical cases in a web application. It means you will have code ready to use. I followed an agile approach, refactoring the generated code again and again until I got the functionality I wanted. After several iterations, part of the code was removed because it was unnecessary while the rest of the code was improved until it covered my initial specifications.
I tried to develop something from the scratch, without the scaffolding, but I found myself cutting and copying the code from other parts of the application. Then I decided to generate the CRUD operations with the scaffolding and modify the automatically generated code.

Web development basics

I have used several MVC implementations -basically Struts-, and some Web frameworks like Sitemesh and Tiles. Struts and Tiles configuration is hell: I tried Xdoclet to relief this pain, but I configuration was still and issue. With Grails you don't have to configure anything. Only plain convention over configuration. And I can't say it took me time to understand how it worked, the learning curve was almost non existent. I had already worked before with Sitemesh, and the concepts behind this framework applies 100% in Grails. I think I really save a lot of time with Grails configuring the layouts of the views.
GSP are almost identical to JSP. Just change the scriptlets code from Java to Groovy and use the Grails tag libraries. The number of tag libraries is big, but honestly I have just coded with the basic ones: conditionals, loops, access to form values, errors...). My HTML designer gave me the pages, and I just modified them to insert the dynamic stuff. I did not use any scriptlet, only the tags libraries. The localization of the pages was like in classic web development: modify the messages_*.properties files and use the g:message tab library.
There is a new scope concept called 'Flash' scope. With this scope, the information only is available for the current request and the next request. It's great to render errors text or warnings.
By the way, I tried to implement my test application avoiding the usage of HTTP sessions.

The controllers

All the controllers were created in the beginning with the scaffolding, but I refactored them iteration after iteration until I get functionality I wanted. Again convention over configuration for the name of the actions helps you to save precious time (compared to Struts this is really an advantage). Closures are key in Groovy and Controllers. You can feel intimidated with the code generated if you don't understand closures and don't know very well the Groovy syntax. I needed some time to understand the structure of a Controller and to double check how render, redirect and direct return of model objects worked. Probably this was my hardest time with Grails: fully understand how to extend Controllers and writing good Groovy code.

Spring to rescue

In my test application I wanted to send an email rendering a Velocity template with some basic information (username, password, url, etc...). I have developed this functionality for several projects in the past with Java, and I have already integrated it with Spring. So I took the Java code and pasted in the project. I updated the resources.xml file of Spring and modified a Controller to reference the bean. The integration of Groovy code and Java code using Spring was perfect. Spring wired the bean automatically, injecting the bean into the Controller. I could recycle my Java code like breeze.

Still not very clear...

I think there are several things to polish:

• If you modify a domain class, the application will enter in an endless loop. Every time I modified a domain class I had to restart my project in debug mode. This was really annoying.
  
• I could not force the change of the language adding a parameter in the url, as the documentation said.
• The application created a session for every single element of the web application -static and dynamic-. It was a reported bug that should have been already fixed.
• The problem with the session made me wonder if Grails was really ready for production. I tracked down the issue in the Jira of the project and it seems they knew and was scheduled to be fixed in RC4. Still, I think it was a serious bug that impacts directly on the scalability.
  

Conclusions

I took me about 40 hours of development to implement my test application. I sized the project with the classic Java web stack of Struts+Tiles+XDoclet+Spring+Hibernate in a range of 60-80 hours. So I was quite surprised with the results. I met a Ruby on Rails developer that told me it took him only 25% of the time to develop a simple web application in RoR compared to the classic Java web stack. I think he was exaggerating, but probably he could improve more than 50%. Now that I have more experience with Groovy and Grails I could improve my performance in future Grails projects reducing from 40 to 30 hours. That's one third of the work needed for a classic Java web stack. Of course, stand alone development cannot be compared with team development. A developer can build in 800 hours more than a team of four people in 200, because of code integration issues, knowledge transfer, meetings, etc. But you can really feel that Grails improves your performance. A lot.
Regarding the adoption of Grails in Enterprise environments, I would like to explain why I think it will be adopted broadly instead of Ruby on Rails. But that's a new story for a new post more focused on management and architectural topics.

Next: Do you need a Software Architect?

Sometimes you only need to wait to understand why things happen. When Steve Jobs said 'Java is not worth building in. Nobody uses Java anymore. It's this big heavyweight ball and chain', we were surprised and disappointed because like me a lot of people think that there is still room for Java in the mobile platforms. Of course we have J2ME, but we can do very few things with it. What we need is a full J2SE stack for mobiles. And the iPhone could do it... if Mr. Jobs wants. But he does not want because Java has become the flag of Google to do developers to follow them. He chose to keep Java out of the iPhone to keep a strict control of the platform, to keep Google out of his business.
I have just reviewed what Google Android is, it's architecture and the SDK and I think it's much more than a smart mobile platform: it's a philosophy. Think of what SWT is to the Java User Interfaces and apply it to the Android Architecture. The access to the physical layer has been developed in C/C++ on top the Linux OS, and on top of it there is a pseudo-Java Virtual Machine called Dalvik virtual machine. As they say, "Dalvik has been written so that a device can run multiple VMs efficiently. The Dalvik VM executes files in the Dalvik Executable (.dex) format which is optimized for minimal memory footprint. The VM is register-based, and runs classes compiled by a Java language compiler that have been transformed into the .dex format by the included "dx" tool."
J2ME is very similar in the approach, but what looks new is the tight integration with the OS and the use of Java 5 extensions of the language. J2ME is almost JDK 1.0, but if you read the source code of the samples, you can recognize some coding practices of Java 5. I still don't know if the Dalvik VM can do reflection or annotations, I will need some time to test it.
The access to the physical elements of the mobile device are performed thanks to the set of android.* packages. So, it will be a task of the device manufacturer to migrate the C/C++ libraries layer for their devices. I have tried to find some information about how to certificate that the libraries has been migrated correctly. This was one of the key elements for the success of J2ME on the mobile phones, and should be done the same way.
The Java applications are what they call 'Third party applications'. It's not possible yet to develop or modify the underlying Linux OS and the C/C++ libraries. Again, Java runs in a sandbox -sounds familiar, eh?-.
I will continue playing with it. Just some random thoughts, will Sun Microsystems get some money for the licensing of Dalvik VM? How much? Will J2ME be replaced by the Dalvik VM?

Last years have been a wonderful time in Amplía Soluciones S.L., but nothing is forever and I think it's time for a change. I will continue as a partner of the company but I will look for new challenges and adventures out there. You can read my public linkedin profile if you want to know a bit about my skills and experience.

Today I have discovered the existence of G2One.com. This is the company created by the people behind Grails. Following the business model 'It is all about the Services, stupid!' they have jumped into the bandwagon of companies that:

1. create a framework
2. work hard to make it first class
3. create or find a necessity in the community
4. offer their services as gurus to the companies

37signals is the paradigm of this approach (they created Rails). They sell products not only services which makes them distinct from Interface21, the creators of Spring. Interface21 only sells professional services as far as I know.

I think G2One potentially can overtake 37Signals. I feels Grails can fit into the Enterprise much better than Ruby On Rails because it works on the existing deployed application servers. If they can guarantee the same productivity of Rails, the same stability of Java and the JVM and the seamless integration with legacy Java code, then they can succeed.

Good luck!

P.D.: Being 37signals and interface21 the path to follow, should they change the name to 'G2fiftyeight?'

I have received information about open positions in Linkedin for Grails developers. This is very interesting because it seems that big companies have realized of the potential of Grails.
I think Linkedin is the biggest professional network site with more than 14 million users (Via wikipedia).
It sounds like a very interesting place to work, a pity I'm not in the Bay Area. :-(

Previous: Setting up the development environment

The strategy

One of the most important reasons why I have chosen Grails and not JRuby/Ruby on Rails is that I'm lazy and I do not want to throw away all my Java knowledge. But now I'm wondering if Grails will change too much the way I plan my projects. And what do I mean with 'the way I plan my projects'? I mean the strategic decisions I usually take when planning my project. OK, and what 'strategic decisions'? A strategy is a long term plan of action designed to achieve a particular goal. Examples of strategic decisions planning a project sorted by (my) relevance are:

1. What development technology I will use: A very common mistake when planning a project is the assumption that the development technology does not affect the development processes and workflows. I will show you later how Grails can modify the development workflow compared to Struts.
   
2. How I will assign the resources: Though I'm a music man playing all the instruments in this project, in bigger projects you should normally find the following resources playing single roles. I will show you later why with Grails I think some of these roles can be redundant or even irrelevant:

• Project Manager
• Software Architect / Technical Lead
• HTML Designers / Developers
• Web layer developers
• Business layer developers
  
• Testers
  

3. What development workflow I will use: Each member of the development team creates small deliverables to be consumed by the next member in the chain. Again, the standard development workflow (analyze-design-implement-test roundtrips) can change.
   
4. How I will design the solution: Some of the design decisions, blue prints and patterns used in other web frameworks like Struts or others MVC implementations do not apply with Grails and it's convention over configuration approach. I will talk about it in future posts in this blog.
   

So yes, I think Grails will change how you plan your project. Here goes the details:

The development technology

I have seen some really big projects failing because of this naive assumption: a development technology can work just like another 'similar' technology. So, even before starting to analyze the project and with my short knowledge of Grails, I can forecast that Grails will constrain my planning in a way or another. Due to my lack of knowledge of Grails I cannot budget how much time and resources I will need as a buffer, so I should be conservative and reserve a good development buffer to manage the uncertain.

The resources assignment

In a classical Web project in Java it's very common to have a HTML Designer/Developer that builds the static pages of the site. Then he/she gives the pages to the Java Web developer. He/she converts them to JSP pages and creates the Action and Form Classes (if he/she works with Struts) and implements the logic regarding HTTP Sessions, cookies and all the web stuff. Finally, the business layer developer(s) build(s) the components that interacts with the database or other third party systems and expose(s) the interfaces to the web developers. This horizontal layered distribution of the developers helps the isolation of roles in teams and encourages the use of clear interfaces and pushes the implementation of automatic unit tests specially in the business layer.
Grails forces a different approach: The Vertical Slice. The Vertical Slice developer works in all the layers crossing the boundaries of the systems. It works changing HTML, GSP, Controllers and Domain classes at once. I guess this is one of the reasons of the productivity of Grails and RoR. Removing the public interfaces between layers avoids repeating unnecessary code. I have always liked the Vertical Slice approach because it fits perfectly with UML Use Cases. A Use Case is normally described crossing different layers, just like the Vertical Slice.
So, the roles of Web and Business layer developers can merge. Hence you need developers with a broader range of knowledge (HTML, Javascript, GSP, Groovy and or Java and HQL) and may be more experienced, but if Grails is more productive you can save in headcounts.
The Software Architect role is not clear in a Grails project. Due to the fact that the configuration is almost null and the architecture is so rigid I think that a full-time architect is an unnecessary resource. I would only consider a full-time architect if I need to integrate Grails with external components or existing Java projects (like OpenID in Sprint 2). If you only need to use the database, you don't need a full-time architect.

The development workflow

As I have said above, the development of a web interface with the classical approach is:

1. A designer creates the look and feel, and then draft the CSS.
2. The HTML Developer builds the pages and the navigation. Javascript code is also added here.
3. The Web developer converts the HTML pages to JSP pages, adding the server side code and creating the Action and Form classes
4. (Optional) The Web developer uses the business logic layer or develops the business logic by herself.
   

One of the most popular features of Ruby on Rails and Grails is the creation of automatic web controllers for basic CRUD actions, the scaffolding. The dynamic generation of code to manipulate classes of the Domain reduce the repetitive tasks of building Create-Retrieve-Update-Delete logic. And this is one of my doubts about the development workflow. Should I modify the HTML pages and convert them to GSPs and then create the controllers and their CRUD actions manually? Or should I use the automatically created scaffolding first, and then merge the HTML page and the automatically generated GSP? I can see pros and cons for both, and I will probably take the decision based on the complexity of the User Story to implement. Help and advice from experience Rails and Grails developers will be welcomed!
There is another workflow that can change: classic Web development takes the bottom-up approach to build the applications. Grails encourages the top-down approach because of its use of Hibernate and Domain classes. With the bottom-up approach, the database entity-relationship model should be implemented before starting with the business logic. With the top-down, the databases ER model arises as result of the Domain classes. This is not a trivial change in the workflow. In classic web development the Data architect tries to build a very stable version of the ER model before starting with the business logic, because the roundtrips 'change ER model, change business logic,change ER model, etc.' are 'expensive'. Developing with Grails (and with Hibernate too) the top-down approach and the features of Hibernate synchronizing the changes of the Domain classes with the ER model automatically set the developer free of the manual tracking of the changes and can build very fast the Domain classes.
There is an exception to the top-down approach with Grails: if you are using a legacy database you cannot take this approach. You must map the database to the domain classes, and probably the naming convention won't match, making necessary the use of hbm.xml Hibernate configuration files or annotations.

These are my thoughts about how to plan the development.In my next post I will talk about the analysis and design phases of the project.

Next: Grails, is it really worth it?

Previous: JRuby on Rails or Grails?

Once I have explained why I will use Grails I will go deeper in the development environment.

Methodology: Scrum

One of the things I would like to test is how Grails works with Agile development methodologies. I'm not a ScrumMaster but I like Scrum and I would like to go deeper in this methodology. So I will divide the development of the project in two Scrum Sprints:

1. Basic functionality of the web site and
   
2. Some extra functionality integrating Spring an OpenID java library for authentication.

Tools

Mingle

Since I cannot guarantee how much of my time I will dedicate to this project every day, the Scrum approach of relative estimations is perfect.
I have been testing a tool from Thoughtworks called Mingle. This tool is not a supertool, but the interface is very simple and it's very configurable. The funny thing about this tool is it has been developed with JRuby on Rails!? The only bad thing about this tool is the slow response of some of the actions. And I guess it's not a problem of JRuby (Charles Nutter commented something about it ), but a problem of the application in itself (a problem with the ActiveRecord pattern?). All the user stories and tasks will be created with this tool, and I will publish them in my blog.

JetBrains IDEA 7.0M2

I was not a Jetbrains IntelliJDEA guy, but I have been testing a little bit the tool with Grails and it's the right tool for me. Sorry Emacs and Vi talibans, I need auto-complete and a lot of windows in my desktop. I will use a 30 days trial license. If the tool convince me,I will probably change from Eclipse.

EMS SQL Manager

My favourite SQL Administration Tool for MySQL. I can do almost everything from it.

CVS

I have a CVS server at home (and who doesn't?) running in my little and beautiful Buffalo Kurobox.

Books

Software Architecture: The basics

And the software products I'm going to use.

Installing the IDE and the Grails development environment

If you are not familiar with IntelliJ like me then you will probably waste a lot of time trying and discovering all the features of the tool. The first thing to do is to download the JetGroovy plugin. Go to 'IDE Settings > Plugins > Available' and find 'JetGroovy'. Click on the right button and download and install the plug-in. You will have to restart the IDE. Now we have to download the Groovy 1.0 libraries and Grails 0.5.6. You can install them anywhere, it seems there is not any directory restrictions. Now go to the Groovy/Grails node in IDE Settings and select the existing directories of the tools.
The current version of IntelliJ IDEA 7.0M2 does not support version 0.6. It seems that the groovy-starter.jar has been re-factored in another jar file and now you cannot run the grails applications. Hopefully it will be fixed in future versions of the IDE.
Update: Read the comments at the bottom to fix this issue. It's possible to use 0.6 with IDEA 7.0M2. Thanks Dierk and Graeme.

Now we have to create a Grails project from the scratch. No problem, go to 'New Project' > Select 'Create Project from scratch' > Choose your favourite name and 'Grails Application > Create a source directory and click 'Finish'. The tool creates all the Grails stuff, just like running the create-app command from the command line. Be patient, it can take some time.
The setup of the server running the code is easy: go to 'Run > Run/Debug Configurations' and 'Add a new Configuration', 'Grails Application'. I have named it 'website'. I have used the default parameters (Run Application, No VM parameters, Make before launch). Click on the 'Run' button and voila! if you connect to http://localhost:8080/website you will see the Grails Welcome Page.
The environment is ready. In future posts I will show how I'm going to plan the development of the project.

Next: How Grails will change my project plan

I must admit it: I'm weak. During last two years I have been trying to avoid the dark attraction of Ruby on Rails. And it was hard. I have been searching for almost seven years something that alleviates my feeling of waste of time when developing a Web user interface. And I have been promiscuous, believe me. I have tried everything in the Java world: Tapestry, Velocity, Cocoon,Altio, GWT, Flex, Laszlo... I even tried ASP.NET!!! But finally I gave up. It took me a lot of time to realize that it's not about the framework, it's about the level of details in the development of Web applications.

So I stopped looking for my Golden Hammer and assumed that Web development is hard. But I still listened the mermaids in the blogosphere claiming for the Holy Grail of Web Development: Ruby on Rails. The approach was different: 'OK, Web development is hard, but may be we can try to avoid the repetitive tasks using a language full of expressiveness -Ruby- and a framework based on conventions and not on configuration. Coming from the Java World this convention over configuration thing was like a heresy, but an attractive heresy.
In my daily job -the one I do for living- I just code after 5PM and/or because there is an emergency. But I still enjoy coding and it's something I do in my spare time. It's like blogging, only if I have time to do it. So I have decided to perform a project to testJRuby on Rails or Grails just for fun. A very simple project, because I still think this frameworks only work for small projects.

But why not pure Ruby on Rails? Here goes my reasons:

• I'm going to do this test just for fun, but I feel it's possible to use these frameworks inside the Enterprise. And the only way to convince IT departments to run these frameworks is reusing all their Java infrastructure. If it runs on top of a JVM and inside the application server X, then it's OK.
• There are TONS of excellent open source software for Java. And we have already built some nice components running with Spring orEJB containers. It makes a lot of sense to reuse it. So, Integration of existing components is an added value for Grails and JRoR.

Now the thing is, should I use JRuby on Rails or Grails? I have decided to use Grails because of the following reasons:

• The main reason is laziness: I have a little notion of Groovy, but very few of Ruby.
• Spring integration. I need to integrate this website with some business logic already developed. Grails have a good Spring integration.
• ActiveRecord pattern. I don't like this pattern. I feel it cannot work in complex models. I prefer the Hibernate approach of Grails.
• The development IDE. I'm fed up of Eclipse. I want to try Jetbrains IDEA 7.0 with Grails support.
  
• I have the feeling that JRuby is not still mature enough. Charles Nutter is making an incredible job, but Groovy works solid as a rock.

So, the first step is done: I will go for Grails. The next is step is to define the project scope and the development methodology. So stay tuned!

Next: Setting up the development environment

Finally it seems that Ruby on Rails is starting to show its real face. Two years of RoR development put into the trash. Wow! Being seven times more productive than Java that means fourteen years of development! Seriously, it's not common to hear of failures of a new and cool software stuff -you can look stupid- so I guess this is the peak of the iceberg of Ruby On Rails projects. In my opinion, Ruby on Rails failed in this project because when dealing with the very small details of a project (that 5% of the project that takes 90% of the time) RoR convention over configuration approach fails. This, and of course integration with legacy systems (considering a database a legacy system). It's clear that any other general purpose language and framework (JEE, .NET) can solve the integration problem better and faster.

A few days ago I made a search in Infojobs.es (the most popular Job Board in Spain) for open positions for Ruby On Rails developers. The result was... one! I was very surprised. Only one? With all the hype around RoR? Only one job offer?
It's obvious that USA is more developed regarding technology than Spain, but the question is: How far are they? Can we measure the distance? Honestly I think it's very difficult and probably comparing two countries is not the right way. But anyway I did the following exercise. I selected a list of key words of software development, and I compared the number of open positions found in dice.com for USA and infojobs.es for Spain. Since absolute numbers are not comparable, I summed the number of open positions per key word and compared the percentages.
These are the key words and the number of open positions:

Keyword	dice.com	dice.com %	infojobs	infojobs %
JAVA	17098	32.87%	1682	32.41%
JAVASCRIPT	6437	12.38%	581	11.19%
J2EE	8304	15.96%	1077	20.75%
.NET	13269	25.51%	935	18.02%
PYTHON	1224	2.35%	49	0.94%
RUBY	602	1.16%	12	0.23%
PHP	2262	4.35%	459	8.84%
RUBY ON RAILS	257	0.49%	3	0.06%
ROR	34	0.07%	1	0.02%
STRUTS	2502	4.81%	391	7.53%
GRAILS	10	0.02%	0	0.00%
ERLANG	3	0.01%	0	0.00%
GROOVY	13	0.02%	0	0.00%
	52015	100.00%	5190	100.00%

I'm not going to talk about the absolute numbers, but the number of open positions in USA for that key words is ten times the number in Spain. Spain is 45 millions and USA is 300 millions. It means that there is an open position every 8000 Spaniards, and one open position every 5800 Americans.
I think we get the following stuff:

1. Java is about 32% in both countries. Does it mean that a mature technology is deployed everywhere in the same way?
2. Javascript 12%. Same thing.
3. J2EE is 21% in Spain and 16% in USA.
   
4. .NET 18% in Spain and 26% in USA. From points 3) and 4) we can conclude that Spaniards are smart guys ;-)
5. Python is not popular in Spain compared to USA. 0.95% vs 2.35%.
   
6. Spain doubles USA in PHP popularity: 8.84% vs 4.35%
7. Ruby, 1.16% USA vs 0.23% Spain. Ruby On Rails 0.49% USA vs 0.06% Spain. RoR 0.07% USA vs 0.02 Spain. Ruby is 5 to 6 times more popular in the States.
8. Struts popularity almost doubles in Spain: 7.53% vs. 4.81%
9. Grails, Erlang and Groovy are nonexistent in Spain, and almost insignificant in USA.

Here goes my conclusions:

• Mature technologies seem to be deployed all around the globe equally.
• Python is not popular in Spain. Why? Can anybody explain it?
  
• PHP is popular in Spain. I guess PHP popularity is not declining yet because the hype of the new scripting languages has not arrived yet, plus the Python thing.
• Ruby and RoR have not caught Spanish developers yet. I guess the problem is the barrier of the language. It takes some time to translate manuals, tutorials and may be the community of Spaniards developers is not big enough to provide good support. Also, RoR is the language of choice of a lot of start ups companies because its productivity. Spain does not have a lot of 'start up' culture (same thing for Python?).
• Struts is still very popular. But in Spain is more dominant. Again, Struts is a mature technology compared to RoR, and RoR is not relevant in Spain yet.
  
• Grails, Erlang and Groovy are almost irrelevant on both sides of the Atlantic ocean. But it's a good idea to follow the trends.

Finally, the mature technologies are distributed equally in both countries, but emerging technologies are evolving faster in USA. The language and the lack of tradition of start ups companies taking risks using promising technologies makes the difference.

There was a time when DBAs dictate how developers should use Their databases. It was early and mid-nineties and Their Word was The Truth. Those poor guys building client-server applications had to bow down before Him/Her and implement the Business Logic inside The Database Manager. Database hardware was expensive, but His/Her Highness could size the system easily because the number of clients connected was predictable. And that's what the IT Manager wanted; predictable figures.

Late nineties came and Web Application started to rule. Suddenly, the number of clients were unpredictable, the behaviour of the applications was radically different and for the first time in years, the DBMS was the bottleneck and The Master of the Data did not realize on time. So He/She asked to the IT Manager for better and bigger hardware.

IT Manager: Will it fix the performance problems?
DBA: We don't know, it depends on the number of clients on peak, the number of users in the critical path, the TV ads campaigns with the URL of the company...
IT Manager: So the bottleneck is the database?
DBA: Yes, the Web servers are almost idle...
IT Manager: Idle? My god! Can they do anything to relief the database?
DBA: Well, we can try move some business logic out to the application layer. If those lazy and hairy web developers could write good SQL...

And then a new trend started: move out of the database as much business logic as possible. The web developers started to code the SQL inside the application to relief the database.

The DBAs were upset, because these web developers coded really poor SQL and they don't even know about triggers, functions or views that could reduce the size or complexity of some of the statements. And they had to get involved in development almost like a Quality Assurance Team, rejecting poor SQL or aggressive tasks against the RDBMS. That was really disgusting!

By nature Good Developers are lazy. Writing SQL was like a pain in the ass, and wrapping the results in object models took a lot of time. The laziest of them all started to write little applications to create automatically the SQL and the code to return the results as object models. These little tools became in Object-Relational Mapping libraries. The Technical Leads explained to the IT Managers how they could save time using them, and the IT Manager was happy because he could show to the CFO and CEO that they were doing something to make that lazy web developers productive.

Then the nightmare of DBA started.

The DBA realized that the number of queries to the databases were increasing, and the complexity of the query was lower. A lot of simple queries... What's going on here? Sounds like a new intern writing crappy code...

DBA: Hi, is there a new intern in the team writing crappy SQL?
Development Lead: No, we don't have new people. What is going on?
DBA: Then somebody of your team is writing really poor code. I can see tons of queries as simple as a line of SQL!
Development Lead: Ah yes! That's the new ORM library we are using!
DBA: Well, that library is rubbish. It generates tons of shitty SQL. Ask the reseller to give your money back ;-)
Development Lead: No, it's opensource and free. And I see a lot of advantages using it. I think we should discuss it with the IT Manager.

IT Manager: Why this tool writes crappy SQL?
Development Lead: It does not write poor SQL. It creates simple queries, that's all. We can configure it to create more complex SQL, but sometimes the number of objects explodes and the application run out of memory.
DBAs: Why don't you use the already created Views?
Dev Lead: We cannot map Views to objects.
DBAs: So you are not going to use views anymore?
Dev Lead: If we can avoid them, yes.
DBAs: No views!? How am I going to optimize your complex queries?
Dev Lead: Well, we are not going to write complex SQL anymore. The ORM will do.
DBAs: And what about triggers and stored procedures?
Dev Lead: Out. Triggers and ORM does not match very well because it's hard to keep under control the changes performed in the DBMS. And Stored Procedures sucks, we have Java.
IT Manager: So, if there is no views, triggers and stored procedures, the DBAs can set your focus on keeping the system healthy and optimized, but not coding processes. Right?
DBAs: Errr... that's not exactly right...
IT Manager: And we can also transfer all these development tasks to the development team. Right?
DBAs: Correct, but...
IT Manager: And if the DBMS has no heavy processes inside, we can save some bucks in hardware, isn't?
DBAs: Probably yes, but if I don't track what is going on in the database system the system can die!
IT Manager: Of course! And that's what you are suppossed to do from now on. Help the development team to optimize how the ORM tools and libraries performs before going live.

Then the DBA role changed and became a slave of the ORM tools.

She could not recommend any more how to code the SQL code because it was an automatic process. And she saw how there was no gain in the performance because of the use of these ORM tools, even worse, she saw how the tuning of the database now was radically different due to the different characteristics of the SQL code. He thought that may be these new problems in the performance could make IT Manager to roll back to the old way, but it never happened. Actually, the IT Manager asked if it was necessary to have that very expensive Enterprise licenses if triggers, views and stored procedures were declining. Moreover, the application developers were implementing some smart caching strategies that really reduce the overhead in the database, saving millions to the company in hardware and licenses.

IT Manager: Should we try an opensource database?
DBA: Damn Gaving King...

I have just read this post in Engadget about rumours of the Google Phone. I will highlight this: '...the phone will run on a Linux variant (which is nothing new), and will be Java Virtual Machine-capable. Additionally, the OS of the phone will be Java-based (as well as the all phone apps itself), and performance is said to be "very responsive."'
It seems the GPhone will look like a Palm Treo or Blackberry with a screen smaller than the IPhone.
If these rumours are true, Google will become a big supporter of Java for interactive devices something the device market needs desperately to set free of the Windows Mobile and CF.NET slavery.

Jonathan Schwartz says in SD-Asia: "...consumer cell phones on the Java software platform will drive demand for Sun servers...". "The marketplace is going to be defined by the consuming public as much as by the traditional IT decision-maker".

Can anybody explain to me how does Sun think this is happening? How end-consumers can drive the adoption of a transparent server-side technology? End-consumer sees server side services as a commodity, they don't care about the technology running the service if the service is good, reliable and cheap.

How can give me a clue?

Evans data on his semi-annual Web Services Development Survey conducted in June says that companies planning the deployment of SOA with a Java platform has increased slightly, but the companies deploying on .NET have declined by almost 20%. .NET is still ahead with a 31% and Java in second place with 28%. 25% will support both.
It seems the reason is the increase of the activity of the Open Source world, specially Eclipse, around SOA.
Read the press release here.

I have been testing Mingle, the new tool from Thoughtworks to manage Agile developments. The application really looks promising, and have a lot of new features that I'm discovering, but it's taking me more time than I expected because it's slow, very slow.

The guys from Thoughtworks claim they have built the first JRuby commercial application. Probably true, but the price they have paid is high. The application is slow, and it's slow even on a 2.8Ghz 2Gb Server. I have checked the default configuration and heap size is set by default to 1 gig. Not too bad for an application being use by a single user...

I hope they make some changes to improve the overall performance of the application. It's not very 'Agile'.

I have just read the news in Slashdot (Yes, I still read Slashdot) and I found this very interesting article about a new technique to detect GPL violations in proprietary obfuscated code. The technique is called Birthmarking and basically it 'observes short sequences of method calls received by individual objects from the Java Platform Standard API, which is part of the Java Runtime Environment. By aggregating sets of short call sequences the otherwise overwhelming volume of trace data becomes manageable.'

In other words, every piece of java code (and a library is not an exception) moves at bytecode level the information coming and going from the stack following a pattern. So, they identify and classify the patterns of some key objects of a specific GPL piece of code and then they check how the proprietary code uses the stack. If it matches... gotcha!

But I will tell you something... I have seen this concept implemented 15 years ago!!! Not for Java, obviously. Where? At the Faculty of Computer Science of the Polytechnic University of Madrid. There was two terms called Computers Architecture and Computers Fundamentals. We had practices to learn how to code in assembler and microcode. It seems that students copied practices from each other, but students where so smart that they just copied piece of code of other students, or renamed the variables (or labels, it was low level programing). But the core functionality was copied intact. So, the department in charge developed a program called 'El Corrector'. This program checked the input and outputs of each practice automatically (just like today we do with automatic unit testing) and they checked how the program move data to and from the stack. All the practices were compared each other (the patterns were the other students). If they detected similarities in the behaviour of the applications with the stack, they examined the source code and then visually tried to identify a possible copy.
A friend of mine was a bit desperate with the practice, so he asked another friend to have a look to his practice 'to get some inspiration'. He gave him his source code, and he copied parts of it. Both of them were caught by 'El Corrector', and both of them had to repeat the term.

So the ideas behind this paper are not so new, but they are applying it for a real world problem.

Surprise, Surprise! You can read in Jonathan Schwartz's Weblog that they will change the NASDAQ symbol of SUN Microsystems SUNW to JAVA!!!!
The arguments are: '... the number of people who know Java swamps the number of people who know Sun...SUNW represents the past, and its not without a nostalgic nod that we've decided to look ahead...Most know Java, few know Sun...Java means limitless opportunity - for our software, systems, storage, service and microelectronics businesses. And for the open source communities we shepherd.'

The day Steve Jobs said 'Java’s not worth building in. Nobody uses Java anymore. It’s this big heavyweight ball and chain' . Being a Java Zealot as I am, I should find easily a fist full of arguments, but may be he is right.

I have been working in the embedded and mobile world for almost four years in Amplia, and I can tell you that Java is far from being the best option for mobile and embedded development. Even for complex mobile applications I think it's not even an option. There is not a decent implementation of the J2SE for ARM microprocessors, on Linux or Windows CE. There are some implementations out there, but believe me, they are far from the quality standards that Java developers expect. Sun Microsystems realized long time ago that building a J2SE for ARM and embedded OS was hard and expensive and they gave up, abandoning the project.

Of course J2ME it's there. But J2ME was designed for devices with a lot of limitations in term of resources. Devices like mobile phones and embedded hardware. But not for PDAs and the new Smartphones with big screens. Try to build an application with a rich UI with J2ME. It's a nightmare compared to CF.NET for example. J2ME has been very successful for games and simple apps for mobile phones which can gain access to the hardware layer, but nothing else.

But Steve Jobs was not talking about the heavyweight ball and chain of Java because his big memory footprint, slow start up and all that techie stuffs. It's a ball and chain because means a marriage with Sun Microsystems. Steve Jobs would love to have a J2ME virtual machine for the iPhone... if it was for free. But it's not. All devices running J2ME must pay a fee to Sun, no matter if they use it or not. And that's a lot of money. For example, the IBM J9 J2ME implementation for ARM costs $6 per license. Why should Apple pay for it? They have the OS for free -a Symbian license costs between $2.5 and $5-, they have the browser for free -I don't know the license costs of Opera-. So probably Apple can save around $15 per device just using their own dog food. So they said:
1) is it possible to build web applications with HTML and Javascript on Safari? Yes, we just need to allow developers to test easily on Safari. Voila! Safari for Windows!
2) Can we migrate Flash to the iPhone? Yes, and it is easier to port it than a full J2ME stack. Games, Video Streaming and Rich UI apps can be done in Flash too.
3) How many applications today need offline behaviour? Not too much, but less than a few years ago with a rise in the trend. And the device have been designed to be 'always on and connected' to the GSM network and WiFi. The 'offline' behaviour is not so important.

And they decided not to include Java because gave nothing to the business model behind iPhone.

But Mr. Jobs forgot something:
1) Mobile devices (and iPhone too) needs to have access to the hardware to control the low level resources like Bluetooth, GPRS modem, SMS messaging, Wifi connection, Serial ports. A web application cannot access the hardware layer.
2) May be the offline behaviour is not so important, but wireless networks are not as reliable as fixed networks. Offline behaviour from the browser is not mature at all.

So Mr. Jobs and the iPhone needs a piece of software to complete his Masterplan. They need an Embedded Application Server. What? Are you crazy? An application server? In a device? I will explain myself. When we think of an application server, we imagine a J2EE implementation with a plethora of functionality. But this app server is different. Here goes a list of what I think it should implement:
- A small web server to deliver content to the local Safari web browser.
- A scripting language to write server side code. Of course this dynamic content should only be served to the local Safari web browser.
- Massive Multithreading is not a must (it's all local).
- A permanent storage in the device for offline operation.
- Management of the low level resources of the device.
- Some kind of queuing system for asynchronous operations between the device and the remote elements.
- A synchronization service between local storage and remote storage.

This embedded application server should implement a scripting language. Why scripting? Because the logic to implement in a devic