Wednesday, October 28, 2009

Setting Version in your manifest using ANT

While the Gant folks will tell you that is not natural to use XML for your makefile as it is nicer to use a real scripting language (and perhaps today’s post will enforce this for you!), it is doable and in my opinion readable. The below is an excerpt of our ant buildfile that we use for upping the manifest version in an interactive way with the the person doing the build.

We added the ant-contrib task in order to enable us to have “if” statements in the XML, so we define the task as follows:

   1: <taskdef resource="net/sf/antcontrib/antlib.xml">

   2:    <classpath>

   3:       <pathelement location="${3rdParty.home}/ant-contrib/ant-contrib-1.0b3.jar" />

   4:    </classpath>

   5: </taskdef>


This will be used below to drive the logic if the user wants to keep the same version of the manifest or change it.

There are two macros that we have defined, loadmanifest – which reads and parses the manifest and createmanifest – which invokes loadmanifest and then interactively decides whether to write a new manifest or not.

Here is loadmanifest:


   1: <macrodef name="loadManifest">

   2:   <attribute name="manifest" />

   3:   <sequential>

   4:     <echo message="about to load file: @{manifest}"/>

   5:         <loadfile property="@{manifest}.build.version" srcFile="@{manifest}">

   6:             <filterchain>

   7:   <filterreader classname="org.apache.tools.ant.filters.LineContains">

   8:                         <param type="contains" value="Implementation-Version:" />

   9:                 </filterreader>

  10:                 <tokenfilter>

  11:   <replacestring from="Implementation-Version: " to="" />

  12:                 </tokenfilter>

  13:                 <striplinebreaks />

  14:           </filterchain>

  15:           </loadfile>

  16:         </sequential>

  17:     </macrodef>


On line 5 we use the loadfile task to read the version number into a property that we will display to the user. We then use the very powerful filterchain to get to the correct line and the correct section of the line with the version number. For lack of a better way, we strip away the text part of the line (Implementation-Version:) on line 11, and are left with just the version number to be stored in the loadfile property @{manifest}.build.version.

Using this property we can now query the user to determine if the version number suits him or not, and then if necessary put out a new manifest file. This is the macro below:



   1: <macrodef name="createManifest">

   2:         <attribute name="manifest" />

   3:         <attribute name="jarDescription" />

   4:         <sequential>

   5:             <loadManifest manifest="@{manifest}" />

   6:             <echo message="Current Version: ${@{manifest}.build.version}" />

   7:             <input message="Enter new version?" validargs="y,n" addproperty="@{jarDescription}.do.newversion" />

   8:             <if>

   9:                 <equals arg1="${@{jarDescription}.do.newversion}" arg2="y" />

  10:                 <then>

  11:                     <input message="Please enter new version number" addproperty="@{jarDescription}.new.build.version" />

  12:                     <manifest file="@{manifest}">

  13:                         <section name="${@{jarDescription}.jar.description}">

  14:                             <attribute name="Implementation-Title" value="${@{jarDescription}.jar.description}" />

  15:                             <attribute name="Implementation-Version" value="${@{jarDescription}.new.build.version}" />

  16:                         </section>

  17:                     </manifest>

  18:                 </then>

  19:                 <else>

  20:                     <property name="@{jarDescription}.new.build.version" value="${@{manifest}.build.version}" />

  21:                 </else>

  22:             </if>

  23:         </sequential>

  24:     </macrodef>


Other than “if”, the rest are all standard ant tasks that you can look up in the manual on the ant site.

The correct thing to do at this point would be to create a parallel GANT site. Unfortunately, i dont have one ready at the moment so this will have to wait until i begin to play with GANT

Monday, October 19, 2009

SVN Build automation with ant

As i indicated in my last post, we use Ant to create our new scripts. In response to some requests, i am sharing here our “commonbuild.xml” file that shows how we do the key parts of this build in a reusable way. Before you get started you will need to make sure you have late version of ant (1.7 please) and you will need to add some libraries to your ant lib folder. These are the subclipse libraries as well as commons-net if you want to send emails at the end of your build.

Here is is a snapshot of all new libraries added:

antlibs

The first 4 are needed for mail functions and the last 4 are added from svn. I am using svn 1.5 due to the fact that i have not yet updated my Intellij past version 7 so only 1.5 is supported.

In addition you need to install collabnet subversion client (windows) for this to work.

For Svn Purposes, we have set up the following task:

  1: <taskdef resource="org/tigris/subversion/svnant/svnantlib.xml">
  2:         <classpath>
  3:             <fileset dir="${3rdParty.home}/svnant-1.2.0-RC1/lib" includes="**/*.jar" />
  4:         </classpath>
  5: </taskdef>

Using it is pretty straightforward. Below is a macro that we use for commit and tag:

  1: <macrodef name="commitAndTag">
  2:         <attribute name="commitMessage" />
  3:         <attribute name="tagName" />
  4:         <sequential>
  5:             <echo message="checking in dist folder and manifest versions" />
  6:             <svn>
  7:                 <commit message="@{commitMessage}">
  8:                     <fileset dir="${trunk.dir}">
  9:                         <include name="dist/**" />
 10:                         <include name="**/build/**.txt" />
 11:                     </fileset>
 12:                 </commit>
 13:             </svn>
 14:             <svn>
 15:                 <copy srcUrl="${svnUrl}/${svn.projectName}/trunk" destUrl="${svnUrl}/${svn.projectName}tags/@{tagName}" message="@{tagName} tag" />
 16:             </svn>
 17:         </sequential>
 18:     </macrodef>

And to invoke this macro is simple:

  <commitAndTag commitMessage="${release.version}" tagName="${release.version}" />

In the next post I hope to show the macro used for editing the manifest file and upping the version number.

Thursday, October 15, 2009

Time to become a Maven?

I was afraid to post about the fact that we have not moved over all my builds to maven, but i saw that the iBatis team only recently moved. And anyway, last year at a Java conference, the presenter about Maven talked about how only recently had the Maven folks gotten their act together to the point where he was not pulling out his hair over new jars that appeared and changed dependencies happening without his knowing. (I really hate things that happen without my knowing!)

I will start by saying that all our current projects are happily using Ant. We have built some pretty nice scripts to create a full build that include using svn (subclipse) to get the latest version of our code, up the version in the manifest, commit, create a tag, and copy the file to the distribution location. Additionally, the start of a project by our small teams have usually been through a project started in an IDE, (Intellij), and after we have something running, we then take our project and create an ant build, using our existing scripts. So, using maven does not easily fit this model since at this point we already have an existing structure. But everybody is doing it nowadays, so we cant easily dismiss it.

So, we have done a few projects starting with Maven. And of course, it meant we had to break our methodology described above. We used Maven to create the original infrastructure for the project as well as the project file for Intellij. And, as the project would progress and added more dependencies, it seems that the simplest way to move forward was not to add the dependencies to the project in Intellij, but to fix the POM and then have maven regenerate the project file. I believe this is fixed in the latest version of Intellij where it knows how to sync with the POM.

So, what are my conclusions?

First of all, the documentation needs improvement! But, notwithstanding that, I think it makes sense to use Maven to create new projects and this will mean that you will have a simple structure where it is obvious where to put all your additional resources, config files. The alternative would be that we need to keep worrying about these “copy” lines in my ant file.

But the power of ant to do what you want, and use legacy processes (like our use of JWSDP 1.6) would mean that we still want ant in our build process. Luckily this is possible, as I saw here.

Of course, Ant will remain a key tool for us as i use it to run apps and more easily build classpaths and pass params to it than alternatives that I know of. And as a big believer in “If it ain’t broke don’t fix it”, i dont see us moving our old applications to Maven.

Wednesday, September 9, 2009

Test Driving IBATIS 3 (and Spring3)

As a big fan and heavy user of iBATIS SqlMaps, i have wanted to give iBATIS version 3 a look for some time, but due to lack of time and limited documentation, I had not gotten around to it. But the beta version is now here, and with it a very detailed and well written document, and I finally had some time between projects, so I figured i could delay no longer. I immediately headed to check the latest version of Spring (3.0M4 as of this post) but i see that there is no built in support for it yet, though there is some JIRA activity on the iBATIS and Spring side trying to make this happen.

The goal of my first encounter with iBATIS 3 was to get a small project up and running, where the iBATIS session would be provided to me by the Spring container. This would give me the datasource and transactions power of Spring even though i would not have the full DataAccessException hierarchy and Runtime Exception changes.

As the iBATIS folks recommended, i am using maven to create the project (a tool i have a lot of reservations about), and am followiing their folder structure. In order to do this with Spring 3 M4, i needed to add the following repository to my pom.xml

   1: <repository>
   2:     <id>spring-releases</id>
   3:     <name>Spring Maven RELEASE Repository</name>
   4:     <url>http://s3.amazonaws.com/maven.springframework.org/release</url>
   5: </repository>

The end result is that my project looks like this:

proj

As recommended in the documentation, I added a new folder within my package structure called “data” where all my configuration files will be stored. These files, are my Spring Application Context (applicationContext.xml), the SqlMapConfig file for Ibatis configuration (minus datasource information), and the Mapper XML file. There is one new file there, an interface with the same name as the SqlMap file. This is a new feature of iBATIS3 that enables type safety. Each mapping will have a function defined in this interface. I may also choose to implement this interface for my dao functions, so the interface could be useful.

Let’s start with the application Context. It seems to me that Spring has come a long way to solve the length of our application context, by enabling the use of annotation in our java code, as well as the shorthand notation in our XML file. So, here is my very simple application Context that contains 2 beans, the datasource and a call to a static method used by iBATIS to read the SqlMapConfig.xml. This file still contains the mapping files as well as global settings, but obviously no datasource settings. Of course, since i will be using JavaConfig, i need to configure the annotation configuration as well. So, here it is

   1: <?xml version="1.0" encoding="UTF-8"?>
   2: <beans xmlns="http://www.springframework.org/schema/beans"
   3:        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   4:        xmlns:context="http://www.springframework.org/schema/context"
   5:        xmlns:aop="http://www.springframework.org/schema/aop"
   6:         xmlns:p="http://www.springframework.org/schema/p"
   7:        xsi:schemaLocation="
   8:     http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
   9:     http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd">
  10:  
  11:  
  12:     <context:property-placeholder location="classpath:properties/config.properties"/>
  13:     <context:annotation-config/>
  14:     <context:component-scan base-package="com.dr.ibatis3app"/>
  15:  
  16:  
  17:     <bean id="dataSource"
  18:           class="org.springframework.jdbc.datasource.DriverManagerDataSource"
  19:                 p:driverClassName="${jdbc.driver}"
  20:                 p:url="${jdbc.url}"
  21:                 p:username="${jdbc.username}"
  22:                 p:password="${jdbc.password}"/>
  23:  
  24:  
  25:     <bean id="ibatisResourceReader" class="org.apache.ibatis.io.Resources"
  26:           factory-method="getResourceAsReader">
  27:             <constructor-arg value="com/dr/ibatis3app/data/SqlMapConfig.xml"/>
  28:     </bean>
  29:  
  30: </beans>

The rest of the beans needed for my project are defined below in the JavaConfig file. The reason is obvious when you see the code:

   1: @Configuration
   2: public class AppConfig {
   3:     @Resource
   4:     DataSource dataSource;
   5:  
   6:     @Resource(name="ibatisResourceReader")
   7:     Reader sqlMapConfigReader;
   8:     
   9:     @Bean
  10:     public SqlSessionFactory sqlSessionFactoryBuilder() throws IOException {
  11:         System.out.println("Calling SqlSessionFactoryBuilder build method");
  12:         SqlSessionFactoryBuilder builder = new SqlSessionFactoryBuilder();
  13:         return builder.build(sqlMapConfigReader);
  14:     }
  15:  
  16:     @Bean
  17:     @Scope("prototype")
  18:     public SqlSession sqlSession() throws DataAccessException {
  19:         try {
  20:             return sqlSessionFactoryBuilder().openSession(dataSource.getConnection());
  21:         } catch (SQLException e) {
  22:             throw new DataSourceLookupFailureException("Unable to open session");
  23:         } catch (IOException e) {
  24:             throw new DataSourceLookupFailureException("Error opening sqlconfig");
  25:         }
  26:     }
  27: }


Rather than using messy method calls in the XML file, i can have a few simple lines of Java that handle the instantiation of the SqlSession, yet the resource file name is injected from the XML file so this is easily changed. The SqlSession is defined as scope prototype as it is not thread safe.

The mapper file is straightforward, with the main change being the required namespace to match the exact location of the interface and XML files.

   1: package com.dr.ibatis3app.data;
   2:  
   3: public interface CardMapper {
   4:     public String getPrimAcctNum(int acctId);
   5: }
   6:  
   7: //XML file is below
   8: <?xml version="1.0" encoding="UTF-8" ?>
   9: <!DOCTYPE mapper
  10:         PUBLIC "-//ibatis.apache.org//DTD Mapper 3.0//EN"
  11:         "http://ibatis.apache.org/dtd/ibatis-3-mapper.dtd">
  12: <mapper namespace="com.dr.ibatis3app.data.CardMapper">
  13:     <select id="getPrimAcctNum" parameterType="int" resultType="String">
  14:         SELECT prim_acct_num FROM acct WHERE acct_id = #{id}
  15:     </select>
  16: </mapper>


And now we use the code:

   1: public static void main(String[] args) {
   2:     
   3:     ClassPathXmlApplicationContext ctx = new ClassPathXmlApplicationContext("com/dr/ibatis3app/data/applicationContext.xml");
   4:     SqlSession session = ctx.getBean("sqlSession", SqlSession.class);
   5:     try {
   6:         CardMapper mapper = session.getMapper(CardMapper.class);
   7:         String  cardNum = mapper.getPrimAcctNum(4185576);
   8:         System.out.println("CARDNUM IS " + cardNum);
   9:     } finally {
  10:         session.close();
  11:     }
  12: }

And that’s it. We have a working application, and can start to kick the tires a bit.

Wednesday, August 19, 2009

Grails and the Domain Model, or why I prefer IBatis

I have spent many hours playing with Grails, to see how much effort it would be to take one of our existing apps and port it to Grails, since, as i have written previously, i prefer not to have to deal too much with GUI, so if Grails gives me a lot automatically, then I would be very happy using it.

At the same time, i have been using Grails to write a customized version of Quicken for my own personal budgetary needs (unfortunately, it has not progressed as well as i would want due to lack of time).

I will start with the second project first. As advertised, Grails is an absolutely amazing framework to take an idea and make it functional really fast. Just set up your domain classes and you automatically get a whole web site up and running, from which you can customize the columns and edits. And in the case of my personal app, the domain model is a few tables without any complex joins.

However, when trying to set up a Grails application on an existing database, this creates a lot of problems. I will start by saying that I am not a big fan of Hibernate either for a similar reason. In most reasonably sized companies, you have a pretty complex database (or set of databases) and in my opinion, it is easier to write queries than to try to “teach” the entire schema to your application. There are solutions like creating views, but then inserts will still be a problem. With Hibernate though, i found it possible with some effort to get around most of the issues. But in Grails, if i want the advantages of the framework, i cannot veer too far from the default domain model. After a lot of frustration trying to make the domain model work with our database, i found the best way to work was to keep tweaking it using the HSQLDB and seeing the scripts generated until they matched. In the end, i opened two  issues that prevented us from moving forward, and both have been addressed, 3616, and 4158. So, I guess now i am ready to resume.

But in the end, i still believe that the model IBatis provides is more appropriate for a company application. It means that the developers wears two hats – one of the SQL developer, writing and tweaking queries until they are just what you want and then putting on the programmer hat to use those queries that are mapped within an XML file. I see that IBatis version 3 is out, though every time i have checked i did not see too much documentation about what is different. If you are new to IBatis, I can tell that in version 2, the documentation is great, and it definitely is easy to get up and running with it, especially if you use Spring together with IBatis. And there are many nice advanced features that can take you quite far.

Technorati Tags: ,

Tuesday, June 30, 2009

Clean Code – Programmers are Authors!

I recently read the book Clean Code, by Robert C, Martin, but I waited to comment on it until i saw how it fundamentally changed how I write code. I see there are quite a few blogs discussing it but i wanted to give my take on it, and specifically, in the short term, as I hope that other aspects will affect me more in the long run as well. (like improving my unit testing!)

The first powerful principle, laid down at the beginning is that “We are authors” and he develops this point two key ways that I thought i would share:

1. We read a lot of code.

Yes, its true. Whether like me, you are now reading others’ code as you have inherited their code, or you need to re-read your own code that you wrote 6 months ago and cannot remember, we are always reading our code. When we write complex code, long functions, etc, even our code takes way too long to remember what it does

2. Our code is our only reliable way to know what the code does. Comments are unreliable. Yes, its true – i see it in our application. I see that someone from our team changed code and left the now irrelevant comment in place. So, what does that mean? That the only effective and reliable way to document code is to write it in a way that it is readable. (yes, there are needs for comments on a class, but on specific lines of code, no)

So, how do we do that? Well, go ahead and read the book! But i will start and say – keep the functions short, keep the names of the functions descriptive so that the name of the function itself makes it clear what you will do in the function (and of course, don’t do too much in the function)

Tuesday, June 16, 2009

Irregular Expressions

A friend sent me the following code that did not work as he expected

   1: public static void main(String[] args) {    
   2:      
   3:     String str = " status=\"The word deleted is in this sentence and no carriage return";    
   4:     String str1 = " status=\"The word deleted is in this sentence and carriage return\n";    
   5:      
   6:     assert str.matches(".*deleted.*");    
   7:     assert str1.matches(".*deleted.*") : "carriage return threw off the regex";    
   8: }
If you run this code you will see that carriage return is not considered “.*” by default. Of course, a little looking around helped us realize that this was documented,

The regular expression . matches any character except a line terminator unless the DOTALL flag is specified

In other words, we need to use the ugly Pattern class in order to make this work, as follows:

   1: public static void main(String[] args) {    
   2:      
   3:         String str = "The word deleted is in this sentence and no carriage return";    
   4:         String str1 = "The word deleted is in this sentence and carriage return\n";    
   5:      
   6:         Pattern p = Pattern.compile(".*deleted.*", Pattern.DOTALL);    
   7:         Matcher m = p.matcher(str);    
   8:         assert m.matches();    
   9:         m = p.matcher(str1);    
  10:         assert m.matches();    
  11:      
  12:     }

The key to making this work is the parameter Pattern.DOTALL used in the compile.

And how about in Groovy? It works of course!

   1: String str = " status=\"The word deleted is in this sentence and no carriage return";    
   2: String str1 = " status=\"The word deleted is in this sentence and carriage return\n";    
   3: assert str =~ /.*deleted.*/    
   4: assert str1 =~ /.*deleted.*/

As written about extensively, one of the really well done features of Groovy is definitely the regular expression syntax but here we see that the implementation was also improved.