Monday, December 28, 2009

Creating a Zip file from a Stream

Given an InputStream generated for me, i wanted to zip it up and email it. From the java almanac i saw that i could easily do this if i write the stream to disk first. But this seemed like an unnecesary step for me. All i needed to do was to attach this outputstream to my email code and get it done. So, after a bit of playing, I got it working. Here is the code:




   1: private byte[] createZipFile(InputStream in, String fileName) throws IOException {



   2:         ByteArrayOutputStream zipOutputStream = new ByteArrayOutputStream();



   3:         ZipOutputStream zippedFile = new ZipOutputStream(zipOutputStream);



   4:         byte[] buf = new byte[1024];



   5:         zippedFile.putNextEntry(new ZipEntry(myFileName));



   6:         // Transfer bytes from the file to the ZIP file



   7:         int len;



   8:         BufferedInputStream bs = new BufferedInputStream(bds[0].getInputStream());



   9:         while ((len = bs.read(buf)) > 0) {



  10:             zippedFile.write(buf, 0, len);



  11:         }



  12:         zippedFile.closeEntry();



  13:         bs.close();        



  14:         zippedFile.close();



  15:         return zipOutputStream.toByteArray();



  16: } 



By wrapping the ZipOutputStream around a ByteArrayOutputStream, i am able to access the byte array and use that to attach to my email and send out.

Monday, November 30, 2009

Gradle, my new build tool?

Last week, i had the pleasure of attending the JavaEdge conference. The keynote speaker was Ted Neward, and you could read his thoughts about the conference here. I enjoyed his lecture very much whose theme was that new programming languages in the VM is the future and we should all get used to it and get comfortable with it.

One of the sessions that i attended was “Your Next Successful Build”, given by Baruch Sadogursky, who used a slick tool instead of Powerpoint for his presentation, though it was a bit dizzying. I was happy that for the first time, i heard someone say some of my heretical thoughts – that while Maven has a lot of positive, it has many warts, mostly for the reasons of lack of documentation as well as dependency management. As a side note, i  was amazed to see that statistics show that 44% of companies are using Maven, which probably means they are spending countless hours debugging build issues!  It was also refreshing to hear him mention the pros of Ant as well as something he wants to be able to continue to use in his new build environment. But the most exciting part was to hear from Baruch that there is a tool out there that answers the call and it is Gradle. On the one hand, we like the standardization of the file layout of maven, but Gradle adds in better documentation (200 pages!!!), and the ability to use a scripting language, Groovy, which also means you automatically have full access to Ant via Groovy. And of course, he says that dependency management has been fixed as compared to Maven 2. I am hoping to try it out in my next project.

Monday, November 23, 2009

Why did Hibernate do that?!

As i have mentioned in the past, i am not a fan of Hibernate and other ORM tools that generate the SQL for me. This week i have yet another reason for not being willing to consider changing my opinion.

Unbeknownst to me, in our Hibernate application when we were using an “update” to update the one or two columns we change during the regular running of our application, we discovered that actually the update was updating all fields as well as cascading to additional tables joined to this object, which included a CLOB that was getting and update called on it and causing a lot of overhead.

After being warned by our DBA’s to fix ASAP, our next step was to reproduce the problem in our environment so we quickly turned on the “hibernate.show_sql” property to see our SQL. And before we even got to the issue we were looking to solve, we discovered another perplexing Hibernate-ism. We found that when we were doing an initial load or find by id, we were seeing an update happening to the record at the same time as the find/load!

After some fishing around the internet, we understood why this was happening. It was because one of out getters was changing the value from null to something more meaningful, as we see here. When we altered the code to not have this happen, then that update was prevented.

Now onto the problem we were meant to deal with (isn’t always the case that this happens!). After not seeing the results we were hoping for through setting some of the parameters mentioned in various forums (dynamic-update and select-before-update) in an attempt to have our Hibernate “update” not cascade unnecessarily through all the objects and update them all, we settled on a very iBatis like solution. We created the update ourselves so that we know what we are updating, i.e:

String hql = "update myTable set field1 = :field1 where id = :id";
Session session = SessionFactoryUtils.getSession(getSessionFactory(), false);
query = session.createQuery(hql);
query.setString("field1", myObj.getField1());
query.setLong("id",myObj.getId());
query.executeUpdate();

Of course, once you are doing this, you have lost the “advantage” of Hibernate.

Monday, November 16, 2009

Java Web Start – putting my application out there

I recently decided to write a small Java Web Start application using Groovy and Swingbuilder (and HttpBuilder). As i had no experience with Java Web Start, there was a bit of a learning curve. I knew that i would need signed jars in order for my application to be able to write to the local file system. However, a true signing certificate can be very expensive so i realized i would best serve my needs with a self signed certificate that i would keep renewing every 6 months. Using Ant, i have been able to have my app easily deployed with new signed jars.

It is important to mention that there is one simple solution that I did not use for my application, which is to use Griffon, as it takes care of all this stuff behind the scenes for you. However, since I had a simple application and had begun before i knew about it, I decided not to go this route and do it myself, which also ensures i have a better understanding of what is going on. Griffon will still be useful to you for reference. I especially found it useful to look over a sample jnlp file, the file used to list dependencies and configuration information

For signing the jars, i created a self signed certificate. However, every 6 months, i need to renew this certificate, which is pretty straightforward:
keytool -selfcert -keystore keystore -alias myAlias
After re-creating the certificate, i use ANT to sign and deploy. As Ant has a built in task for this, it is trivial:



   1: <target name="signJars" description="sign the jars" depends="makeJar">



   2:         <mkdir dir="signed"/>



   3:         <mkdir dir="signed/lib"/>        



   4:         <signjar destDir="signed"



   5:                  alias="mastertorah" keystore="keystore"



   6:                  storepass="changeit"



   7:                  keypass="mastertorah"



   8:                  >



   9:             <path>



  10:                 <fileset dir="dist" includes="**/*.jar"/>



  11:             </path>            



  12:         </signjar>



  13:         <signjar destDir="signed"



  14:                          alias="mastertorah" keystore="keystore"



  15:                          storepass="changeit"



  16:                          keypass="mastertorah"



  17:                          >



  18:                     <path>



  19:                         <fileset dir="dist" includes="**/*.jar"/>



  20:                     </path>            



  21:         </signjar>



  22:     </target>


In case you are wondering, line 3 above creates a second folder since my signed dependency jars sit separately from the main code; i just left this repeated signing command out of the snippet above.

As you see, line 4 uses the built in ant task signjar and it references my keystore. So, now i have a simple build process to easily redeploy my application.

For users reporting problems after deployment, one thing i have found helpful is to tell them to enable debugging, which is set in the control panel on Windows machines (in the java option there). This way they can send you a log of what went wrong.

I am left with one question. Why should i use SwingBuilder when i can drag and drop widgets using an IDE and get the format of my layout more simply.

Wednesday, October 28, 2009

Setting Version in your manifest using ANT

While the Gant folks will tell you that is not natural to use XML for your makefile as it is nicer to use a real scripting language (and perhaps today’s post will enforce this for you!), it is doable and in my opinion readable. The below is an excerpt of our ant buildfile that we use for upping the manifest version in an interactive way with the the person doing the build.

We added the ant-contrib task in order to enable us to have “if” statements in the XML, so we define the task as follows:

   1: <taskdef resource="net/sf/antcontrib/antlib.xml">

   2:    <classpath>

   3:       <pathelement location="${3rdParty.home}/ant-contrib/ant-contrib-1.0b3.jar" />

   4:    </classpath>

   5: </taskdef>


This will be used below to drive the logic if the user wants to keep the same version of the manifest or change it.

There are two macros that we have defined, loadmanifest – which reads and parses the manifest and createmanifest – which invokes loadmanifest and then interactively decides whether to write a new manifest or not.

Here is loadmanifest:


   1: <macrodef name="loadManifest">

   2:   <attribute name="manifest" />

   3:   <sequential>

   4:     <echo message="about to load file: @{manifest}"/>

   5:         <loadfile property="@{manifest}.build.version" srcFile="@{manifest}">

   6:             <filterchain>

   7:   <filterreader classname="org.apache.tools.ant.filters.LineContains">

   8:                         <param type="contains" value="Implementation-Version:" />

   9:                 </filterreader>

  10:                 <tokenfilter>

  11:   <replacestring from="Implementation-Version: " to="" />

  12:                 </tokenfilter>

  13:                 <striplinebreaks />

  14:           </filterchain>

  15:           </loadfile>

  16:         </sequential>

  17:     </macrodef>


On line 5 we use the loadfile task to read the version number into a property that we will display to the user. We then use the very powerful filterchain to get to the correct line and the correct section of the line with the version number. For lack of a better way, we strip away the text part of the line (Implementation-Version:) on line 11, and are left with just the version number to be stored in the loadfile property @{manifest}.build.version.

Using this property we can now query the user to determine if the version number suits him or not, and then if necessary put out a new manifest file. This is the macro below:



   1: <macrodef name="createManifest">

   2:         <attribute name="manifest" />

   3:         <attribute name="jarDescription" />

   4:         <sequential>

   5:             <loadManifest manifest="@{manifest}" />

   6:             <echo message="Current Version: ${@{manifest}.build.version}" />

   7:             <input message="Enter new version?" validargs="y,n" addproperty="@{jarDescription}.do.newversion" />

   8:             <if>

   9:                 <equals arg1="${@{jarDescription}.do.newversion}" arg2="y" />

  10:                 <then>

  11:                     <input message="Please enter new version number" addproperty="@{jarDescription}.new.build.version" />

  12:                     <manifest file="@{manifest}">

  13:                         <section name="${@{jarDescription}.jar.description}">

  14:                             <attribute name="Implementation-Title" value="${@{jarDescription}.jar.description}" />

  15:                             <attribute name="Implementation-Version" value="${@{jarDescription}.new.build.version}" />

  16:                         </section>

  17:                     </manifest>

  18:                 </then>

  19:                 <else>

  20:                     <property name="@{jarDescription}.new.build.version" value="${@{manifest}.build.version}" />

  21:                 </else>

  22:             </if>

  23:         </sequential>

  24:     </macrodef>


Other than “if”, the rest are all standard ant tasks that you can look up in the manual on the ant site.

The correct thing to do at this point would be to create a parallel GANT site. Unfortunately, i dont have one ready at the moment so this will have to wait until i begin to play with GANT

Monday, October 19, 2009

SVN Build automation with ant

As i indicated in my last post, we use Ant to create our new scripts. In response to some requests, i am sharing here our “commonbuild.xml” file that shows how we do the key parts of this build in a reusable way. Before you get started you will need to make sure you have late version of ant (1.7 please) and you will need to add some libraries to your ant lib folder. These are the subclipse libraries as well as commons-net if you want to send emails at the end of your build.

Here is is a snapshot of all new libraries added:

antlibs

The first 4 are needed for mail functions and the last 4 are added from svn. I am using svn 1.5 due to the fact that i have not yet updated my Intellij past version 7 so only 1.5 is supported.

In addition you need to install collabnet subversion client (windows) for this to work.

For Svn Purposes, we have set up the following task:

  1: <taskdef resource="org/tigris/subversion/svnant/svnantlib.xml">
  2:         <classpath>
  3:             <fileset dir="${3rdParty.home}/svnant-1.2.0-RC1/lib" includes="**/*.jar" />
  4:         </classpath>
  5: </taskdef>

Using it is pretty straightforward. Below is a macro that we use for commit and tag:

  1: <macrodef name="commitAndTag">
  2:         <attribute name="commitMessage" />
  3:         <attribute name="tagName" />
  4:         <sequential>
  5:             <echo message="checking in dist folder and manifest versions" />
  6:             <svn>
  7:                 <commit message="@{commitMessage}">
  8:                     <fileset dir="${trunk.dir}">
  9:                         <include name="dist/**" />
 10:                         <include name="**/build/**.txt" />
 11:                     </fileset>
 12:                 </commit>
 13:             </svn>
 14:             <svn>
 15:                 <copy srcUrl="${svnUrl}/${svn.projectName}/trunk" destUrl="${svnUrl}/${svn.projectName}tags/@{tagName}" message="@{tagName} tag" />
 16:             </svn>
 17:         </sequential>
 18:     </macrodef>

And to invoke this macro is simple:

  <commitAndTag commitMessage="${release.version}" tagName="${release.version}" />

In the next post I hope to show the macro used for editing the manifest file and upping the version number.

Thursday, October 15, 2009

Time to become a Maven?

I was afraid to post about the fact that we have not moved over all my builds to maven, but i saw that the iBatis team only recently moved. And anyway, last year at a Java conference, the presenter about Maven talked about how only recently had the Maven folks gotten their act together to the point where he was not pulling out his hair over new jars that appeared and changed dependencies happening without his knowing. (I really hate things that happen without my knowing!)

I will start by saying that all our current projects are happily using Ant. We have built some pretty nice scripts to create a full build that include using svn (subclipse) to get the latest version of our code, up the version in the manifest, commit, create a tag, and copy the file to the distribution location. Additionally, the start of a project by our small teams have usually been through a project started in an IDE, (Intellij), and after we have something running, we then take our project and create an ant build, using our existing scripts. So, using maven does not easily fit this model since at this point we already have an existing structure. But everybody is doing it nowadays, so we cant easily dismiss it.

So, we have done a few projects starting with Maven. And of course, it meant we had to break our methodology described above. We used Maven to create the original infrastructure for the project as well as the project file for Intellij. And, as the project would progress and added more dependencies, it seems that the simplest way to move forward was not to add the dependencies to the project in Intellij, but to fix the POM and then have maven regenerate the project file. I believe this is fixed in the latest version of Intellij where it knows how to sync with the POM.

So, what are my conclusions?

First of all, the documentation needs improvement! But, notwithstanding that, I think it makes sense to use Maven to create new projects and this will mean that you will have a simple structure where it is obvious where to put all your additional resources, config files. The alternative would be that we need to keep worrying about these “copy” lines in my ant file.

But the power of ant to do what you want, and use legacy processes (like our use of JWSDP 1.6) would mean that we still want ant in our build process. Luckily this is possible, as I saw here.

Of course, Ant will remain a key tool for us as i use it to run apps and more easily build classpaths and pass params to it than alternatives that I know of. And as a big believer in “If it ain’t broke don’t fix it”, i dont see us moving our old applications to Maven.

Wednesday, September 9, 2009

Test Driving IBATIS 3 (and Spring3)

As a big fan and heavy user of iBATIS SqlMaps, i have wanted to give iBATIS version 3 a look for some time, but due to lack of time and limited documentation, I had not gotten around to it. But the beta version is now here, and with it a very detailed and well written document, and I finally had some time between projects, so I figured i could delay no longer. I immediately headed to check the latest version of Spring (3.0M4 as of this post) but i see that there is no built in support for it yet, though there is some JIRA activity on the iBATIS and Spring side trying to make this happen.

The goal of my first encounter with iBATIS 3 was to get a small project up and running, where the iBATIS session would be provided to me by the Spring container. This would give me the datasource and transactions power of Spring even though i would not have the full DataAccessException hierarchy and Runtime Exception changes.

As the iBATIS folks recommended, i am using maven to create the project (a tool i have a lot of reservations about), and am followiing their folder structure. In order to do this with Spring 3 M4, i needed to add the following repository to my pom.xml

   1: <repository>
   2:     <id>spring-releases</id>
   3:     <name>Spring Maven RELEASE Repository</name>
   4:     <url>http://s3.amazonaws.com/maven.springframework.org/release</url>
   5: </repository>

The end result is that my project looks like this:

As recommended in the documentation, I added a new folder within my package structure called “data” where all my configuration files will be stored. These files, are my Spring Application Context (applicationContext.xml), the SqlMapConfig file for Ibatis configuration (minus datasource information), and the Mapper XML file. There is one new file there, an interface with the same name as the SqlMap file. This is a new feature of iBATIS3 that enables type safety. Each mapping will have a function defined in this interface. I may also choose to implement this interface for my dao functions, so the interface could be useful.

Let’s start with the application Context. It seems to me that Spring has come a long way to solve the length of our application context, by enabling the use of annotation in our java code, as well as the shorthand notation in our XML file. So, here is my very simple application Context that contains 2 beans, the datasource and a call to a static method used by iBATIS to read the SqlMapConfig.xml. This file still contains the mapping files as well as global settings, but obviously no datasource settings. Of course, since i will be using JavaConfig, i need to configure the annotation configuration as well. So, here it is

   1: <?xml version="1.0" encoding="UTF-8"?>
   2: <beans xmlns="http://www.springframework.org/schema/beans"
   3:        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   4:        xmlns:context="http://www.springframework.org/schema/context"
   5:        xmlns:aop="http://www.springframework.org/schema/aop"
   6:         xmlns:p="http://www.springframework.org/schema/p"
   7:        xsi:schemaLocation="
   8:     http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
   9:     http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd">
  10:  
  11:  
  12:     <context:property-placeholder location="classpath:properties/config.properties"/>
  13:     <context:annotation-config/>
  14:     <context:component-scan base-package="com.dr.ibatis3app"/>
  15:  
  16:  
  17:     <bean id="dataSource"
  18:           class="org.springframework.jdbc.datasource.DriverManagerDataSource"
  19:                 p:driverClassName="${jdbc.driver}"
  20:                 p:url="${jdbc.url}"
  21:                 p:username="${jdbc.username}"
  22:                 p:password="${jdbc.password}"/>
  23:  
  24:  
  25:     <bean id="ibatisResourceReader" class="org.apache.ibatis.io.Resources"
  26:           factory-method="getResourceAsReader">
  27:             <constructor-arg value="com/dr/ibatis3app/data/SqlMapConfig.xml"/>
  28:     </bean>
  29:  
  30: </beans>

The rest of the beans needed for my project are defined below in the JavaConfig file. The reason is obvious when you see the code:

   1: @Configuration
   2: public class AppConfig {
   3:     @Resource
   4:     DataSource dataSource;
   5:  
   6:     @Resource(name="ibatisResourceReader")
   7:     Reader sqlMapConfigReader;
   8:     
   9:     @Bean
  10:     public SqlSessionFactory sqlSessionFactoryBuilder() throws IOException {
  11:         System.out.println("Calling SqlSessionFactoryBuilder build method");
  12:         SqlSessionFactoryBuilder builder = new SqlSessionFactoryBuilder();
  13:         return builder.build(sqlMapConfigReader);
  14:     }
  15:  
  16:     @Bean
  17:     @Scope("prototype")
  18:     public SqlSession sqlSession() throws DataAccessException {
  19:         try {
  20:             return sqlSessionFactoryBuilder().openSession(dataSource.getConnection());
  21:         } catch (SQLException e) {
  22:             throw new DataSourceLookupFailureException("Unable to open session");
  23:         } catch (IOException e) {
  24:             throw new DataSourceLookupFailureException("Error opening sqlconfig");
  25:         }
  26:     }
  27: }


Rather than using messy method calls in the XML file, i can have a few simple lines of Java that handle the instantiation of the SqlSession, yet the resource file name is injected from the XML file so this is easily changed. The SqlSession is defined as scope prototype as it is not thread safe.

The mapper file is straightforward, with the main change being the required namespace to match the exact location of the interface and XML files.

   1: package com.dr.ibatis3app.data;
   2:  
   3: public interface CardMapper {
   4:     public String getPrimAcctNum(int acctId);
   5: }
   6:  
   7: //XML file is below
   8: <?xml version="1.0" encoding="UTF-8" ?>
   9: <!DOCTYPE mapper
  10:         PUBLIC "-//ibatis.apache.org//DTD Mapper 3.0//EN"
  11:         "http://ibatis.apache.org/dtd/ibatis-3-mapper.dtd">
  12: <mapper namespace="com.dr.ibatis3app.data.CardMapper">
  13:     <select id="getPrimAcctNum" parameterType="int" resultType="String">
  14:         SELECT prim_acct_num FROM acct WHERE acct_id = #{id}
  15:     </select>
  16: </mapper>


And now we use the code:

   1: public static void main(String[] args) {
   2:     
   3:     ClassPathXmlApplicationContext ctx = new ClassPathXmlApplicationContext("com/dr/ibatis3app/data/applicationContext.xml");
   4:     SqlSession session = ctx.getBean("sqlSession", SqlSession.class);
   5:     try {
   6:         CardMapper mapper = session.getMapper(CardMapper.class);
   7:         String  cardNum = mapper.getPrimAcctNum(4185576);
   8:         System.out.println("CARDNUM IS " + cardNum);
   9:     } finally {
  10:         session.close();
  11:     }
  12: }

And that’s it. We have a working application, and can start to kick the tires a bit.