Tuesday, August 17, 2010

Tips For Maintainable Code

Due to a need of late to maintain a lot of code (where of course not all of it has been written by me), I have been able to give a lot of thought to which tips I have read about in the past that strike me as the most important to have maintainable code. In many scenarios, companies simply rewrite their applications every few years and thus this issue may not be relevant but in my case where the lack of manpower makes this impossible, (and the application works so well there is no business justification for it), it has led me to think carefully how to set up a project and practices to have in place so that an application is maintainable and customizable in a reasonable way. Much of what i will be listing is based on the book Clean Code which I already blogged about previously here.

1. Keep functions short and simple – and if you need to add in more functionality, refactor it into a new function that clearly defines in its name what it does. In the book Clean Code, the insistence is that all functions do exactly one thing.

2. Building on number 1, use small functions as a way to comment “funny” things you need to do. Sometimes, we have small things we needed to do – set some parameter to some funny value or the like and the person who makes this change may or may not comment this, since at the time it was so obvious and it may not even make the comment for the check in, since many files were checked in simultaneously, but if you get in the habit of creating a small function, such “nullOutParameterSoThatRequestHandlesCorrectly”, this comment is forever part of the code and will not get lost. It also means that when someone comes along later and sees it he can be sure it is there on purpose and not something wonder – it seems to me that removing this line is ok but i am not sure…

3. Comprehensive Unit Tests. This is a great way to document your code since we can see the kind of input you expect to receive for a given function and how you expect to handle it. As has been written about extensively on the ‘net, when we see these unit tests, we have greater confidence to make changes since we have a baseline to evaluate our changes. One other advantage of the unit tests, is that if you added a new class with dependencies, and you are using Dependency Injection, like Spring, even if the code never made it to production we have a working example to build off for later use. This actually happened to me, where code was written and not released, but lacking an example of how to set up this class in my Spring application context led to my lack of confidence in using this class.

4. Regression Tests. While Unit Tests are imperative, So are regression Tests. These are automated end to end tests so that we have a baseline of how the application works, in dailsychaining multiple functions together, what data was passed from beginning to end and what was received. I have used TestNG to make up regression tests due to the amazing features of paramterized testing, enabling to set up use cases that run multiple times with different inputs. (I hope to write more about this in an upcoming blog)

5. use the Source Control and DONT save pruned code. Remove it. Don’t worry about it since you can get it from the source control. It just creates confusion.

Remember, that YOU may be the one left to maintain the code so its in your best interest to get it right when developing it…

Monday, June 7, 2010

Testable Code Vs. OO code

Can we ever have a situation where the desire to have testable code is not 100% compatible with our Object Oriented Design? As i was working on some new code, i came across this issue and was happy to find that this is not a new issue as we see here. Even if you are not willing to accept the entire thesis, at the very least, we may want to use the thought of testing our code to drive some of our design decisions, since a robust, easy to run test suite is so important in software development, especially as projects grow in size and length and we can hardly remember what our code does anymore. An example of this in my project was my decision to implement the template method pattern where the template methods implemented by the child classes provide data back the parent class rather than invoke helper methods and provide greater functionality. This means that it is easier to test each new implementation and is less time consuming.

An additional aspect of this issue can be seen based on the book Clean Code, discussed in a previous post. While the focus there was on readability of the code, testability seems to overlap quite a bit with readability. The simpler we make our classes and our functions, the simpler for testing and finding the problems when test fail. Another aspect discussed at length there is the issue of parameters to function (or better yet, not having them). It is recommended instead to have members the contain the necessary data rather than pass it from function to function. (Beware, as this means your classes will not be thread-safe and will require a new instance for each use). From what i recall (not having access to this book right now), the main issue was readability, but here too, another fringe benefit is testability, especially if you are using TestNG (or JUnit 4). The reason for this, you create this class and populate the member data correctly and then can run tests repeatedly with little typing other than the method names. Writing and running the tests is now easier since you need not figure out all the parameters of a given function. Also, running multiple tests with different data (using TestNG DataProvider) is easier to provide.

There are two other ways that testability impacts design in my current project:
The first is in using interfaces even when i have no plans to implement another implementation of a member class. It is much easier to mock behavior and isolate a test if we use interfaces and then plug in a simpler implementation of our interface for test purposes. Of course, use of Spring here and the Spring application context makes this trivial

The second aspect is in determining the scope of functions in order to enable testing of private functions as discussed here. I agree wholeheartedly with Bob Lee in his post that your code will be better if private functions are tested individually and you do not leave all your testing to the public API.

I realize that much of what i write here is well known but i see that many of my peers are still not recognizing how important repeatable tests are. It is hard to see down the line when you are creating a new project from scratch. But, there are two ultra important reasons that we need to have a repeatable test suite. The first is documentation. As written about in Clean Code, Javadocs are not reliable. It happens too often in projects that programmers do not make sure to update javadocs as things change, and this is true for comments as well. The best documentation in addition to well written code with good naming (as discussed in my previous post about the book Clean Code), is the tests of the API, that show exactly how a function was meant to be used by the original author. It enables us to also what results it was meant to return and all error cases. The more tests on a function the more information provided. The second reason is maintainability. As our software has gotten larger/bloated, while staff has been pared down, a rewrite of our software is not truly possible. The confidence to make changes stems from good, repeatable tests.

A second part of the confidence to make changes in a larger project comes from a good regression suite, which i hope to discuss in an upcoming post.


Wednesday, April 14, 2010

Using a non default Calendar in the Google API

All the examples in the Google Calendar API documentation involve using a default calendar. But i was looking at a project where i would have a shared calendar among users and this calendar would not be the default one. The simple code below demonstrates how to do this in Java. There are three parameter you will need to inject into this class (in my case via Spring) -- the username, password, and calendarname. Using these three you have what you need to work with the non default calendar.


   1: public class CalendarUtil {



   2:   private final static Logger logger = Logger.getLogger(CalendarUtil.class.getName());



   3:   private String username;



   4:   private String password;



   5:   String calendarName;



   6:   private URL calendarUrl;



   7:  



   8:   private CalendarService service;



   9:  



  10:   // The base URL for a user's calendar metafeed (needs a username appended).



  11:   private static final String METAFEED_URL_BASE =



  12:           "http://www.google.com/calendar/feeds/";



  13:  



  14:   private static final String CALENDAR_URL_SUFFIX = "/private/full";



  15:  



  16:   public void init() {



  17:     service = new CalendarService("DR-Appt-App-1");



  18:     try {



  19:       service.setUserCredentials(username, password);



  20:     } catch (AuthenticationException e) {



  21:       throw new RuntimeException("Could not authenticate to Google Calendar with username: " + username, e);



  22:     }



  23:     String calendarId = null;



  24:     try {



  25:       calendarId = getCalendarId();



  26:     } catch (IOException e) {



  27:       throw new RuntimeException("Error caught trying to connect to calendar: ", e);



  28:     } catch (ServiceException e) {



  29:       throw new RuntimeException("Error caught trying to connect to calendar: ", e);



  30:     }



  31:     try {



  32:       calendarUrl = new URL(METAFEED_URL_BASE + calendarId + CALENDAR_URL_SUFFIX);



  33:     } catch (MalformedURLException e) {



  34:       throw new RuntimeException("Bad URL: " + calendarUrl.toString(), e);



  35:     }



  36:     if (logger.isDebugEnabled()) {



  37:       logger.debug("Url is " + calendarUrl.toString());



  38:     }



  39:   }



  40:  



  41:  



  42:   //get id of correct calendar



  43:   private String getCalendarId() throws IOException, ServiceException {



  44:     String calendarId = null;



  45:     URL metafeedUrl = new URL(METAFEED_URL_BASE + username);



  46:     if (logger.isDebugEnabled()) {



  47:       logger.debug("Url is " + metafeedUrl.toString());



  48:     }



  49:     CalendarFeed resultFeed = service.getFeed(metafeedUrl, CalendarFeed.class);



  50:     for (int i = 0; i < resultFeed.getEntries().size(); i++) {



  51:       CalendarEntry entry = resultFeed.getEntries().get(i);



  52:       if (calendarName.equalsIgnoreCase(entry.getTitle().getPlainText())) {



  53:         calendarId = new File(entry.getId()).getName();



  54:         break;



  55:       }



  56:     }



  57:     return calendarId;



  58:   }



  59:  


I have left out of the above the getters and setters to make it more concise.
Using the above class, you now can access the non primary calendar simply using the syntax:
calendarUtil.getService().insert(…)
or any other service function.



Monday, February 22, 2010

Not a Maven yet….

Notwithstanding my post about using Gradle as a build tool, when i needed to start a new project i decided to use maven due to its amazing integration with an IDE (in my case Intellij), as well as its unbiquitousness.

So, as i have written previously, it is really nice to use Maven as you get started, since with no writing of any config files you have the build process working for you. You have a clean, compile, jar, etc with no work and no opening of an XML file. But that is all theoretical. As i attempted to get going with my project i encountered two really serious issues that took a while to figure out, as well as one issue that needs better resolution.

I will start with the more minor issue. When we use standard libraries that we need to add to our project, then things are pretty straightforward, in fact even better than not using maven. All you need to do is go to http://mvnrepository.com and find the item you want and then it gives you the lines to add to your pom, and voila, you have added your dependency to your project. Of course, in one case, this was not so simple. I wanted to use log4j, so i said, ok, I will just use the latest version. That turned out to be a mistake since all of sudden the compile was failing with libraries i had never heard of unable to be downloaded and installed. As it turns out, after a bit of googling, i learned that i needed to stay away from this latest version. But that is not even the issue i was referring to above. The issue i ran into was when i needed to use non standard libraries that are not in the mvnrepository site. This became difficult as i did not have an easy time figuring out what repository to use and what to add to my POM. In my case, i wanted to use Spring 3 and as far as i can tell, there is no browser for the Spring 3 release repository and thus no instructions for what my artifact ids would look like. (Hint to SpringSource – mention in the javadocs which jar is needed for a package). I wound up guessing based on one item i found somewhere and since i had downloaded the spring 3 distribution i was able to guess how the others would look and solved that issue. The second set of non standard artifacts were the google calendar ones. Here, i had to manually install them in my repository to get it going. Since it was a lot of jars, it was tedious, thought here is a nice link to make it easier, just be sure to use version 2 and not 1 for those things are now in version 2.

Now on the two major issues that i encountered. The first one was that my project would not compile successfully even though there was very little code in it other than use of the Google Calendar API. Finally after not getting anywhere for a while, i realized the issue must be Maven since it did compile in my IDE. As it turned, it was a documented Jira issue, but as the guys in the comments wrote there, who would have thought that default of maven today would not be java 5 at least?

The second one related to testing. Even though my most recent work has been using TestNG, i thought maybe i would give JUnit a try. When my JUnit tests were not being recognized, i figured out that this is because, once again, the default is junit 1.3. After a bit of searching i found what i need to do to add Junit4 to be the test runner. But, since i found the documentation of JUnit 4 lacking, i had decided to switch back but figured i would leave both dependencies in the POM. Well, that turned out to be a mistake too, as i eventually found here

But with these problems behind us, we trudge forward assuming that all should now be smooth sailing, until we need some other complex library….

Wednesday, January 27, 2010

Asynchronous Groovy

In my previous post, i showed how simple asynchronous code in Spring 3 is. I wanted to see what it takes to write the same code in Groovy. Well, it is quite straightforward, using the GPARS library. The docs there show it to be quite powerful, but to be loyal to our simple example, here is the code:



   1: Asynchronizer.withAsynchronizer(4) {ExecutorService service ->



   2:       (0..9).each { num->



   3:         service.submit({



   4:         Thread.sleep(2000);



   5:         println("Running for ${num} thread ${Thread.currentThread()}")}as Runnable)



   6:       }



   7: }



See how you specify the number of threads in the parameter to the closure withAsynchronizer.
There are lots more examples at the GPARS site.

Wednesday, January 13, 2010

Asynchronous Code with Spring 3 – Simple as an annotation

After reviewing some of the features of Spring 3, i decided to test out the asychronous features via annotation. And below is all the code i needed in order to have a function executed asynchronously in my application:



   1: public class Asyncher {



   2:     @Async



   3:     public void myMethod(int i) {



   4:         try {



   5:             Thread.sleep(2000);



   6:         } catch (InterruptedException e) {



   7:             e.printStackTrace();



   8:         }



   9:         System.out.println("Running for " + i);



  10:     }



  11:  



  12:     public static void main(String[] args) throws InterruptedException {



  13:         ApplicationContext ctx = new ClassPathXmlApplicationContext("appContext.xml");



  14:         Asyncher a = ctx.getBean("asyncher", Asyncher.class);



  15:         for (int i=0; i<10;i++) {



  16:                 a.myMethod(i);



  17:         }      



  18:  



  19:         ThreadPoolTaskExecutor t = ctx.getBean("myExecutor",ThreadPoolTaskExecutor.class);



  20:         while (t.getActiveCount() > 0) {



  21:             System.out.println("Number of tasks left: " + t.getActiveCount());



  22:             Thread.sleep(2000);



  23:         }



  24:         t.shutdown();



  25:     }



  26: }



Hard to believe, but all i needed to do was add the Async annotation on line 2 to my function and it works. of course we need a few lines in our application Context to make this work, but here they are:


   1: <?xml version="1.0" encoding="UTF-8"?>



   2: <beans xmlns="http://www.springframework.org/schema/beans"



   3:        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:task="http://www.springframework.org/schema/task"



   4:        xmlns:context="http://www.springframework.org/schema/context"



   5:        xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-3.0.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">



   6:  



   7:     <task:annotation-driven executor="myExecutor" />



   8:     <task:executor id="myExecutor" pool-size="2"/>



   9:     <bean id="asyncher" class="Asyncher"/>



  10:  



  11: </beans>


This same method can be used to create scheduled jobs as well, by changing or adding on line 7 the scheduler attribute and then adding in a scheduler bean

<task:scheduler id="myScheduler" pool-size="10"/>

You can read about it in the docs here

Monday, December 28, 2009

Creating a Zip file from a Stream

Given an InputStream generated for me, i wanted to zip it up and email it. From the java almanac i saw that i could easily do this if i write the stream to disk first. But this seemed like an unnecesary step for me. All i needed to do was to attach this outputstream to my email code and get it done. So, after a bit of playing, I got it working. Here is the code:




   1: private byte[] createZipFile(InputStream in, String fileName) throws IOException {



   2:         ByteArrayOutputStream zipOutputStream = new ByteArrayOutputStream();



   3:         ZipOutputStream zippedFile = new ZipOutputStream(zipOutputStream);



   4:         byte[] buf = new byte[1024];



   5:         zippedFile.putNextEntry(new ZipEntry(myFileName));



   6:         // Transfer bytes from the file to the ZIP file



   7:         int len;



   8:         BufferedInputStream bs = new BufferedInputStream(bds[0].getInputStream());



   9:         while ((len = bs.read(buf)) > 0) {



  10:             zippedFile.write(buf, 0, len);



  11:         }



  12:         zippedFile.closeEntry();



  13:         bs.close();        



  14:         zippedFile.close();



  15:         return zipOutputStream.toByteArray();



  16: } 



By wrapping the ZipOutputStream around a ByteArrayOutputStream, i am able to access the byte array and use that to attach to my email and send out.