Grails AS400 RPG Connection Pool

In my last post I said: "For the most part, I did not have to do anything overly special to work with the AS400" in the development of the AS400-based Kettler USA retail site. The one area that took a bit of Grails-magic was some direct calls to RPG and some indirect calls to RPG via messages. The direct calls were done via JDBC stored procedures, so the standard JDBC connection pool set up by the Grails DataSource.groovy configuration worked fine. But for the message-base communication, I used AS400 data queues -- which are not supported by JDBC. So I needed to create an AS400 connection object. You really don't want to create an AS400 connection object for each request. Instead you want to use IBM's AS400 Connection Pool.

Once the AS400 Connection Pool is available, I'm able to use it in my service class like so:

class PreAuthorizationService implements Serializable {
  def as400ConnPool
  String preAuth(Cart cart){
    AS400 conn = getConnection()
    ...

But where do you create the AS400 Connection Pool and when? Grails makes that easy.

In grails-app/conf/spring/resources.groovy place the following code:

import org.codehaus.groovy.grails.commons.ConfigurationHolder as CH
beans = {
  if (CH.config.dataSource.driverClassName == 
      'com.ibm.as400.access.AS400JDBCDriver') {
      as400ConnPool(AS400ConnPoolFactory) {bean ->
          ip = CH.config.iseriesIPAddress
          userId = CH.config.iseriesUserId
          password = CH.config.iseriesPwd
      }
  }
}

I put the IP, username, and password in Config.groovy (hence the use of ConfigurationHolder. Also note that, because I often use a local MySQL-based AS400-connection-less test environment, I predicate the creation of the AS400 connection so it is only created if the IBM driver was active. The IBM AS400 Connection Pool needs a bit of Spring infrastructure, which I coded in my AS400ConnPoolFactory wrapper class:

import org.springframework.beans.factory.FactoryBean
import org.springframework.beans.factory.InitializingBean

import com.ibm.as400.access.AS400
import com.ibm.as400.access.AS400ConnectionPool

class AS400ConnPoolFactory implements FactoryBean {
  String ip
  String userId
  String password

  public Object getObject() throws Exception {
    // note: try/catch removed for brevity
    AS400ConnectionPool as400ConnPool = new AS400ConnectionPool()
    as400ConnPool.setMaxConnections(128)
    as400ConnPool.fill(ip, userId, password, AS400.COMMAND, 5)
    return as400ConnPool
  }
  public Class getObjectType() {AS400ConnectionPool.class}
  public boolean isSingleton() {true}
}

It's all pretty simple. AS400 connection pools made easy by Grails!

Grails 400 Experiences

I received a number of responses to my post about the launch of the retail site www.kettlerusa.com. I was happy to hear that there are folks out there that are either considering or using Grails on the AS400 (a.k.a. iSeries, System-i.) I will be doing a series of posts on my experiences Grails and the AS400.

For the most part, I did not have to do anything overly special to work with the AS400. DB2/400 supports all the features that Grails (via Hibernate) uses. The biggest issue was working with tables that have composite keys. Note that, although Grails has facilities for working with composite keys, rather than keying the required GORM code, I used the Systemi Grails Domain Plugin. This plugin has a script that takes an AS400 table and generates the Grails domain class. I have used that plugin (which I wrote with help from Mike Brown) to generate hundreds of Grails domain classes for tables on about a dozen different AS400s (all at different companies.) You can read more about the tool in my article Rapid IBM i Web Dev with Open Source Tools (note that article will require a web subscription to www.systeminetwork.)

KETTLER USA was deployed to a 64-bit Windows server that is running on Tomcat. The Tomcat server is behind a firewall with an Apache server running outside the firewall as a proxy server. The Tomcat server is able to connect to the AS400 via JDBC. The Tomcat server can only be accessed from the Apache server so it is a secure AS400 connection.

Most of the application is written using Grails domain classes to access DB2/400, but there are some points where the application does direct calls to RPG via JDBC stored procedures or indirect RPG calls via AS400 data queues. I will be elaborating on this in a future post.

I've been coding 12x7 for so many months getting the Kettler retail site up that I've forgotten some of the slick things that I was able to do with Grails (where coding the solution in Java would have been far more difficult.) But one thing, off the top of my head, that I loved being able to do was to put the Grails domain classes in their own Grails project and pull that project into the UI project as a Grails plugin. The reason why I needed to do this was that I had written a Business-to-business Grails application for Kettler last year. And the retail application used many of the same Grails domain classes. As they say "there shall be no duplicate code." And I certainly didn't want to maintain the 85 domain classes required for Kettler in two places. With one of the more recent versions of Grails (we are on 1.3.5 now) Grails made using your own project Grails plugins very easy. Instead of having to generate a Grails zip archive and import it to the project that requires that plugin -- everytime that supporting project is changed -- you simply put a reference to that project in your grails-app/conf/BuildConfig.groovy:

grails.plugin.location.'shared-domain-plugin' = "../SharedDomainPlugin"

Again, I will be added more Grails400 posts in the future.

Grails 400 Retail Site Now Online

I haven't posted in months because I was developing a Grails-based retail web site www.kettlerusa.com. I've been primarily developing Grails applications for the past 4 years now and I'm still impressed with Grails. This is the first retail site that I've developed -- all my prior sites have been internal, or business-to-business password based applications that I couldn't easily share. The back-end database is DB2/400 running on an iSeries/AS400 box. The application has some direct integration with RPG via stored procedures and data queues but otherwise it uses the Grails domain classes and GORM to work with the DB2/400 database.

If you are interested in Fitness, Bikes, Toys, Patio, or Table Tennis, check out www.kettlerusa.com.

In future posts I plan to blog about issues I had in the development of this application that were elegantly solved with Groovy and/or Grails.

Building a Browser for Amazon S3 with Grails

Are you using Amazon S3 and considering learning Grails? Or are you a Grails developer and beginning to look at Amazon S3?
Check out my Amazon article:
Building a Browser for Amazon S3 with Grails

DRY with Grails namedQueries and a Super-Dynamic Search Strategy

I just finished developing a simple query engine with Grails 1.2.1 for one of my clients. As usually, Grails did most of the hard work. The requirement was to provide a query engine for customer-specific flat files that contained all information about transactions that occurred in the last year. Previously the flat files were delivered to the clients as Microsoft Access binary files. The problem was that the clients never took the initiative to write apps to search and aggregate the information. So I was brought in to use Grails to rapidly develop a web application that provided search and aggregation of the client-specific web flat files.

The resulting Grails application is hosted on Amazon EC2 with MySQL. Anyway, I wanted to share with you my use of Grails 1.2's namedQueries and my simple strategy to support a dynamic search in the standard list.gsp.

The application's list page shows all the columns of the table. To do that it needs to know the properties of the domain. Each client has a custom domain class specific to their flat file (which, for some reason, have different column names.) The domain classes have a getColumnProperties():

static List getColumnProperties() {
def columnProperties = []
CustDomainName.properties.declaredFields.each {prop ->
if (prop.modifiers == 2            &&
!NO_SHOW_COLS.find {it == prop.name} ) {
columnProperties << prop
}
return columnProperties
}

The column properties are passed to list.gsp as [properties:domain.columnProperties] so the page can list all columns:

<tbody>
<g:each in="${domainInstances}" status="i" var="domainObj">
<tr class="${(i % 2) == 0 ? 'odd' : 'even'}">
<g:each in="${properties}" var="property">
 <td>${domainObj[property.name]}</td>
</g:each>
</tr>
</g:each>
</tbody>

Each column has a search added to the the top of the table:

<tr>
<g:each in="${properties}" var="property">
<td>
<% if (property.type.name != 'java.util.Date') { %>
<input type="text" name="${property.name}" value="${(params[property.name])}"
<% if (property.type.name == 'java.lang.String') {%>
    title="Enter a character prefix to filter ${property.name}"
<% } else { /* assume numeric */ %>
   title="Enter a numeric value to filter ${property.name}"
<% } %>
/>
<% } else { %>
<richui:dateChooser format="MM/dd/yyyy" name="${property.name}" value="${(params[property.name])}" />
<% } %>
</td>
</g:each>
</tr>
<tr>
<g:each in="${properties}" var="property">
<td>
<% if (property.type.name == 'java.util.Date') { %>
 <richui:dateChooser format="MM/dd/yyyy" name="${property.name}To" value="${(params[property.name+'To'])}"  />
<% } else if (property.type.name != 'java.lang.String') {
 def propOp = property.name+'_Op'
%>
<g:select name="${propOp}" value="${params[propOp]}"
   from="${['eq', 'gt', 'ge', 'lt', 'le', 'ne']}"
   valueMessagePrefix="critera.operator"  />
<% } %>
</td>
</g:each>
</tr>

Note how dates have a from and to with the RichUI dateChooser and numerics have operators. Strings work as prefixes to the SQL like clause. The list action is then implemented as follows:

def list = {
params.max = Math.min(params.max ? params.int('max') : 10, 100)
params.offset = params.offset ? params.int('offset') : 0
params.sort = params.sort?:'id'
params.order = params.order?:'desc'

List domainInstances = domain.dynaCrit(params) // .list(params) ignors sort
    .findAllByIdGreaterThan(0,
         [max: params.max, offset:params.offset,
          sort:params.sort, order:params.order])

int count = domain.dynaCrit(params).count()

[domainInstances: domainInstances, count:count,
properties:domain.columnProperties, params:params]
}

Prior to namedQueries (with Grails 1.2.) I've always had to copy-and-paste criterion logic from the search to the count query. This violated the Don't Repeat Yourself (DRY) principle. Now I use a named query. Note that I did have an issue with the sort facilities in the list method so I did a hack with the findAllByIdGreaterThan (perhaps this is fixed with Grails 1.3.) Finally, here's the namedQuery in the domain class:

static namedQueries = {
dynaCrit {params ->
params.each {filterProp ->
  if (filterProp.value) {
    def property = CustDomainName.columnProperties.find {it.name == filterProp.key}
    if (property) {
      if (property.type == String) {
        ilike ("$property.name",  filterProp.value+'%')
      } else if (property.type == Date) {
        between ("$property.name",  filterProp.value, params[filterProp.key+'To'])
      } else {
        "${(params[filterProp.key+'_Op'])}" ("$property.name",  property.type.newInstance(filterProp.value.trim()))
      }
    }
  }
}
}
}

The application does provide a number of other handy features:

• The ability on the list page to remove (and re-add) columns
• Dynamic summary reports with prompt pages prompting for groupBy and sum columns along with filter criterion.
• Excel download option of the user-defined report
• The ability to save user-defined queries.

What I found amazing during the development of this application was how Grails made it ridiculously easy to develop dynamic reporting.

Handling Select-Multiple with Grails

When a page sends multiple values with the same HTTP parameter name, Grails stuffs them into an ArrayList. That is very handy but there's an issue when the user only selects one value. When only one value is passed Grails does not put it into an array. And your code has to detect whether or not the parameter is an array.

Perhaps the best solution is to use a Command object. The following, for example, handles the user selection of one or multiple items (orders, qtys, etc.):

class BuildReturnItemsCommand {
   String[] itemNo
   int[] orderNo
   int[] qty
   BigDecimal[] unitPrice
   String[] desc
}

[recent add: check out http://www.intelligrape.com/blog/2010/06/14/getting-params-attribute-as-list/ for another solution]

But, I'm lazy. If I have simple input where no validation is required, I don't want to take the time to create a command object. The following shows the preamble of a Controller closure that handles a single selection of a multiple-select list of regions:

def salesReport = {
  params.regions = (params.regions)?[params.regions].flatten():null

Groovy to the rescue!
The ternary, if there is a regions parameter, stuffs the regions into a List. But, because the parameter might already be a list, it is flattened.

The following Groovy snippet tests this code:

def params = [:]
params.regions = 'West'
params.regions = (params.regions)?[params.regions].flatten():null

assert params.regions == ['West']

params.regions = ['East', 'North', 'South', 'West']
params.regions = (params.regions)?[params.regions].flatten():null

assert params.regions == ['East', 'North', 'South', 'West']

Groovy wrapping of long table td text with no blanks

Subtitle: Groovy's a Wrap

Just today I had a client -- who had just tested a fairly complex GSP -- ask to have text in a table data element wrap. He had tested it with a long string of characters with no spaces. He wanted:

asldfjas;fjasodfsaodjfsaodfdsadfa;ljlj

To show as:

asldfjas;fj
asodfsaod
jfsaodfdsa
dfa;ljlj

And not just the first 12 characters that fit in the td.


Quick solution: Hey buddy, use spaces and the browser will wrap for you.
Groovy solution: Insert thespaces for them.

I took the Groovy option. What I did was extend the String class in BootStrap to have a wrapAt method:

class BootStrap {

def init = { servletContext ->
String.metaClass.wrapAt = {width ->
matcher = (delegate =~ /\w{1,$width}/);
def that = ''
matcher.each {
 that += "${it.trim()} "
}
return that
}
}
def destroy = {
}
} 

Then, in my GSP (where the em width was 12) I added:

<td>${option.value.wrapAt(12)}</td>

I tested the code with the following:

String.metaClass.wrapAt = {width ->
println delegate
matcher = (delegate =~ /\w{1,$width}/);
def that = ''
matcher.each {
that += "${it.trim()} "
}
return that
}

def str = 'one two thetimehascomeforalgoodmentocometotheaidoftheircountry'
assert "one two thetimehas comeforalg oodmentoco metotheaid oftheircou ntry " == str.wrapAt(10)

Sure, not a perfect solution, but all the text the user keyed is viewable on the page (and the optional PDF that is generated from that page.) It looks like this:

one two
thetimehasco
meforalgoodm
entocometoth
eaidoftheirc
ountry

Also, it gave me a use for Groovy meta programming

Kindle for Technical Content

I'm a reader. I started reading novels at a very young age. As a developer and reader, I'm addicted to reading technical books. I am also a technical writer so I'm constantly buying and reading books. And I'm not a fan of reading on-line. When I sit in front of my PC, I expect the instant gratification of coding or writing. I find it difficult to read a PDF book cover-to-cover (or should that be byte-to-byte?) Case in point: To save money, I opted for the PDF version of Venkat Subramaniam's "Programming Groovy: Dynamic Productivity for the Java Developer." Great book, by the way. But I ended up getting frustrated and printed the book -- which ended up costing me more in paper and ink than that print version would have.

A month ago I was at Barnes and Nobles and I became intrigued by the Nook. The Nook is B&N's book reader. I almost bought it. Luckily I backed out of the store and began researching the Nook and compared it with Amazon's Kindle.

The Nook is pretty neat but, unlike the Kindle, it does not have a keyboard. With the Nook, you can bookmark your documents -- but you can't highlight text or add notes. The Kindle has an integrated keyboard and you can highlight and add notes.

I got my Kindle yesterday. And, so far, I love it.

Book Availability:

I am currently researching Cloud Computing for use in Grails applications using both Amazon Web Services and Google App Engine. To that end I've been wanting to buy two books: "Programming Amazon Web Services" and "Programming Google App Engine" (both from O'Reilly.) Curiously, the Amazon book was not available for Kindle at Amazon's site but the Google book was. So I bought the GAE book from Amazon and went to the O'Reilly site for the AWS book. At O'Reilly I found that they do make an eBook download available that is Kindle compatible (.mobi.) O'Reilly gives you both the Kindle and PDF versions of the book. To get the AWS book's mobi file on my Kindle I simply plugged in the Kindle's USB and copied the mobi file from my PC to the Kindle's documents directory.

My next issue was that I wanted to get some of my PDF books on the Kindle. All I had to do was plug in the USB and copy the PDFs from the PC to my Kindle. There are two issues with PDF on the Kindle: 1) It shows the entire page on the screen, which requires a very small font (but it is readable). 2) While you can add bookmarks to a PDF, you can't add notes or highlight text. Note that I believe there are PDF to mobi converters available. But, for right now, I'm happy with Kindle's PDF reader.

Reading Experience:

I was reticent to get an electronic reading device because I thought I needed the spacial context of a printed book. When I first get a printed copy of technical book, I put a half-dozen sticky notes inside the front cover. Then I hook a highlighter and a mechanical pencil on the back cover. I use the sticky notes to bookmark important sections and my current page. Those sticky notes stay in the book forever. When I go to my book shelve, I usually find what I'm looking for by going to the various bookmarked pages.

As I read my print books, I use the highlighter on important sections and I use the pencil to make notes. If I'm doing research for an article, I also write notes on the front cover with page references.

The Kindle does all that for me only in a cleaner, greener fashion. First off, I love that when I put the Kindle down (which has been difficult) and power it off and then later return to the Kindle and power it back on, it goes right to the page of the document that I was last reading. I also love that I can make bookmarks with notes and then Kindle shows me my bookmarks, notes, and highlights in an easy to read index. For article research, the notes are invaluable. For programming, the bookmarks and highlights are extremely useful.

I was also concerned about the display of tables and code on the Kindle. But I discovered that the Kindle allows you to change the view from portrait to landscape (a feature the Nook does not provide.) The Kindle also allows you to change your font size (which the Nook allows as well.)

At any rate... I'm loving my Kindle

Free Book on Grails

Three years ago, when I first began developing with Grails, I benefited from a book by Jason Rudolph called "Getting Started with Grails" The book was well written, short, and available as a free PDF. I recommended it to everyone interested in Grails. But, as Grails rapidly improved, the book began to become outdated.

No longer.

Jason Rudolph teamed up with Scott Davis (author of "Groovy Recipes") to update the book. It now covers Grails 1.2 and, once again, I highly recommend it.
You can download the book from:
http://www.infoq.com/minibooks/grails-getting-started
But note that InfoQ will ask you to register with their site before you can download the PDF. But it is well worth it as InfoQ is a great site with lots of technical information, in form of articles and videos, on application development.

Comment on IE

Half my work is creating Web solutions for my clients.
The other half of my work is getting it to work on Internet Explorer....

Real World Observation about Grails

In the past 3 years I've developed 7 Grails applications for a variety of customers. And I have to make an observation: Maintaining Grails applications is a breeze. It is far more easy than with any other development platform I've used. I've had to revisit Java applications that I had written. But it always takes me a while to figure out the design strategies. For instance: what MVC pattern did I use?

With Grails applications -- whether written by myself or not -- the MVC is always the same. The directory structure is always the same. It is fabulous being able, in Eclipse, to switch workspaces from one Grails project to another and have the structure look identical. They look so similar sometimes, when I come back from lunch or something, I have think "OK, what project is this."

This is an observation of the benefits of the Grails philosophy of "Convention over Configuration." This Grails feature provides a huge ROI when someone else can take over a Grails project and know where everything is. In fact, I benefited from this myself two years ago when I joined an existing Grails project team. The project was Circuit City's return system (I have some gift cards if anyone would like to buy them). The return system had been in production for a year and it was very complex (for example it had DB connections to Informix, Oracle, and DB2i.) Yet I was able to be productive in a very short time after joining the project. By the way, that Grails app is the only Grails app that I worked on that is no longer in use -- for obvious reasons....

Grails makes life easier.

Book Review: Grails: A Quick-Start Guide

It's been 5 months since my last post. I have dozens of times thought "I need to post this." But -- as I was working 12X7 on a major Grails-based order entry system and two other smaller Grails projects and authored 3 articles on Grails and delivered a multi-city seminar tour on Grails -- I simply didn't have the time. But things are slowing down now.

I recently burned through The Pragmatic Programmer's new book: "Grails: A Quick-Start Guide" by Dave Klein. I was impressed. Attendees to my seminars and Grails mentees usually ask "What's a good book to buy on Grails?" Well, "The Definitive Guide to Grails" authored by the creator of Grails, Graeme Rocher, is a bit too long for beginners. I think of it as the Grails bible. "Grails in Action," by Glen Smith and Peter Ledbrook, is fabulous as it is very written. But its 487 pages can be a bit daunting. "Grails: A Quick-Start Guide" is very approachable. It is just barely over 200 pages and it very succinctly written. It is now my recommended book for Grails beginners.