Monday, February 16, 2015

History of the World, Part II: The Rise of Languages


Much like Mel Brooks’ movie, “History of the World, Part I” this blog post lacks historical accuracy and is tainted by personal experience and views. Basically I’m just brain dumping my thoughts on where languages are today, how they got there, and how they’ve affected me.


FORTRAN:



A great language for mathematics and science and a decent language for teaching. John Backus of IBM premiered this language in 1957. Strangely, continually revised versions of FORTRAN are still around today. I learned FORTRAN in college and what I liked most about FORTRAN was that I got an easy A while my brilliant older brother had to work his ass off to get a C. This was the first time that I thought maybe I could make a career out of programming.


Basic:



Basic came out of Dartmouth in 1964. It was a crappy language really only good enough for kids to learn how to program. It should have died decades ago but some guy called Bill Gates liked it and it morphed into a monstrosity called Visual Basic. I’m pretty sure the term “spaghetti code” was coined to describe Visual Basic applications.


RPG:



RPG is an IBM language that appeared in 1959. It helped catapult the S/3, S/36, S/38, AS/400, iSeries, and IBMi into a leading business platform for Fortune 100. The RPG language was originally developed so each statement could easily fit on an 80 column card. I remember, as an operator in the 80s, retyping statements in a card machine when programs would not load from the card-reader due to a bad card. RPG morphed into a modern language but the wealth of well-written 70s and 80s code haunts the RPG language and gives the development community the incorrect view that the IBMi/AS400 is an antiquated platform.


COBOL:



COBOL, like RPG, appeared in 1959. It was designed to read like the English language -- which did not work as it resulted in overly wordy statements and programs. But, I have to thank COBOL for two things:


1) COBOL programmers, because of their ridiculously long printouts monopolizing printers, forced me to learn how to develop (I was doing RPG at the time) without program listings. That’s what you did: Made changes, attempted a compile, printed the compile listing, reviewed code with a high-lighter, made changes again, and tried another compile. The only way I could get anything done was to go paperless.


2) During my two years of programming in COBOL,  I was pretty much forced to finally move from a hunt-and-peck two-fingered typist to a full ten-fingered typist at 25-30 words a minute (my sons were typing at 50-70 WPM by the time they were 15, but whatever.)


C:



For some reason, the C language fascinated me. I studied the crap out of it and was excited when IBM finally made a C compiler for the AS400. In fact, The first article I ever published was called “Freedom of Choice in C Compilers” it compared two C compilers that were newly available for the IBM AS/400 platform:




The book “The C Programming Language” is arguably one of the best technical books ever written. Kernighan and Ritchie patterned their book after Strunk and White’s “The Elements of Style.” Both books are extremely thin and yet full of useful information. Many, many programming language books have attempted to emulate “The C Programming Language” with limited success. I’ve thrown out a dozen books on writing and dozens and dozens of books on programming yet these two remain on my shelf and are regularly referenced.


My research into C (and my article) got me a job as a systems-programmer at ASNA, where I worked for 5 years as a C and C++ programmer.


dBase III:



This great language was built for the IBM PC and appeared abou 1980. It forced me to learn modular programming and coerced me into learning better database design.


Assembler (MI):



Assembler has been around since forever. I did learn some MS/DOS-style assembler but I used the AS400 MI (Machine Interface) language on a daily basis during my 5 years as a systems programmer at ASNA. The best thing I can say about assembly language programming is that it made me appreciate C. I’ve met some folks that bragged to me about how they program in MI regularly and I would think “Why?” I mean, the last programming I did at ASNA was to replace all use of MI with C-API calls. There is rarely a need to use assembler level coding anymore.


C++:



I loved Stroustrup's monstrosity. And I got very good at it. I created a Windows source-level debugger for RPG and Cobol using C++ (that debugger was called Extermin8 for Windows.) I also worked with a team of C++ programmers at Circuit City that used an Object Oriented Database.


Question: Does Linus Torvalds write in C++?
Answer: No, he uses C.


Java:



Following Jimmy Fallon’s lead in thank you notes:


Thank you Java, for wasting 10 years of my life while building the infrastructure for modern languages


I’ve said in interviews that I’ve been doing Java since “before it was real.” I downloaded and played with IBM’s Java environment in 1996. At that point IBM was saying Java was going to be the next big thing. Lot’s of folks jumped on the bandwagon. But it wasn’t until the Java Servlet API was readily available in 1998 that Java became “real.”


I was considered a leading expert of Java on the AS/400 (a.k.a. iSeries and IBMi.) I published several books and hundreds of articles on Java. I was the Java Subject Matter Expert for Common (www.common.org/) I was the editor of both a Java print magazine called NetJava Expert, I was the editor of a popular email newsletter called Eclipse Tips. And I delivered week long seminars in most major US cities.


But, here’s what I think of Java now: It is a systems programming language. It is too complex of a language for business applications. Even if the developer is brilliant at Java, they would develop faster with a different language.


Java, however, gave us a lot including:


The Java Virtual Machine, with optimizations and excellent thread support
Testing frameworks
APIs for most anything you can think of from email services to search optimizers
A platform for dozens of other languages, from Scala to Rhino.


PHP:



Or, as it was first called, Personal Home Page. PHP was developed as a thin veneer on top of the Perl language to simplify web development. I liked Perl so I gave PHP a several year whirl. And I still try stay on top of it. I did a series of articles for IBM Developerworks on advanced PHP:

Leveraging PHP 5.3's lambdas and closures:
Leveraging PHP V5.3 namespaces for readable and maintainable code
PHP V5.3 invigorates object-oriented programming with late static binding


Nevertheless, I feel PHP is a sloppy language. It does have many great frameworks but there are so many of them that the developer base has splintered. I think PHP is the VB of the Web. It is all to easy for folks to write bad code in PHP. I’m thinking Kent Beck, however, has written some fine PHP as a contractor for Facebook. So you certain can do great things with PHP and, as a dynamic language, you can do things easily that are next to impossible with static language like Java.


Here's an article I wrote called "Get Real with PHP" where I prescribed strategies for development: 
http://iprodeveloper.com/rpg-programming/get-real-php


Groovy:



Groovy is a dynamic language that runs in the JVM. I love the Groovy language. A Groovy source file compiles to become a Java class -- so it will run in any Java container. When the Grails framework (a Ruby on Rails wannabe that is build using Groovy) became available, I immediately focused all my work on Grails. I was able to coerce a dozen or so clients into allowing me to implement Web apps for them in Grails. In fact, the first Grails app I developed, 8 years or so ago, is still running fine.


The Groovy language is very elegant and, of all the languages that run on the JVM, Groovy integrates the best with existing Java classes. I found that I was able to write 50-80% less code with Groovy than it would have taken with Java. And I was also able to develop much more quickly with Groovy and Grails than with Java and standard Java frameworks.


I’ve published a bunch of articles on Groovy and Grails and here’s one I did specifically for Amazon Web Services:

Building an Amazon S3 Browser with Grails : Articles & Tutorials : Amazon Web Services


Ruby:



Ah, Ruby, this has been my current love for about 4 years now. As I said, I did Grails for many years but, all that time, I still had Rails-envy. But as soon as I was able to coerce a client into using Rails instead of a Java, I never looked back.


Ruby was written for programmers that love to code. It allows you to do more with less. You can do things in Ruby extremely easily that would otherwise take hundreds of lines of code and the use of complex design patterns in Java. With Ruby, you can do things that you had thought of while using Java but never did because it would have taken too much time.


Ruby has a number of environments. One is MRI: Matz Ruby Interpreter (Matz is the creator of Ruby.) MRI is also known a CRI as it is a C language implementation of a Ruby environment. One example of RI alternatives is IronRuby with is Ruby for the .Net platform. But the most interesting alternative RI, to me, is JRuby. JRuby is very successful because it runs on top of the heavily optimized JVM. And that JVM provides true threads to Ruby applications -- which is something that is lacking in the other Ruby Interpreters. JRuby also allows you to easily integrate with Java APIs. The first time I used JRuby I did so because I needed to use a JDBC driver to access a database that did not, at the time, have a Rails ActiveRecord adapter. I wrote about JRuby in “Get on the Fast Track with Rails”


Rails is the best web application development frameworks I’ve used. And I’ve used dozens. Rails brought us Convention-Over-Configuration (CoC) and a myriad of other well-thought out concepts.


To learn more about CoC you can read my article:


What next:


Functional programming languages?

Tell me about your language experiences.

Sunday, February 8, 2015

AWS SES Mini-App with Bounce Notification Handling



As promised in my last blog entry on http://denoncourt.blogspot.com, this my first in a series of instructive Rails mini-apps. The purpose of each application will be to illustrate the use of one technique, feature, or utility. And the README of that application will provide instructive details (https://github.com/dondenoncourt/aws_ses_bounce_mini_app). [And, actually, this blog post displays better on github so I suggest you click on the prior link and continue reading.]
Recently I had to code a Rails application to handle bounce notifications from AWS SES. Amazon Simple Email Server is ridiculously easy to configure and use in a Rails application. As the instructions at the github gem page, https://github.com/drewblas/aws-ses, explain, all you need do is add the aws-ses Rails gem and create a configuration file called config/initializers/amazon_ses.rb that contains your Amazon credentials (soft-coded of course.)
But, if you need your application to handle bounce, complaint, or delivery notifications, things get a little more complicated. It took me a bit to figure out how to properly configure AWS-SES bounce notifications and to write the code required to handle AWS callbacks. So I figured it would be helpful if I built a mini-app with a README that details the process.

Bounce Notification Flow

Let me walk through the two-step flow of bounce notifications:
First: your Rails mailer sends an email and, subsequently, AWS-SES fails to be able to deliver that email -- perhaps because the email address was incorrect, or maybe the mailbox was full, or it could have failed for a couple of other reasons that are covered in http://docs.aws.amazon.com/ses/latest/DeveloperGuide/notification-contents.html#bounce-types.
Second: because you have configured AWS-SES to perform callbacks to your application when bounces occur, AWS-SES sends a POST to your application with a JSON payload that contains an array of bounced email addresses.

Handling AWS SES Callback Confirmation HTTP Post Request

The first part of that second step, configuring AWS-SES bounce callbacks, is easier said than done. The AWS configuration screens are confusing and, as I found out the hard way, to make things easy, your application should be coded to handle notifications before you do the AWS-SES configuration.

Mini-App’s AWS-SES Bounce Confirmation Handling

My mini-app’s routes.rb includes the following:
  get 'mail_it' => 'simple_mail#mail_it'
  post 'aws_sns/bounce' => 'simple_mail#bounce'
And here’s the controller code:
class SimpleMailController < ApplicationController
  skip_before_filter :verify_authenticity_token # so AWS callbacks are accepted

  def mail_it
    logger.info "mail_it called with #{params}"
    @email = params[:email]
    SimpleMailer.mail_it(@email, 'original text').deliver
    render text: 'mail sent'
  end

  def bounce
    json = JSON.parse(request.raw_post)
    logger.info "bounce callback from AWS with #{json}"
    aws_needs_url_confirmed = json['SubscribeURL']
    if aws_needs_url_confirmed
      logger.info "AWS is requesting confirmation of the bounce handler URL"
      uri = URI.parse(aws_needs_url_confirmed)
      http = Net::HTTP.new(uri.host, uri.port)
      http.use_ssl = true
      http.verify_mode = OpenSSL::SSL::VERIFY_NONE
      http.get(uri.request_uri)
    else
      logger.info "AWS has sent us the following bounce notification(s): #{json}"
      SimpleMailer.mail_it('dondenoncourt@gmail.com', json).deliver
      json['bounce']['bouncedRecipients'].each do |recipient|
        logger.info "AWS SES received a bounce on an email send attempt to #{recipient['emailAddress']}"
      end
    end
    render nothing: true, status: 200
  end

end
The controller’s mail_it method is self-explanatory. It take a url like:
/mail_it?email=dondenoncourt@gmail.com
and responds by sending an email to the specified address. It is the bounce method that needs a bit of explanation. As I mentioned earlier, to make AWS-SES configuration simple, your application should be coded to respond to an AWS-SES bounce callback confirmation request before configuring AWS-SES. Let me explain: When you configure AWS-SES bounces, you provide AWS the URL of your application. AWS will put that bounce configuration in a pending status until it is able to send a confirmation request to your application and gets a positive response. Anyway, I recommend that you add the route and the controller action and have the app running so it is ready to handle the AWS POST HTTP request -- before you configure AWS-SES.

Heroku

Note that I put my AWS-SES Mini-App on Heroku mostly because AWS-SES callback needs to be able to POST to the URL of an addressable server. My good old localhost:3000 wouldn’t work without doing some router configuration that I wasn’t willing to do.

Configure Verified Senders

When your application is ready to respond to AWS-SES callbacks, it’s then time to configure AWS-SES. The first step is to add and verify sender email addresses. From the AWS menu, select Services and click on SES.
Alt text
Then, on the left panel of the SES page, click on the Email Addresses link and, in the center panel, click on the Verify a New Email Address button. Enter the address and click the Verify This Email Address button.
Alt text
The following popup will be displayed:
Alt text
Note, in the background, in a highlighted color, that the Status of the email address is “pending verification.”
An email will be sent to that address with the subject of “Amazon SES Address Verification Request.” The body of that email will say:
Dear Amazon Simple Email Service Customer,
We have received a request to authorize this email address for use with Amazon SES in region US East (N. Virginia). If you requested this verification, please go to the following URL to confirm that you are authorized to use this email address:
After you click that link the status of the Verified Sender:Email will go to verified.

Add the Bounce Topic

With the email addresses verified, the next step is to create a bounce topic. From the AWS menu, select Services and click on SNS.
Alt text
Click the SNS Dashboard link on the left panel and then click the Create New Topic button in the center panel. Enter Topic and Display Names of “bounce” and then click the Create Topic button.
Alt text
In the subsequent panel Alt text Click the Create Subscription button and key an endpoint name that matches your application’s bounce route and click the Subscribe button.
You will see a pop panel that says:
“Subscription request received! A confirmation message will be sent to the subscribed endpoint. Once the subscription has been confirmed, the endpoint will receive notifications from this topic. Subscriptions will expire after 3 days if not confirmed.”
Alt text
Click the Close button on that popup and note the SubscriptionId column on the page still says “PendingConfirmation.” Click refresh and, if your application was available to successfully respond to the URL specified in the bounce endpoint, the SubscriptionId should be set to a value like:
arn:aws:sns:us-east-1:294894041652:bounce:30e9ca4b-0723-4078-86a5-0d1d2573d101

Selecting the Simplified JSON Notification Format

To get a simplified version of the AWS-SES bounce JSON string I (which is also the format I expect in SimpleMailController) I changed the raw type. You should do that as well by clicking on the Subscriptions link in the left panel, then clicking the checkbox of the bounce topic in the center panel, and, in the popup, click Raw Message Delivery True, and click the Set Subscription Attributes button.
Alt text

Enabling Bounce Notifications

Next you need to assign the newly created “bounce” SNS topic to your SES Verified Senders. From the AWS menu, select Services and click on SES. Then, on the left panel, click on Email Addresses under Verified Senders. You had previously entered an email address here but, this time, we need to specify that emails sent through the Verified Senders will have bounce processing enabled.
Alt text
Then click on the Verified Sender that you use as your application’s from address, expand the Notifications twirly, click on the Edit Configuration button, and, in the select box for Bounces, pick the ‘bounce’ topic you created earlier. And click the Save Config button.
Alt text
Note that you can configure bounce notifications by domain as well as sender email address. Also note that, after a successful bounce notification configuration, AWS post to your bounce handler an “AmazonSnsSubscriptionSucceeded” bounce notification.

Test Bouncing

The free and default version of AWS SES only lets you mail to verified address. So it is a bit of a problem to test bounces. No problem, though, as AWS provides a set of test email addresses you can use: (http://docs.aws.amazon.com/ses/latest/DeveloperGuide/mailbox-simulator.html)
That list contains an email that you can use to tests your bounce code. From my app I used:
https://fast-cove-3541.herokuapp.com/mail_it?email=bounce@simulator.amazonses.com
The simple_mail_controller#bounce method handled that bounce by sending myself an email the body of which contains the following ASW-SNS bounce JSON string:
{"notificationType"=>"Bounce", "bounce"=>
{"bounceSubType"=>"General",
 "bounceType"=>"Permanent",
 "bouncedRecipients"=>
 [{"emailAddress"=>"bounce@simulator.amazonses.com", 
   "status"=>"5.1.1", 
   "action"=>"failed", 
   "diagnosticCode"=>"smtp; 550 5.1.1 user unknown"
 }], 
"reportingMTA"=>"dsn; a8-34.smtp-out.amazonses.com", "timestamp"=>"2015-02-07T17:40:39.338Z",
 "feedbackId"=>"0000014b65210ac9-b9f36242-8ade-413e-8597-1112a631244f-000000"}, "mail"=>{"timestamp"=>"2015-02-07T17:40:38.000Z", 
"source"=>"dondenoncourt@gmail.com", 
"destination"=>["bounce@simulator.amazonses.com"],
 "messageId"=>"0000014b652108a4-38938047-2f1b-4d2b-a1ca-28b58ed6fdd5-000000"}}
The JSON contains an array of bouncedReceipts. My bounce method did not really “handle” the bounce in a useful manner. All it does is log the email addresses. Your application would, undoubtedly, do something a bit more useful.
Amazon lists the JSON structure for bounce notifications at http://docs.aws.amazon.com/ses/latest/DeveloperGuide/notification-contents.html. You probably will want to look at the bounce types and process their handling accordingly.
In a production application, I coded RSpec bounce handling tests with JSON built from the AWS samples.

Monday, February 2, 2015

Looking for a Rails Job

I’m looking for a Rails job, and the process is not going as easily as I had expected. In this blog entry I will describe why I’m looking, how I’m doing it, and what I’m going to do to improve my job prospects in the future.


I not just some one-off guy hoping to cash in on the Ruby and Rails bandwagon. I’ve been building web applications since the last century and I’ve been developing Rails applications for the last 3-4 years. Cool apps. One was theshoplift.com -- a social shopping site targeting the fashion industry. Another was a K-12 online education application that is used worldwide. And another was the Rails Active Record adapter for the IBM AS/400 (a.k.a. IBM i.) Collaborating directly with IBM, I worked on the low-level Ruby and C code required to access IBM DB2 for i from Rails. So why am I having problems finding a job? I’ll get to that. But first let me describe how I’m looking for a job or consulting gig. And, please, post suggestions. If not for me, then for others that may read this post as they are also looking for Rails work.


How I’ve Been Looking:



Rather than creating the standard Microsoft Word document version of a resume I’ve been using two Web sites as they provide, essentially, dynamic resumes: careers.stackoverflow.com and linkedin.com. And then, in cover letters, I paste in the following tiny URLs: bit.ly/1zuir1u and lnkd.in/d5MYeKq. When I need a hard-copy of a resume I simply export my LinkedIn profile to PDF. Perhaps I should get it professionally enhanced but I believe the LinkedIn format is pretty decent.


I’ve been searching for jobs from the following sites:


  • careers.stackoverflow.com
    • This is my preferred site as it does not include head hunters and the companies making the job post seem more interested in quality development (in that they often are not the monolithic fortune 100 IT organizations.)
  • Indeed.com
    • I’m not too crazy about this site but I have receive a few calls from head hunters that found me on indeed.com
  • dice.com
    • Dice’s search is pretty good and, as they are well known, they havemany job posting.
  • hired.com
    • Hired is a great idea but it is limited to jobs in major metropolitan areas.
  • toprubyjobs.com
    • There are not a lot of jobs on this site but it is a solid and genuine job site.
  • cybercoders.com
    • Cybercoders has a stable of head hunters that work with you. There are a lot of jobs on this site and they also have a decent email alert system for newly added jobs. When you select a job to apply for, cybercoders.com prompts you to flesh out responses to specific questions -- which you normally would flesh out in a cover letter. I like that strategy as the prospective company gets immediate feedback on questions for which they are most interested. It does, however, remove some opportunity to humanize a response that you might otherwise have put in a cover letter.
  • flexjobs.com
    • This site actual costs $14 or so a month but, as it is specifically adapted for telecommuting, I went ahead and paid $14 for one month. Probably a third of their viable jobs, though, are already in cybercoders.com. Nevertheless, I have responded to some job posts that were on flexjobs and not on other sites.


Salary



So what salary should I ask for? Difficult question as I’ve been making 150-165K for the last 3 years or so. On the other hand, I’ve had to struggle with invoicing and looking for subsequent gigs. Also, understand that working for yourself doesn’t mean you are your own boss: It means you have many bosses.


Anyway, to figure out the salary I’d be comfortable with, I used the following algorithm:
  • Start with 150K
  • Minus 50-100% of the cost of my health insurance (which almost 1K a month)
  • Minus my average consulting income for 3-5 weeks with consideration for vacation and holidays -- which I almost never took working for myself
  • Minus 8K for self-employed FICA


After doing the above math and, considering the relief from the headaches of working for myself, I came up with 1xxK. But is 1xxK a reasonable salary for a Rails developer with my experience? The only salary site I could find that listed Rails salaries (payscale.com/research/US/Skill=Ruby_on_Rails/Salary) listed a Senior Software Engineer salary of 1xxK. But that was the median salary. When I clicked on the Years of Experience option, I discovered that the median salary for my years of experience is 125K. I also did an empirical survey by looking at a dozen or so salaries ranges of Rails jobs on cybercoders.com and found that 1xxK was regularly within scope.


What Am I Doing Wrong?



I had thought that It would be easy for me to find a job. After all I leave a long trail of successful projects and I have over 20 years experience. My problem is that, I short, I got lazy with my marketing. And I sort of knew it. For the past half-dozen years or so I’ve been too busy immersing myself in Convention-over-Configuration frameworks (I did Groovy and Grails for 5 in the Java marketplace years before moving on to Rails 4 years ago.) I pretty much thought I was that guy in “Field of Dreams” who heard “If you build it, they will come.” Well, I built it. A half-dozen times I built it. But they are NOT coming. The reality is: They won’t come unless they’ve learned about it from some form of marketing.


For my recent 6 years of freelancing, all my work has come from word of mouth. I’ve come to rely on that. I guess I even expected it. Now, I am getting interviews and I’ve come close to getting a couple of jobs. But I’m limited by a couple of things:


1) I been working remotely since 1998, and I want to continue that
2) My desired salary is close to the high end
3) The breadth of my resume may make some managers consider me overqualified
4) The length of my resume clearly shows that I’m in my 50s and there may be some age discrimination
5) My github profile lacks public Ruby and Rails projects


That last item -- my lack of open-source Ruby code for companies to review -- ends up being a big deal.  These days, many companies start the applicant filtration process by looking at github code. My issue with that is that, while I have plenty of production Rails application code on my github account, those applications are all private and I legally can’t make them public. What I offer to prospective employers is to walk-thru my git Ruby code in a web meeting. But, before I have the opportunity to suggest the walk-thru, most companies have already pressed delete on my resume submission due.


What am I Going to Do About It?



This past Saturday I listened to a Ruby Rogues Rails podcast “187: Marketing Yourself as a Software Developer” (devchat.tv/ruby-rogues/187-marketing-yourself-as-a-software-developer-with-john-sonmez).
The podcast featured John Sonmez, author of the book “Soft Skills: The software developer's life manual” and creator of simpleprogrammer.com. The podcast told me what I already knew: I had neglected marketing myself for the past 6 years.


I have a blog, denoncourt.blogspot.com, but my last entry was December 12, 2010. And that post pre-dated my entry to the Rails market.
Why didn’t I keep up my blog? I was too busy billing 40-60 hours a week and having a blast writing cool Rails applications. One project turned into another and, after those projects completed, I was, for the first time in 5 years, in a situation where my word-of-mouth market failed to provide my next gig.


John Sonmez, in the podcast I had listened to, said that the standard reticence to author a blog is coders belied that they can’t write. Well. I can. I’ve published several books, over 200 articles, for several years I was a full-time editor of a technical magazine, and, in the early 2000s, I was also the editor of the highly successful Eclipse Tips email newsletter, which, pretty much, is the same thing as a blog.


Bottom line. I’m blogging again.


What am I going to blog about?


The more I thought about blogging on Rails topics, the more excited I got. Here’s a short list



  • Rails versus Grails: After doing Grails full-time for over 5 years and Rails for 4 years, it’s time for a comparison
  • Ruby gems that I’ve used to solve problems
  • Fill the void of of strategies for elegant development of CSS with SASS. 
  • IBMi-specific solutions to Rails development issues


I will, in a subsequent post, provide a more detailed list of blog topics I plan to cover.


The Gist of Open Source Git Code



What am I going to do about my lack of open source code on github? In association with Ruby and Rails blog entries, I will be building mini Rails applications that solve one problem. The Read Me for each of those applications will essentially be the content of the blog post where it walks you through the manual creation of the application. The completed version of the mini-application will, of course, be available for online review and download. This is a good idea even if no one ever uses my mini apps. It's a good idea because I’ll have an online resource that will show how I figured out a coding technique or the proper use of a Rails gem. But, in terms of marketing, those git apps will be searchable.

In the late 90s and early 2000s, while traveling around the country delivering Java seminars, I would often say: “I’m very good at writing trivially tiny applications that do but one thing -- teach a technique” Well, I have that skill and now it’s time for me to put it back to use.


The gist of programming: github also has a thing called gists. github, a the page https://help.github.com/articles/about-gists, says “Gists are a great way to share your work. You can share single files, parts of files, or full applications. You can access gists at https://gist.github.com.”
So should I have a technique that doesn’t warrant a full Rails application, I will write a gist.


Anyway, I’m back. I’m back in the blogosphere.


Monday, December 13, 2010

Grails AS400 RPG Connection Pool

In my last post I said: "For the most part, I did not have to do anything overly special to work with the AS400" in the development of the AS400-based Kettler USA retail site. The one area that took a bit of Grails-magic was some direct calls to RPG and some indirect calls to RPG via messages. The direct calls were done via JDBC stored procedures, so the standard JDBC connection pool set up by the Grails DataSource.groovy configuration worked fine. But for the message-base communication, I used AS400 data queues -- which are not supported by JDBC. So I needed to create an AS400 connection object. You really don't want to create an AS400 connection object for each request. Instead you want to use IBM's AS400 Connection Pool.

Once the AS400 Connection Pool is available, I'm able to use it in my service class like so:

class PreAuthorizationService implements Serializable {
def as400ConnPool
String preAuth(Cart cart){
AS400 conn = getConnection()
...

But where do you create the AS400 Connection Pool and when? Grails makes that easy.

In grails-app/conf/spring/resources.groovy place the following code:

import org.codehaus.groovy.grails.commons.ConfigurationHolder as CH
beans = {
if (CH.config.dataSource.driverClassName ==
'com.ibm.as400.access.AS400JDBCDriver') {
as400ConnPool(AS400ConnPoolFactory) {bean ->
ip = CH.config.iseriesIPAddress
userId = CH.config.iseriesUserId
password = CH.config.iseriesPwd
}
}
}

I put the IP, username, and password in Config.groovy (hence the use of ConfigurationHolder. Also note that, because I often use a local MySQL-based AS400-connection-less test environment, I predicate the creation of the AS400 connection so it is only created if the IBM driver was active. The IBM AS400 Connection Pool needs a bit of Spring infrastructure, which I coded in my AS400ConnPoolFactory wrapper class:

import org.springframework.beans.factory.FactoryBean
import org.springframework.beans.factory.InitializingBean

import com.ibm.as400.access.AS400
import com.ibm.as400.access.AS400ConnectionPool

class AS400ConnPoolFactory implements FactoryBean {
String ip
String userId
String password

public Object getObject() throws Exception {
// note: try/catch removed for brevity
AS400ConnectionPool as400ConnPool = new AS400ConnectionPool()
as400ConnPool.setMaxConnections(128)
as400ConnPool.fill(ip, userId, password, AS400.COMMAND, 5)
return as400ConnPool
}
public Class getObjectType() {AS400ConnectionPool.class}
public boolean isSingleton() {true}
}


It's all pretty simple. AS400 connection pools made easy by Grails!

Thursday, December 9, 2010

Grails 400 Experiences

I received a number of responses to my post about the launch of the retail site www.kettlerusa.com. I was happy to hear that there are folks out there that are either considering or using Grails on the AS400 (a.k.a. iSeries, System-i.) I will be doing a series of posts on my experiences Grails and the AS400.

For the most part, I did not have to do anything overly special to work with the AS400. DB2/400 supports all the features that Grails (via Hibernate) uses. The biggest issue was working with tables that have composite keys. Note that, although Grails has facilities for working with composite keys, rather than keying the required GORM code, I used the Systemi Grails Domain Plugin. This plugin has a script that takes an AS400 table and generates the Grails domain class. I have used that plugin (which I wrote with help from Mike Brown) to generate hundreds of Grails domain classes for tables on about a dozen different AS400s (all at different companies.) You can read more about the tool in my article Rapid IBM i Web Dev with Open Source Tools (note that article will require a web subscription to www.systeminetwork.)

KETTLER USA was deployed to a 64-bit Windows server that is running on Tomcat. The Tomcat server is behind a firewall with an Apache server running outside the firewall as a proxy server. The Tomcat server is able to connect to the AS400 via JDBC. The Tomcat server can only be accessed from the Apache server so it is a secure AS400 connection.

Most of the application is written using Grails domain classes to access DB2/400, but there are some points where the application does direct calls to RPG via JDBC stored procedures or indirect RPG calls via AS400 data queues. I will be elaborating on this in a future post.

I've been coding 12x7 for so many months getting the Kettler retail site up that I've forgotten some of the slick things that I was able to do with Grails (where coding the solution in Java would have been far more difficult.) But one thing, off the top of my head, that I loved being able to do was to put the Grails domain classes in their own Grails project and pull that project into the UI project as a Grails plugin. The reason why I needed to do this was that I had written a Business-to-business Grails application for Kettler last year. And the retail application used many of the same Grails domain classes. As they say "there shall be no duplicate code." And I certainly didn't want to maintain the 85 domain classes required for Kettler in two places. With one of the more recent versions of Grails (we are on 1.3.5 now) Grails made using your own project Grails plugins very easy. Instead of having to generate a Grails zip archive and import it to the project that requires that plugin -- everytime that supporting project is changed -- you simply put a reference to that project in your grails-app/conf/BuildConfig.groovy:

grails.plugin.location.'shared-domain-plugin' = "../SharedDomainPlugin"


Again, I will be added more Grails400 posts in the future.