History of the World, Part II: The Rise of Languages

Much like Mel Brooks’ movie, “History of the World, Part I” this blog post lacks historical accuracy and is tainted by personal experience and views. Basically I’m just brain dumping my thoughts on where languages are today, how they got there, and how they’ve affected me.

FORTRAN:

A great language for mathematics and science and a decent language for teaching. John Backus of IBM premiered this language in 1957. Strangely, continually revised versions of FORTRAN are still around today. I learned FORTRAN in college and what I liked most about FORTRAN was that I got an easy A while my brilliant older brother had to work his ass off to get a C. This was the first time that I thought maybe I could make a career out of programming.

Basic:

Basic came out of Dartmouth in 1964. It was a crappy language really only good enough for kids to learn how to program. It should have died decades ago but some guy called Bill Gates liked it and it morphed into a monstrosity called Visual Basic. I’m pretty sure the term “spaghetti code” was coined to describe Visual Basic applications.

RPG:

RPG is an IBM language that appeared in 1959. It helped catapult the S/3, S/36, S/38, AS/400, iSeries, and IBMi into a leading business platform for Fortune 100. The RPG language was originally developed so each statement could easily fit on an 80 column card. I remember, as an operator in the 80s, retyping statements in a card machine when programs would not load from the card-reader due to a bad card. RPG morphed into a modern language but the wealth of well-written 70s and 80s code haunts the RPG language and gives the development community the incorrect view that the IBMi/AS400 is an antiquated platform.

COBOL:

COBOL, like RPG, appeared in 1959. It was designed to read like the English language -- which did not work as it resulted in overly wordy statements and programs. But, I have to thank COBOL for two things:

1) COBOL programmers, because of their ridiculously long printouts monopolizing printers, forced me to learn how to develop (I was doing RPG at the time) without program listings. That’s what you did: Made changes, attempted a compile, printed the compile listing, reviewed code with a high-lighter, made changes again, and tried another compile. The only way I could get anything done was to go paperless.

2) During my two years of programming in COBOL,  I was pretty much forced to finally move from a hunt-and-peck two-fingered typist to a full ten-fingered typist at 25-30 words a minute (my sons were typing at 50-70 WPM by the time they were 15, but whatever.)

C:

For some reason, the C language fascinated me. I studied the crap out of it and was excited when IBM finally made a C compiler for the AS400. In fact, The first article I ever published was called “Freedom of Choice in C Compilers” it compared two C compilers that were newly available for the IBM AS/400 platform:

http://www.mcpressonline.com/programming/general/freedom-of-choice-in-c-compilers.html

The book “The C Programming Language” is arguably one of the best technical books ever written. Kernighan and Ritchie patterned their book after Strunk and White’s “The Elements of Style.” Both books are extremely thin and yet full of useful information. Many, many programming language books have attempted to emulate “The C Programming Language” with limited success. I’ve thrown out a dozen books on writing and dozens and dozens of books on programming yet these two remain on my shelf and are regularly referenced.

My research into C (and my article) got me a job as a systems-programmer at ASNA, where I worked for 5 years as a C and C++ programmer.

dBase III:

This great language was built for the IBM PC and appeared abou 1980. It forced me to learn modular programming and coerced me into learning better database design.

Assembler (MI):

Assembler has been around since forever. I did learn some MS/DOS-style assembler but I used the AS400 MI (Machine Interface) language on a daily basis during my 5 years as a systems programmer at ASNA. The best thing I can say about assembly language programming is that it made me appreciate C. I’ve met some folks that bragged to me about how they program in MI regularly and I would think “Why?” I mean, the last programming I did at ASNA was to replace all use of MI with C-API calls. There is rarely a need to use assembler level coding anymore.

C++:

I loved Stroustrup's monstrosity. And I got very good at it. I created a Windows source-level debugger for RPG and Cobol using C++ (that debugger was called Extermin8 for Windows.) I also worked with a team of C++ programmers at Circuit City that used an Object Oriented Database.

Question: Does Linus Torvalds write in C++?

Answer: No, he uses C.

Java:

Following Jimmy Fallon’s lead in thank you notes:

“Thank you Java, for wasting 10 years of my life while building the infrastructure for modern languages”

I’ve said in interviews that I’ve been doing Java since “before it was real.” I downloaded and played with IBM’s Java environment in 1996. At that point IBM was saying Java was going to be the next big thing. Lot’s of folks jumped on the bandwagon. But it wasn’t until the Java Servlet API was readily available in 1998 that Java became “real.”

I was considered a leading expert of Java on the AS/400 (a.k.a. iSeries and IBMi.) I published several books and hundreds of articles on Java. I was the Java Subject Matter Expert for Common (www.common.org/) I was the editor of both a Java print magazine called NetJava Expert, I was the editor of a popular email newsletter called Eclipse Tips. And I delivered week long seminars in most major US cities.

But, here’s what I think of Java now: It is a systems programming language. It is too complex of a language for business applications. Even if the developer is brilliant at Java, they would develop faster with a different language.

Java, however, gave us a lot including:

The Java Virtual Machine, with optimizations and excellent thread support

Testing frameworks

APIs for most anything you can think of from email services to search optimizers

A platform for dozens of other languages, from Scala to Rhino.

PHP:

Or, as it was first called, Personal Home Page. PHP was developed as a thin veneer on top of the Perl language to simplify web development. I liked Perl so I gave PHP a several year whirl. And I still try stay on top of it. I did a series of articles for IBM Developerworks on advanced PHP:

Leveraging PHP 5.3's lambdas and closures:

http://www.ibm.com/developerworks/opensource/library/os-php-lambda/index.html

Leveraging PHP V5.3 namespaces for readable and maintainable code

http://www.ibm.com/developerworks/library/os-php-5.3namespaces/index.html

PHP V5.3 invigorates object-oriented programming with late static binding

http://www.ibm.com/developerworks/library/os-php-53static/index.html

Nevertheless, I feel PHP is a sloppy language. It does have many great frameworks but there are so many of them that the developer base has splintered. I think PHP is the VB of the Web. It is all to easy for folks to write bad code in PHP. I’m thinking Kent Beck, however, has written some fine PHP as a contractor for Facebook. So you certain can do great things with PHP and, as a dynamic language, you can do things easily that are next to impossible with static language like Java.

Here's an article I wrote called "Get Real with PHP" where I prescribed strategies for development: 
http://iprodeveloper.com/rpg-programming/get-real-php

Groovy:

Groovy is a dynamic language that runs in the JVM. I love the Groovy language. A Groovy source file compiles to become a Java class -- so it will run in any Java container. When the Grails framework (a Ruby on Rails wannabe that is build using Groovy) became available, I immediately focused all my work on Grails. I was able to coerce a dozen or so clients into allowing me to implement Web apps for them in Grails. In fact, the first Grails app I developed, 8 years or so ago, is still running fine.

The Groovy language is very elegant and, of all the languages that run on the JVM, Groovy integrates the best with existing Java classes. I found that I was able to write 50-80% less code with Groovy than it would have taken with Java. And I was also able to develop much more quickly with Groovy and Grails than with Java and standard Java frameworks.

I’ve published a bunch of articles on Groovy and Grails and here’s one I did specifically for Amazon Web Services:

Building an Amazon S3 Browser with Grails : Articles & Tutorials : Amazon Web Services

http://aws.amazon.com/articles/Amazon-S3/4000

Ruby:

Ah, Ruby, this has been my current love for about 4 years now. As I said, I did Grails for many years but, all that time, I still had Rails-envy. But as soon as I was able to coerce a client into using Rails instead of a Java, I never looked back.

Ruby was written for programmers that love to code. It allows you to do more with less. You can do things in Ruby extremely easily that would otherwise take hundreds of lines of code and the use of complex design patterns in Java. With Ruby, you can do things that you had thought of while using Java but never did because it would have taken too much time.

Ruby has a number of environments. One is MRI: Matz Ruby Interpreter (Matz is the creator of Ruby.) MRI is also known a CRI as it is a C language implementation of a Ruby environment. One example of RI alternatives is IronRuby with is Ruby for the .Net platform. But the most interesting alternative RI, to me, is JRuby. JRuby is very successful because it runs on top of the heavily optimized JVM. And that JVM provides true threads to Ruby applications -- which is something that is lacking in the other Ruby Interpreters. JRuby also allows you to easily integrate with Java APIs. The first time I used JRuby I did so because I needed to use a JDBC driver to access a database that did not, at the time, have a Rails ActiveRecord adapter. I wrote about JRuby in “Get on the Fast Track with Rails”

http://iprodeveloper.com/application-development/get-fast-track-rails

Rails is the best web application development frameworks I’ve used. And I’ve used dozens. Rails brought us Convention-Over-Configuration (CoC) and a myriad of other well-thought out concepts.

To learn more about CoC you can read my article:

(http://iprodeveloper.com/system-i-network/coc-frameworks-php-and-java)

What next:

Functional programming languages?

Tell me about your language experiences.

AWS SES Mini-App with Bounce Notification Handling

As promised in my last blog entry on http://denoncourt.blogspot.com, this my first in a series of instructive Rails mini-apps. The purpose of each application will be to illustrate the use of one technique, feature, or utility. And the README of that application will provide instructive details (https://github.com/dondenoncourt/aws_ses_bounce_mini_app). [And, actually, this blog post displays better on github so I suggest you click on the prior link and continue reading.]
Recently I had to code a Rails application to handle bounce notifications from AWS SES. Amazon Simple Email Server is ridiculously easy to configure and use in a Rails application. As the instructions at the github gem page, https://github.com/drewblas/aws-ses, explain, all you need do is add the aws-ses Rails gem and create a configuration file called config/initializers/amazon_ses.rb that contains your Amazon credentials (soft-coded of course.)
But, if you need your application to handle bounce, complaint, or delivery notifications, things get a little more complicated. It took me a bit to figure out how to properly configure AWS-SES bounce notifications and to write the code required to handle AWS callbacks. So I figured it would be helpful if I built a mini-app with a README that details the process.

Bounce Notification Flow

Let me walk through the two-step flow of bounce notifications:
First: your Rails mailer sends an email and, subsequently, AWS-SES fails to be able to deliver that email -- perhaps because the email address was incorrect, or maybe the mailbox was full, or it could have failed for a couple of other reasons that are covered in http://docs.aws.amazon.com/ses/latest/DeveloperGuide/notification-contents.html#bounce-types.
Second: because you have configured AWS-SES to perform callbacks to your application when bounces occur, AWS-SES sends a POST to your application with a JSON payload that contains an array of bounced email addresses.

Handling AWS SES Callback Confirmation HTTP Post Request

The first part of that second step, configuring AWS-SES bounce callbacks, is easier said than done. The AWS configuration screens are confusing and, as I found out the hard way, to make things easy, your application should be coded to handle notifications before you do the AWS-SES configuration.

Mini-App’s AWS-SES Bounce Confirmation Handling

My mini-app’s routes.rb includes the following:

  get 'mail_it' => 'simple_mail#mail_it'
  post 'aws_sns/bounce' => 'simple_mail#bounce'

And here’s the controller code:

class SimpleMailController < ApplicationController
  skip_before_filter :verify_authenticity_token # so AWS callbacks are accepted

  def mail_it
    logger.info "mail_it called with #{params}"
    @email = params[:email]
    SimpleMailer.mail_it(@email, 'original text').deliver
    render text: 'mail sent'
  end

  def bounce
    json = JSON.parse(request.raw_post)
    logger.info "bounce callback from AWS with #{json}"
    aws_needs_url_confirmed = json['SubscribeURL']
    if aws_needs_url_confirmed
      logger.info "AWS is requesting confirmation of the bounce handler URL"
      uri = URI.parse(aws_needs_url_confirmed)
      http = Net::HTTP.new(uri.host, uri.port)
      http.use_ssl = true
      http.verify_mode = OpenSSL::SSL::VERIFY_NONE
      http.get(uri.request_uri)
    else
      logger.info "AWS has sent us the following bounce notification(s): #{json}"
      SimpleMailer.mail_it('dondenoncourt@gmail.com', json).deliver
      json['bounce']['bouncedRecipients'].each do |recipient|
        logger.info "AWS SES received a bounce on an email send attempt to #{recipient['emailAddress']}"
      end
    end
    render nothing: true, status: 200
  end

end

The controller’s mail_it method is self-explanatory. It take a url like:
/mail_it?email=dondenoncourt@gmail.com
and responds by sending an email to the specified address. It is the bounce method that needs a bit of explanation. As I mentioned earlier, to make AWS-SES configuration simple, your application should be coded to respond to an AWS-SES bounce callback confirmation request before configuring AWS-SES. Let me explain: When you configure AWS-SES bounces, you provide AWS the URL of your application. AWS will put that bounce configuration in a pending status until it is able to send a confirmation request to your application and gets a positive response. Anyway, I recommend that you add the route and the controller action and have the app running so it is ready to handle the AWS POST HTTP request -- before you configure AWS-SES.

Heroku

Note that I put my AWS-SES Mini-App on Heroku mostly because AWS-SES callback needs to be able to POST to the URL of an addressable server. My good old localhost:3000 wouldn’t work without doing some router configuration that I wasn’t willing to do.

Configure Verified Senders

When your application is ready to respond to AWS-SES callbacks, it’s then time to configure AWS-SES. The first step is to add and verify sender email addresses. From the AWS menu, select Services and click on SES.

Then, on the left panel of the SES page, click on the Email Addresses link and, in the center panel, click on the Verify a New Email Address button. Enter the address and click the Verify This Email Address button.

The following popup will be displayed:

Note, in the background, in a highlighted color, that the Status of the email address is “pending verification.”
An email will be sent to that address with the subject of “Amazon SES Address Verification Request.” The body of that email will say:
Dear Amazon Simple Email Service Customer,
We have received a request to authorize this email address for use with Amazon SES in region US East (N. Virginia). If you requested this verification, please go to the following URL to confirm that you are authorized to use this email address:
After you click that link the status of the Verified Sender:Email will go to verified.

Add the Bounce Topic

With the email addresses verified, the next step is to create a bounce topic. From the AWS menu, select Services and click on SNS.

Click the SNS Dashboard link on the left panel and then click the Create New Topic button in the center panel. Enter Topic and Display Names of “bounce” and then click the Create Topic button.

In the subsequent panel Click the Create Subscription button and key an endpoint name that matches your application’s bounce route and click the Subscribe button.
You will see a pop panel that says:
“Subscription request received! A confirmation message will be sent to the subscribed endpoint. Once the subscription has been confirmed, the endpoint will receive notifications from this topic. Subscriptions will expire after 3 days if not confirmed.”

Click the Close button on that popup and note the SubscriptionId column on the page still says “PendingConfirmation.” Click refresh and, if your application was available to successfully respond to the URL specified in the bounce endpoint, the SubscriptionId should be set to a value like:
arn:aws:sns:us-east-1:294894041652:bounce:30e9ca4b-0723-4078-86a5-0d1d2573d101

Selecting the Simplified JSON Notification Format

To get a simplified version of the AWS-SES bounce JSON string I (which is also the format I expect in SimpleMailController) I changed the raw type. You should do that as well by clicking on the Subscriptions link in the left panel, then clicking the checkbox of the bounce topic in the center panel, and, in the popup, click Raw Message Delivery True, and click the Set Subscription Attributes button.

Enabling Bounce Notifications

Next you need to assign the newly created “bounce” SNS topic to your SES Verified Senders. From the AWS menu, select Services and click on SES. Then, on the left panel, click on Email Addresses under Verified Senders. You had previously entered an email address here but, this time, we need to specify that emails sent through the Verified Senders will have bounce processing enabled.

Then click on the Verified Sender that you use as your application’s from address, expand the Notifications twirly, click on the Edit Configuration button, and, in the select box for Bounces, pick the ‘bounce’ topic you created earlier. And click the Save Config button.

Note that you can configure bounce notifications by domain as well as sender email address. Also note that, after a successful bounce notification configuration, AWS post to your bounce handler an “AmazonSnsSubscriptionSucceeded” bounce notification.

Test Bouncing

The free and default version of AWS SES only lets you mail to verified address. So it is a bit of a problem to test bounces. No problem, though, as AWS provides a set of test email addresses you can use: (http://docs.aws.amazon.com/ses/latest/DeveloperGuide/mailbox-simulator.html)
That list contains an email that you can use to tests your bounce code. From my app I used:
https://fast-cove-3541.herokuapp.com/mail_it?email=bounce@simulator.amazonses.com
The simple_mail_controller#bounce method handled that bounce by sending myself an email the body of which contains the following ASW-SNS bounce JSON string:

{"notificationType"=>"Bounce", "bounce"=>
{"bounceSubType"=>"General",
 "bounceType"=>"Permanent",
 "bouncedRecipients"=>
 [{"emailAddress"=>"bounce@simulator.amazonses.com", 
   "status"=>"5.1.1", 
   "action"=>"failed", 
   "diagnosticCode"=>"smtp; 550 5.1.1 user unknown"
 }], 
"reportingMTA"=>"dsn; a8-34.smtp-out.amazonses.com", "timestamp"=>"2015-02-07T17:40:39.338Z",
 "feedbackId"=>"0000014b65210ac9-b9f36242-8ade-413e-8597-1112a631244f-000000"}, "mail"=>{"timestamp"=>"2015-02-07T17:40:38.000Z", 
"source"=>"dondenoncourt@gmail.com", 
"destination"=>["bounce@simulator.amazonses.com"],
 "messageId"=>"0000014b652108a4-38938047-2f1b-4d2b-a1ca-28b58ed6fdd5-000000"}}

The JSON contains an array of bouncedReceipts. My bounce method did not really “handle” the bounce in a useful manner. All it does is log the email addresses. Your application would, undoubtedly, do something a bit more useful.
Amazon lists the JSON structure for bounce notifications at http://docs.aws.amazon.com/ses/latest/DeveloperGuide/notification-contents.html. You probably will want to look at the bounce types and process their handling accordingly.
In a production application, I coded RSpec bounce handling tests with JSON built from the AWS samples.