Gliffy updated their site!

December 09, 2008

Though I'm no longer working for Gliffy, I'm excited to see that they updated their site with some of what I worked on! Awesome!

Specifically, I worked on the folder organization system that they added to replace the tagging system. This fell out of the API work that I did (which I'm assuming is in private beta right now, but I'm not sure). I also worked on a feature that, while painful as a developer, is my favorite new thing about Gliffy: the basic account no longer has a five-diagram limit! That means for free, you can create unlimited diagrams. The catch is that your diagrams are all public, but I think it's a great way to enhance the functionality while subversively getting their name out to more people.

EMMA and TestNG for Simple Java Code Coverage

December 07, 2008

Although RCov makes code coverage in Ruby dead simple, I wasn't sure how easy this would be to achieve with Java. The first free tool I found is called EMMA and it was surprisingly easy to setup, especially since the documentation isn't geared toward getting coverage during tests (but getting it during execution).

EMMA works by instrumenting the classfiles to analyze coverage. Although it can do just-in-time instrumentation, that didn't seem to work for recording coverage via TestNG. The offline instrumentation makes is pretty easy to use with anything. Basically, you want your ANT file to:

  1. Compile your code
  2. Use EMMA to instrument your classes to a different directory
  3. Run your tests, using the instrumented classes first in your classpath
  4. and passing a few system properties to your running code
  5. Run EMMA's report generator on the output
At first, I was getting some runtime errors because interfaces are not instrumented (and don't show up in the location you tell EMMA to put them). The solution is to put both your instrumented classes directory and your regular, non-instrumented classes directory in the classpath, making sure the instrumented ones are first.

Here's my test.xml I'm using in my fork of ImportScrubber that shows it all working together. All in all, it only took about 15 minutes to set up and debug. Of course, now, the tests that came with ImportScrubber provide almost no coverage, but that's another story....

I can haz job

December 01, 2008

So, I am finally employed and I didn't even have to settle. After a refreshingly protracted and detailed interview process, I'm finally schlepping myself to a job that I'm more or less excited about. That's saying something, since I've spent the last 8 months at home (6.5 of them working for Gliffy) in my perfect environment: waking up whenever, using my dual-monitor mac, Rudy close by. My first day was a net win, despite having to bring in my own computer, and overall I'm not complaining because I get to use a Mac at work thank GOD.

Pluses so far:

  • Smart people I can have a conversation with
  • Meaningful product (i.e. not another CRUD app for a government agency [not that there's anything wrong with it])
  • Not only have they heard of javadoc, they use it!
  • Database migrations!
  • Clean looking code and tests that actually pass on a fresh checkout!
  • No M$ exchange server or other shitbox mail system (they use Google Apps)
  • Damn close to home; I should be biking in real soon
  • Relaxed environment
  • I'm one person away from a bonafide window with the shades open!
Honestly, it's almost a 100% on my interview rubric (which I took down for a while, because some HR person read it and gave me shit about not liking having a dress code. I mean, does anyone really like putting on a suit and tie to site and write code? Or to do anything? We're talking levels of tolerance, and mine is low, mostly because I believe dress codes indicate a deeper organizational problem of priority management).

Negatives so far:

  • Kinda noisy office (fortunately few people seem to have phones)
  • Subversion (it looks like they aren't going nuts with branches, so git-svn should preserve my sanity in this regard)

On the fence so far:

  • Maven - The only reason this isn't a negative is that it's better than the pile of shit ant script everyone else uses, and the build does work pretty painlessly.
  • Spring - I haven't used Spring for anything real, and I can't say it gets me excited (nor have I ever thought it sounded all that great), but I'm optimistic about it. I figure if it, in fact, is great, I'm happy. If it sucks, I have fodder for ranting. It's a win/win. I do fear the XML situps tho.

Ruby's awesomely dangerous awesomeness

October 31, 2008

While packaging my resume creator/formatter as a Ruby Gem, I noticed my command line parser was preventing my code from running. It has this hook that sets up an "auto run" sequence (your code just defines a main method):

def self.inherited(child_class)
  @@appname = caller[0][/.*:/][0..-2]
  @@child_class = child_class
  if @@appname == $0
    __set_auto_run
  end
end
the problem is that $0 from gem is the /usr/bin location and @@appname is the actual location in the Gem repository. Not sure why this check was there, but I had a problem.

Not feeling like overhauling the command line interface code and not being able to patch OptionParser, I opted instead to use the dangerous-sounding ruby feature of replacing the method with my own implementation:

class CommandLine::Application
  class << self
    alias old_inherited inherited

    def inherited(child_class)
      @@appname = caller[0][/.*:/][0..-2]
      @@child_class = child_class
      normalized_appname = @@appname.gsub(/^.*\//,"");
      normalized_dollar0 = $0.gsub(/^.*\//,"");
      if normalized_appname == normalized_dollar0
        __set_auto_run
      end
    end
  end
end
Now that's pretty cool. My code just dynamically patches the broken Gem! The power of open source and dynamic languages. Of course, this feature allows so much awful code to be written....

Interview Rubric really needed?

October 24, 2008

It's been a few weeks of job hunting and I haven't once had to make a decision that my post on interviewing would've helped resolve. The fact is, you get a feel almost immediately for a place, and the reasons you say "yes" or "no" have more to do with who you'll be working with than the number of monitors you get. Usually, a place that's using ancient COTS products has put up red flags far earlier in the process. And the jobs I get excited about have nothing to do with their use of Git.

Some lowlights:

  • Left waiting for so long, I had to walk out of the interview in order to meet an appointment. The receptionist explained "Well, they have a lot of work to get done!" Good luck with getting it done.
  • A guy giving a tech interview (for a J2EE position) couldn't understand how a Swing front-end to an EJB/JPA backend could possibly be called J2EE.
  • Being asked a logic question and, as soon as I used the right word while thinking out loud (in this case "tree"), was cut off and we moved to the next question. After 10 minutes of this, he walked out without telling me anything about the company or job.
  • Being offered a job after having No technical questions asked of me. Gee, who else is working there?
  • Interviewing in a place where every single aspect of the work environment was crappier than the crappiest house I've ever lived. If I can afford a fresh coat of paint every few years, shouldn't some "global enterprise solution consulting firm" be able to swing it?
  • A Java development shop using....Visual Source Safe. I think cp Foo.java Foo.java.bak might be better
Of course, there's been some legitimate highlights as well:
  • Being asked some challenging questions about concurrency and data structures. You may not ever have to implement a linked-list, but anyone should know it and when I'm asked, it's a definite plus that the people on the other end know what they are doing
  • Being asked to write code. So far, exactly two positions have asked me write code in the interview. Thank god for them, or my faith would be shaken; seriously. I'm always very nervous when I'm applying for a job where I have to write code and no one seems to need any proof that I can do it. It makes me wonder who else is working there
  • Solid explanations of the business or project. I used to think this was a no-brainer, but more often than not, I come away from a second interview with NO IDEA what I'd be doing if I took the job.

Criteria is only useful when you have to narrow down a lot of choices. I'd love to have so many great opportunities that I could just pick the one where I can develop on a Mac, or the one with the nicest office (all other things being equal). Sadly, that is not the case around here. It seems very few of my colleagues are being too particular, and I can't help wondering what effect it might have on, well, the world if clueless developers were not as employable as it seems they are.

Things I'd like to leave behind

October 14, 2008

  • Subversion - Git is so much better in so many ways (it goes without saying that CVS should be allowed to die)
  • JUnit - TestNG does all that JUnit does and more; what does JUnit even have to offer these days?
  • Java 1.4 - Java without generics is just so much pain
  • Java Web Frameworks - As far as I can tell, none of them adequately address one of the fundamental problems in web development, which is to simplify the creation of the UI.
  • Ant - Ant has always been the world's worst build automation language; who creates a build tool without variables, loops and conditionals? We can do better (and Maven doesn't appear to be it)
  • Checked exceptions - java.lang.Exception is possibly the worst class in the Java library; should never be caught nor thrown1
  • Misuses of XML - XML has a purpose, and it's not as a programming language or configuration file format. Any notion that XML is anything other than a binary format is misguided

1API developers should be allowed to declare that they throw Exception to allow subclasses to throw whatever they want with impunity, however Throwable is the preferred thing to catch for catch-alls, and the entire exception mechanism in Java is woefully broken.

Ruby and dead simple code coverage

October 14, 2008

I haven't used a code coverage tool for Java, but in my spare time I've been working up some Ruby code (mostly to learn the language). I'm using Test Driven Development, which is slightly simpler with Ruby than with Java (mostly due to Ruby's interpreted nature).

I had heard about rcov and decided to give it a shot. Within literally 5 minutes I had it installed and a report showing my tests were not covering all my code! Amazing. I could then easily see exactly where I need to to test and, sure enough, found some bugs that would've gone unnoticed.

Even the best tool with Java would've required some painful ant tweaking (not to mention hopes and prayers that it worked with TestNG). I already cannot imagine writing tests without being able to view the coverage....

Interviewing the Interviewer: A Rubric

October 07, 2008

Sad to say, my time at Gliffy is at an end (:sniff:), so I'm heading back into the job pool. I was lucky to get some time in at Gliffy, because, living in Washington, DC, my opportunities for sexy cutting-edge jobs are about zilch. Instead, I'm facing a huge market of "Senior JBoss Portal Maintenance Archiect" type jobs.

I guess I should feel lucky that there's lots of positions out there, but I really don't want to be the cog in a huge machine. Gliffy has shielded me from the Horror That Can Be Consulting, so I need to keep my perspective in such trying times. So, calling on my experiences before Gliffy, I've made this handy rubric to make sure I explore all facets of a potential position.

I don't expect anyone to get all positives and no negatives (I've certainly never been anywhere that perfect), bit it's always good to know. For example, if I have to put up with PVCS, I better be sitting on an Aeron chair and be using a normalized database. Further, this is obviously in addition to standard questions regarding what the project is about (i.e. is it interesting) and what the people are like. A whole lot of this can be forgiven by being part of a great team or working on a really cool product.

QuestionPoints forPoints against
How do you fare on the Joel Test? · High score
· Good explanations for missing items
· Low score
· Never heard of it
Describe your development process · Structured
· Easily described
· Overly draconian
· Lack of
· Not easily described
What kind of computer will I be using to develop? · Two monitors
· Mac
· Linux
· Administrator access
· Vague answer
· Small monitor
· Windows
· Locked-down
Do you block certain sites or applications on your network? · Open network · Closed network
What is the physical environment like? · Good Chairs
· Private office
· Natural light
· Reasonable Temperature
· Old, crappy
· Bullpen style
Am I required or encouraged to use Windows? · No, few devs use Windows · Yes
What collaboration tools do you use? · Wikis
· IM
· Bug tracking
· Sensible PM
· Email word documents
· MS-project
· SharePoint
· Other proprietary crap (e.g. eRoom, Documentum)
· No tools
What are some of your HR policies? · Few, if any
· Loose dress code
· Flexible schedule
· Draconian
· Dress code
· Core hours
Can I see the code I will be working with? · Letting me see it
· Meaningful javadocs/API documentation
· Structured, consistent style
· Sensible class names and file organization
· Sane build process
· Not letting me see it
· Mix of styles
· No javadocs
· Empty javadocs
· Convoluted file organization
· Broken build file
How do you do testing? · Have testers
· Do unit tests
· Maintain tests
· TDD
· Bug tracker
· Ad hoc
· Lip service to test-first
· No unit tests
What is your approach to configuration management? · Having an approach
· Git
· Database migrations
· Know the versions of 3rd party software/have a baseline configuration
· Organized
· No approach
· CVS
· Perforce, ClearCase, other closed crud
· Shared drives
· Can't describe versions/configuration
Can I see the database schema I'll be working with? · Letting me see it
· Normalized
· Sane names (no TBL_* bullshit)
· Synthetic numeric keys
· Referential integrity
· Documented!
· Versioned!
· Not letting me see it
· Unnormalized
· Dumb names
· Incorrect types
· String-based keys
· No constraints
· Undocumented

What else am I missing?

Are you emailing yourself your log errors? You should be.

September 26, 2008

Time and time again, users complain about an application crashing on them or otherwise not working. They don't provide you any info and it's hard to repeat. You check out the log, but there's thousands (or millions) of entries and you have no clue where their error occured. Worse, if you are deploying a RIA, the log may be on their computer and not available.

On my last project we experienced this scenario so much that we instituted two things

  • All messages logged with Level.ERROR in log4j would be emailed to us
  • All exceptions caught on the client would be packaged and sent back to the server and logged at Level.ERROR level (thus emailing them to us
After the initial deluge of emails, we found a lot of bugs. I mean a lot of bugs. The annoying, intermittent kind that are hard to reproduce. Further, by judicious use of logging, we discovered a lot of mis-configured environments and other problems without having to get users to mail us their logs.

At Gliffy, they are doing the same thing. Right now, we're testing a bunch of new features and the stage instance just sent me a bunch of emails, all indicating configuration problems, which is the exact kind of thing that can be hard to track down.

Setting it up using log4j is dead simple:

log4j.appender.mail=org.apache.log4j.net.SMTPAuthenticateAppender
log4j.appender.mail.SMTPHost=@SMTP_HOST@
log4j.appender.mail.UserName=@SMTP_USER@
log4j.appender.mail.Password=@SMTP_PASS@
log4j.appender.mail.Authenticate=true
log4j.appender.mail.From=errors@gliffy.com
log4j.appender.mail.To=@SMTP_LOGGER_FAILURE@
log4j.appender.mail.Subject=Errors from @SMTP_DESC@
log4j.appender.mail.BufferSize=1
log4j.appender.mail.Threshold=ERROR
log4j.appender.mail.LocationInfo=true 
log4j.appender.mail.layout=org.apache.log4j.PatternLayout 
log4j.appender.mail.layout.ConversionPattern=%d %p%n%t%n%c:%M:%L%n---%n%m%n---%n%n 
In my previous job I even created a customized layout to format the emails in such a way that our code was highlighted and GMail didn't compress things into threads.

If you aren't doing this, you should be. Now.

Getting Rake's PackageTask to depend on generated files

September 17, 2008

Been playing with Rake lately and decided to use it to package up the PHP Client Library for the Gliffy integration API. Didn't seem to make sense to use ant for something that amounts to creating a tarball. make would be appropriate here, too, but I figured it would be cool to use Rake and there's not really much harm in doing so. A large annoyance is Rake::PackageTask. This is a seemingly handy task that creates tars, zips, etc. and is pretty useful. It's not a task in and of itself, but it creates the :package task:
Rake::PackageTask.new("gliffy-php-client",GLIFFY_VERSION) do |p|
    p.need_tar = true
    p.need_zip = true
    p.package_files = SRC_FILES + EXAMPLE_FILES + DOC_FILES
end
Unfortunately, this doesn't do what it seems to do. DOC_FILES is the list of documentation files output by phpDocumentor and are not checked into version control. The syntax of the PackageTask makes it appear that the code in the block will run when the :package task executes, however this is not the case. This code is initialization code. So, I tried:
# DOC_DIR is the dir generated by phpdoc, a task elsewhere

# uses this to kick off phpdoc

task :package => DOC_DIR
The result is that the tarball and zip files are created and then the documentation is generated. The reason is that PackageTask.new creates a set of tasks and the actual creation of the tarball/zip file is done via a file task, upon which :package is dependent. So, the real dependency I created was:
task :package => "gliffy-php-client.zip" "gliffy-php-client.tgz" DOC_DIR
Examining the source code, a task named for the directory created by :package is created. This task is dependent on the package_files set up in the constructor. So this is the task I need to use:
# Have to keep a reference to the PackageTask object

package_task = Rake::PackageTask.new("gliffy-php-client",GLIFFY_VERSION) do |p|
    p.need_tar = true
    p.need_zip = true
    # Executed BEFORE any other tasks; DOC_FILES don't exist  yet

    p.package_files = SRC_FILES + EXAMPLE_FILES
end

file package_task.package_dir_path => DOC_DIR

file DOC_DIR => SRC_FILES + EXAMPLE_FILES do |t|
    system("phpdoc #{PHP_DOC_ARGS}");
    doc_files = FileList.new(DOC_DIR + "/**/**");
    # Have to add these files to the package_task file list

    package_task.package_files = package_task.package_files + doc_files
end
This is definitely a hack, because I'm depending on the internal implementation of the PackageTask. It really needs a facility for including generated files. In make, I could just send the directory DOC_DIR to tar and it would pick up everything. In Ant, I'd probably have to spawn another ant, since ant sets all property vaules at startup time.