Dependency Inversion Principle

Dependency Inversion is one of the five SOLID OO principles that’s become so popular in recent years (sometimes referred to as DIP). My opinion is that it’s a highly valuable concept, and is not well-named. “Dependency inversion” doesn’t mean a great deal on its own, and a lot of attempts to explain it tend to get very heady.

(Speaking of unfortunate naming, “hexagonal architecture” is another concept that I find poorly named. Ironic that the concept is about using abstraction to avoid unnecessary coupling, yet the name itself couples the idea to the number 6, which is totally unrelated to the idea. Fortunately, it’s slowly being renamed as “ports and adapters” in most discussions, which is a much better description.)

My personal preference is towards explanations that are more intuitive, and I’d like to put my two cents in for Dependency Inversion.

To me, a way of describing Dependency Inversion is to use classes to separate the features of the application (the parts that a user might use) from the technology that makes the feature work.

For example, let’s say you have a feature where a user can sign themselves up, and once the user’s information is saved to the database, you have some more steps that need to take place. You could make an interactor called


and it will handle the actions necessary (instead of using a callback).

Let’s say one of the things it needs to do is send an email welcoming the user. Let’s also pretend that you use SendGrid to manage emails. That would mean we could end up with something like

class CompleteUserSetup
  def self.perform "subjectline", "body of email"


Will this run? Absolutely. Is it a good example of code that meets the criteria for Dependency Inversion? No.

What’s happening here is that the feature (user setup) is directly mentioning the technology that implements it (SendGrid). It literally has the name of the tool in the code that is defining the feature.

Literally Rob Lowe

Since there are probably lots of places in the code that send email, if you ever need to switch away from SendGrid, you need to change all those places. Usually it’s not as simple as find-replacing all the instances of “SendGrid” from the code, so a better way is to abstract all of the email interactions into your own class.

class CompleteUserSetup
  def self.perform

class OurEmailClient
  def self.send_welcome_email "subjectline", "body of email"


Now the feature doesn’t mention the implementation technology. If you kept this pattern going, “SendGrid” would only ever appear in your email client class, and this means that the technology would be decoupled from the features. This is an Adapter pattern, which is basically the first half of Dependency Inversion. (IMHO this is the most common implementation of DI, and this makes testing really easy)

To close the loop of “Dependency Inversion” is to actually pass the client an instance of an email client so it can use more than one.

class CompleteUserSetup
  def self.perform(client)

class OurEmailClient
  def send_welcome_email 'subjectline', 'body of email'

class OurOtherEmailClient
  def send_welcome_email 'subject, 'body of email'

email_client =

# or

email_client =

Now we’re actually passing the dependency into the location where it’s needed. This means the CompleteUserSetup interactor is totally decoupled from which messaging system it will use.

The reason this matters is that now we can choose any type of email provider we currently support, and we can also add new types of email providers that we didn’t previously use.

To be fair, in Ruby, this isn’t quite the same as in Java. Ruby will allow duck typing, which means that we don’t have to write an Interface. Also, Ruby doesn’t support Interfaces at all, so that aspect of this is missing. Still — the principle behind decoupling does make it very easy to write readable code, and testable code, and I always love that.

How to specify the schema using Spark’s Java client

I’ve been working with Spark recently. It’s awesome. Unfortunately, the Spark community leans towards Scala, but the Java client was the best choice for our team. This means that sometimes the documentation and examples out there aren’t great. One thing that took me longer than I liked was figuring out how to specify the schema for the data.

Spark has a nice default behavior where it will infer the schema of your data when you load it. The trouble with this is that if you change the data over time (adding new attributes for example), you can run into issues with your code only working with certain versions of the data and not others.

Fortunately, you can specify the schema, so that the fields will exist as nulls. I found lots of examples for how to do this in Scala, but it was hard to find examples in Java. So, here’s how:

Let’s pretend the following file is the schema of `whatever.json`

  "address":{"street":"Spear St","city":"San Francisco"},
  "rank": 100 

This would correspond to the following code:

SparkConf conf = new SparkConf().setAppName("MyFunApp");
JavaSparkContext sparkCtx = new JavaSparkContext(conf);
HiveContext context = new HiveContext(;
String sourceUrl = "whatever.json";

StructType schema = DataTypes.createStructType(Arrays.asList(
  DataTypes.createStructField("company_name", DataTypes.LongType, false),
  DataTypes.createStructField("address", DataTypes.createStructType(Arrays.asList(
    DataTypes.createStructField("street", DataTypes.StringType, true),
    DataTypes.createStructField("city", DataTypes.StringType, true)
  )), true)

DataFrame changesRaw =;

The third createStructField param is “can the value be null?”

How to hire engineers

From IRC:


There’s a saying that you are the average of the 5 people you surround yourself with. If we apply that logic to hiring, it means that hiring is really important (but we knew that already).

My friend’s question was an interesting one. As an engineer, I know about writing code,  and engineering fit. If I put my bizdev/marketing hat on, I start to think about the process of interviewing, and how to best allocate my resources (time) in order to qualify a target and convert them (hiring).

There’s one concept that I think is important to share with my engineering friends is the idea of a conversion funnel. The gist is that in a lot of business situations, you have a group of people, you have an action you want some percentage of those people to do, and you have a series of steps that they go through (the stages of the funnel).

A (hilariously simple) conversion funnel for Apple might be

  1. Find out about new iPhone (begin funnel)
  2. Read about new iPhone on internet
  3. Try new iPhone in store
  4. Purchase iPhone (end of funnel)

Hiring engineers (or any role) can be thought of in this way. For example:

  1. Create awareness of your company/the job. This could be through blogging, buying ads, or posting on Craigslist.
  2. Create enough desire to get the person to submit an application. This could be talking up perks, describing the work environment and technologies used, or having a really great application process. Anything that makes your company attractive to work at goes in here.
  3. Review applications/resumes. This is early in the funnel, so you want to spend very little time on. Are they worth any time at all or are they totally not a fit? I recommend having a non-interviewer conceal the person’s name from the reviewers. There is strong evidence that people with good intentions can have something as simple as a name affect their judgement.
  4. Send them a screener problem, and review answers to it. Ideally you should be able to decide to move to the next step (or reply with “no thanks”) with about 15-20 minutes of effort. Again, do this with names concealed, the goal is to focus totally on the code.
  5. 1-2 hour interview. Be sure to talk about their screener problem and understand their engineering sense. Also, you should start getting a sense of what the person might be like to work with, but try and stay open minded about this one until the next step. I like to start the discussion by asking the person how they feel about interviewing — if someone is nervous about being on-the-spot, spending a minute or two to talk about that feeling of nervousness can help them get rid of the feeling so they can focus, and this means you will get a better picture of what they are really like. Also, I typically do this interview via Skype, and ask them to screenshare with me and write a blog using whatever tools or resources they normally would. You can quickly learn a lot about an engineer by spending 15 minutes watching them code in their own comfortable environment.
  6. 4-8 hour pairing session. This should be in-person, unless you’re hiring a remote engineer. The best way to find out what someone is like to work with is to work with them under the most realistic circumstances possible. If possible, ensure that you’ll encounter specific scenarios so that you can gauge skills consistently from one candidate to another. The more objective you can be here, the better your results will be.

Notice that the intent is to minimize effort at the beginning of the process, and do the more intense quality screening at the end.

Above all — customize your funnel in a way that makes sense for you and your situation. There’s not necessarily a right or wrong answer. Some people prefer take-home interviews, and I can see the merit of that as well. If you document your process for this, you can experiment with it over time and end up with a formula that gets engineers who are a great fit for what you need.

The difference that focus makes

Over the past 48 hours, we had some things happen at with respect to email delivery. I found some of the takeaways interesting, and felt like writing about it, and how it connects to larger business and strategy ideas. With respect to Ship and the companies involved, I’m going to stay light on the details, and look more at the concepts I see behind the issue.

Mandrill and SendGrid are two very big players in the email delivery space.

Mandrill is owned by Rocket Science Group. Rocket Science also controls MailChimp, TinyLetter, Gather, which are all very marketing-focused products.

SendGrid is a company that focuses on developers as customers. They aggressively brand themselves as developer-focused, and show up at every hackathon they can (one of the Ship developers has three different SendGrid shirts that he’s been given at hackathons).

A marketing focused company is going to attract marketing people, who think of the universe through a marketing lens. A developer company is going to attract engineer-minded people, who think of the universe through an engineering lens.

As Porter taught us, one way of analyzing a company’s strategy (and their strengths and weaknesses) is to look at what their team has done in the past. Past experiences will inevitably shape future decisions.

If you were choosing an email service provider, which one would you choose? Which company do you think understands your view of the world? Which one do you think will create features that lend themselves to your use case?

The more software you have, the more software you need.

Jevons Paradox says that as a resource can be used more efficiently, more of it ends up being used overall. That is, the better the deal you’re getting, the more you’re interested in buying.

This example of real-world compounding reminded me of another example: software. There’s an interesting quirk about software — the more you have of it, the more you need.

Let’s trace through one recent path of the software industry. We start with a website.

A website is a great idea. Websites (code) make it easy to distribute information. There’s so much information that people create dynamic websites (more code).

Dynamic websites are useful, but once you’ve got dynamic data, you want the data available via API (even more code) so that outside developers can build applications (lots more code).

To help with this, you build an API management layer (tons more code), which produces information about all these other applications.

You want that data combined in a dashboard (still even more code) alongside data from all of the other software that relates to your business (which is even more code than all of the other code so far).


Anything that cannot go on forever must stop. But there doesn’t seem to be any reasonable end in sight. This is one small piece of the software industry that points to a bigger trend, which seems to contradict common sense. People intuitively understand supply and demand. But here, supply creates more demand.

It’s a good time to be a programmer.

Voxeo, Tropo, & ORUG

I went out to ORUG tonight. Voxeo was presenting a thing they’re working on, Tropo. Disclosure: they bought us dinner. Full disclosure: I think this thing is really tight.

I used to help set up phone systems in high school, and phone trees have always seemed like kind of a mystery. Tropo lets you build whole phone apps, and it’s ridiculously easy. It’s basically a phone system DSL. They handle text-to-speech, speech-to-text, playing recorded sound files; there’s lots of convenience things for capturing different types of inputs, handling error cases, recording calls, transferring calls, etc. They give local phone numbers in different area codes, they’ve also got Skype integration, and a few other ways to connect to the system. The very cool part is that it’s all free to play around with, but once you start using it for commercial reasons, then you have to pay.

Ever hear of Google’s Grand Central? With this, you could easily make your own. I’ve been playing around with a few things using Tropo’s Ruby setup, and I’ve put the demo code on GitHub. Very cool stuff.

You can write apps in Ruby, PHP, Python, Javascript, and Groovy (“Java++”). There’s a bunch of example code on their site, and development is really easy to do. For example:


digits = $currentCall.callerID.to_s.split('')

area_code = digits[0..2]
city_code = digits[3..5]
subscriber_number = digits[6..9]

# single dashes get spoken as 'dash', use doubles for a pause.
# Double commas don't work, neither do extra spaces
say "-- -- -- S-up. Your phone number is -- #{area_code.join(',')}--#{city_code.join(',')}--#{subscriber_number.join(',')}"


There is a debugger that you can print messages to. Right now there’s a *ton* of output to it, but you’ll find your messages in there.

One thing: I was getting a message that the caller was “not accepting calls at this time”. I realized this was a parse/compile error in my script. So, if you can’t get something to load, check it. The debugger doesn’t seem really helpful with this, I got a generic seeming Java Exception for a variable name typo. They use Java under the hood for tons of stuff, so even though I’m writing Ruby code, it gets interpreted in Java.

I did learn a cool fact about these phone trees. You know how a lot of phone trees suck when you try and talk to them? Well, for speech-to-text conversion, you can only get around an 80% success rate. The reason is that phones are only around 64kbps of data. There’s too much loss for the algorithms to work well. That’s why apps that run on the local computer/phone are able to do better — they embed part of the recognition algorithm in the client.

And, on a final note: skateboarding through downtown is awesome.

The HTML structure of webmail interfaces: Gmail, Hotmail, and Yahoo Mail

As part of the Zentact project I’ve been working on, we were asked to integrate with various webmail clients. This makes it easy to manage your contacts while sending email.

Doing this was a bit of a pain. Since all code is minified, and they all use Javascript events differently, there was a good bit of working to figure out the details. I wanted to share this info in a blog post for programmers who come along in the future. If you don’t know/care about HTML, Javascript events, the DOM, YUI, or AJAX, this post is not for you. Please enjoy one of my other fine posts, perhaps this post on military code names.

Before I begin: there was a ton of info learned (and already forgotten) about this process. This is not a complete guide, but is mostly a brain dump from implementing UI integration on three different webmail interfaces.

  • Gmail uses 6-character strings, [A-Za-z0-9] for all its classes. These classes remain the same from load-to-load, but I believe that they may change over time with minification. IDs are not as constant, and many are dynamically assigned. These start with a colon.
  • When you’re working with events, you may get inconsistent results. Some events are not fully propagated, they get captured and you can’t find out about them. If onclick doesn’t work, try listening for onmousedown or onmouseup. One of them may get you notified of the event you want. Same advice goes for onkeydown, onkeyup, and onkeypress. That being said, once you get into these, be sure to realize that these three events will occur in particular orders. Make sure you’ll be getting notified at the right time.
  • All of the webmail UIs use iframes. This lets them keep their code for loading the UI separate from the code to display the UI. I know there’s some cross-site scripting implications in this, but I’m not sure of all the details. Gmail’s loading screen (the loading bar they show you) is a different iframe than the one that shows you the inbox. All of these iframes are at the root of the document, and there’s nothing else in there.
  • You could use Firebug break points to pause the code and examine what’s going on, but nearly all JS is minified. Since breakpoints can only be set by line, and there’s multiple functions defined per line, it ends up not being helpful.
  • For its UI, Yahoo seems to use YUI, plus some other stuff on top of that. There’s some weird results because of this. The body of the email editor is a group of DIVs, some are invisible, some are for border decoration, and others are for the background of the editor.
  • When we inserted elements into Yahoo Mail using regular DOM operations, they would appear behind other page elements, until another part of the UI was interacted with, when the screen would redraw and then they would bump into place. YUI seems to have its own redraw/repaint functionality, and it won’t play nice with DOM manipulations.
  • Hotmail is strangely one of the less-exotic interfaces. They use consistent IDs. I don’t think they’re hand-coded, however, because they submit to a naming scheme that seems too machine-generated. But still, they are there, and you should take advantage of them.
  • When you’re using events, and you get notified of an event, use the event.originalTarget property to find out where in the DOM you are. That’s useful information when you’re dealing with a DOM tree of nonsense class names and IDs.
  • When you’re trying to figure out where in a DOM tree you are, don’t hesitate to go up several levels and check a great grandparent node, or a “cousin” node. Once you get a single point of reference, you can generally work out where everything else is, relative to it.
  • Some UIs open each message in its own iframe, which means that IDs are consistent since they’re in their own namespace.

Also, thanks to Nate Koechley for helping me get through some of the Yahoo details.

If you’ve got other questions, shoot me an email. I remember more stuff, but might need a good question to shake it loose.