Biden on judgement

When the subject of Trump came up aboard Air Force Two, Biden referred to a well-worn story about how, as a freshman senator, he saw Jesse Helms, the archconservative North Carolina Republican, ripping into a piece of disabilities legislation. Biden was furious about it and began attacking Helms to Mike Mansfield, the Democratic Senate majority leader. Puffing on his pipe, Mansfield asked Biden if he knew that Helms and his wife had adopted a disabled 9-year-old boy no one else would take. “Question a man’s judgment, not his motives,” Mansfield instructed.

I wish to hell I’d just kept saying the exact same thing – Joe Biden

Dependency Inversion Principle

Dependency Inversion is one of the five SOLID OO principles that’s become so popular in recent years (sometimes referred to as DIP). My opinion is that it’s a highly valuable concept, and is not well-named. “Dependency inversion” doesn’t mean a great deal on its own, and a lot of attempts to explain it tend to get very heady.

(Speaking of unfortunate naming, “hexagonal architecture” is another concept that I find poorly named. Ironic that the concept is about using abstraction to avoid unnecessary coupling, yet the name itself couples the idea to the number 6, which is totally unrelated to the idea. Fortunately, it’s slowly being renamed as “ports and adapters” in most discussions, which is a much better description.)

My personal preference is towards explanations that are more intuitive, and I’d like to put my two cents in for Dependency Inversion.

To me, a way of describing Dependency Inversion is to use classes to separate the features of the application (the parts that a user might use) from the technology that makes the feature work.

For example, let’s say you have a feature where a user can sign themselves up, and once the user’s information is saved to the database, you have some more steps that need to take place. You could make an interactor called


and it will handle the actions necessary (instead of using a callback).

Let’s say one of the things it needs to do is send an email welcoming the user. Let’s also pretend that you use SendGrid to manage emails. That would mean we could end up with something like

class CompleteUserSetup
  def self.perform "subjectline", "body of email"


Will this run? Absolutely. Is it a good example of code that meets the criteria for Dependency Inversion? No.

What’s happening here is that the feature (user setup) is directly mentioning the technology that implements it (SendGrid). It literally has the name of the tool in the code that is defining the feature.

Literally Rob Lowe

Since there are probably lots of places in the code that send email, if you ever need to switch away from SendGrid, you need to change all those places. Usually it’s not as simple as find-replacing all the instances of “SendGrid” from the code, so a better way is to abstract all of the email interactions into your own class.

class CompleteUserSetup
  def self.perform

class OurEmailClient
  def self.send_welcome_email "subjectline", "body of email"


Now the feature doesn’t mention the implementation technology. If you kept this pattern going, “SendGrid” would only ever appear in your email client class, and this means that the technology would be decoupled from the features. This is an Adapter pattern, which is basically the first half of Dependency Inversion. (IMHO this is the most common implementation of DI, and this makes testing really easy)

To close the loop of “Dependency Inversion” is to actually pass the client an instance of an email client so it can use more than one.

class CompleteUserSetup
  def self.perform(client)

class OurEmailClient
  def send_welcome_email 'subjectline', 'body of email'

class OurOtherEmailClient
  def send_welcome_email 'subject, 'body of email'

email_client =

# or

email_client =

Now we’re actually passing the dependency into the location where it’s needed. This means the CompleteUserSetup interactor is totally decoupled from which messaging system it will use.

The reason this matters is that now we can choose any type of email provider we currently support, and we can also add new types of email providers that we didn’t previously use.

To be fair, in Ruby, this isn’t quite the same as in Java. Ruby will allow duck typing, which means that we don’t have to write an Interface. Also, Ruby doesn’t support Interfaces at all, so that aspect of this is missing. Still — the principle behind decoupling does make it very easy to write readable code, and testable code, and I always love that.

Highs and Lows

I was talking with an engineering manager the other day who told me one key technique he uses for his teams. I thought it was one of those simple-yet-powerful techniques that I love so much.

The simplicity is this — when he does his weekly checkin with his reports, he asks them “in the past week, not just at work but in your life as a whole, what was your high and what was your low?”

I think this is a great approach to management. It’s highly touchy-feely, but I think that’s an important part of managing others. If someone is having a good time in their personal life, it can and will have an effect on their work performance. Likewise, if someone is having a bad time in their personal life, it can and will affect their work.

I think it’s worth being mindful that the effect might not be what we intuitively expect. Someone who is having bad things happen in their personal life might react by being distracted at work, or they might use work as a means to distract themselves from the unpleasantness and focus on it strongly. Or, perhaps there might be another reaction. There’s no way to know without discussing it and paying attention.

(To be clear, I’m not suggesting that people should use a dark pattern to drive their employees by offering work as a distraction from things that might be difficult at home, only saying that the connection between personal and work does exist, and is unique for each person.)

By understanding the person as a unique individual, and as a whole, a great manager can work with the natural rhythms of the lives that their team members live. I think this maximizes happiness and results, and minimizes mistakes and turnover, which are all desirable.

How to specify the schema using Spark’s Java client

I’ve been working with Spark recently. It’s awesome. Unfortunately, the Spark community leans towards Scala, but the Java client was the best choice for our team. This means that sometimes the documentation and examples out there aren’t great. One thing that took me longer than I liked was figuring out how to specify the schema for the data.

Spark has a nice default behavior where it will infer the schema of your data when you load it. The trouble with this is that if you change the data over time (adding new attributes for example), you can run into issues with your code only working with certain versions of the data and not others.

Fortunately, you can specify the schema, so that the fields will exist as nulls. I found lots of examples for how to do this in Scala, but it was hard to find examples in Java. So, here’s how:

Let’s pretend the following file is the schema of `whatever.json`

  "address":{"street":"Spear St","city":"San Francisco"},
  "rank": 100 

This would correspond to the following code:

SparkConf conf = new SparkConf().setAppName("MyFunApp");
JavaSparkContext sparkCtx = new JavaSparkContext(conf);
HiveContext context = new HiveContext(;
String sourceUrl = "whatever.json";

StructType schema = DataTypes.createStructType(Arrays.asList(
  DataTypes.createStructField("company_name", DataTypes.LongType, false),
  DataTypes.createStructField("address", DataTypes.createStructType(Arrays.asList(
    DataTypes.createStructField("street", DataTypes.StringType, true),
    DataTypes.createStructField("city", DataTypes.StringType, true)
  )), true)

DataFrame changesRaw =;

The third createStructField param is “can the value be null?”

Keeping standards high when hiring engineers

My last post on hiring engineers got a lot of attention, and I wanted to do a followup with a technique that you can use to keep hiring standards high.

Typically engineering hires happen in a situation where there’s a serious need of another set of hands to do some work. I say this in contrast to the idea that sometimes gets mentioned in the tech industry of “if you find someone really good, just hire them and then figure out what to do with them”.

This idea matters because when there is a serious need, the people closest to the situation are the ones who know what skills are needed, and these are the people who are brought in to handle the interview. This is entirely reasonable, but it also means that the people tasked with the hiring decision all have an incentive for a hire to occur — they want the position to be filled so someone can take over the work. When incentives are strong enough, it sometimes leads people to make decisions that they otherwise might not — hiring a candidate who isn’t as good as they should be to avoid some short term pain.

If a team is really committed to hiring very good candidates, there is a straightforward way of minimizing this type of mistake. When the interview team is put together, you should include one person who isn’t directly impacted by the hire. This person won’t have the same time pressure as the people who are directly impacted. Their role is to be a detached observer and help keep standards high in the face of the (relatively) short term pressure to make any hiring decision.


How to hire engineers

From IRC:


There’s a saying that you are the average of the 5 people you surround yourself with. If we apply that logic to hiring, it means that hiring is really important (but we knew that already).

My friend’s question was an interesting one. As an engineer, I know about writing code,  and engineering fit. If I put my bizdev/marketing hat on, I start to think about the process of interviewing, and how to best allocate my resources (time) in order to qualify a target and convert them (hiring).

There’s one concept that I think is important to share with my engineering friends is the idea of a conversion funnel. The gist is that in a lot of business situations, you have a group of people, you have an action you want some percentage of those people to do, and you have a series of steps that they go through (the stages of the funnel).

A (hilariously simple) conversion funnel for Apple might be

  1. Find out about new iPhone (begin funnel)
  2. Read about new iPhone on internet
  3. Try new iPhone in store
  4. Purchase iPhone (end of funnel)

Hiring engineers (or any role) can be thought of in this way. For example:

  1. Create awareness of your company/the job. This could be through blogging, buying ads, or posting on Craigslist.
  2. Create enough desire to get the person to submit an application. This could be talking up perks, describing the work environment and technologies used, or having a really great application process. Anything that makes your company attractive to work at goes in here.
  3. Review applications/resumes. This is early in the funnel, so you want to spend very little time on. Are they worth any time at all or are they totally not a fit? I recommend having a non-interviewer conceal the person’s name from the reviewers. There is strong evidence that people with good intentions can have something as simple as a name affect their judgement.
  4. Send them a screener problem, and review answers to it. Ideally you should be able to decide to move to the next step (or reply with “no thanks”) with about 15-20 minutes of effort. Again, do this with names concealed, the goal is to focus totally on the code.
  5. 1-2 hour interview. Be sure to talk about their screener problem and understand their engineering sense. Also, you should start getting a sense of what the person might be like to work with, but try and stay open minded about this one until the next step. I like to start the discussion by asking the person how they feel about interviewing — if someone is nervous about being on-the-spot, spending a minute or two to talk about that feeling of nervousness can help them get rid of the feeling so they can focus, and this means you will get a better picture of what they are really like. Also, I typically do this interview via Skype, and ask them to screenshare with me and write a blog using whatever tools or resources they normally would. You can quickly learn a lot about an engineer by spending 15 minutes watching them code in their own comfortable environment.
  6. 4-8 hour pairing session. This should be in-person, unless you’re hiring a remote engineer. The best way to find out what someone is like to work with is to work with them under the most realistic circumstances possible. If possible, ensure that you’ll encounter specific scenarios so that you can gauge skills consistently from one candidate to another. The more objective you can be here, the better your results will be.

Notice that the intent is to minimize effort at the beginning of the process, and do the more intense quality screening at the end.

Above all — customize your funnel in a way that makes sense for you and your situation. There’s not necessarily a right or wrong answer. Some people prefer take-home interviews, and I can see the merit of that as well. If you document your process for this, you can experiment with it over time and end up with a formula that gets engineers who are a great fit for what you need.

Hard Work

I think this is a great representation of the hard work that goes into producing great results, and the lengths that it takes. This guy is working on a commercial for cereal, and to make sure that it looks good for the camera, he’s sorting pieces of cereal by hand.

I think that more often than we realize, great results are a matter of working hard at the most obvious stuff. Do you want a bowl of cereal to look good? Then sort through a few boxes until you have enough great-looking pieces of cereal to make a bowl. There’s no genius-moment-of-inspiration here, there’s no clever trick…just hard work.

Sorting cereal for commercial shoot

Time management for fathers

This is for the dads out there…

I read Getting Things Done years ago, and it had an immediate impact on my life. One of the tips saved me so much time each day that I decided to work out the savings over the course of my life. I found that with this one tip, I would save over two years of my life, a number that surprised me.

If a doctor walked into a room and told you they had found a drug that would, with no side effects, give you an extra two years of quality life, they would be hailed as a hero the world around. As a result, I’m always keeping an eye out for ideas that might help me get a happier, healthier, and more productive life.

I recently had my first kid, and while being a father is awesome in many ways, a child does require a lot of time, especially in the first 5-6 years. I quickly found that there were some ways in which I wasn’t making the most of my time.

I want to clarify — I don’t mean “making the most of my time” in an unpleasant sense of “grinding maximum productivity out of every moment”. Sometimes making the most of time can include drumming, watching TV, going for walks, or playing games (have you seen the new Arkham Knight?). These things are considered by some to be wastes of time, although there is a material benefit to them if they bring you joy. Even Steve Jobs went home and sat in front of the TV.

Given that being a parent has many unique constraints, a new book caught my eye that seemed like it might be relevant to me. The thing that surprised me a bit was that it was a book written for women: I Know How She Does It.

To me, there are many similarities when it comes to women and men who are trying to be successful in business and in their personal lives, so I decided to read it. I’ve got to say, it’s worth a read, even for men.

Any sociologist will tell you there’s a difference between how people self-assess their own behavior when compared to a more objective measurement. This is what made me so interested in this book. The author, Laura Vanderkam, gathered time logs from a large group of women who were both mothers and successful in their careers.

These time logs are recorded by her subjects every 30 minutes during their days, of what they were doing during that time. There’s no right or wrong here, just simply recording what it was. This is far more objective than anecdotal stories of how people spent their time, and in some cases, the women keeping the logs were surprised to find out how they spent their time.

Laura then takes this data, and presents her analysis of it, as well as some great case studies that both support her conclusions, as well as giving the reader an opportunity to mirror the techniques.

I don’t want to attempt to reproduce the contents of her book here, only to share how fresh and valuable I think her approach is, in taking hard data and applying it to the question of how to live a happy and productive life as a parent, for mothers and fathers.