Meta-analysis: best interview questions to spot ideal employees

It’s too hard to find the best interview questions. Even though it’s widely acknowledged that “hiring right” is the make-or-break duty of every growing business, finding the perfect interview question requires sifting through a compendium of books, blogs, and brains.

As a boss, what I want is a broad list of great interview questions, organized by the quality I wish to evaluate (e.g., “hard worker,” “focused,” “team player”). So that’s what I’ve compiled, below.

Read more about the methodology at the bottom of this post, following the questions. tl; dr it involved selectively extracting from all the best books and blog posts I could find in a week of research.

Part one: the most sought-after traits in employees

The most sought-after traits have been broken into four tiers, based on the frequency with which they were mentioned in first-page search results. If you only have an hour for your interview, you probably don’t want to go very in-depth on more than 3 different areas. A popular strategy at bigger companies is to share different areas of evaluation between the team. It helps to ensure that you won’t miss a fatal weakness of the applicant.

If you are going to interview collaboratively, remember that some questions are duplicated between sections. Don’t be that interviewer who repeats a question from the previous interviewer.

Tier One

There is one talent that almost every blog & book regards as critical. Apparently, businesses don’t want to hire a candidate that causes drama and resists management?

  • Team player. Are they friendly, agreeable and upbeat?

Tier Two

These qualities were cited by 75% of publications surveyed, which is to say they are important to virtually every company. “Honesty” is one of the hardest qualities to evaluate in an interview context, but I’ve found a handful of questions to penetrate the interview defenses.

Tier Three

Qualities that were mentioned by roughly half the sources surveyed. Many of the most critical talents Bonanza seeks are located here.

  • Communicator. Do they know how to wield language and be tactful?
  • Leadership. Does this candidate have the tools to be a leader?
  • Ambitious. Every business wants a team that is self-motivated and hard-working
  • Focused. Can they possibly eschew Facebook, Twitter, IM, Hacker News, et al. for eight hours?
  • Problem-solving. Can they work autonomously, surmounting challenges and making decisions lacking manager intervention?
  • Intelligent. Does this candidate have the raw smarts to be proficient at their job?
  • Organized. Can they keep track of the details and prioritize among choices?

Tier Four

These qualities weren’t mentioned as consistently across articles surveyed. It is my personal judgement that they are still important to assess, depending on the role being filled.

  • Passionate. Are they excitable? Do they love their work?
  • Creative. Can they figure out novel solutions to problems?
  • Stable. Loyalty and dependability are essential to team morale.
  • Invested in us. Have they done the homework on the company they’re appyling to?
  • Meat & potatoes. Miscellaneous questions not to forget.

Part 2: My favorite interview questions

This list is the culmination of a week spent combing the internet and popular books for great interview question. I have attempted to choose my favorite versions of the classic questions (e.g., “What’s your weakness?”) such that the version chosen gives the candidate the opportunity to talk in the realm of specific examples.

Do note: some questions apply to more than one category, so are listed more than once. Don’t make a fool of yourself in a collaborative interview by asking the same question twice. Use your communication skills to coordinate questions with your teammates.

Team player, friendly, agreeable, upbeat

  1. What do you think makes a person likeable?
  2. How would a co-worker who likes you describe you?
  3. How would a co-worker who dislikes you describe you?
  4. What qualities in your co-workers bother you most? Do you appreciate most?
  5. If I were to ask your current boss what your greatest strength is, what would he or she tell me?
  6. What do you have Strong Opinions about?
  7. Describe the boss who would get the very best work from you.
  8. What changes have you made in working with others to be more effective at work?
  9. What could you have done to improve your relationship with your worst boss?
  10. What’s a time you disagreed with a decision that was made by your boss? How did you handle it?
  11. Tell me about a time you had a conflict with a co-worker, and how you handled it?
  12. How do you deal with a difficult customer?
  13. What do you feel an employer owes an employee?
  14. Give an example of a time you misjudged someone
  15. When was the last time you were angry?
  16. Can you describe a time when your work was criticized?

Adaptable, flexible, eager to learn

  1. What do you think are the most important attributes of successful people? How do you rate yourself in those areas?
  2. If you could change one thing about yourself by snapping your fingers, what would it be and why? Do you have a plan for
    making that improvement?
  3. Tell me about competitor (or similar) businesses you kept up with in your past position. What did you glean from them?
  4. When have you failed? Describe what happened and what you learned from it.
  5. How do you take advantage of your strengths?
  6. How do you compensate for your weaknesses?
  7. What’s the most interesting thing you’ve learned in the past year?
  8. If you were starting college today, what classes would you take? (Alternative: What would you like to learn more about in the next year?)
  9. What books are you reading? (Alternative: What industry publications are your favorites?)
  10. Tell me about the last time you took a risk? Was it the right decision? What did you learn?
  11. How do you keep up with changes in your field?
  12. What was the most useful criticism you received?
  13. How do you measure your own success?

Trustworthy, honest

  1. Have you worked in a position where you felt you weren’t given enough to do?
  2. What was the most useful criticism you received?
  3. If I were to ask your current boss what your greatest weakness was, what would he or she tell me?
  4. Tell me about the last time you had to make an unpopular decision
  5. When have you failed? Describe what happened and what you learned from it.
  6. Tell me about a work incident in which you were totally honest, despite a potential risk or downside.
  7. What’s your greatest fear about this opportunity?
  8. Where does your boss think you are now? [Ed: answer doesn’t matter, smell of truthfulness in response does]
  9. If I looked through your web browser history, what would I learn that isn’t on your resume?
  10. Tell me about some of the gaps in your employment. How did these come about?
  11. May I contact your current employer? May I contact your references?
  12. Why shouldn’t I hire you?

Culture fit, compatible personality

  1. Describe the boss who would get the very best work from you
  2. What will make you love coming to work here everyday?
  3. What type of work environment do you prefer?
  4. What gets you excited outside of work?
  5. If I looked through your web browser history, what would I learn that isn’t on your resume?
  6. Assume that you come to work here. One year from now you go home one Friday evening thinking that accepting this job was the best thing you ever did. What happened during the year for you to feel that way?
  7. How much time per week do you spend working alone? Do you think it should be more or less?
  8. How competitive are you?
  9. Do you have a favorite quote?
  10. What unique experiences separate you from other candidates?
  11. What are the most important rewards you expect to get from your career?
  12. Where do you see yourself in five years?
  13. Which areas of your work are most often praised?
  14. Tell me about the funniest role you’ve had on a job/project?
  15. When you are on vacation, what do you miss most about work?
  16. How would you describe yourself to a stranger at a party?
  17. How do you incorporate fun into your day?

Communicator, tactful

See also: “Team player

  1. Why are you leaving your current job?
  2. Have you given presentations to your peers, either at work or in other contexts?
  3. See that picture frame on the wall? Sell it to me.
  4. What’s a time you disagreed with a decision that was made by your boss? (Alternate: If you know your boss is 100% wrong about something, how would you handle it?)
  5. If you feel strongly about an idea, but a person doesn’t agree with you, what tactics would you use to persuade them?
  6. What’s your greatest fear about this opportunity?
  7. Describe a time where you put in extra effort to make sure a person with whom you were communicating had really understood your point.
  8. Describe a situation in which you dealt with a difficult customer
  9. Why shouldn’t we hire you?
  10. What do you think we could do better or differently?


  1. How do you motivate people?
  2. What do you think are the most important attributes of successful leaders?
  3. Tell me about the last time you had to make an unpopular decision
  4. Tell me about the last time you fired someone
  5. How do you decide what tasks to delegate to whom?
  6. How do you bring other team members into your process? (Alternate: How do you get others on board with your decisions?)
  7. Did you inaugurate new processes in your previous position? Tell me about them
  8. Have you been in charge of a budget? How did you balance priorities?
  9. What’s the most difficult part of being a manager for you?
  10. What do your reports find most difficult about your management style?
  11. How do you set expectations with your reports or peers? How do you handle a situation where a report fails to meet expectations (e.g., consistently missing deadlines)?
  12. Describe a situation where you successfully resolved a conflict with a team member.
  13. Would you rather be liked or feared?
  14. Describe a crisis you faced at work. What was your role? How did you resolve it? What were the results?
  15. Do you have a management philosophy? No.

Ambitious, self-motivated, hard-working

See also: “Invested in us

  1. What would your first 30, 60, 90 days look like in this role?
  2. What extracurricular activities were you involved with in college? Since?
  3. What side projects have you done? What was the biggest?
  4. Have you been promoted in past jobs? How did these promotions come about?
  5. Describe a situation in your past job where you were proactive? (Alternate: Give me an example from a previous job where you’ve shown initiative)
  6. When did you first start doing [the job’s requirement]? [Ed note: if they started doing it before required in school, suggests self-starter]
  7. How do you keep up with changes in your field?
  8. How competitive are you?
  9. What are your short range goals and objectives?
  10. Give me an example of a goal you set in the past and how you went about reaching it?
  11. Where do you see yourself in five years?
  12. Do you take work home with you?
  13. How ambitious are you? No.

Focused, tenacious

  1. What are your strategies for making the most of your day?
  2. What do you do when things are slow? (You are lacking for projects)
  3. What do you do when you’re burned out to rekindle your spark?
  4. If you’re struggling with an idea, how do you break through and find a solution?
  5. How do you pass time while waiting for [job specific pause point, like compiling a program]?
  6. Describe a time where you put in extra effort to improve a business process to be more efficient?
  7. What percentage of your time is spent on each of your job responsibilities? [Answer matters less than whether they seem to keep track of it]

Problem-solving, self-sufficient, autonomous, decisive

See also: “creative.”

  1. Describe a time when you were asked to do something you weren’t trained to do. How did you handle it?
  2. Tell me about a time when you made a decision without all the information you needed.
  3. Can you tell me about a time when you discovered a more efficient way to do a routine task? Have you done this repeatedly?
  4. What are some of the Google tricks you use to find answers to your questions?
  5. How do you decide whether to message your boss when you have a tough problem?
  6. How many dentists are there in Poland?
  7. How would you build a toaster for a blind person?
  8. See that picture frame on the wall? Sell it to me.
  9. Tell me about the last time you took a risk? Was it the right decision? What would you have done differently?
  10. Can you think of a time where you were forced to take charge when your manager wasn’t available?
  11. Have you worked in a position where you felt you weren’t given enough to do?
  12. What was the most important task you’ve ever had?

Intelligent, proficient, competent

  1. What are you especially good at compared to others?
  2. Have you been promoted in past jobs? How did these promotions come about?
  3. What’s the one accomplishment you’re most proud of? Why?
  4. What is your greatest professional achievement?
  5. What’s your greatest fear about this opportunity?
  6. How would you rate yourself in terms of speed, quality, and self-direction? [Ed: Can they answer a multi-part question?]
  7. In what college courses did you get the worst grades? Why? How do you think it’ll effect your job performance?
  8. Tell me about the last time you took a risk? Was it the right decision? What would you have done differently?
  9. What do you consider to be the most important aspect of your craft?
  10. What do you consider to be the most misunderstood aspect of your craft?
  11. If I asked a co-worker friend what made you most valuable to the team, what would they say?

Organized, detail-oriented

  1. What would your first 30, 60, or 90 days look like in this role?
  2. Tell me about the first 60 minutes of your typical day
  3. What are your strategies for making the most of your day?
  4. Have you been in charge of a budget? How did you balance priorities?
  5. What do you do when you begin a project? How does this help the project go successfully?
  6. What tools do you use to maintain organization? How do you keep lists?
  7. Tell me about a project where you noticed a small detail that made a big difference?
  8. What percentage of your time is spent on each of your job responsibilities?
  9. Describe the top of your desk
  10. Do you always double-check your work? What’s your system for deciding?

Passionate, excitable

  1. What excites you about our company?
  2. What gets you excited outside of work?
  3. Describe the boss who would get the very best work from you.
  4. What are the most important rewards you expect to get from your career?
  5. If you had unlimited money, what would you be doing?


  1. Tell me about the last time you took a risk? Was it the right decision? What would you have done differently?
  2. Tell me about the last time you found a unique solution to a long-standing problem.
  3. Tell me about the last time you figured out how to save your company money? Did they heed your advice?
  4. Did you inaugurate new processes in your previous position?
  5. What improvements might you make to our business?
  6. Where do you draw inspiration for your craft from?

Stable, loyal, dependable

  1. Why are you leaving current job? Have you brought your concerns with your job to the attention of your boss?
  2. Is this the first time you’ve thought about leaving? What made you stay before?
  3. Why have you had so many jobs?
  4. How do we know you’ll stick around?
  5. Will your current job be surprised to hear you’re leaving?
  6. What aspect of our job appeals to you least?
  7. How does your ideal job stack up to what we’re offering?
  8. Have you been absent from work more than a few days in any previous position?

Invested in us

  1. What about this job most excites you?
  2. What about this company most excites you? (Alternative: What do you know about the company?)
  3. What do you think we could do better or differently?
  4. What advantage do we have over our competitors? (Who are our competitors?)
  5. How does your ideal job stack up to what we’re offering?
  6. Where do you see yourself in five years?
  7. What criteria are you using to choose companies you interview with?

Misc, administrative

  1. Tell me about yourself
  2. Which areas of your work are most often praised?
  3. Is there any question I haven’t asked you that I should?
  4. Salary range expectations?
  5. When can you start?
  6. Are you considering other offers?


Settling on the definition for what makes a “great” interview question is a subjective affair. There were a couple rules I applied. I wanted questions that lead the candidate to talk about a specific example; questions like “what’s your management style?” (a common question amongst my sources [1]) are too vague to expect worthwhile insights in return. When in doubt, I considered whether I would be annoyed to get asked the question in an interview (e.g., “Give us one word that describes you“). If so, it’s not on the list. 80% of questions evaluated are not on the list.

Once I decided what I was after, I purchased a handful of Amazon’s top-rated interview books, and visited tens of Google’s top ranked pages on interviewing. Most of my sources are linked in the footnotes [1][2].

Building a dataset from my research, I sought to make two determinations:

  1. What are the top 15 qualities that employers seek in an employee?
  2. What are the most succinct questions to evaluate those qualities?

The first goal was tricky, because often, sources will refer to the same idea in slightly different ways (e.g., “adaptable,” “flexible,” or “willing to learn”). Some qualities, like “problem-solving” and “creative” feel distinct, but related. In these cases, I added links between the two categories to help the reader find related questions.

Still here? Hello!

I’d wager you would pass the “hard-working” and “tenacious” evaluation phases of the interview. So why not check out the many jobs Bonanza is hiring for at the moment? Everyone loves an interview where they know the questions in advance. :-)

Also: if you have a favorite interview question that I haven’t listed here, please share it in the comments! Around 10% of the questions above were from my personal collection, indicating there are likely still more good questions lurking out there.


[1] Bibliography:

[2] I didn’t start documenting until around half way through my research, so the list isn’t quite comprehensive

Evernote QA Report: Merge & Sync

Hi Evernote,

Your sync and merge has long been broken, and I hope to provide you enough detail to get it fixed here. All the below should be reproducible, let me know if you find otherwise.
List Fruits 1

It starts simple. Create a note in Evernote webview. List some fruits. If you’re anything like me, you probably make a couple lists like this per day. All seems well. Later that day, you remember this list, and have a burning need to add “Starfruit” to it. All looks well?

2015-11-05 16.06.47
But things are not so well, because when we return to the webview, there’s some bad news awaiting:
List Fruits Double Note               ^– Now two notes, instead of one

That’s right, my fruit list has split itself in two lists with edits from desktop and mobile. Similar problems persist as I try to use the document.

Unless one is fastidious enough to sync every note when finished, and then manually sync when resuming on another platform, you are liable (at rate inversely proportional to speed of your internet connection) to soon find yourself with something like this:

Two Syncs Later

Every time I switch devices, I must remember “ensure save,” then pre-sync on the new device (and remember to confirm sync finished). Granted, there are a number of trigger points for syncs, but it’s impossible to be sure which, if any, have triggered/completed. One too-fast move, and I’m back in the ghetto of Mergetown.

I usually don’t even know I’ve been losing list items (to duplicated versions), until I notice the note lags in load/save time. And then I realize there are 10 copies of the list in my note, below the (already long) version I was treating as the “real list.”

I sympathize that getting sync to work reliably across disparate platforms is a hard problem [2]. So why not, instead of fully fixing the sync, just spend a few days fixing the merge?

If you could automatically add new lines to both documents without creating duplicate copies, that would be a good start. If you kept the note differences in the area they were found (not bottom of document), that would help. Source control tools have spent 20+ years building great systems to merge two documents. You can probably crib most of your implementation from a plugin/library in that domain? Or at least mark “duplicate documents” so I know I’ll need to hand merge them?

Thanks for considering these improvements. I take the time to write this because I still believe you’re the best solution in your space. If you could just address this fundamental stuff, I’d reap benefits greater than the (sizeable) time it took to create this report.

[1] Lists featuring fruits.
[2] Albeit the hard problem you ostensibly built your business to solve

Remove green checkbox from Dropbox on OS X

As of November 2015, here’s how to do it with one line of code in console:

mv /Applications/ /Applications/

Then you just need to restart Finder. You can do this by opening Finder and choosing “Force quit” from the Apple menu.

The Cnet instructions didn’t work for me.

Fixed: My i7 Intel Dell Laptop is Ridiculously Slow

Most of the Google results I found when digging around on this subject pointed to usual boring causes of slowness: too many programs being run on startup (which you can test with ms-config if you’re running Windows), anti-virus software, and other boring stuff of that sort. In my case, I had been running Ubuntu so most of those tips are moot. But to be thorough, I did remove practically any and every resident program that was running on what should have been a zippy Dell Latitude E6520 with a i7-2720QM (2.20GHz, 6M cache) processor.

And yet, running a utility that averaged about 5 seconds on my desktop consistently took 30 seconds on my laptop. Except for every once in awhile, when it would take 6 or 7 seconds.

Before splurging for a new laptop, I decided to take a peek through my BIOS settings and managed to stumble across the culprit: the Intel “Speed Step” feature. On my Dell, this was under the “Performance” settings. I guess that the idea of Speed Step is that the i7 powers itself down when it decides you’d like your system to perform like a 486. Whatever the logic is that determines when to power down was clearly NOT working as intended on my laptop. After disabling Speed Step, I have been running for the entire day at speeds very similar to my desktop.

Hopefully someone else thinks to Google for this problem and find themselves helped by a similar approach. FWIW I suppose that this might mean that the laptop uses more battery, but you can be an informed consumer about whether you want to run fast or power-efficiently.

Is Gmail secure from its employees?

The recent essay from Paul Graham, “Don’t Talk to Corp Dev,” reminded me of a question I’ve wondered from time to time over the years: what would stop Google from accessing a startup’s Gmail (or Google Apps) if it were advantageous to their business negotiations? A part of me always cringes at this thought; it’s too paranoid. It goes against their culture. Still, given how bad an internal breach could be, I was curious to see what language Google would use to assure its users that their Gmail was in safer hands than, say, an Uber itinerary.

The first result I came upon in my Googling was a guy (Christopher Nguyen [1]), who on Quora who gives a clear and explicit description of what Google has done to protect Gmail from internal intrusions. The author describes a tough privacy policy that inspired several upvotes, not to mention at least two separate articles from tech sites quoting his answer as proof of “Google’s policy”. Nguyen concludes: “… ultimately, an internal culture of respecting users’ privacy helps keep [us] in check.” It sounded pretty good, like the sort of approach I’d expect Google to take.

My problem with the answer? It’s not from Google. Dropbox is explicit about how employees access data; I find their policies well-thought out and comprehensive. Evernote’s security is moderately explicit [2] that it has more lax policies [3]. Either way, I can evaluate the security of my accounts from an internal employee who might be bored some Wednesday night.

Meanwhile, Google itself is mum on who internally can read my Gmail, and for what purposes. “Gmail security”-related queries yield results ranging from how to secure your account against external entities to how Google keeps us safe from open wifi. They also assure us that they aren’t manually reading every one of our emails to serve us ads (Really?? So do they contract hamsters to pick all those ads? [4]). But nothing in my Googling can locate a Google-authored document that describes their internal Gmail security policy at all.

Does this lack of explicitness matter? I’m torn. Google’s generic privacy policy paints the picture of a company that cares about security. They seemed to get pretty upset when the NSA spied on their users. And yet, I’ve probably had more conversations in Gmail over the past 10 years than I’ve had in the real world. If there were any service I’d like to see a clear, tough, explicit policy on all aspects of data security, it would be Gmail.

I think their policies are probably as good or better than Dropbox, but at this point they are forcing me to assign an awful lot of credibility to that guy on Quora.


[1] Christopher Nguyen says he worked at Google between 2003 and 2008.
[2] See the “Customer Account Access” section
[3] “This [admin access] tool allows our customer service and platform administration teams to resolve customer issues.” Which customer support personnel are accessing? What do they access?
[4] Blog sarcasm

Determine Ruby GC Garbage Collection Settings

Ever wish that you could determine what garbage collection settings your current running version of Ruby was using? Me too, but Google was not much help for it. There are two techniques for getting the current GC settings.

Technique 1: Use sh

In Rails console (or debugger), you can run shell commands by surrounding your command in the “ characters. So you can do:


The pros of this approach is that it maps directly to the way that most people configure their GC settings: by setting environment variables. This is a means to be certain that the files you think are getting sourced when you start Ruby are being loaded correctly.

Technique 2: Use GC.stat

As of Ruby 1.9, running “GC.stat” can tell you all sorts of information about your current GC stats. The drawback of this approach is that there isn’t clear mapping between the environment variables you set and the data that GC.stat returns. Anecdotally, it appears that GC.stats[:heap_free_num] corresponds to RUBY_HEAP_MIN_SLOTS. I’m not sure yet how the other variables get reflected, but feel free to chime in with a comment if you know how the standard Ruby GC settings map to the hash keys returned by GC.stat

TBTTY: Freedom February

[This post was originally sent to the TBTTY list (4), but I thought others might enjoy it too so it’s reposted here]

My favorite thing this year is February.

Last year, Bonanza started a tradition of swapping Seattle for somewhere tropical every February. We call it “Freedom February.” Its birth came from asking ourselves some pointed questions:

  • We’re an Internet company, we’re small-ish (1), why do we have to work in the same place every month?
  • How will it impact productivity if we give our team the freedom to work on their own schedule?
  • What was that bright, glowy thing that seemed to hang in the sky all day, making things warm and pleasant last Summer? (2) Can we hunt it down?

Curiosity got the better of us, so we booked some plane tickets, a month of lodging in Costa Rica, and we did it. It was more fun than I’d hoped for. There were monkeys.

Inline image 1

In response to our first question, we learned that there was really no reason we couldn’t get away with doing this every February. All we need is a connection to the Internet and this business can run itself fine. We were all friends already, but this trip reinforced our friendships and our combined sense of mission. To the second question, we got more code checked in that month than we did the month before or after, in part because of a collective spirit of working nights and weekends, even though we worked less during daylight hours. As a nice kicker, our sales also jumped about 15% that month. There was a lot to like about this idea.

This brings us to February 2014. Even before we departed, it had the makings of a TBTTY email when this victory dropped into our lap:

Inline image 2

In their annual survey of more than 12,000 sellers (the biggest/only third-party survey on marketplaces), Bonanza beat out Etsy, eBay and Amazon in whether sellers would recommend it to a friend! We had scored in the top four of this survey during previous years, but this was the first year we won both “Most Recommended” and “Top Score Overall.” Etsy, eBay and Amazon finished 2nd, 3rd and 4th to us in cumulative score. February was off to a fine start.

And then, the trip. This year, since Bonanza has a new tyke in tow (our developer’s criminally adorable daughter, Kira), we decided to visit the north shore of Oahu, where we’d have easy access to modern health care if anything important came up. Here’s Kira enjoying Freedom February 2014:

Inline image 3

In terms of Februarys, this year was even better than last year. Bonanza has grown since 2013, so we had more people = more opportunities for epic BBQs = more fun. The availability of our own hot tub this year was a nice touch too.

Inline image 4

Enjoying BBQ dinner

Inline image 5

A good time was had by all this year. But my favorite part of the trip was seeing how happy my parents were to be in Hawaii. Until I started dragging them around the world with me a couple years ago, they hadn’t been on a vacation since their honeymoon (25+ years ago). My mother in particular has more energy and enthusiasm than any of my 30-something friends, and her happy-go-lucky spirit becomes infectious in a group setting. Here’s her with my pa in front of our house:

Inline image 6

Even when I was sitting upstairs with ocean surf drowning out most sound, I could still hear my mom’s laughs echoing above everyone else’s yapping downstairs. It made me realize that I probably have her to thank for much of my willingness to fail (in jokes, in crazy ideas, in business). When you grow up with someone that thinks most everything you say is hiiii-larious, I think there’s a bit of irrational confidence that sticks to you, failures be damned.If any of you run a small-ish company and would like to give this idea a try, DM me and I’ll share what I’ve learned from my two years experience. Generally speaking, it’s less work than I had expected (3). I think “inertia” was the biggest reason it took us a couple years to try it out, but now I think that this is exactly the sort of benefit a startup deserves to compensate for all the hard work we put in throughout the year.

(1) About 15 people, many of which are remote, some part-time. So far the participants have just been folks working out of our Seattle office, but I hope to keep slowly growing the trip as long as we can employ hard workers who GSD with minimal oversight.

(2) I’m not telling. You’re just going to have to try this yourself or wait a few months and hope.

(3) The hardest part is finding the big enough+affordable lodging that’s conducive to work (i.e., has a view of the ocean). Besides that, everything else tends to fall into place, especially if you get your team involved in the planning process.

(4) Still hasn’t been posted to the list though after posting it there yesterday, so not sure exactly how “alive” the list still is

Rails 3.2 Performance: Another Step Slower

Having a large codebase means that we don’t upgrade our version of Rails very often (we’re averaging once every two years, with about 1-2 weeks of dev time per upgrade). Every time we do upgrade, though, one of the first things that I’m curious to inspect is the performance delta between versions.

For our previous upgrade, I documented our average action becoming about 2x slower when we moved from Rails 2.3 to Rails 3.0, with an action that had averaged 225ms climbing to 480ms. Luckily, in that episode we were able to pull out some tricks (GC tuning) such that we eventually got the same action down to 280ms. Still around 25% slower than Rails 2.3, even implementing fancy new tricks, but we could live with it.

When we finally decided we had to move from Rails 3.0 to 3.2 to remain compatible with newer gems, I was understandably anxious about what the performance drop was going to be based on our past experience. With the numbers now in hand, it looks like that apprehension was warranted. Here is the same action I profiled last time (our most common action – the one that displays an item), on Rails 3.0 before upgrade:

Most common action before upgrade, averaging 301 ms over 3 hours time window


And here it is now:

After upgrade, same time period as last week, averaging 423 ms over 3 hour time window


The problem with 3.2 is that, unlike last time, we don’t have any more tricks to pull out of our hat. We’ve already upgraded to the latest and greatest Ruby 2.0. We’ve already disabled GC during requests (thanks Passenger!). When we made these upgrades, they sped up our Rails 3.0 app around 25%. That performance improvement has now been overshadowed by the 40% slower controller and view rendering we endure in Rails 3.2, making us slower than we were in 3.0 before our Ruby optimizations.

Suffice it to say, if you have a big app on Rails, you have probably learned at this point to fear new versions of Rails. I fully empathize with those who are forking over bucks for Rails LTS. If we didn’t need compatibility with new gems, staying on 2.3 would have left us about 100% faster than Rails 3.0, which in turn is about 40% faster than Rails 3.2.

New Rails trumpets improvements like “ability to build single-page web apps” and “tighter security defaults” and “streamlining, simplifying” the constituent libraries. The closest we’ve seen to a performance improvement lately was that 3.2 made loading in development faster (1). This was certainly a fabulous improvement (took our average dev page load from 5+ seconds to 1-2), albeit one we already had in Rails 3.0 thanks to active_reload.

My sense is that performance has become the least of the concerns driving Rails development these days, which, if true, is a shame. If Rails put equal time into analyzing/improving performance as it does to “streamlining, simplifying,” it’s hard to believe that we would keep swallowing 40%-100% performance setbacks with each release. Maybe a partnership with New Relic could help the Rails team to see the real world impact of their decisions on the actual apps being built with their platform? If others’ experience is similar to ours, that would be a lot of pain felt by a lot of people.

I admit I’m a bit reluctant to make this post, because Rails has given so much to us as a platform, and our business is too small at this point to be directly involved in improving performance within Rails. We will, however, continue to post any salient optimizations that we discover to this blog and elsewhere.

My primary concern though, and the reason I am posting this, is that if Rails keeps slowing down at the rate it has, it makes me wonder if there will be a “point of no return” in the 4.x or 5.x series where it simply becomes too slow for us to be able to upgrade anymore. Each new release we’ve followed has been another step toward that possibility, even as we buy ever-faster servers and implement ever-more elaborate optimizations to the compiler.

Has anyone else out there upgraded a medium-to-large webapp from Rails 2 -> 3 -> 4? I’d be very curious to hear your experience? The lack of results when Googling for “Rails performance” has always left me wanting for more details on other developers upgrade experiences.

(1) New caching models may improve performance as well in some scenarios, as could the dynamic streaming when used with compatible web servers. For the purposes of this post I’m focusing on “performance” as it pertains to dynamic web apps that run on a server, which means stuff like interpreting requests, interacting with the database, and rendering responses.

Copy Ubuntu (Xubuntu) to new partition with minimum pain

Every time I upgrade computers or hard drives I have to re-figure out how to get my Ubuntu and Xubuntu OS reinstalled in the minimum number of steps. The following is the sequence I used to get the job done here in 2013 on Raring Ringtail.

  1. Create a Live Ubuntu Startup Disc. This has consistently been more painful than I think it ought to be. Last time I tried this, usb-creator-gtk wouldn’t recognize my USB drive. This time, usb-creator-gtk would crash with various Segfaults before it completed copying my iso to my flash drive. Eventually I discovered UNetbootin and all was well. It can grab the OS installs for you, or you can give it the path to an ISO you want to burn. The only trick with it for me (perhaps because of my balky flash drive) was that I had to plug and unplug the flash drive from my computer a few times before UNetbootin (or Xubuntu) would recognize it. As far as OS goes, I put the most recent Ubuntu LTS on my flash drive since I figured it would have the best toolset for modifying partitions.
  2. Boot from the Live Startup Disc. In the menu, pick “Try Ubuntu.” Click the Ubuntu icon (upper left) and search for “Gparted.” Fire it up. If you’re lucky, your new drive is larger than the partition you want to copy. If not, you’ll have to resize the partition you’re copying such that it can fit onto the new disk.
  3. Copy the partition. Since you live booted, neither partition should be mounted, so you should be able to click “Copy” on the old partition and “Paste” in on the new drive (if it has an existing partition, you’ll need to delete that first). Apply change. Wait an hour.
  4. Run some esoteric crap to change UUID of new partition. Start terminal then
    sudo blkid # shows the list of all your drives, observe that the new and old drive have same UUID, that won't do!
    sudo tune2fs -U random /dev/sdXX # XX is the letter+number of the new partition you copied to. This will assign it a new UUID
  5. Open the file manager. The top left choices should be your various mountable drives. You should recognize the UID for one of them as the UID you randomly created in the last step. Click on that drive to mount it (UUID of drive may not show until you click it, that’s fine. Just keep clicking until you find the drive with the new UUID).
  6. Run some more esoteric crap to update your grub and fstab config. Grub is the bootloader. It lives in /boot/grub/grub.cfg on the drive you just mounted (should be /media/UUID/boot/grub/grub.cfg). Do a find-and-replace of all the old UUIDs with your new UUID. Also change the line “menuentry Ubuntu” to “menuentry UBUNTU” so you can be sure that you’re booting into the right grub after step 7. Save that file. Then open /media/UUID/etc/fstab and update the UUID there as well. More detailed (longer-winded) version of these instructions can be found in step 5 here.
  7. Ensure drive is bootable. Still in the Live Ubuntu Trial, go to System -> Administration -> Disk Utility. Pick your new disk and unmount it. Then click “edit partition” and choose the “Bootable” checkbox.
  8. (Optional) Update your MBR. If your new partition is on a new drive, you can just setup your BIOS to try to boot off the new drive first and you should be GTG. Otherwise, you can follow the instructions in Step 6 of the aforementioned link to update your MBR. If you don’t see the upper-cased UBUNTU after you point to the new drive, that’s probably a sign you need to update MBR.

After that, reboot and you should be GTG. Seems like a lot of steps for something that ought to be simple, but the tricky bit is to get your Grub/bootup stuff able to disambiguate between two drives that look identical on a byte-for-byte basis.

Ruby slice to end of an array

It’s popular enough to be a Google-suggested search, but not popular enough to have a good result yet.

If you want to slice to the end of a Ruby array, and/or get the end of a Ruby array, what you want is

arr[1, -1] # -1 means all the rest of the array