05/9/12

Create a Custom Thumbnail Processor in Carrierwave

The latest in my of “shouldn’t this be better covered in Google and docs if people really use CarrierWave?”-series. Creating a custom thumbnail processor in Carrierwave is pretty straightforward, but not from any search query I could construct. The gist:

class MyUploader < CarrierWave::Uploader::Base
  version :custom_thumbnail do
    process :some_fancy_processing
  end
 
  def some_fancy_processing
    # Here our context is the CarrierWave::Uploader object, so we have its full
    # assortment of methods at our disposal.  Let's say we want to open an image with
    # openCV and smooth it out.  That would look something like:
    cv_image = OpenCV::CvMat.load *([ self.full_cache_path ]) # See past blog post on the origins of full_cache_path
    cv_image.smooth(5,5)
 
    # The key to telling CW what data this thumb should use is to save our output to
    # the current_path of the Uploader, a la
    cv_image.save_image(self.current_path)
  end
end

Just call #custom_thumbnail.url on your Uploader instance, and you should get the path to the custom result you created.

Using this framework you should be able to get CW to perform whatever sort of custom image processing you want. Thanks to these fellows for helping me decode the #current_path magic here.

05/9/12

Howto: Store a CarrierWave File Locally After Uploading to S3

Have now needed to do this twice, and both times have required about an hour of sifting through Google morass to figure out how to pull this off. The situation we’re assuming here is that you have some CarrierWave file that you’ve stored to a remote location, and you want to get a copy of that file locally to manipulate it. With promising method names like “retrieve_from_cache!” and “cache!” and “move_to_cache” you too may become entangled in the maze of what the hell you’re supposed to be calling to accomplish this. Here’s what.

Step 1: Retrieve from your store (the external place your file exists) to cache (your local machine).

my_cw_uploader.cache_stored_file!

After running that, if you call

my_cw_uploader.cached?

You should get a long string that is the filename for the file that got cached. Note that the cache status of a file is only saved on a per-object basis, so if you try something like

Image.find(1).my_cw_uploader.cache_stored_file!

… you will never again see the object that did the caching, and you will never be able to use that cache. Only call #cache_stored_file! on an object you are keeping around.

Step 2: Go access the file that you cached. Not quite as easy as it sounds. For this, I mixed in the following into my uploader class

module CarrierWave::Uploader::Cache
	def full_cache_path
		"#{::Rails.root}/public/#{cache_dir}/#{cache_name}"
	end
end

So now when you call

my_cw_uploader.full_cache_path

You’ll get the full pathname to the file you downloaded.

Hope this helps save someone else from an hour of Google hell.

04/15/12

Ubuntu, Dark Side of Simplicity

The following is my take on how the current passion for UI “simplicity” may be to blame for Unity, the downfall of Ubuntu as I’ve known & loved it. It’s wide-ranging, so I won’t be able to fully substantiate its every point, but if you simply click the area you’d like to know more about, I will wish that you were being shown detailed examples in support of my point. Many of the newly introduced usability issues in Unity are shared by Gnome3, so it seems that now is as good a time as ever for people who care to try to persuade some Linux distro to retain the high standard of developer usability we’ve become accustomed to.

#—————————

Steve Jobs broke my OS, and I don’t even use a Mac. It began about 10 years ago, around the time Jobs had re-joined Apple, and the software industry was smitten with building UI that had every button and link you could need. “If they might need it, why not include it?” seemed to be the effective rationale.

good-ole-days

Windows XP represented the pinnacle of the art; complete with a “Start” button, that, when clicked, exploded into two columns. These columns in turn had menu options like “All Programs” that could themselves balloon out to several more (overlapping) columns. In the case of the “All Programs” specifically, the user was treated to an unordered list of every program they had ever installed (often hundreds). It was so…terrible…yet quick to an advanced user (e.g., those that figured out how to sort it). For new users, well, you could probably figure out some of the basics within a week.*

But soon people began to decide this was arrangement was not ideal. Or even OK. I noticed this in full force with the release of the first iPhone. It was a device that was so stripped down that it didn’t include a feature (secure network access for business email) that would could have increased its user base significantly. It launched anyway, was quickly dubbed the “Jesus Phone,” and has managed to sell a couple gajillion units since.

Gone forever were the days of “the most commercially successful” products were “the most feature-full” ones.

This evolution, which I’d pin as starting around 2006 (first iPhone) has continued expanding its base of believers to present day. Now, in addition to the set of Apple devices, the default aesthetic for web consumer products has become “huge buttons / huge text / nothing that requires reading.”

In the context of the web, I think that this growing obsession with simple UI is usually a great thing. Like it or not, our attention is fragmented and life’s too short to read the manual for a product that I simply want to entertain me (see: Twitter, Instagram).

The problem is when the momentum for this trend** pushes it out to use cases where it makes no sense. Sometimes, a detail-rich interface is required to get the job done efficiently. In the case where an app is used by novice and sophisticated*** users, a balanced curve of “level of detail shown to user” vs “user expertise level” might look something like this

balance_complexity1

That is, this balanced approach would dictate that novice users were exposed only to the most essential 10-20% of all UI complexity. The UI should appear very basic to them. As the user’s needs become more sophisticated, the UI reveals contextual choices and/or configuration menus that accommodate their needs as power users of the product. Novice users are happy because they don’t see the complex pieces. Sophisticated users are happy because they can use it with maximum productivity so long as they’re willing to read a handful of configuration menus.

Products rarely end up this balanced. Windows XP threw the user in the deep end, both in terms of the learning complexity, and the vast sea of choices/links presented to even the notice user. OS X freed users from this soup of links and options, though before they got smart about context sensitivity, it often came at the expense of more clicks.

Ubuntu, pre-Unity, was arguably even worse than XP to the poor novice:

balance-pre-unity

No oversized buttons or contextual UI reveal here. The reason the project thrived was only because the Ubuntu audience is made up largely of users who have advanced expectations for their OS. Many are programmers. They have to juggle IDEs, web browsers, web browser tools, and a smattering of terminal shells. Usually across multiple high resolution monitors, over multiple workspaces. To them, if complexity is the price that must be paid for configurability, then it shall be paid****.

This isn’t the sort of thing a novice will understand, let alone feel comfortable with, but the software did meet the needs of it sophisticated user base.

Then came tablets, the momentum of simplicity, and Ubuntu’s loving ode to it all: Unity.

balance-post-unity

Because this blog needs to get finished before tomorrow morning, I am forced to gloss over a detailed analysis of the functionality lost between pre- and post-Unity versions of Ubuntu. A couple salient examples include: well-integrated dual monitor support, multiple/configurable taskbar support, desktop customization, and ability to keep program menus within each program. For the quantitative-minded amongst you, the compared market share of Ubuntu vs. Mint makes the point more compellingly than my mini-list.

If it’s not already obvious, I love the trend toward simplicity. It was one of the main points of opportunity I saw in starting Bonanza back in 2008 — we sought to build a version of eBay that would be usable to busy people and non-experts. Simplicity continues to be something that I push for as often as anyone on my team, and I think it continues to be a big difference between our platforms.

But I don’t believe that “simplicity” should be the same thing as “dumbed down,” and I wonder if Unity’s pared down featureset could be a result of the Ubuntu designers mistakenly conflating “simple” and “feature-sparse”?

tl; dr

“Simple” and “effective” are closely related for novice users and for simple products. But they can be inversely related when “simple” gets in the way of “configurability,” which begets effectiveness for power users. In the case of Ubuntu, the users are largely geeks who use complex features to maximize productivity. Give the pared-down version of Ubuntu to novice users, but don’t let it rob the majority of your users the functionality they need.*****

Update: Finally inspired to post this to HN after reading Linus’ comments about Gnome3 being a detriment to usability. Given that Gnome3 has traveled a very similar path to Unity in terms of degrading the user experience (for sophisticated users) with its newest release, I am hoping that perhaps a sympathetic designer of Unity or Gnome3 might find this.

Footnotes

* Though as a computer repair guy, I often saw the concept take far longer to sink in. And don’t even get me started on trying to teach my grandparents exactly what a file system was and “what it did”

** Regarding use of “trend” to label the simplicity movement: I mean only that it is influencing all corners of design (web, native apps, mobile, and beyond) — not that it is ephemeral or irrational.

*** “Sophisticated” here means “more advanced,” or “more demanding,” not somuch the “better looking” or “more expensive” connotations of the word.

**** Of course, something doesn’t need to be complex to be configurable. Progressively revealed / contextual UIs can often deliver much of the best from both worlds. But it’s also easy to get implement rather intricate revealing schemes incorrectly and be worse off than if you had simply built a cluttered but static interface.

***** What makes it doubly insulting is that until Ocelot, we could get the functionality we needed by choosing the “Classic Ubuntu” login. What explanation is there to chop a feature that’s already been built…and provided the main lifeline to advanced users after Unity’s release?

04/8/12

Fixed: My i7 Intel Dell Laptop is Ridiculously Slow

Most of the Google results I found when digging around on this subject pointed to usual boring causes of slowness: too many programs being run on startup (which you can test with ms-config if you’re running Windows), anti-virus software, and other boring stuff of that sort. In my case, I had been running Ubuntu so most of those tips are moot. But to be thorough, I did remove practically any and every resident program that was running on what should have been a zippy Dell Latitude E6520 with a i7-2720QM (2.20GHz, 6M cache) processor.

And yet, running a utility that averaged about 5 seconds on my desktop consistently took 30 seconds on my laptop. Except for every once in awhile, when it would take 6 or 7 seconds.

Before splurging for a new laptop, I decided to take a peek through my BIOS settings and managed to stumble across the culprit: the Intel “Speed Step” feature. On my Dell, this was under the “Performance” settings. I guess that the idea of Speed Step is that the i7 powers itself down when it decides you’d like your system to perform like a 486. Whatever the logic is that determines when to power down was clearly NOT working as intended on my laptop. After disabling Speed Step, I have been running for the entire day at speeds very similar to my desktop.

Hopefully someone else thinks to Google for this problem and find themselves helped by a similar approach. FWIW I suppose that this might mean that the laptop uses more battery, but you can be an informed consumer about whether you want to run fast or power-efficiently.

03/15/12

Fix it: Ubuntu CTRL-SHIFT-V Won’t Paste Into Terminal

Ugh, just spent an hour traveling from forum to forum trying to figure out why I couldn’t CTRL-C in Rubymine and CTRL-SHIFT-V into terminal. As many forum posters were eager to point out, it is possible to use CTRL-SHIFT-INSERT or middle click to paste into terminal, just not CTRL-SHIFT-V. Unfortunately, those workarounds were not OK since my insert key is in the arctic circle of my keyboard, and I don’t want to touch the mouse.

Luckily, the folks at Rubymine helped me figure out the answer where Google couldn’t.

The problem is that when running Rubymine via OpenJDK, the clilpboard used is not compatible with the clipboard used in terminal. The solution was to run Rubymine with Oracle Java instead. In Rubymine, this is a matter of first installing Oracle Java (I’ll let you Google that one, you have to add a repository) and then adding the following lines to the beginning of your Rubymine startup script:

export JAVA_HOME=/usr/lib/jvm/java-6-sun
export JDK_HOME=$JAVA_HOME
export RUBYMINE_JDK=$JAVA_HOME

After that you should be golden. In my hour of Googling I read that many other IDEs (Netbeans in particular) seem to be subject to the same problem with CTRL-SHIFT-V not working. I’d reckon that if these users were to change their application to use Oracle Java it would probably resolve the problem in other IDEs as well.

11/12/11

Linux Mint Firefox & Chrome :: Remove Search Branding

I’m all for Mint Linux making some bucks via their Chrome and Firefox searchers, but not if it comes at the expense of basic usability. <quickie rant> If I were the Mint maintainers, I’d take a long look at whether it was desirable (let alone essential) that they hijack my CTRL-K functionality and replace standard Google results with their poorly formatted, functionality-impaired substitute.</quickie rant>

Anyhow, if you are here, you’re probably trying to figure out how to remove the Mint branded search from Firefox and/or Chrome. And I’m here to tell you how.

Remove Search Branding from Firefox

  1. Click on the Google search icon in your title bar
  2. Click “Manage Search Engines”
  3. Click the link to “Get more search engines”
  4. Choose a Google, any Google, from Mozilla’s choices. I chose Google SSL, which worked nicely.
  5. After you install your Google SSL (or other version of Mozilla version of Google), click “Manage Search Engines” again, and move your new Google Search to the top of the list.
  6. Voila!

Remove Search Branding from Chrome

There are probably an assortment of ways to accomplish this. I chose to Google “Chrome deb package” which led me to Google’s official distributions of Chrome, which can be found here. After following Google’s instructions to install my Chrome package, all was well (though that meant that I was running “Chrome” rather than “Chromium.” Whatevskis.)

Other than the annoying search stuff, so far Mint Linux seems to be an easy-to-setup iteration on the developer utopia that Ubuntu was built as, before it decided to go the way of the mandatory Unity.

07/10/11

Ruby debugger list local variables

Short and sweet. You want to list local variables in ruby-debug? Try

info locals

You can also get ruby-debug to list your stack, instance variables, and much more. Type in plain old “info” into debugger to see a full list of what ruby-debug is willing to reveal to you.

05/17/11

Likes & Dislikes: The Product Edition

Anyone who has followed my blogs over the last couple years knows that I’m a very big fan of the like/dislike list. But I generally try to exclude products from my lists since they don’t have that “essence of life” quality that I’ve strived for in my lists.

But products are important, too. So here you have it: a like/dislike list dedicated to the products I use or have used. I’ll actually split this particular list into four levels of like because I can quantify more precision when it comes to products.

Love

  • VMWare. Being able to seamlessly run Ubuntu & Win7 side-by-side (and have both of them performant) still feels like the coolest thing ever, even after doing it a year.
  • Rubymine. See: favorite Rails tools
  • New Relic. See: favorite Rails tools
  • Quora. Today it is very good. And if they don’t mess it up, next year it is going to be the oracle that has an intelligent answer for everything.
  • Google Search. So easy to take for granted, given how it is woven into every minute of our lives. But can you imagine a world without it? Try using any other search engine for solving programming problems if you want to remember why Google search deserves your love.

Like

  • Windows 7. Terrible for programming Rails, great for UI/usability and productivity.
  • Ubuntu. Great for programming Rails, passable for usability and productivity (Gimp and Openoffice sure as hell ain’t no Photoshop and MS Office)
  • Microsoft Onenote. I have found no better tool for mapping whatever arbitrary structure/idea from my brain into tangible existence.
  • Firefox 4. It has Firebug.
  • Google Chrome. Introduced to the world the realization that we were browsing at half-speed. Low memory footprint.
  • Github. The world of open source programming could accurately be talked about in terms of “BG” and “AG”. It is not an overstatement to say that, along with Git (see below), Github has completely and utterly revolutionized the world’s ability to collaborate on complex projects. The residual impact of that change is hard to grasp.
  • Stackoverflow. Opportunity for them to move to “love” if ever they could build a half-decent search… I still use Google to find answers that I suspect are somewhere on Stackoverflow
  • Amazon. Like Google Search, above, it is such a part of our daily lives that it’s easy to take for granted. But also like Google, imagine shopping for commodities without it. Not to mention their efforts to lead the cloud computing movement.

Deeply Divided

  • Git. The “deeply divided” category exists specifically for git. On one hand, I love what it lets me do (effectively manage source control). On the other hand, I despise how unnecessarily arcane the syntax is, and how the documentation feels like it was written by a seemly unbathed newsgroup

Dislike

  • Rhythmbox. Happy to remove them from this list if they can ever pull off the herculean feat of not considering the word “The” when sorting artists by name. Update: ho, shit! A hero!
  • Dell’s website. Inconsistent, buggy, hard to shop. Do like: Dell’s products, esp the pricing of them)
  • eMusic. Every company has a right to pivot and drop their early adopters — it is business. But the manner in which eMusic made this pivot (with an utter lack of advance notice and concerted effort to mask what was really happening) still rubs me the wrong way when I think about it.

Despise

  • Quicken. Everything feels like it takes 5x longer than necessary.
  • Microsoft Money. Try to use it sometime.
  • Playstation 3. Already hated their constant 45 minute system updates; and days after I wrote that blog post, they give away my CC and password to the Internet. Bang up job, guys.
04/22/11

EC2 vs. Heroku vs. Blue Box Group for Rails Hosting

Not to kick a company when it’s (literally) down, but today’s 12+ hours EC2 outage has finally driven me to write a blog post I’ve been holding in my head for several months now: comparing a few of the major players within today’s Rails hosting ecosystem. In this post, I’ll compare and contrast EC2, Heroku, and Blue Box Group. I’ve chosen these three not only because of their popularity, but also because I believe each has a value proposition distinct from the other two, which which makes each ideally suited for different types of customers. In the interests of full disclosure, we are a current Blue Box Group customer, but we have spent a great deal of time looking at our choices, and I think that all have their advantages in specific situations. The facts and opinions below are the result of weighing data gleaned from Googling, popular opinion, and three years running a Rails site that has grown to serve more than 2m uniques per month.

Heroku vs. EC2 vs. Blue Box Group, Round 1: Price

Let’s start by establishing a pricing context. We’ll use a dedicated (not-shared resource/virtual) 8 GB, 8 core server with 1TB bandwidth as our baseline, since it is available across all three services (with some caveats, mentioned below).

Heroku EC2 (High-CPU XL) BBG
Dedicated 8-Core Server with 8 GB $800.00 $490.00 $799.00*
1 TB Bandwidth $0.00 $125.00 $0.00
Total $800.00 $615.00-695.00** $799.00

* In the case of BBG, their most current prices aren’t on their pricing page. They should fix that.

** This doesn’t include I/O costs for Amazon EBS. While these are fairly impossible to predict (varying greatly from app to app), it sounds from Amazon like you’d be talking about something more than $40 for this. Give that we’re comparing a “high end machine” here, perhaps $80 might be a more accurate estimate, that would make the price more like $700.

Various minor approximations were made to try to get this as close to apples-to-apples as possible, but the biggest caveat is that the Heroku instance (Ika) only has about 25% the CPU as the EC2 and BBG instances (though it has the same amount of memory. They don’t configure their DBs with comparable CPU muscle). The next highest Heroku instance (Zilla) is $1600 per month, and more comparable to the other two in terms of CPU, but has twice as much memory as they do. Note that EC2 and BBG make offer discounts when committing to a year of service — I couldn’t find a comparable offer from Heroku, which is not to say that it doesn’t exist (readers?). These discounts typically range from 10-25% off the no-commitment price.

Heroku vs. EC2 vs. Blue Box Group, Round 2: Setup

Heroku is ridiculously get started with, the runaway winner of the bunch when it comes to hitting the ground running with zero red tape. Per their homepage, all you do is run a couple rake commands and you’re in business. Even cooler, they offer a vast and useful collection of add-ons to make it easy to get started on whatever the specific thing is that you app is supposed to do.

Setting up Rails with EC2 is not quite the same walk in the park, but it’s not necessarily bad. Amazon handles configuring the OS for you, so in terms of getting your app server setup, you are essentially just getting Ruby and Rubygems installed, and letting Bundler take care of the rest. If you managed to set up your development environment in Linux or on a Mac, chances are you won’t have too much trouble using packages to fill in the gaps for other non-Ruby elements to your application (like Sphinx). When EC2 gets trickier is when you start figuring out how to integrate EBS (Amazon Elastic Block storage, necessary for data that you don’t want to disappear) and the other 20 Amazon web services that you may or may not want/need to use to run your app. It can ultimately amount to quite a research project to figure out which tools you want, which ones you need, and how to tie them all together. That said, you may end up using these tools (S3 in particular) even if you use BBG or Heroku, so that cost is not entirely unique to using EC2.

Ease of getting started on Blue Box is somewhere in between EC2 and Heroku. There is no high-tech set of tools that automatically build stuff like Heroku, but unlike EC2, you have a friendly and qualified team willing to help you get your server setup in the best possible way. In my experience, when they have setup new servers, they will ask in advance how we plan to use the server, and then automatically handle getting all of the baseline stuff we’ll need installed such that we can just focus on deploying our app. Which brings me to Round 3…

Heroku vs. EC2 vs. Blue Box Group, Round 3: What’s Best Suited for What?

For pet projects, small sites, or newly started sites, I think that hosting with Heroku is a no-brainer. You can be up and running immediately, you get a huge variety of conveniences with their add-ons, and there is a wealth of Google help behind you if you should happen to encounter an trouble, given the immense user base Heroku has managed to establish. All three of these these services can scale up available hardware within hours/minutes-not-days (yay for clouds!), but Heroku is probably the most straightforward to rapidly grow an application with their “Dynos” approach. However, given their highest cost amongst the choices, and their lower-than-BBG application-specific support, the significance of those advantages will erode as your application grows into the 10′s of thousands of monthly visitors.

I believe that EC2′s greatest selling point is its price, with its scalability and ubiquity (= generally good Googlability) being close seconds. As detailed above, on balance, EC2 tends to run 20-30% cheaper than other choices by leveraging their immense scale. Nifty features like auto-scale have the promise of making instant growth possible if you get flooded after your Oprah appearance. The trade-off for those advantages is that you will get 0 application-specific support, and even getting generic system-wide support can be hit-and-miss, as folks who suffered through today’s EC2 outage learned firsthand. Transparency is not Amazon’s strong suit at this point in their evolution, which can be a real problem if you have real customers who depend on your product and want to know when they can expect to see your website again during an outage. Also, as mentioned in the setup section, figuring out your way around the Amazon product ecosystem can be dauting at first.

I would consider EC2 the best choice for intermediate-sized businesses, particularly if 100% uptime is not imperative to their existence. EC2 is a great option for bootstrapped startups who want to get online as cheaply as possible, and are willing to put in the extra work setting up their servers in exchange for those cost savings. Also, since you will probably be unclear about what kind of resources your app is going to consume as it scales, EC2 is a great proving ground to get a sense for what kind of resources you might need if you decide to venture beyond Amazon for improved reliability and service. I would also take a long look at EC2 for huge businesses that can afford their own IT department, which diminishes the significance of EC2′s lack of application-specific support or monitoring.

While their prices are competitive with EC2, I would assert that the real differentiator with Blue Box is their focus on service. Included by default for business-sized BBG customers is 24/7 pro-active human monitoring of all services, including the ability to bring servers back online if they should happen to crash and you’re not around. Having gone through a fair number of web hosts in our day, we we have come to realize that, once you are signed up at a given host, it can be a huge pain to change. Most hosts use this knowledge to their advantage, and after a very romantic honeymoon period, become inattentive to their customer’s needs after it becomes clear the customer would be hard-pressed to move.

At Blue Box, their customer-focused attitude has not diminished a bit over time. We still regularly find them answering our questions…on the weekend…within minutes of the question being sent. Equally important, Jesse Proudman (BBG CEO) has built a team around him that gives the customer the benefit of the doubt. In more than a year of being hosted at BBG, I can not ever remember them “blaming” us for server changes we’ve made that have caused havoc (not the case at some of our past hosts). Instead, BBG has a solution-focused team that is consistently personable, reasonable, and most importantly, effective when it comes to solving tricky application and server problems.

While BBG offers small VPS instances, as well as cloud servers that can quickly scale, I consider their sweet spot to be businesses that have grown beyond the point of being able to easily maintain their server cluster themselves, but they don’t want to hire an on-staff IT guy. Or maybe they do have an IT guy, but they really need two. Over the past couple years, BBG has been our “IT guy,” working to implement systems for us ranging from a Mediawiki server, to load balancers, to Mysql master-master failover clusters. And compared to having a real IT guy on staff, the price is a huge bargain (not to mention the savings on health insurance, taxes, etc.)

Another nice benefit for those that have been stung by EC2/Heroku uptime hiccups: in 20 months with BBG, our total system downtime has been something between 1-2 hours (excluding downtime caused by our mistakes).

Conclusion

The best host for a particular Rails app depends on a number of factors, including phase of development (pre-launch? newly launched? rapidly growing? already huge?), need for 100% uptime, makeup of team, and cash available. Hopefully this post will be helpful to someone trying to figure out which host makes most sense for their unique situation. Please do post any questions or anecdotes from your hosting experience for future Google visitors.

Update: Response from BBG

Blue Box emailed me after this post, with a few extra details that I believe are pertinent:

  • 4 x 300GB 15k SAS outperforms EBS (which both Heroku and Amazon rely on) by almost 30% based on our client benchmarks.
  • Neither Heroku nor Amazon provide 24 x 7 monitoring with proactive human response. This can be is a *key* differentiator, particularly when comparing costs.
  • All of our dedicated database instances are running bare metal, meaning you gain consistent and reliable performance, and aren’t subject to similar massive outages caused by the muli-tenancy of a SAN.

If anyone from Amazon or Heroku would like to provide extra details of what makes them a strong choice, I’d be only too happy to post those as well.