This morning at Windy City Rails, Dinshaw Gobhai’s talk “Meet the SLAs: Rails at Constant Contact” started with an intimidating consideration: their application, which they intended to build on Rails, would have to serve multiples more requests than most Rails apps. With the “Rails doesn’t scale” idea making the rounds, how were they going to make sure that it did?

Well, at first, by writing two Rails apps: one for the UI and one for the logic. This required separate integration stubs, a node.js gateway, and a bunch of other workarounds that made the next 6 months very frustrating.

So instead, they switched back to one rails app, and learned an important lesson:

Don’t build for scale unless you need to scale. Don’t create problems by solving problems that you don’t have.

That way, Dinshaw pointed out, a new person could walk onto the team and understand the app architecture. I’d point out, too, that this is a scaling consideration as well. Maybe the app can’t scale yet, but if it’s all complicated, the team can’t scale. That’s a problem, too.

Constant Contact moved to a cell architecture, and they seem happy with it.

But how do you go about scaling the app when the time comes?

Start with profiling and benchmarks. Ruby provides a built-in benchmark wrapper. Jesse Stroimer provides a primer. 

When Constant Contact did this, queries were the first thing to optimize. So, the team hand-wrote them with Preload, Eagerload, Includes, and Joins.

Then came a controversial step: using composite primary keys. The thinking behind it came from the fact that multiple-column index is faster than multiple indices, but the internet said “if you’re going to use composite primary keys, don’t use rails). They decided to try it. Rails integrated well with the technique, but…

…the requests were never finishing.

So, what was happening?

In-line caching. Saving output after it is returned to avoid having to do the lookup again the second time it is called. Fine, but if you’re never hitting a class that a method has called before, the program will just keep doing the lookup because it can’t find the cache path. Sound farfetched? Well, in Ruby, an extension is treated as its own, new class. And thus the problem is created. Confused? Here’s an article that lays out the whole thing.

So how else can we speed up the app?

• Database partitioning: reducing the amount of data the program has to sift through on any one request. Be careful what you divide on, though: if you have to make cross-partition queries,  the performance benefits go out the window.
• Serialize: Skip your ORM. What? Here’s a rundown. It advises agains using rails on this case, but the article is comprehensive and worth consideration.
• Avoid extra work: instantiating arrays, downcased copies of strings, other digestion.
• Memoize to speed things up—Gavin Miller offers a good explanation, in Ruby no less, though the lessons are applicable across languages.
• view/action caching: a railscast on the latter, for your viewing pleasure
• Exceptions as part of normal application flow: return a message, rather than an exception. This is more elegant and a way to avoid the “ugly” if statement, as Dinshaw calls it (Dinshaw, let’s talk). My addendum: if you’re going to do this, check out use cases for the and and or operators in Ruby. This great video breaks it down. 
• Log carefully—actually, for a whole host of reasons, please log carefully
• Use delete to handle objects you don’t need anymore: consider it manual trash collection.
• Let the database handle requirements for uniqueness, to avoid doing a read ahead of time, which slows down the system.
• Reconsidering callback functionality: callbacks stop the program momentarily. Are there other ways to do what you want to do? Here’s some wisdom, complete with code samples, on callbacks and how to use them. It’s about JavaScript, but the lessons are universally applicable.
• MEASURE EARLY, MEASURE OFTEN. Don’t solve problems you don’t know you have.