CodeBaby collects data on this interaction, and each animation has many events that need to be captured and stored. I joined the project to help build a system that will collect this data and make it available for analysis and reporting.

After diving into the problem, it quickly became apparent that storing the data in a normal database would not work. Currently, the animation event data is being stored in a MySQL database. This works - to a point. CodeBaby is growing quickly, and after analyzing some numbers, it was clear that the data storage requirements would reach the limits of MySQL or any relation database system pretty quickly.

So I began to investigate other alternatives. CouchDB quickly made its way to the top of the list, so I set out to build a prototype system using CouchDB and Rails.

The first step was to setup a CouchDB server and learn how it works. I was able to get CouchDB 0.9.1 up and running pretty quickly on both Amazon EC2 and Rackspace Cloud Servers using Ubuntu 9.04. Once running, I was able to use the web interface to CouchDB - Futon. Futon is actually a very good utility. You can do most everything you need to in the browser, including changing server config values (you occasionally have to restart the server via the command line).

Once I had CouchDB running, I needed to load sample data into a database. Using Ruby and the couchrest gem, this turns out to be very easy. In short order I had a sample database setup and populated with about 100K documents. At several hundred MBs, this is a nice size to work with for development purposes. I was able to try out CouchDB's replication support between the EC2 server and the Rackspace server. Replication in CouchDB is easy and painless. CouchDB supports much more advanced replication functionality, but it's very good at the basics.

At this point, I should note that CouchDB is a good answer for certain problems, but not others. If you plan on building a typical Rails web app, I would still recommend MySQL or a traditional relational database. CouchDB is the better choice for problems that don't have a well defined schema, or where you need to store a lot of data that does not change often, and need to be able to read and sort on that data - a pretty good description of what CodeBaby needs.

After getting CouchDB setup and populated with data, it was time to learn about CouchDB views. If you are like me, and have spent your development career using SQL and a relational database, be prepared to spend a solid two weeks learning views. A CouchDB view is similar to a view in MySQL, in that it lets you retrieve a subset of data, but the similarity ends there. CouchDB views are built using Javascript and the MapReduce algorithm. Results of the MapReduce calculations are indexed and stored in a b-tree on disk. Essentially, this means that the first time you run a view in CouchDB, it takes time. Minutes or hours, depending on how many document you are storing. After that, views are very fast, and this is the real power of CouchDB. If you have a CouchDB database containing hundreds of millions of documents, you will be able to filter and view those documents much faster than if you had the same data in MySQL.

CouchDB seemed to be a great fit. I was able to build a prototype system with CouchDB and Rails pretty quickly on Amazon's EC2 infrastructure. But when it came time to run performance tests, I discovered some issues.

Because of the way CouchDB indexes and stores views on disk, writes to the database are serialized. This was a bottleneck. Using specific data from CodeBaby, I was able to get about 300 document inserts per second. This was after optimizations such as generating each document's '_id' value, rather than letting CouchDB handle it. 300 inserts per second turned out to be too slow, and short of setting up a complicated system of multiple databases and intelligent load balancers (using a consistent hashing algorithm), I came to the conclusion that CouchDB would not work fro this project.

To be honest, I was a bit disappointed. I like everything about CouchDB, from its simple API to the way you write views in Javascript. CouchDB is a solid product and I'll definitely consider using it again in a future project.

Quiltivate.com is a community for quilters. You can browse hundreds of quilt block patterns and see what other quilters have built.

Today, Quiltivate launches a new service called the Quilt Builder.

What do I know about quilting? A lot more than I used to. My wife Kacie is the real expert though. I've been working with her over the past year or so to build a service for quilters to make their life easier. When it comes to planning a quilt, graph paper and colored pencils still rule the day. And if you are unlucky enough to spend money on the software available to 'help' you plan a quilt, you'll be ready to gouge your eyes out with those colored pencils after you try to figure out the software. The leading software available for quilters today costs well over $100 and you have to take a class to learn how to use it. And it requires Windows.

So we decided to create something simple and easy for quilters. Give it a try. I'm willing to bet if you are reading this, like me, you don't know much about quilting. But I'm willing to be you can figure out the Quilt Builder.

Quiltivate is built on Rails, of course. The Quilt Builder is a Flex application, talking to Rails behind the scenes. I had quite a bit of help building the Flex app from Daniel Wanja over at OnRails.org.

I've invested a lot of time, energy, and money in this project, and I'm really excited to see it launch. Kacie and I have built this service by continually reminding ourselves that good design matters and that 'less is more' - we are definitely trying to under-do the competition. We think the Quilt Builder does one thing extremely well. If you know any quilters out there, be sure to let them know about Quiltivate.com!

Once you have a merchant account setup, it tends to run fairly smoothly. You pay the monthly fee and money shows up in your account, minus the occasional chargeback.

But what happens when you need a second merchant account? Why would you even need a second merchant account, or a third or fourth for that matter?

If I told you I can bring $250,000 in revenue to your business every month, you would probably do whatever it takes to work with me. But it's not that way with merchant account providers. Surprisingly, merchant accounts have a monthly cap on the total amount of transactions. So if your business is doing $75,000 per month in transactions and your growth shows that you will be at $100,000 next month, and your merchant account is capped at $80,000 per month, you had better find a second merchant account soon.

You would think that the more money you send to a company the better, but not with merchant account providers. It turns out they do not like chargebacks. The more transactions you send their way, the more likely they are to have to process and pay chargebacks - which eventually get billed to you, but not immediately.

So if your business is processing a high dollar amount in transactions each month, you will end up with multiple merchant accounts. This causes some interesting problems in whatever application or billing system you are running. First, refunds must go through the same merchant account as the original transaction. Merchant account providers will scream if you charge a customer $20 on a separate merchant account and refund the customer on their system - and for good reason. It's easy to commit fraud when you can't trace the flow of money through a system. Second, if you 'use up' the monthly limit on one account half way through the month, you must send all new transactions to a different system. You will end up creating what amounts to a 'load balancer' for merchant accounts to ensure that you do not reach the limit on one merchant account too soon.

The client I am working for processes a large amount of transactions each month and we have done what I've outlined above. Our team has essentially learned on-the-fly and built a system that can handle refunds and charges correctly. A load balancer for merchant accounts. Along the way, we've added a half dozen merchant accounts to the system and more are on the way. Once again, the ActiveMerchant library has been great. Some of the new merchant accounts were supported already, and with others, we've had to add support. Here are two that we are using that will someday make it into the ActiveMerchant library -

ActiveMerchant Support for the JetPay Gateway

ActiveMerchant Support for the FirstPay Gateway

An interesting side note on this issue. Many companies deal with this multiple merchant account issue and it can be very tricky. In particular, the chargebacks can really cause problems. A few years ago, the Denver-based airline Frontier had to file for bankruptcy. The cause, in simple terms, had to do with merchant accounts and chargebacks. So many flyers were telling their card company not to pay for already purchased tickets (for weather delays, cancellations, etc) that Frontier ended up having cashflow issues with their merchant providers. The only solution was to file for bankruptcy and fix system.

So what have I been doing? Simply put, I've been building a large system using both Ruby and Rails. Remember that small project for ID Watchdog (IDW) I built? Well, it turned into something much larger.

After launching the new signup system, the team I'm working with continued to iterate, build, and maintain applications for IDW. Legacy systems needed to be migrated to the new system we had built. New processes had to be designed and implemented. Then the real fun began: IDW had an IPO in August 2008, and since then IDW's growth has been off-the-charts. All of the systems we built in the first half of 2008 needed to be modified to handle this tremendous growth. New third party APIs needed to be added. Databases had to be optimized. Servers were added. More developers were hired. In short, the project grew very fast.

Some of the highlights while working with IDW in 2008:

I designed and built the core functionality for IDW's monitoring process using Ruby and BackgrounDRb. I designed and built a distributed billing system in BackgrounDRb. I wrote code to interface with a handful of third party APIs. Some of the code is set to become open source patches to the ActiveMerchant library. Our team made the migration from Subversion to Git, and never looked back. We built an API for the IDW system, which is now in use by hundreds of companies every day. We added servers as needed to handle the load. Finally, as the system began to push the limits of BackgrounDRb, I moved the background processing components to the delayed_job queue system.

In a nutshell, this is what I do. I build applications and systems for companies. Often I will start with a small piece of the puzzle, and work with the company over time to make sure the system will grow and scale as the company grows. Both Ruby and Rails are perfect for companies like IDW, where flexibility and short iterations are necessary to keep pace with the company's growth.

I've been doing this for over 10 years now, and it's what I love to do. I'm passionate about building applications and systems. I'm fortunate that I can work on projects that are interesting, using tools I love, with a great team of developers.

If you are working with Merlin API data inside a Ruby or Rails application, be sure to try the library.

If you are building a Ruby or Rails application and need to work with IDology data, have a look at the ruby-idology library.

In late December 2007, a project I've been working on for the past few months was released to the public. The company I've been working with, ID Watchdog (IDW), needed a better way to process signups for their identity theft prevention service.

Enter the new signup system.

Working with IDW's design team, I built the new signup system and added several handy features. As signup systems go, this one was pretty complex. Most signup systems take care of the basics - gather the customer information, collect a payment, and send the customer on to the application. IDW needed something a bit more complex.

When it comes to people's identity, security has to be the top priority. So when asking for a potential customer's information in order to monitor their identity, the application must make sure the customer is indeed who they say they are. If you were to signup for an IDW service, you would be asked questions to verify your identity. Questions that someone who is trying to impersonate you would not know.

This screening system is part of what I developed for this project. There are also a few other behind-the-scenes extras that were developed to keep customer information secure. I should note that I worked with a great team of Rails developers on this project, though most of what I am describing here is specific to my direct involvement with the project.

The other members of the team developed several internal applications for IDW to use along with the new signup system. These applications interface with an existing CRM system and also give IDW control over the identity monitoring process for each customer.

So what about the Rails-specific parts of this project? There were actually several problems that were a fun challenge.

First, we wanted to encrypt all customer information in the database (we used PostgreSQL on this project). This was important for PCI compliance and I would argue it's common sense when it comes to storing identity information. Using the Sentry plugin and OpenSSL I generated a very secure public / private keypair that is used to encrypt all data in the database. If you ever need to encrypt data with Rails, not just hash data (which is not nearly as secure), Sentry is the best tool out there.

There were also several third party APIs that had to be integrated with this project. IDology and Merlin are two services that deal with personal information. Unfortunately, neither service had a Ruby library available to interface with their API. We needed a solid interface for these two services, so I spent some time with the documentation for each. I made use of rspec and hpricot and developed two great libraries for communicating with IDology and Merlin. I'm working on getting each released as a gem or plugin. Get in touch with me if you are interested.

I was also in charge of the infrastructure for this project. Without revealing too much sensitive information, I worked with a hosting company and setup a multiple machine platform for secure deployments via VPN. I made heavy use of Capistrano, and the site runs on Apache, Mongrel, and PostgreSQL.

It was great to launch this project in December. IDW actually wanted to hold off until we had a few more features implemented, but our team was able to convince them that releasing early and often with short iterations is a better approach. If we had waited until everything was finished, the project would be well into March before anything was released. Needless to say, there is much to do, and I am looking forward to working with IDW for the next few months as we add features and make their service even better.

NationwideSpeakers.com contacted me about maintenance for their existing site. It had been developed using Rails way back when Rails was in version 0.9 or so. My immediate task was to add some new features, but not long after getting access to the code, I realized the site needed a major upgrade.

With Rails 2.0 just around the corner, it was time to upgrade the application from pre-1.0 Rails coding standards.

After talking with Nationwide and explaining that an upgrade was badly needed, I brought another developer to the team and we started digging through the Rails application line-by-line looking for old, broken, or deprecated code. It took us about a week to get everything converted.

One major issue we discovered during the upgrade was that the older Rails code was using file-based storage for session data. That had caused over 3 million files to build up on the server, and needless to say, it was starting to impact performance. We moved session storage to the database and added some housekeeping scripts to cleanup old sessions.

During the conversion, while prepping the server for deployment with Capistrano, I realized that Nationwidespeakers.com was not running any backups on the production data or source code. I used the upgrade opportunity to ensure that production databases are backed up nightly to a secure off-site server. I also moved the Subversion source code repository to a secure hosted service, separate from the main application server. When it comes to backups, my rule is not to put all the important data on one server.

After a few weeks of work, the upgraded NationwideSpeakers.com site was deployed successfully.

So while most of what I do involves creating new applications, I will occasionally take on projects that are more maintenance-related. NationwideSpeakers.com is a perfect example of what my team and I can do for your existing Rails application.

Give me a call at 719.966.4313 if you would like to talk about your existing Rails application.

No doubt half the people in the audience are spacing out and dreaming about the iPhone release tomorrow...

Here are the slides.

I told my former client that they need to find someone passionate about what they do. Right now, if you are involved in the Rails community, you are more than likely passionate about what you do. And if you do freelance work like I do, that passion matters.

I would much rather hire someone who is passionate about design or development than a "scripter" or "coder" who punches the clock and is content using what Microsoft or Sun give them. When I hire other developers to work on projects with me, I look for passion. The same goes for designers.

My former client was posting their Rails development jobs on Monster and a few other mainstream job sites, and was surprised that all they got in response were form letters from code sweat-shops in India or Romania. I told my client to look at sites like jobs.rubynow.com, the 37signals job boards, and other sites that attract the readership of passionate developers.

I'm not sure they liked what I had to say, but it needed to be said. I also told them that having a good developer - someone who understands Rails - interview prospective developers is key. Interviews are always difficult, interviewing a prospective developer is even more difficult, and having a non-technial person - who does not understand Rails - interview someone for a very technical position is a recipe for disaster.

Also, if you are hiring a Rails developer and they are using Windows, you had better start looking elsewhere soon. It all comes back to passion. Passion for the tools used to develop web applications. Ruby on Rails is a tool. The Mac and OS X are tools. They happen to be the best tools out there right now to develop web applications. If you are hiring a Rails developer, you need to find someone who has passion for their tools, who lives and breathes their work. That is the type of person you want to hire.

I'll stop before I descend into much more of a rant. Hopefully you understand at least part of my passion. I'm available for work. Get in touch with me - give me a call at 719.966.4313 - if you are looking for someone who is passionate about Rails and their work.

RailsConf 2007 was great. It was bigger than last year's event, by about 1000 attendees. The event felt bigger, and seemed to have less of a community feel to it. Last year, when all the attendees were in the same hotel, everyone gathered at the bar and discussed Rails-related issues, hacked away on projects, or played 8-way Mario Kart on the DS Lite. This year was missing some of that, but it was still good.

By far the best event at RailsConf 2007 was the RejectConf - held at the Portland Free Geek building. Free Geek is a pretty cool service to the community. I would like to see more of them around the country. RejectConf was a gong-show style event, where anyone could present their idea, as long as you agreed to leave if you were booed off the stage. I don't think anyone got booed this year (the Microsoft representative did last year), and I got to see some amazing presentations. People presented their plugins, gems, projects, and ideas to a packed room. The most interesting project I saw presented was Rubinius. I had heard about it at last year's RubyConf, but did not know how far it had progressed since then.

In addition to RejectConf, there were a few presentations during the conference tracks that are worth mentioning. Tom Preston did a great job talking about his FixtureScenarios plugin. I love test driven development, so fixtures are important to my development process. They have some issues in the current version of Rails, and Tom's plugin does a great job of making them more flexible. Another presentation worth attending was the Twitter guys talking about scaling their Rails application. Twitter is arguably the biggest Rails app out there right now, and they are having some issues scaling. What I found interesting is that Rails allows Twitter to focus on the 'real' scaling issues, not the mundane stuff. In other words, Rails does scale by taking care of the mundane, and lets Twitter deal with real problems of scaling - issues that any technology would have on a site as big as Twitter.

The issue of Twitter and scaling ties into the third great presentation I attended - Jason Hoffman of Joyent. Jason presented all of the concrete numbers and costs associated with building a large-scale Rails appclication. I get a lot of prospective clients who like to ask "will my app scale?" To me, this question is a "shark-attack" question. Sure, you could get attacked by a shark if you go swimming in the ocean, but you should probably worry about other things first, like rip-tides, water temperature, etc. If you ask me this question, I will usually respond with numbers. It's hard to argue with concrete numbers, and that's what the Joyent presentation did a good job with. If Twitter is getting 11,000 requests per second at peak, they need roughly 32 cores to handle the traffic. Is your app going to be getting 11,000 requests per second? How about 1000?

All told, RailsConf 2007 was great. Portland was great, especially Stumptown Coffee. I wish I could attend RailsConf Europe in Berlin, but it looks like I will have to pass this year. It was great to meet a bunch of new people, pretty much all of whom were doing some type of Rails freelance work. The Rails market is strong, and is only going to get better. I have a feeling that RailsConf 2008 will be upwards of 2500 people.

There's not much I can say about SportRPM in terms of the business, and unfortunately, most of the best work is behind the wall of NDAs and passwords.

What I can say is that working with a small team of Rails experts - including Toby Sterrett at 120db.com - we were able to build a killer Rails application using the latest features in Rails. I was inspired by David's talk at RailsConf 2006, and decided that building the new SportRPM application using REST principals would be the best way to approach the project.

It worked out great. We developed using Edge Rails and were able to launch the application right around the time Rails 1.2 was delivered. The timing was great. We were ahead of the curve and able to take advantage of some of the new Rails 1.2 features such as 'map.resources' from day one. I am convinced that by using REST we were able to keep a very complex project manageable. By sticking with conventions, we were able to keep the code simple and easy to manage. The best part of using REST principals was that SportRPM got a free API with their application, something they can use in the future to monitor and interact with their application.

We also decided to use Engine Yard for hosting and in the process discovered a great service. A site like SportRPM.com, which frequently experiences a large number of transactions in a short period of time, cannot have any downtime. The guys at Engine Yard helped us setup several slices to keep the app constantly available.

All told, it was a great project and I look forward to using what I learned about REST principals in future applications.

My topic for the evening was two Amazon web services - Simple Storage Service (S3) and Elastic Compute Cloud (EC2).

These two services are very popular in the Rails community, and it showed this evening. I answered close to two-dozen questions about EC2 and S3.

I used Rick Olson's 'beast' application to demonstrate deploying a Rails application to an EC2 server via Capistrano. It actually worked as planned (the WiFi is a bit flaky at our meeting location).

I am convinced that S3 and EC2 are a glimpse into the future of web application development. I highly recommend checking out EC2 and S3 if you have not already.

Here are the slides.

"mime_type.rb:48: warning: already initialized constant CSV"

Here's why. CSV was added as a pre-defined mime type to Rails 1.2, but a very minor bug causes the mime type to not be available for use. So you have to define it yourself using the code below (usually in environment.rb):

Mime::Type.register 'text/csv', :csv, %w('text/comma-separated-values')

This code then causes the above warning to appear.

I did some digging and found the error in mime_type.rb. I considered submitting a ticket / patch, but right now the Rails Trac is down and I can't do much with it. I was able to look at the trunk source and see that mime_type.rb has been re-written, so this problem will go away in future versions of Rails (or if you are using Edge Rails).

Here's a quick fix that I am using in one of my production application's (which is frozen to Rails 1.2.2) environment.rb -

Mime::SET << Mime::CSV unless Mime::SET.include?(Mime::CSV)

The above line will fix the issue for Rails 1.2.2, and will be ignored if the specific bug is ever fixed in the current structure of mime_type.rb.

I just received my copy of the Rails Cookbook. The author chose to pull the book and revise it, against the wishes of O'Reilly, and was rewarded for his hard work with a release date that coincided perfectly with the release of Rails 1.2. Not an easy task for a fast-changing technology like Rails!

I think the Rails Cookbook should be in every Rails developer's library. If you are trying to solve a problem with Rails, there is probably a starting point somewhere in this book.

Here's a list of the Ruby / Rails books I have on my desk. I have the PDF version for most of these books as well, which is great for when I need to look up something when I am away from my desk.

-Agile Web Development with Rails - the Rails Bible

-Programming Ruby - the "Pickaxe"

-Ruby for Rails - the most helpful book when learning Rails aside from AWDwR

-Ruby Cookbook - a great companion to the Pickaxe

-Rails Cookbook - as mentioned above

-Rails Recipes - another great resource