mindscratch.org

Shared Mongo Configuration Settings in Rails

2011-12-18T21:58:00-05:00

I've developed a handful of Rails (3.x) applications which all used MongoDB as the persistence layer. In fact, I've yet to build a SQL backed Rails application during my "day job". Thus far I've preferred Mongoid vs. MongoMapper, primarily because of the documentation. Although, I haven't gone back to look at MongoMapper in almost a year...but at this point, I don't have a need to change.

Typically with Mongoid, you'll have a mongoid.yml file that has your database configuration. That's really super when you only use Mongoid to interact with your database. However, if you end up using another library/gem that needs to work with your database you'll have to duplicate your database configuration. NOT! Being a star developer (you know you are), you'll come up with some solution so you don't have to copy your database configuration around to multiple files.

I ran into this situation when I began using Qu. Qu is a library for "queueing and processing background jobs". It's easy to use, has a dandy API and works with either MongoDB or Redis. It might be worth mentioning here that Brandon Keepers, the creator of Qu, worked for Ordered List which was recently acquired by GitHub.

Anyways, back to my solution for not duplicating database configuration. I've recently started using the settingslogic gem on my Rails project to manage my application configuration. I have a "application.yml" file that now contains my database configuration. Using settingslogic is not a requirement, in fact Rails has a built-in method for dealing with application configuration, so use what you like.

Once I had my database configuration in a file, I setup two initializers: one for Mongoid and one for Qu. The best way I found was to create a MongoDB URI formatted string. Check out the following gist:

# this is only a partial YAML file from a Rails application # this YAML file would be loaded and used with the settingslogic gem defaults: &defaults database: &database conn_uri: mongodb://username:password@localhost:27001/ db_name: myapp_default job_queue: &job_queue db_name: qu development: <<: *defaults database: <<: *database db_name: myapp_development job_queue: <<: *job_queue db_name: qu_development test: <<: *defaults database: <<: *database db_name: myapp_test job_queue: <<: *job_queue db_name: qu_test

Now Mongoid and Qu use the same database configuration settings and it all comes from one file. Of course, you might want Mongoid and Qu to use different configuration settings, but hopefully you can see the benefit to my approach.

Now that I've checked the time, I'm going to get 5 hours of sleep. So, like a chef, I'm cooked.

Map/Reduce With Hadoop and Ruby

2011-12-18T21:09:00-05:00

Unless you've been passed out at a bar you've probably heard about Map/Reduce, Hadoop or just the "cloud". Afterall, it's the "next big thing", if you're into that sort of stuff.

At work the "cloud", Map/Reduce, analytics are all the hotness. I figured I'd try out the Hadoop Streaming API by piping data in and out with Ruby. I started out with a very small text file. Next I wrote the mapper which reads lines of text from STDIN and emits results through STDOUT.

The reducer would be as trivial as the mapper except that it needs to keep track of when the key changes. Unlike the standard reducer API which provides a key with all of its values, the streaming API doesn't do that. Besides that the reducer (at least in my trivial example) is rather simple.

The best part, is you can do all of this without ever setting up or using Hadoop! Now wait a minute, I thought I was trying out the Hadoop Streaming API. Well, you can do all the work and even test your mapper and reducer without using Hadoop. By simply piping data into and out of the mapper and reducer you can simulate what Hadoop will do. If you want to get real fancy pants you can throw in an intermediate sort before piping data to your reducer.

Check out my gist which does the minimal thing before getting Hadoop involved.

Pretty cool stuff I think. Now I just run this with a real cloudy Hadoop setup and everything will be real groovy!

Becoming Enumerable

2011-08-18T20:19:00-04:00

My first post on RubySource, hasn't been published as of 08/18/2011, covers using mixin's in Ruby. Shortly after writing that article I ran across a presentation titled "The Enumerable Module" presented by Haris Amin at Cascadia Ruby Conf 2011. The general idea is that you can make your classes enumerable (e.g. have lots of cool methods you're used to seeing when working with Arrays, Hashes, etc).

Since the process of making a class enumerable requires the use of mixin's I figured I provide a brief demonstration of how to do it.

class Followers
    include Enumerable

    def initialize
        @followers = []
    end

    def add_follower(name)
        @followers << name
    end

    def degrees_of_separation(name)
        # calculate
    end

    # must implement the each method
    def each
        # simple in this example
        @followers.each do |val|
            yield val
        end
    end
end

The Followers class stores an array of values which we iterate over in our implementation of the #each method.

Check it out:

f = Followers.new
f.add_follower 'shaun'
f.add_follower 'cliff'
f.add_follower 'sean'
f.add_follower 'nick'
f.add_follower 'tim'

f.find_all {|val|
  val.match /^s/i
}
# output: ["shaun", "sean"]

f.collect do |val|
  val.upcase
end
# result: ["SHAUN", "CLIFF", ...]

Sorry for the lame example (e.g. "followers") but I think you get the point.

Writing for rubysource

2011-08-13T19:50:00-04:00

I've been writing articles for various online sites (and publications) for the past four years. It really all started after I got to know Scott Delap, the guy behind Client Java. I was a Java 2D and Swing developer from 2003 - 2008, and the Client Java site was a great resource. I ended up doing some consulting type work for Scott and he ended up asking me about writing for InfoQ. I did some trial type posts and ended up getting the gig.

Catching Up with Scala was my first post and was published on August 22, 2007. That post was published jointly with Scott, but my first "solo" post was titled "Has JPA Killed the DAO?". I ended up writing 32 news articles and 2 featured articles during my 2+ years with InfoQ (my last news article was published on December 12, 2009, titled "IntelliJ IDEA 9: Java EE6, OSGi, Flex and More").

Before I stopped writing for InfoQ I got in contact with Michael Kimsal, founder of webdev publishing, GroovyMag, JSMag and plenty of other things, about an opportunity to work on the GroovyMag publication (a digital magazine, in PDF format). Here's the original email I sent to Michael on February 4, 2009:

Hi, I recently started subscribing to/reading GroovyMag and I really enjoy it. I am a software engineer by day (java/swing/flex/python/jython...groovy/grails hopefully) and Java Editor for InfoQ by night/weekends (part-time). I am learning Groovy and Grails because I believe there is a lot of potential and usefulness and I hope to incorporate them into my day job sometime (sooner than later hopefully).

Anyhow, I saw there opportunities to write for GroovyMag and I was wondering how I might be able to get involved. If it seems like a possibility hit me back and we'll go from there.

Thanks,

Craig

I officially got started as a technical editor for both publications (GroovyMag and JSMag) on February 12, 2009 and I continued through April/May 2010. During that time I also wrote several articles:

Easy Behavior Driven Development, Using easyb for BDD (sept 2009)
Metaprogramming with Groovy, part 1 (dec 2009)
Metaprogramming with Groovy, part 2 (jan 2010)
Catching Up with Griffon, Q&A with Andres Almiray (feb 2010)

Working on GroovyMag was interesting and frustrating. The frustration usually came from waiting for authors to submit their posts for review and having to sort through some pretty poor grammar and spelling (and believe me, I'm nowhere near perfect myself).

This leads me to my latest endeavor, writing for RubySource. I've chosen to take on this opportunity because I've been working with Ruby for about 6 months and I'd like to share my experiences and learn more (through the process of writing). I've worked with Aaron Osteraas to get started and I'm looking forward to having my first post on the site in the next couple of weeks.

On a side note, I also signed up to be a moderator for the super-awesome RailsCasts site. Ryan Bates has published 278 free screencasts covering all sorts of Rails aspects including lots of 3rd party gems/plugins. Ryan has used moderators to help fight spam, keep the comments formatted properly and help find any mistakes in the videos or the associated source code.

In short, here's the timeline of my writing/editing career:

InfoQ: 08/2007 - 12/2009
GroovyMag/JSMag: 02/2009 - 05/2010
RailsCasts Moderator: 07/2010 - ?
RubySource: 08/2011 - ?