With the rise of mobile applications came new developer’s needs. In order to facilitate their life and speed up apps delivery a new type of platform named Backend as a service try to bring such solutions. Let’s begin our journey by introducing two services (among many others) named Parse (recently bought by Facebook) and Kinvey.

Relational Databases have been used for several years with success but the rise of the Internet and new computer/mobile/embedded applications revealed new challenges for data handling. This is why new tailored tools appeared in order to answer each of these issues (query speed, data structure, data graphs, data volume, … ) through optimal means.

Among all of these new tools, let’s see today what’s called document dbs and more especially mongo db.

Whether your log purpose is more technical (aiming performance, number and kind of accesses, or as a comprehension and correction mean for bugs) or if it’s more targeted towards retrieving business data (as the usage of some application feature, or a/b testing), logs need to be performed transparently and in a non invasive way for your existing code base in such way that performances of your application, user experience (which involve availability, responsiveness, reliability and security of your system), as well as your code to a lesser extent are not going to be impacted.

As pursued goals of logging are totally external to those followed by the main application, it’s thus preferable for this mechanism to be the least intrusive and invasive in the code of the monitored application. Moreover, as the handling of these data require quite some computation power, algorithms, screens and graphics, the separation of these applications (the main one and the logging system) and the minimisation of the impact of their connections are features that all well done logging systems should look for in priority. The chosen implementation of this in your system should aim for these logical features.

Rails implementations

Keeping all of this in mind, let’s see some possible rails implementation !

Each of these issues relate to an appropriate answer:

Audited

If you are more interested about technical indicators on your application, you should consider set up the audited gem. It will basically log all changes (CRUD) made on Rails models with additional information such as who did the change and how did it impact the application code.

Go on the github page for further information about this gem:

• https://github.com/collectiveidea/audited

Notifications

On the other side, if you are more interested in business feature logging and want to keep in mind this non invasive way of doing, let’s consider the approach developed by 37signals based on ActiveSupport::Notifications, statsd and graphite.

In this implementation, the main application will produce diverse events according to subscribed features chosen through configuration. These messages received by the logging component of your application will then trigger the logging information mechanism (as in the observer pattern), sending this information via udp to the side logging system. Using events, the impact of logging on the main application’s code will be minimised since the implementation of the linked listener containing all the log logic will be well divided and put apart from the rest of the code. Using proper configuration, you could easily enable or disable desired features logging without touching almost anything. Furthermore, the udp usage instead of tcp allow that the main application will never be stuck during the log data recording. Finally, the usage of a side application tailored to treat these log information will allow much more analysis, comparisons, … without any impact on the main application performances.

In addition, this split between the main and log application could allow you to gather information from many sources and then allow you to combine all these sources into a much richer information report. Or in a simpler way, if your company sysadmins already use a logging system, this strategy allow a painlessly integration.

Read this really interesting blogpost for further details:

• http://37signals.com/svn/posts/3091-pssst-your-rails-application-has-a-secret-to-tell-you

Get also some clue in these additional documents:

• http://codeascraft.etsy.com/2011/02/15/measure-anything-measure-everything/
• http://graphite.wikidot.com/

And as a final and opening discussion point, here is a talk about proper logging, flexible configuration, a sane exception hierarchy, and useful documentation coming from Aloha Ruby Conf:

• http://confreaks.com/videos/1246-aloharuby2012-building-a-ruby-library-the-parts-no-one-talks-about

Imagine that your application manages a store, and that you have a model that represents the number of items in stock for each product:

1
2
3
4
5
6
7

class ProductStock < ActiveRecord::Base
  belongs_to :product
  validates :product, :presence => true
  validates :stock_size,
            :presence => true,
            :numericality => { :only_integer => true, :greater_than_or_equal_to => 0 }
end

When a clerk sells a product item, your application will typically call a method like this on the corresponding ProductStock model:

1
2
3
4

def decrement(number_of_items = 1)
  self.stock_size = stock_size - number_of_items
  save
end

Now what if two clerks sell the same product at the same time? This situation might seem very unlikely but, if there are a lot of clerks and your product is a real bestseller, it can happen sooner than you think (imagine an Apple Store on the new iPhone release date). Unless you limited the server to handle only one request at a time (probably not a very practical decision), both requests will be treated by concurrent threads. A possible sequence of events is the following:

1. Thread 1 loads the model from the database, with a stock_size value of, say, 10.
2. Thread 2 loads the model from the database, also with a stock_size value of 10.
3. Thread 1 decreases the stock_size value by one. On its own copy of the model, this value is now 9.
4. Thread 2 decreases the stock_size value by one. Again, on its own copy of the model, this value is now 9.
5. Thread 1 saves its modified version of the model.
6. Thread 2 saves its modified version of the model.

What you end up with is an incorrect inventory: your application now says there are 9 items in stock, while obviously there are only 8 on the shelves. The problem is that the second thread never knew that the value was changed by the first. This is a typical example of what is known as a race condition.

Simply calling reload at the beginning of your method won’t make the problem disappear: thread 1 could still save the new value right after the model copy of thread 2 is reloaded (You can easily simulate this by calling sleep(30.seconds) just after the call to reload and play with two parallel Rails consoles). What you really need is a way to prevent outdated data from being written in the database by implementing a locking strategy. Fortunately for you, Rails makes it really easy to use the two most well-known, respectively called pessimistic locking and optimistic locking.

Pessimistic locking

The idea of pessimistic locking is to prevent more than one process to access a record in the database at the same time: when a process wants to load an object in order to modify it, it puts a lock on the corresponding record¹, forcing any other process to wait for this lock to be released before they can load the record. Basically, the purpose is thus to bring atomicity to a series of operations.

In ActiveRecord, when you are inside a transaction, you can load models with the :lock => true option or call lock! on an already loaded model to put a lock on the corresponding record (if you are not inside a transaction, the lock is released as soon as it is acquired). You can also start a transaction and acquire the lock in one go by calling with_lock with a block:

1
2
3
4
5
6

def decrement(number_of_items = 1)
  with_lock do
    self.stock_size = stock_size - number_of_items
    save
  end
end

Note that placing a lock on a model will automatically force it to be reloaded.

This strategy is not without its problems, however. Indeed, a process can potentially acquire a lock and keep it for as long as it wants or even never release it, forcing all the other processes that need access to the locked record to wait indefinitely (this is called starvation). Another potential problem are deadlocks: process A locks record 1, then tries to lock record 2, but record 2 has already been locked by process B which now needs to lock record 1 to complete. Both processes are unable to complete, each waiting for the record the other has locked.

Optimistic locking

In optimistic locking, a version number is assigned to each row. When a model is updated, its version number is checked against the one in the database. If they are the same, the changes are committed and the version number of the row is incremented (within the same atomic operation); if not, that means another process has updated the row since you loaded the model, and the update fails. In this case, you need to reload the model and try again.

To enable optimistic locking in Rails, you only need to add a “lock_version” column on your table:

1
2
3
4
5

class AddLockVersionToInventory < ActiveRecord::Migration
  def change
    add_column :product_stocks, :lock_version, :integer, :null => false, :default => 0
  end
end

If your record is outdated, ActiveRecord will raise a ActiveRecord::StaleObjectError; it is then your responsibility to deal with the conflict. Note also that for optimistic locking to work across all web requests, you should add lock_version as a hidden field to your form, and to the list of attr_accessible (or to the filtered set of attributes for strong_parameters).

The main drawback of optimistic locking is that it can cause a lot of updates to fail if the same record is often accessed concurrently (this could be the case in our example, actually), which can be quite tedious from the end user point of view.

References

• http://api.rubyonrails.org/classes/ActiveRecord/Locking/Pessimistic.html
• http://api.rubyonrails.org/classes/ActiveRecord/Locking/Optimistic.html
• https://blog.engineyard.com/2011/a-guide-to-optimistic-locking
• http://en.wikipedia.org/wiki/Locking_(computer_science)#Database_locks
• http://en.wikipedia.org/wiki/Race_condition
• http://en.wikipedia.org/wiki/Deadlock

Notes

[1] Note that some RDBMS put locks on the whole table, not on the records.

Let’s have a look at the mail_form gem ! As described by it’s author: José Valim. The gem allow you to send e-mail straight from forms in Rails with I18n, validations, attachments and request information. Really helpful if you need for instance to quickly create a contact form or any other particular mail form !

Here is a typical mail_form model you simply have to call via controllers and appropriate views:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

class ContactForm < MailForm::Base
  attribute :name,      :validate => true
  attribute :email,     :validate => /\A([\w\.%\+\-]+)@([\w\-]+\.)+([\w]{2,})\z/i
  attribute :file,      :attachment => true
  attribute :message
  attribute :additional_field,  :captcha  => true

  append :remote_ip, :user_agent, :session

  def headers
    {
      :subject => "My Contact Form",
      :to => "your.email@your.domain.com",
      :from => %("#{name}" <#{email}>)
    }
  end
end

and then, the way you send some mails:

1
2
3

c = ContactForm.new(:name => 'Sam', :email => 'sam@email.com', :message => 'Cool!')
c.request = request
c.deliver

As seen in these piece of code, you can validate presence of information as well as format of these ones and furthermore check if it’s spam or not via the captcha option (with the “Honeypot” captcha strategy).

Further information:

• https://github.com/plataformatec/mail_form
• http://everydayrails.com/2011/09/07/rails-contact-form.html

Some time ago, as we were facing a performance drop in one of our projects, the decision was made to cache some intensively used data pieces and thus improve the overall system read efficiency.

After some research, we found some interesting key-value stores (as MemCached, Redis,…) which could perfectly do the trick. Since one has to be chosen, we finally picked up Memcached.

MemCached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) coming from results of database calls, API calls, or page rendering.

The system uses a client–server architecture wherein servers maintain a key–value associative array and clients populate this array and query it. To do so clients use client-side libraries to contact servers which, by default, expose their service on port 11211.

Using MemCached in your applications will allow to layer your requests, resulting in faster responses coming from RAM when available and falling back on a slower store such as a database otherwise.

But pay attention, values stored in MemCached are only available until the server runs out of memory. At which point, oldest data is discarded, a mechanism known as LRU. Therefore, clients must treat Memcached as a transitory cache. They cannot assume that data stored in Memcached is still there when they need it.

And as quite everything is well thought and done in our ruby world, a MemCached client implementation exists through the ruby Dalli gem (Which is the officiall MemCached client since Rails 4).

Installation and usage

So, what do we need precisely to incorporate of all these features into our application?

First of all, an up and running MemCached server (The set up could differ from one OS to another):

• for Mac OS (via HomeBrew):

1

brew install memcached

• for debian-based linux distributions:

1

apt-get install memcached

Then you can simply install the Dalli gem (or put it into your gemfile):

1

gem install dalli

and use it into your ruby/rails project as shown in this piece of code:

1
2
3
4

require 'dalli'
dc = Dalli::Client.new('localhost:11211')
dc.set('abc', 123)
value = dc.get('abc')

Some tips

• If you pass an array of strings, for instance: ["offices", "BE"], as key. Dalli will concatenate all those using backslashes as separator, resulting into "offices/BE" being really sent to the MemCached server.

• You can even contact a MemCached server without ruby or rails using telnet.

• If you’re in a rails project, you can even directly use MemCached as rails cache store via this additional configuration put into the desired environnement configuration file (rails 3 feature) :

1

config.cache_store = :dalli_store

We’ll certainly talk about some other no sql alternatives, their advantages, drawbacks and use cases in further blog posts, so stay tuned !

References

• http://memcached.org/
• https://github.com/mperham/dalli
• http://en.wikipedia.org/wiki/Memcached

Tmux, as the acronym might (maybe?) suggest, is a “terminal multiplexer”, which makes it seamless to manage multiple terminal windows and split panes within a single session of your terminal app of choice.

Those are just a mere glimpse into what tmux is capable of. Let’s start with the essential, shall we ?

Installing on OS X

Just install the package through Homebrew :

$ brew install tmux.

Check the result by executing this command :

$ tmux -v.

First steps

Creating your first session

One of tmux’s useful features is named sessions. As we often have to manage various contexts (different projects, different clients…) at the same time, tmux allows us to create one persistent session for each.

Let’s say I want to create a session for the projects I’m working on at Belighted.

$ tmux new-session -s belighted

This command can be shortened as such :

$ tmux new -s belighted

This attaches a session, serparated from my original terminal session. It can hold multiples windows (equivalent to tabs), and split planes within a window.

Detaching from the session

Once created, you can leave (“detach”) this session, and return later to find everything as you left it. To do so, hit the following keys :

Ctrl-b then d

The command prefix

As you have just witnessed, every tmux command is prefixed with Ctrl-b, which I’ll simply call Prefix from now on. For example, I’ll write Ctrl-b then d as Prefix d.

The key mappings are entirely customizable, that will be the subject of a next post in this series.

Managing sessions

Listing sessions

While outside a tmux sessions, run the following command to get a list of your running sessions.

$ tmux list-sessions

or the shorter

$ tmux ls

Reattaching to a session

Say you have detached from a running session, of course you’d like to be able to get back.

$ tmux attach -t belighted

The -t argument being the name of your previously created session. If you only have one session running, it’s not necessary to pass an argument.

Killing a session

When you’re done with your session, just execute the following, which will close every windows within your session, and kill the session itself.

$ tmux kill-session -t belighted

That’s it for the basics of tmux. In the next post, we’ll review windows, panes, and start our configuration customization.

See you next time !

You have a bunch of Pdf documents and you want to concatenate them all into a resulting one, using the Prawn gem and this small piece of code, it’s now easily doable !

1
2
3
4
5
6
7
8
9
10
11
12
13

require 'prawn'

pdf_file_paths = ["1.pdf", "2.pdf"]
Prawn::Document.generate("result.pdf", {:page_size => 'A4', :skip_page_creation => true}) do |pdf|
  pdf_file_paths.each do |pdf_file|
    if File.exists?(pdf_file)
      pdf_temp_nb_pages = Prawn::Document.new(:template => pdf_file).page_count
      (1..pdf_temp_nb_pages).each do |i|
        pdf.start_new_page(:template => pdf_file, :template_page => i)
      end
    end
  end
end

If you want this feature fully integrated in the next Prawn version, just support my issue on Github !

Today I wanted to try out Puma on a Heroku project. Works great, but I quickly realized my app wasn’t reporting to New Relic, which is kind of a bummer. Here’s a compilation of what I found to be working with the latest version, 1.6.2 at the time of writing.

1. Add an environment variable.

1

NEWRELIC_DISPATCHER=puma

Or, if you use something like Figaro, just add the following to your application.yml, and run the rake figaro:heroku task.

1

NEWRELIC_DISPATCHER: puma

2. Add it to your New Relic config.

Paste the followig in your newrelic.yml config file.

1

dispatcher: 'puma'

There, two easy steps and it’s all better.

See you next time !

We’ve seen last time with Better Errors an interesting tool to debug your application. But what about support on a fully working app with maybe some possible improvments on data requests, memory consumption or view rendering? Here comes the Rails Footnotes gem, which will display a bunch of information for easy debugging and development, such as sessions, request parameters, cookies, routes, queries, etc. and other potentially useful things to help troubleshoot and optimize your Rails application.

Just add this in your gemfile and without any additional effort, you’ll get the all feature package:

1
2
3

group :development do
  gem "rails-footnotes"
end

Furthermore, you can add your own custom footnotes and customize the way they are displayed. The gem also allow you to jump directly from the stack trace through the code.

For further information:

• https://github.com/josevalim/rails-footnotes

Another interesting tool which offer quite the same features is the RailsPanel gem, a chrome extension which requires the “meta_request” gem but which is, in our sense, less powerful and customizable than the previous one even if it has the advantage to not break your application interface.