Jun 15, 2011 - 1 minutes

Ruby on Rails: Averaging Large Data Sets

Graphing objects client side is a great way to avoid generating them server side (since client side scales infinitely). You do however run into issue when you get into thousands, or hundreds of thousands of points (for example displaying 5 minute intervals in a month: 8928). When graphing this many points javascript can hang or cause the browser to seem like its not responding.

This is a simple solution that I’ve been using for a while to average data points from a Active Record model:

 1 def generic_graph(column, hours, multiplier = 1)
 2     beginning = Time.now.advance(:hours => -hours)
 3     x = YourModel.where("created_at > ?", beginning
 4     arr = []
 5     timeoffset = Time.zone.utc_offset/(60*60)
 6     Time.now.dst? ? timeoffset += 1 : 0
 7 
 8     if hours >= 48 #or whatever number works for you
 9       x.collect.each_with_index do |s,y|
10         tmp_averaged = x[y..y+24].map{|ss| ss[column] } # collect 24 (or however many you want) records, then average them
11         arr << [s.created_at.advance(:hours=>timeoffset).to_i*1000, tmp_averaged.average] # this is for going into the flot.js graphing library
12       end
13     else
14       x.collect { |s| arr << [s.created_at.advance(:hours=>timeoffset).to_i*1000, s[column] ] }
15     end
16 
17     arr.to_s #output for flot
18   end

comments powered by Disqus