Recently I’ve been getting into more Java and (attempting to) Scala development. I always got annoyed with the Scala ecosystem for development and would get fed up and just go back to writing straight Java (coughsbtcough). Today I decided to write down everything I did and get a sane process going for Scala development with SBT.

I decided to write a small Scala client for OpenWeatherMap - here is what I went through.

Testing is one of the most important parts of software development and helps to ensure bugs don’t get into production and that code can be refactored safely. If you’re working on a team with multiple people with different skill sets, you might have people doing testing who only know windows and development is only using OSX or Linux. We want everyone to be able to test - someone in QA who is familiar with Windows shouldn’t have to throw away all that knowledge, install Linux, and start from scratch. Enter JRuby and John.

Let’s start out with a simple directory structure:

1.
2├── plugin.rb
3├── main.rb
4└── plugins
5    ├── cat.rb
6    └── dog.rb
7
81 directory, 3 files

All the plugins we will use for our library will be loaded from plugins. Now lets make a simple Plugin class and register our plugins.

 1
 2class Plugin
 3  # Keep the plugin list inside a set so we don't double-load plugins
 4  @plugins = Set.new
 5
 6  def self.plugins
 7    @plugins
 8  end
 9
10  def self.register_plugins
11    # Iterate over each symbol in the object space
12    Object.constants.each do |klass|
13      # Get the constant from the Kernel using the symbol
14      const = Kernel.const_get(klass)
15      # Check if the plugin has a super class and if the type is Plugin
16      if const.respond_to?(:superclass) and const.superclass == Plugin
17        @plugins << const
18      end
19    end
20  end
21end

We’ve now made a simple class that will contain all of our plugin data when we call register_plugins.

Why setuid is Bad

setuid allows a binary to be run as a different user then the one invoking it. For example, ping needs to use low level system interfaces (socket, PF_INET, SOCK_RAW, etc) in order to function properly. We can watch this in action by starting ping in another terminal window ( ping google.com ) and then using strace to see the syscall’s being made:

sudo strace -p PID and we get the following:

Table of Contents

Initial Setup

Setting up iptables and Fail2Ban

Fail2Ban

iptables rules

Make shared memory read-only

Setting up Bastille Linux

Configuring Bastille

sysctl hardening

Setting up a chroot environment

Securing nginx inside the chroot

Extras

Initial Setup

Let’s login to our new machine and take some initial steps to secure our system. For this article I’m going to assume your username is ubuntu.

Rb RFO Status is a simple system to post status updates to your team or customers in a easy to understand format so there is no delay in reporting a reason for outage. It is modeled slightly after the Heroku Status Page .

Source: https://github.com/bluescripts/rb_rfo_status

Download: https://s3.amazonaws.com/josh-opensource/rb_rfo_status-0.1.war

It is licensed under the MIT License so do whatever you want with it!

I’ve already opened up a few issues on Github that are enhancements, but this serves as a super simple application to deploy to keep your customers and team informed of system states.

Chef is awesome. Being able to recreate your entire environment from a recipe is an inredibly powerful tool, and I had started using Chef a few months ago. When I had initially configured the Chef server I hadn’t paid much attention to the couchdb portion of it until I had a chef-server hiccup. Here are a few things to watch out for when running chef-server:

• Setup CouchDB compaction - Chef had a CouchDB size of 30+GB (after compaction it was only a few megabytes).
• When resizing instances, make sure you setup RabbitMQ to use a NODENAME . If you don’t you’ll run into an issue with RabbitMQ losing the database’s that were setup (by default, they’re based on hostname… so if you resize a EC2 instance the hostname may change, and you’ll either have to do some moving around or manually set the NODENAME to the previous hostname).
• Client’s may fail to validate after this - requiring a regeneration of the validation.pem, which is fine since this file is only used for the initial bootstrap of a server.
• Make sure you run your chef recipes you setup (for instance monitoring) on your chef-server.

I hope these tips will be helpful to other people when they run into a Chef/CouchDB/RabbitMQ issue after a server resize or hostname change. Another really helpful place is #chef on freenode’s IRC servers.

Before we dive into the benchmarks of Resque vs Sidekiq it will first help to have a better understanding of how forking and threading works in Ruby.

Threading vs Forking

Forking

When you fork a process you are creating an entire copy of that process: the address space and all open file descriptors. You get a separate copy of the address space of the parent process, isolating any work done to that fork. If the forked child process does a lot of work and uses a lot of memory, when that child exits the memory gets free’d back to the operating system. If your programming language (MRI Ruby) doesn’t support actual kernel level threading, then this is the only way to spread work out across multiple cores since each process will get scheduled to a different core. You also gain some stability since if a child crashes the parent can just respawn a new fork, however there is a caveat. If the parent dies while there are children that haven’t exited, then those children become zombies.

I saw the question of “How can I prevent a class from being reopened again in Ruby?” pop up on the Ruby mailing list. While this is somewhat against the nature of ruby, it can be accomplished:

{% codeblock lang:ruby %} class Foo def Foo.method_added(name) raise “This class is closed for modification” end end

class Foo def testing p “test” end end {% endcodeblock %}

This will raise an exception anytime someone tries to reopen the class.

<a href="https://github.com/bluescripts/reddit-cli">View Source on Github</a>

When you work on your code and are finished for the day, is what you have committed worry free? If another developer were to push your code in the middle of the night, would they be calling you at 3am?

Let’s see how we can improve our development cycle with testing so we can avoid those early morning calls. We’ll go over some of the basics with a simple project to start.