Wojtek Mach

On Ruby & Rails.

Sharing Examples in Minitest

| Comments

Last time I wrote about enforcing Liskov principle via tests. It was pretty simple to do in Minitest using just class inheritance. Sometimes, however, we can’t inherit test methods because the framework forces us to inherit from a test case class like:

  • ActiveSupport::TestCase
  • ActionDispatch::IntegrationTest

etc. In these cases we need to find some other way to share behavior and with Minitest’s design the answer is pretty simple – modules.

Example

Let’s write a simple data store library inspired by Moneta

1
2
3
4
5
6
7
8
9
10
11
12
13
class DataStore
  def initialize(adapter)
    @adapter = adapter
  end

  def get(key)
    @adapter.get(key)
  end

  def set(key, value)
    @adapter.set(key, value)
  end
end

Now let’s write an adapter:

1
2
3
4
5
6
7
8
9
10
11
12
13
class DataStore::InMemoryAdapter
  def initialize
    @hash = {}
  end

  def get(key)
    @hash[key]
  end

  def set(key, value)
    @hash[key] = value
  end
end

Let’s write a test for this. Knowing we will later reuse test methods, we start with a module:

1
2
3
4
5
6
7
8
9
10
module DataStore::AdapterTest
  def test_get_not_found
    assert_equal nil, @adapter.get(:invalid)
  end

  def test_set
    @adapter.set(:foo, 42)
    assert_equal 42, @adapter.get(:foo)
  end
end

Now the actual DataStore::InMemoryAdapter test (note, I’m using Minitest::Test which comes from minitest 5):

1
2
3
4
5
6
7
class DataStore::InMemoryAdapterTest < Minitest::Test
  include DataStore::AdapterTest

  def setup
    @adapter = DataStore::InMemoryAdapter.new
  end
end

Running this we see that two examples have been “inherited” from the shared module:

1
2
3
4
5
6
7
8
9
10
~% ruby shared.rb
Run options: --seed 18221

# Running:

..

Finished in 0.001126s, 1776.1989 runs/s, 1776.1989 assertions/s.

2 runs, 2 assertions, 0 failures, 0 errors, 0 skips

With this foundation it’s pretty easy to add new adapters and we don’t really have to write new tests. Including shared module in the test is enough to have confidence that an adapter is conforming to an interface.

Let’s say we package the data store as a gem. We can ship the AdapterTest as an integral part of the gem distribution and let the users write their own application specific adapters. Just as Rails ships with ActionDsipatch::IntegrationTest.

minitest/spec

It’s actually pretty easy to use shared modules with minitest/spec. It’s simple because minitest/spec is really just a DSL on top of minitest/test (minitest/unit). A describe block creates a new Minitest::Test class, an it block defines a new test_ method. With this in mind we can start with our custom DSL like this:

1
2
3
4
5
6
7
8
9
10
module DataStore::AdapterSpec
  it "returns nil for an invalid key" do
    @adapter.get(:invalid).must_equal nil
  end

  it "can set a value" do
    @adapter.set(:foo, 42)
    @adapter.get(:foo).must_equal 42
  end
end

And the spec:

1
2
3
4
5
6
7
describe DataStore::InMemoryAdapter do
  include DataStore::AdapterSpec

  before do
    @adapter = DataStore::InMemoryAdapter.new
  end
end

Running this will result in error like:

1
<module:AdapterSpec>: undefined method 'it' for DataStore::AdapterSpec:Module (NoMethodError)

Let’s fix this; we basically have to implement Module#it for it to work:

1
2
3
4
5
class Module
  def it(description, &block)
    define_method "test_#{description}", &block
  end
end

Tests should be passing now.

I mentioned before that minitest/spec is just a DSL. In fact, there’s literally a Minitest::Spec::DSL module that Minitest::Spec is including. The DSL module is so good in fact that it can be included both in classes and in other modules:

1
2
3
class Module
  include Minitest::Spec::DSL
end

and it just works! We now can do stuff like:

1
2
3
4
5
6
7
8
9
10
11
12
module SomeTest
  before { "..." }
  after { "..." }

  let(:foo) { "..." }

  it "returns this" do
  end

  it "returns that" do
  end
end

etc.

The way Minitest::Spec::DSL is implemented is actually pretty simple. It doesn’t do anything special; it just defines a bunch of methods like setup, teardown, foo, test_returns_this etc. It means that after the “DSL” phase we end up with just a ruby module that we can include (or not), and nothing is evaluated until the module is included somewhere.

Conclusion

Minitest’s simple design allows us to extend it with standard tools we use in day to day ruby programming. We can use the same exact constructs like classes, modules, inheritance & mixins for both the production & test code. As a consequence of this design writing minitest extensions is imho pretty easy too!

Updates

As Piotr points out including Minitest::Spec::DSL in every Module pollutes the namespace. I’m not very concerned about this in my tests code (maybe there’s a performance hit) but it’s still a good point. While proposed solution works fine, there’s one more way you can do this.

Instead of adding DSL to every Module, we can create a special kind of module with that DSL. Modules are really just classes, and you can subclass them (http://solnic.eu/2012/08/13/subclassing-module-for-fun-and-profit.html):

1
2
3
4
5
6
7
8
9
10
11
class Minitest::SharedExamples < Module
  include Minitest::Spec::DSL
end

SomeExamples = Minitest::SharedExamples.new do
  it "does something"
end

describe "Something" do
  include SomeExamples
end

Taking this further, we can do a very minimal implementation of:

shared_examples_for/it_behaves_like in 23 LOC

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class Minitest::SharedExamples < Module
  include Minitest::Spec::DSL

  def self.find(name)
    @shared_examples ||= {}
    @shared_examples[name]
  end

  def self.register(name, &block)
    @shared_examples ||= {}
    @shared_examples[name] = Minitest::SharedExamples.new(&block)
  end
end

def shared_examples_for(name, &block)
  Minitest::SharedExamples.register(name, &block)
end

class Minitest::Spec
  def self.it_behaves_like(name)
    include(Minitest::SharedExamples.find(name))
  end
end
1
2
3
4
5
6
7
shared_examples_for "Some test" do
  it "does stuff"
end

describe "Foo" do
  it_behaves_like "Some test"
end

Liskov Principle & MiniTest

| Comments

What is Liskov Principle?

In layman’s terms Liskov Substitution Principle says that if class Foo inherits from class Bar, then you should be able to use (substitute) derived class in every place that the base class is used. For a better definition and further references check out The Liskov Substitution Principle by Uncle Bob.

Testing LSP with MiniTest

MiniTest has a really simple design. A test case is a class and an example is a method of that class. After requiring minitest/autorun every subclass of MiniTest::Unit::TestCase is instantiated and test methods are executed one by one.

One very nice result of this design, which is kind of obvious when you think about it, is that you can not only inherit helper methods (eg. you subclass ActionController::TestCase to have get, post etc) but you may as well inherit whole examples! This is a perfect way to test LSP because, again, you should be able to substitute base class with a derived class.

Example

Let’s re-implement Ruby’s built-in Set class. I’ll write a test first:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
require 'minitest/autorun'

class SetTest < MiniTest::Unit::TestCase
  def setup
    @set = Set.new
  end

  def test_size
    assert_equal 0, @set.size
    @set.add 42
    assert_equal 1, @set.size
  end

  def test_include?
    refute @set.include? 42
    @set.add 42
    assert @set.include? 42
  end

  def test_add
    @set.add 13
    @set.add 13
    assert_equal 1, @set.size
  end

  def test_to_a
    @set.add 1
    @set.add 4
    @set.add 2

    ary = @set.to_a

    assert_equal 3, ary.size
    assert ary.include? 1
    assert ary.include? 2
    assert ary.include? 4
  end
end

Note I didn’t write the exact result of Set#to_a because a cannonical set is unordered. A Ruby 1.9 built-in Set is actually ordered, it simply preserves the order of insertion.

A basic implementation is very easy using Hash like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class Set
  include Enumerable

  def initialize
    @hash = {}
  end

  def size
    @hash.size
  end

  def add(obj)
    @hash[obj] = true
  end

  def include?(obj)
    @hash.include? obj
  end

  def each(&block)
    @hash.keys.each(&block)
  end
end

Let’s run it:

1
2
3
4
5
6
7
8
9
10
~% ruby set.rb
Run options: --seed 59316

# Running tests:

....

Finished tests in 0.000589s, 6791.1715 tests/s, 15280.1358 assertions/s.

4 tests, 9 assertions, 0 failures, 0 errors, 0 skips

Now, let’s write a SortedSet that will keep values sorted. Again let’s write a test and run it first:

1
2
class SortedSetTest < SetTest
end
1
2
3
4
5
6
7
8
9
10
~% ruby set.rb
Run options: --seed 54235

# Running tests:

........

Finished tests in 0.000944s, 8474.5763 tests/s, 19067.7966 assertions/s.

8 tests, 18 assertions, 0 failures, 0 errors, 0 skips

We now have exactly twice assertions because all test methods have been inherited. Let’s now build a simple SortedSet class and adjust the test, so that we actually use the derived class:

1
2
3
4
5
6
7
8
9
10
11
class SortedSetTest < SetTest
  def setup
    @set = SortedSet.new
  end
end

class SortedSet < Set
  def each(&block)
    @hash.keys.sort.each(&block)
  end
end

Sure enough all tests passes and we’re now certain that a Set object can be substituted with a SortedSet object.

Let’s also test the unique behaviour of the SortedSet. We won’t just define test_to_a method, because we would overwrite assertions from the base test. We’ll pick a different name instead:

1
2
3
4
5
6
7
8
9
10
11
12
13
class SortedSetTest < SetTest
  def setup
    @set = SortedSet.new
  end

  def test_to_a_sorted
    @set.add 1
    @set.add 4
    @set.add 2

    assert_equal [1, 2, 4], @set.to_a
  end
end

Now, we could stop it right here, but you propably noticed some duplication between test_to_a and test_to_a_sorted. Again, because we’re using just classes and methods, we can actually write:

1
2
3
4
5
6
7
8
9
10
class SortedSetTest < SetTest
  def setup
    @set = SortedSet.new
  end

  def test_to_a
    super
    assert_equal [1, 2, 4], @set.to_a
  end
end

I’m not sure if it’s that useful and you should use it, but you must agree it’s pretty neat!