I noticed a couple of interesting things when I experimented with Mutant. If you are unfamiliar with Mutant, basically it parses your ruby code into an AST and performs modifications to your code to see if your code breaks. The idea is that with tools like SimpleCov, you can see what lines your tests run, but it does not show that you are actually testing the behavior correctly. Mutant modifies your programs AST to create valid ruby that should break your tests.

The project

I am going to be using a little movie management app as my test app. You can find caketop and look at the code if you like, but really I only expect you to understand what a movie is.

Discoveries

lazy tests

Found

Some tests are just lazy, I found a couple examples of tests like this:

it 'should parse text with markdown and put it in contents on save' do
  page = create(:page)
  expect(page.content).to_not eq(nil)
end

Solution

This kind of test has a very simple solution. Just expect something. In this case I went with a regular expression. This should be more robust than trying to match the whole page content which is factory generated and could change.

 it 'should parse text with markdown and put it in contents on save' do
   page = create(:page)
-  expect(page.content).to_not eq(nil)
+  expect(page.content).to match(/This\ is\ a\ test\ page!/)
 end

Missing special cases

Interestingly, not all of the changes I made were in my tests! I was expecting only making changes in my tests, but in reality I ended up making at least equal amounts of changes in my code. And I think the code is better for it. I don’t know if the creator of mutant intended for the tool to expose poor code as well as poor coverage. Either way I found mutant to be a helpful tool for code introspection as well.

Found

One of my favorite finds was a non-dry method that was missing a test. So Mutant threw up on this in a big way.

The method was,

def self.update(params)
  s = Setting.get(params[:setting])
  s.update_attribute(:content, params[:content])
  s.update_attribute(:boolean, (params[:boolean] == 'true'))
  Setting.get('admin-pass').update_attributes(content: Digest::SHA256.hexdigest(params[:admin_pass])) if params[:setting] == 'admin'
end

In addition to not fully really understanding ruby’s self it also has this special case for setting the admin password that breaks the typical pattern above.

Solution

so First thing I did was extract the special behavior around admin-pass and the unnecissary references to Setting

+    def update(params)
+      s = get(params[:setting])
+      s.update_attribute(:content, params[:content])
+      s.update_attribute(:boolean, (params[:boolean] == 'true'))
+      admin_pass(params[:admin_pass]) if params[:setting] == 'admin'
+    end
+
+    private
+
+    def admin_pass(new_password)
+      get('admin-pass').update_attributes(content: Digest::SHA256.hexdigest(new_pass))
+    end

-  def self.update(params)
-    s = Setting.get(params[:setting])
-    s.update_attribute(:content, params[:content])
-    s.update_attribute(:boolean, (params[:boolean] == 'true'))
-    Setting.get('admin-pass').update_attributes(content: Digest::SHA256.hexdigest(params[:admin_pass])) if params[:setting] == 'admin'
end

Then I added the missing test case,

describe 'admin' do
  it 'will sha the admin_pass' do
    Setting.update(setting: 'admin', admin_pass: 'mypwd')
    expect(Setting.get('admin-pass').content).to eq Digest::SHA256.hexdigest('mypwd')
  end
end

Constants over static methods

Found

First thing I ran into was a a fun little method for on the model that listed the different sort orders you could use when sorting movies.

class Movie < ActiveRecord::Base

...

def self.sort_orders
  [
    ['Title (asc)', 'title asc'],
    ['Title (desc)', 'title desc'],
    ['Release Date (asc)', 'release_date asc'],
    ['Release Date (desc)', 'release_date desc'],
    ['Runtime (asc)', 'runtime asc'],
    ['Runtime (desc)', 'runtime desc'],
    ['TMDB Rating (asc)', 'vote_average asc'],
    ['TMDB Rating (desc)', 'vote_average desc'],
    ['Revenue (asc)', 'revenue asc'],
    ['Revenue (desc)', 'revenue desc'],
    ['Added (asc)', 'added asc'],
    ['Added (desc)', 'added desc']
  ]
end

The first thing I thought was, “wow this method should be a constant.” The tests that Mutant ran on this file inserted nil or self throughout the this array of arrays. I realize that I didn’t care about testing that because this array will always be exactly like this as long as those are the fields on a Movie you can sort.

Solution

 class Movie < ActiveRecord::Base
+  SORT_ORDERS = [
+    ['Title (asc)', 'title asc'],
+    ['Title (desc)', 'title desc'],
+    ['Release Date (asc)', 'release_date asc'],
+    ['Release Date (desc)', 'release_date desc'],
+    ['Runtime (asc)', 'runtime asc'],
+    ['Runtime (desc)', 'runtime desc'],
+    ['TMDB Rating (asc)', 'vote_average asc'],
+    ['TMDB Rating (desc)', 'vote_average desc'],
+    ['Revenue (asc)', 'revenue asc'],
+    ['Revenue (desc)', 'revenue desc'],
+    ['Added (asc)', 'added asc'],
+    ['Added (desc)', 'added desc']
+  ]
+
   has_many :genres

   has_many :encodes
@@ -15,23 +30,6 @@ class Movie < ActiveRecord::Base
     "/backdrops/#{id}.jpg"
   end

-  def self.sort_orders
-    [
-      ['Title (asc)', 'title asc'],
-      ['Title (desc)', 'title desc'],
-      ['Release Date (asc)', 'release_date asc'],
-      ['Release Date (desc)', 'release_date desc'],
-      ['Runtime (asc)', 'runtime asc'],
-      ['Runtime (desc)', 'runtime desc'],
-      ['TMDB Rating (asc)', 'vote_average asc'],
-      ['TMDB Rating (desc)', 'vote_average desc'],
-      ['Revenue (asc)', 'revenue asc'],
-      ['Revenue (desc)', 'revenue desc'],
-      ['Added (asc)', 'added asc'],
-      ['Added (desc)', 'added desc']
-    ]
-  end

When I realized that there really isn’t a practical way to change this array without changing the purpose too. I moved the method into a constant. Next step would normally be to add the method back and just reference the constant to maintain the interface, but I found only one use of this method, so I just referenced it directly there.

Conclusion

The best thing I have found about using mutant is that I actually learned some new patterns to watch for in my future code. I am sure I could find more patterns if I test more things with mutant, but I have already found some small improvements I will be looking for in my code and tests in the future.