“Fixin' Fixtures”

– Chris on July 30, 2007
The main problem with fixtures, for me, has always been how unfun they are. They literally suck the fun out of anything they’re around. You throw them in your test/ directory, then suddenly testing is, like, work. But it’s not work, dammit. This is Ruby, dammit.

So, keep them fun. Write your fixtures in Ruby.

Step 1: FixtureScenarios

Tom Preston-Werner has this FixtureScenarios plugin. The project home page explains:

This plugin allows you to create ‘scenarios’ which are collections of fixtures and ruby files that represent a context against which you can run tests.

Pretty cool. Instead of just test/fixtures, you can have (for instance), test/fixtures/users and test/fixtures/beta. Then, in your test class, you call scenario :users in place of the normal fixtures :users, :posts, :images call. This will pull all the YAML files from test/fixtures/users into your test database. When you need a completely separate set of fixtures for another feature, it’s as easy as scenario :beta.

Scenarios can also be nested. You can create a test/fixtures/users/banned directory filled with YAMLy files and call it with scenario :banned. This’ll pull in both the test/fixtures/users fixtures and all the test/fixtures/users/banned fixtures. It will also, by default, pull in all the test/fixtures fixtures.

The real beauty of this is when you load up your code six months down the road, recently returned from your jaunt up Everest, and need to add a feature. Fast! Just add a new scenario—forget about worrying on your old tests, they won’t be affected.

This plugin is also great when BDDin’ with test/spec or RSpec. You can have separate contexts in one spec file, all using different fixture scenarios as needed. Maybe you want to assert hardcore pagination behavior—you can load up 10,000 records! But only for that one scenario, leaving your other scenarios sane.

Grabbit:

$ script/plugin install \ http://fixture-scenarios.googlecode.com/svn/trunk/fixture_scenarios

Step 2: FixtureScenarioBuilder

Okay, like I said, we need to write our fixtures in Ruby.

By default, FixtureScenarios will load any Ruby files it finds in your fixture directories. Cool, but not really sufficient. We need transactional fixtures, we need the database cleared out after test runs, we need a clean database when tests start. We really need all the cool features YAML fixtures offer.

And we can get them, with FixtureScenarioBuilder. This is an add-on to Tom’s FixtureScenarios plugin. AKA you need FixtureScenarios for it to work.

Grabbit:

$ script/plugin install \ svn://errtheblog.com/svn/plugins/fixture_scenarios_builder

Step 3: scenarios.rb

After you’ve installed the plugin, create a scenarios.rb file and place it in your test/fixtures directory. It is in this file you build your fixture scenarios. In Ruby.

Like this:
```
scenario :blog do
  %w( Tom Chris Kevin ).each_with_index do |user, index|
    User.create! :name => user, :banned => index.odd?
  end
end
```
As soon as I run a test, test/fixtures/blog will get created with a users.yml file inside it. It’ll look like this:
```
chris:
  name: Chris
  id: "2"
  banned: "1"
  updated_at: 2007-05-09 09:08:04
  created_at: 2007-05-09 09:08:04
kevin:
  name: Kevin
  id: "3"
  banned: "0"
  updated_at: 2007-05-09 09:08:04
  created_at: 2007-05-09 09:08:04
tom:
  name: Tom
  id: "1"
  banned: "0"
  updated_at: 2007-05-09 09:08:04
  created_at: 2007-05-09 09:08:04
```
Notice we’ve got intelligent fixture names. FixtureScenarioBuilder did its best to guess these by trying name, username, then title. If it can’t figure out a name, it falls back to stories_001 or whatever.

As for quick, iterative, agile development: whenever you modify scenarios.rb, your YAML fixtures will get re-built. Any errors in your scenarios.rb file will be clearly surfaced when building.

FixtureScenarioBuilder also supports nesting fixtures:
```
scenario :blog do
  %w( Tom Chris ).each do |user|
    User.create! :name => user
  end
end

scenario :banned_users => :blog do
  User.create! :name => 'Kevin', :banned => true
end
```
As you may guess, this creates test/fixtures/blog and test/fixtures/blog/banned_users. Rake-like and tasty.

Ruby fixtures are super because you can easily throw around associations, forget about those updated_at & created_at fields, easily refactor fixtures after making sweeping schema changes, and don’t need to worry about join tables.

I’ve been doing fixtures this way for a few months now and really dig. FixtureScenarios and FixtureScenarioBuilder are like a one-two knock out punch of… uh… test data.

Step 4: Real Life And All

For a more complex example, check out this scenarios.rb file. It uses nesting, multiple scenarios, and all sorts of Ruby goodness (for a code review site, ya heard).

You can get a lot of this stuff by ERBing your YAML files, but really, that sucks. I like having my habtm join tables generated for me, I am comfortable defining associations and whatnot in Ruby. It’s how I think.

These days, many people are hacking their database or mocking to get around the fixture pain. Other fixture solutions are the fixture references plugin, this patch, and QuickFix. I’m sure there are countless others. Got a good solution? Leave a comment, plz.

Step 5: Get Out of Here

For further explanation, check out the README. Browse the code. Find a bug. Etc.

This lil’ plugin is fairly new, so any bugs and all suggestions are appreciated.

Thanks, and goodnight.

Gonna have to check this out sooner rather than later. Your opening paragraph sums up exactly how I feel about fixtures. With a flat fixturespace every fixture results in hand-wringing while I try to think up bulletproof names and minimize the number of fixtures used. Then later when I’m adding new tests I have to wade through all the old fixtures to find what I need.

Ultimately I think mocking is the way to go for large apps with comprehensive test suites, but this is a very happy middle ground that makes standard Rails fixtures much much more useful.

We used to use fixture scenarios last fall(Tom gave me a peak at RubyConf) and we actually moved away from it after a while. I think we were most likely misusing them in unit tests. Our test scenarios devolved into things like “Billing Run Scenario” where coworkeres would add data to the scenario rather than address underlying code issues. This is obviously wrong, but shit happens. This also led to people under testing their models since they felt they had good coverage. “Well if I didn’t break the build with my modifications, I’m ok right?” So I guess if you’re going to use scenarios, use them responsibly and sparingly. My personal belief is that if you’re really testing first, scenarios are considerably less appealing than they first seem. No disrespect to Tom though, it does what it was intended to very well. :)

I definitely dig the ruby generation of fixtures you demo above. Just make those datetime’s output in .to_s :db so the oracle folk can use them too. ;)

Nice article as always.

Sweet. The idea is great, I’ll have to check it out :) Oh, and by the way, the title doesn’t compile…

The fundamental problem with fixtures, in my mind, isn’t so much how much they suck to maintain or even the fact they it prevents “true” unit testing. The problem is that they are just too damn slow, and anyone project that gets to be of a decent size will suffer as the amount of fixtures grow.

I’m sure scenarios can help that by loading just what you need for the context, but for now I lean towards more and more Mocha from the start to keep tests running fast.

Chris, you rock as usual! Fixtures are dead, long live fixtures!!!

The tools look nice and I think I’ll give them a try, although I never felt tired and stuff with fixtures.

To me, if the fixtures are so complicated that you have to resort to some tools to build them, you probably overdoing them. Maybe you should for some ways of refactoring your code to make the fixtures simpler? This is just my gut feeling, I don’t have anything specific on my mind :)

I also fail to see how this scenario thing is much better than creating the necessary objects in your setup() or before() method?

@szeryf Mostly I think he’s talking about annoying things like habtm/hmt relationships which really are just nose grinding and aren’t fun. They’re fine if you’re only testing one habtm relationship, but that’s not how my sites run so I’m uncomfortable doing that. I want a scenario with 100k entries (and one that I can use to quick test when I’m developing).

Preach it, Chuck!

Here is a solution I like: Fixin’ Fixtures with Factory

I’ve used an alternative solution to Dan Manges’ Factory approach for a few weeks now: http://pastie.caboo.se/84188

Usage is exemplified at: http://pastie.caboo.se/84190

I don’t like the use of

eval

. I’ll look into making Object.send work.

I’m attempting to incorporate all this fixture fixin’ goodness into my project. I would report my issues in plugin bugs but the fixturescenariobuilder isn’t listed there yet. I have two issues. 1. I have a custom validation for a model. This custom validation instantiates a related temporary object, captures its validation errors, and then appends them to the original objects validation errors. Even though the temporary instantion is in a rescue block, fixturescenariobuilder barfs. I work around this by temporarily disabling the validation which brings me to my second problem. 2. The custom validation is also in the super class of the class I’m trying to instantiate in 1. Single table inheritance here. I do not create a table for the derived class and this causes the fixturescenariobuilder to barf. Thoughts? Need more info?

The second issue has been resolved by educating myself on testing Single Table Inheritance (STI), (http://blog.headlondon.com/archives/testing-single-table-inheritance-models-in-ruby-on-rails/) and (http://wiki.rubyonrails.org/rails/pages/SingleTableInheritance).

I’m just curious how to use with the latest rspec. I tried something alow the line:

describe SomeClass do scenario :valid_state ...

This seems to be calling the wrong scenario method. And throws a exception like this:

/lib/scenario_builder.rb:8:in `scenario’: undefined method `scenario’

Anyone has any idea how to fix this?

The plugin is pretty useful… but there is something that does not convince me, and is that the only way it sets a fixture name is via the FixtureScenarioBuilder , I always use fixtures names like…

============== [in client_markets.yaml]

nestle_cada: id: 2 number: 23304 client_id: 1 # Nestle market_id: 2 # Cada

[in bill_client_markets.yaml]

bcm_first_quarter_january_2013_nestle_cada: id: 10 number: 10 amount: 114.31 bill_client_id: 8 # first_quarter_january_2013 of Nestle client_market_id: 2 # Nestle providing Cada a Service

This way a keep a story of the different parts I want to test, something like “hey the case where nestle_cada is in january 2013 is faling, can you check that?”

Is there any way to set the fixture name before a Mode.create!() ?

How do you deal with models with callbacks or observers then? I don’t want things getting mailed out or anything like that.

Do you all not realize that folder-based fixtures already exist?

./test/fixtures/load_test/people.yml

fixtures “load_test/people”, :companies, :companies_people

No? I have used this extensively.

I don’t want things getting mailed out or anything like that.

Hi. Just wondering, how do you use it (builder) with fixtures under SVN? The fact that the builder removes the directory (with the fixtures) when rebuilding an updated scenario which is under SVN already causes problems for me. Thanks.

Hi. Just realized that I’m an idiot – those directories should be added to the SVN ignore list cause they’re generated and that’s why should not be under version control, sorry :)

This is really neat, but I’m having some trouble. In the scenarios.rb file, say we have a :users scenario, and then a :posts scenario, which has :users as a parent. And we’d like to create some posts in the :posts scenario that reference users that were created in the :users scenario. As far as I can tell, we can’t do that, because the database is cleared out after the execution of each scenario.

Ideally, I think, the data created by parent scenarios would be available in the child scenarios. And, to stay consistent with the FixtureScenarios plugin, the fixtures in the root directory would be available to all scenarios.

Michael

Hi, thanks for the interesting article. This might become very useful in our dev team since soon we will be dealing with a huge variety of potential scenarios, which all need to be covered, and the scenario plugin makes it possible to group them in a way that makes them more readable and understandable.

But: I’ve encountered some difficulties with the Builder. Since we are using foreign keys in our DB, the “delete_tables” method call when preparing to build the fixtures on line 53 in scenario_builder.rb makes the whole thing blackout with a foreign key constraint MySQL failure message. Maybe there is another way to wipe the contents of the test DB tables without doing it “table by table”? I thought of maybe trying rake db:test:clone_structure for this, since you get a vanilla DB with that one, too, but it might slow down things more than anything.

Greetings, Christoph

Is RSpec intented to work? I’ve got the following error

vendor/plugins/fixture_scenarios_builder/lib/scenario_builder.rb:8:in `scenario’: undefined method `scenario’ for # (NoMethodError)

Greetings Sebastian

Thanks for this great plugin. A few points:

Peter – I think scenarios are easier because you just specify the folder and it loads all the fixtures rather than explicitly specifying them.

Eugene – Yes you are right but it brings up an interesting annoyance in that you have to remember to always svn:ignore new scenario folders.

I have a couple of requests. I think the plugins should be merged. If you didn’t want the ruby generation then it doesn’t need to be used but I think they are useful as a single package. This would allow my second request (I suspect): Have a separate scenarios/ folder under test/. This would allow it to be svn:ignored and then you wouldn’t have to ignore every new folder.

Thanks again for another great ‘err’!

One other thing. If you have a yml for a model in test/fixtures/ and you create a scenario that includes the same model the yml created by the scenario won’t be seen by the tests. I think this is because it gets overwritten by the yml loader as it deletes the table data before each test, right?

Me Again! One issue I just had was that a country_id attribute was not getting set in the fixtures by the generator. I eventually traced it back to the initializer. It was setting the country something like Country.find(‘GB’). In fact I’m not even sure why. Probably histerical reasons! Anyway. I guess the countries fixture wasn’t loaded yet as I got nothing in the generated YML. Altering this to a hardcoded ID (Yea I know) fixed the problem. Now time to find why on earth it’s doing that in the first place.

Eivind Uggedal (and other fans of the factory): You might want to look at FixtureReplacement plugin, found on rubyforge:

http://replacefixtures.rubyforge.org/

I couldn’t get that to work with RSpec and the Rails 2.0 preview, but I did throw this into my spec_helper file so I could get just a little of the functionality:

  def scenario(name)
    files = []
    Dir.glob("spec/fixtures/#{name}/*").each do |file|
      files << file.gsub("spec/fixtures/","").gsub(".yml","")
    end
    fixtures files
  end

None of the ruby goodness or the nesting, but a quick fix until all of those plugins mature a bit.

Scratch that – the example above only works for fixtures named the same as the parent directory.

Are people still using this. It seems Toms plugin is not being maintained either. Rails 2 fixes some of the issues but doesn’t address the scenario idea properly unless I’ve misread. Are there now better alternatives?

Eivind Uggedal (and other fans of the factory): You might want to look at FixtureReplacement plugin, found on rubyforge: http://www.come-in-china.com

Sorry, no more comments :(

Step 1: FixtureScenarios

Step 2: FixtureScenarioBuilder

Step 3: scenarios.rb

Step 4: Real Life And All

Step 5: Get Out of Here

bcm_first_quarter_january_2013_nestle_cada: id: 10 number: 10 amount: 114.31 bill_client_id: 8 # first_quarter_january_2013 of Nestle client_market_id: 2 # Nestle providing Cada a Service