Fixturies: The speed of fixtures and the maintainability of factories

 

We had a rails app. We used factories in our tests, and it took ten minutes to run them all.  That was too slow. (spoiler alert: by the end of this blog post, they will run in one minute.)

We suspected that we could speed up the test run time by using fixtures instead, but worried that fixtures would be much more difficult to maintain than our factories.

As it happens, we are not the first developers to deal with the issue that factories are slow and fixtures are hard to maintain.  I cannot explain the issue any better than the following folks do, so I’ll just give you some quotes:

“In a large system, calling one factory may silently create many associated records, which accumulates to make the whole test suite slow …”

“Maintaining fixtures of more complex records can be tedious. I recall working on an app where there was a record with dozens of attributes. Whenever a column would be added or changed in the schema, all fixtures needed to be changed by hand. Of course I only recalled this after a few test failures.”

“Factories can be used to create database records anywhere in your test suite. This makes them pretty flexible and allows you to keep your test data local to your tests. The drawback is that it makes them almost impossible to speed up in any significant way.”

In our case, 99% of our tests were using identical records.  For example, we were calling FactoryGirl.create(:user) hundreds of times, and every time, it was creating the exact same user.  That seemed silly.  It was great to use the factory, because it ensured that the user would always be up-to-date with the current state of our code and our database, but there was no reason for us to use it over and over in one test run.

So we wrote the gem fixturies to solve the problem this way:  Each time we run tests, just once at the beginning, we execute a bunch of factories to create many records in the database.  The fixturies gem then dumps the state of the database to fixtures files, and our tests run blazingly fast using those fixtures.

We saw a 10x improvement in run times, from ten minutes down to one.  We still use factories here and there in our tests when we need a record with specific attributes or when we want to clear out a whole table and see how something behaves with a certain set of records in the database.  But in the vast majority of cases, the general records set up in that single run at the beginning are good enough.

If you are using factories in your tests to re-create the same records over and over again, and your tests are running too slowly, give fixturies a try and let us know how it goes.  It only took us about half a day to refactor 700 tests to use fixturies instead of traditional factories, so there is a good chance it will be worth your time.

Pedago releases 3 AngularJS projects to the open source community

In the past week, Pedago has released 3 open source projects on our github page.

In the past week, Pedago has released 3 open source projects on our github page.

iguana

Iguana is an Object-Document Mapper for use in AngularJS applications.  This means that it gives you a way to instantiate an instance of a class every time you pull down data over an API.  It’s similar to Ruby tools like activerecord or mongomapper.

super-model

Iguana is dependent on super-model, which should someday include much of the functionality that activemodel provides for Ruby users.  For now, however, it only provides callbacks.

a-class-above

Both iguana and super-model depend on a-class-above, which provides basic object-oriented programming (OOP) functionality. A-class-above is based on Prototype’s class implementation, and also provides inheritable class methods and some convenient helpers for dealing with enumerables that are shared among classes in an inheritance hierarchy.

This is our first foray into the management of open-source projects, so we’ll be learning as we go along.  We’re trying hard to make these useful to the community, so we have packaged them up as bower components and spent time writing what we hope is useful documentation.  We used groc for the documentation and focused on documenting our specs in order to provide lots of useful examples, rather than documenting each method in the API.  We hope that this will be more helpful than more traditional API documentation would have been, and would love to hear comments on how it’s working for folks.

We hope that other AngularJS users will find iguana, super-model, and a-class-above to be useful and decide to contribute.

Enjoy!


Curious about Pedago’s interactive education? Enter your email address to be added to our beta list.

Questions, comments? You should follow Pedago on Twitter.