Fixturies: The speed of fixtures and the maintainability of factories

 

We had a rails app. We used factories in our tests, and it took ten minutes to run them all.  That was too slow. (spoiler alert: by the end of this blog post, they will run in one minute.)

We suspected that we could speed up the test run time by using fixtures instead, but worried that fixtures would be much more difficult to maintain than our factories.

As it happens, we are not the first developers to deal with the issue that factories are slow and fixtures are hard to maintain.  I cannot explain the issue any better than the following folks do, so I’ll just give you some quotes:

“In a large system, calling one factory may silently create many associated records, which accumulates to make the whole test suite slow …”

“Maintaining fixtures of more complex records can be tedious. I recall working on an app where there was a record with dozens of attributes. Whenever a column would be added or changed in the schema, all fixtures needed to be changed by hand. Of course I only recalled this after a few test failures.”

“Factories can be used to create database records anywhere in your test suite. This makes them pretty flexible and allows you to keep your test data local to your tests. The drawback is that it makes them almost impossible to speed up in any significant way.”

In our case, 99% of our tests were using identical records.  For example, we were calling FactoryGirl.create(:user) hundreds of times, and every time, it was creating the exact same user.  That seemed silly.  It was great to use the factory, because it ensured that the user would always be up-to-date with the current state of our code and our database, but there was no reason for us to use it over and over in one test run.

So we wrote the gem fixturies to solve the problem this way:  Each time we run tests, just once at the beginning, we execute a bunch of factories to create many records in the database.  The fixturies gem then dumps the state of the database to fixtures files, and our tests run blazingly fast using those fixtures.

We saw a 10x improvement in run times, from ten minutes down to one.  We still use factories here and there in our tests when we need a record with specific attributes or when we want to clear out a whole table and see how something behaves with a certain set of records in the database.  But in the vast majority of cases, the general records set up in that single run at the beginning are good enough.

If you are using factories in your tests to re-create the same records over and over again, and your tests are running too slowly, give fixturies a try and let us know how it goes.  It only took us about half a day to refactor 700 tests to use fixturies instead of traditional factories, so there is a good chance it will be worth your time.

How do I read the AngularJS message: [$rootScope:infdig] 10 $digest() iterations reached. Aborting! Watchers fired in the last 5 iterations

I’ve been using Angular every day for over a year, but have always been too intimidated by this error message—and the crazy list of information that comes along with it—to really dig into it and find out how to use it to my advantage.

Building a new product at Pedago, I see this error happen from time to time in production (possibly related to a user using an old browser), but never when developing locally. So I have the error message from our error logs, but I can’t reproduce it or debug it by making changes in the code.

After researching the issue, here’s what I found out on my own.

After the colon there are two brackets. (…’atchers fired in the last 5 iterations: [[{“msg’) The first bracket is the beginning of a json block. Copy from the first bracket to the end of the error and find a way to pretty-print that json. (Use your favorite code editor or an online json formatter.)

Now you have a pretty-printed array with 5 entries in it. Each entry represents an iteration in the digest cycle, i.e. one pass through all of the active watchers in your app, looking for changes. Angular will repeat iterations until it does one in which no watcher has a changed value, or until it hits 10 iterations, at which point it will error. That’s what happened in this case.

There were 10 iterations before the error, and 5 are included in the error message. Presumably that means there are 5 more iterations that happened earlier than what is included in the error message. The first entry in the error message is the 6th iteration, and the last entry in the message is the 10th iteration.

The entry for each iteration is also an array. In this case it is an array of objects, and each object represents a watcher whose value changed during this iteration. Each object will give you the text or the function that defines the watcher, the old value for the watcher before this iteration and the new value after this iteration.

Read it from top to bottom like a story, adding commentary based on what you know about your app. In my case, I was able to see how the changes in each iteration caused new watchers to be created, requiring yet another iteration. “In the 6th iteration, this watcher changed, causing this new stuff to be rendered on the page, creating new watchers which were assigned values in the 7th iteration, and then …” There was no infinite loop or anything. In fact, if Angular had been willing to do just 1 or 2 more iterations, it would have finished.

I hope this is helpful to anyone else experiencing this issue.

Pedago releases 3 AngularJS projects to the open source community

In the past week, Pedago has released 3 open source projects on our github page.

In the past week, Pedago has released 3 open source projects on our github page.

iguana

Iguana is an Object-Document Mapper for use in AngularJS applications.  This means that it gives you a way to instantiate an instance of a class every time you pull down data over an API.  It’s similar to Ruby tools like activerecord or mongomapper.

super-model

Iguana is dependent on super-model, which should someday include much of the functionality that activemodel provides for Ruby users.  For now, however, it only provides callbacks.

a-class-above

Both iguana and super-model depend on a-class-above, which provides basic object-oriented programming (OOP) functionality. A-class-above is based on Prototype’s class implementation, and also provides inheritable class methods and some convenient helpers for dealing with enumerables that are shared among classes in an inheritance hierarchy.

This is our first foray into the management of open-source projects, so we’ll be learning as we go along.  We’re trying hard to make these useful to the community, so we have packaged them up as bower components and spent time writing what we hope is useful documentation.  We used groc for the documentation and focused on documenting our specs in order to provide lots of useful examples, rather than documenting each method in the API.  We hope that this will be more helpful than more traditional API documentation would have been, and would love to hear comments on how it’s working for folks.

We hope that other AngularJS users will find iguana, super-model, and a-class-above to be useful and decide to contribute.

Enjoy!


Curious about Pedago’s interactive education? Enter your email address to be added to our beta list.

Questions, comments? You should follow Pedago on Twitter.

Tinkering Toward Learning

Given how useful the tinkering approach is for keeping learners motivated, how do we apply a similar approach to a subject like Finance?

man holding bicycle
By Artaxerxes (Own work) [CC-BY-SA-3.0], via Wikimedia Commons
My friend Alfredo builds bikes as a hobby. He started by replacing a broken chain on his own bike. Then he upgraded his brakes. After a few more repairs, he understood the whole bike system well enough that he could gather all the parts and build one from scratch.

Experienced programmers generally learn new languages in a similar way. We get assigned to a new project for which there is an existing codebase that needs to be maintained or extended. Everything is mostly working, but something needs to be tweaked or added. So we tweak it. After working on five or ten features, we know the new language well enough that we could start a new project ourselves.

In more traditional educational environments, however, we tend to learn things the other way around. We start with simple, contrived building blocks and slowly work our way up to the point where we can comfortably manipulate a more complex and realistic system.

For example, a course that teaches the principle of the “Time Value of Money” is likely to start with a question like “if someone offered you $90 today or $100 a year from now, which one would you take?” This is, to say the least, an unrealistic scenario. But it is an introduction into the concept. After working through a number of similar examples in order to allow the student to master the math, the course will hopefully move on to a more reasonable explanation of how this concept is used in practice.

By Anna reg (Own work) [GFDL or CC-BY-SA-3.0-at], via Wikimedia Commons
By Anna reg (Own work) [GFDL or CC-BY-SA-3.0-at], via Wikimedia Commons
Not that it was a bad course. I actually quite liked it. But this would be like if Alfredo had first worked on pedals, then wheels, then built himself a unicycle before moving on to gears and brakes. It would have been years before he had anything he could ride on. Knowing Alfredo, he would have had no hope of staying motivated for such a long time with no bike to show for it.

Given how useful the tinkering approach is for keeping learners motivated, how do we apply a similar approach to Finance? It turns out this is difficult to do because it often involves risking real money and waiting years to see any results. What a learner really needs is a safe environment to develop intuition around the long-term consequences of her decisions and to discover for herself the places where she needs to dig deeper.

At Pedago, developing alternative approaches to teaching tough topics is what we’re passionate about. Stay tuned over the coming months to see us tackle similar problems.

 

This post has been updated to include a clearer example. Thanks to Earthling for the feedback!