You’re Not Refactoring «
»


Code: , , ,
5 comments

Changing code is a great way to break it, especially in really subtle ways that you won’t pick up on for weeks or months. The maintainer of Unangband, Andrew Doull, wrote that Refactoring Is Hell. I sent him a note in response to that blog post that I think I may just as well have blogged. So:

I wanted to write to say you’re not refactoring. The term refactoring
was created in the context of extreme programming [er, I should have said “popularized”] and it was defined as
reworking code that’s under test. Without the tests, you’re just
rewriting, working without a net. That’s why your code ends up buggier.

I don’t write to be pedantic. (Well, OK, maybe a little. I am a
programmer, after all.) I write because I also used to wonder what all
the fuss was about ‘refactoring’ when it really just seemed like a
dangerous bunch of nonsense that left my code with new bugs. I didn’t do
any automated testing at the time and didn’t realize how valuable it can
be.

I still haven’t drunk the XP kool-aid, but having a good test suite
means I don’t code in fear that I’ve broken things, or feel awful when a
bug slips into production because I didn’t remember to hand-test some
weird situation. I’ll even write tests on the personal projects I’m just
hacking away privately at — yeah, I probably will just do a little unit
testing instead of a really comprehensive, robust system, but even a bit
helps.

He responded that a game like Unangband is especially hard to test because it has so many moving parts and much of the gameplay is generated. It does have a very high-level test, an automated player. I wrote back:

I don’t really think anything is too complex for test cases. You have a
‘pick up’ command that, if the player is standing on an object and has
inventory space, removes it from the world and puts it in their
inventory. There’s a fair amount of data structure you have to mock
(which is not the easiest thing in the world in C), but you can know the
basic command works even because you test all the really weird corner
cases about when the player is polymorphed into a creature without hands
or something. I’d bet as simple as the command usually is, you’ve broken
‘pick up’ at least a few times. It’s a safety net you build one strand
at a time, like a spider web. You may still be able to fall through
(especially at first), but sometimes you’ll catch a strand and be glad
you built it for yourself.

The other part of complexity is that thinking about testability changes
how you structure your programs. You force yourself to decouple
different parts of code because you want to be able to test them in
isolation from each other — that this improves your overall design is a
nice side-effect. That you can spend less time worrying about or kicking
yourself for breaking something ‘so obvious!’ is the real one.

You may enjoy the book Refactoring: Improving the Design of Existing
Code
, it’s the classic in the space. Though my money’s on the later book
Working Effectively with Legacy Code, which is a couple hundred pages
on how to go from cursing your predecessors to a reliable codebase.
Saved my bacon at a previous job. I know Unangband is your hobby rather
than your job, but I suggest them because they’ve helped keep my hobby
code more fun than frustrating.

Part of what keeps developers from testing more is friction. You have to pick a test suite, read the docs, learn the API, blow an hour or few chasing a bug because you misunderstood something, figure out the philosophy of it, rewrite your tests so they’re clear and paradigmatic…

The friction seems to be a lot higher in C/C++ than in dynamic languages. There are good technical reasons for it, but I think at least part of it is cultural. I’ve noticed a less testing in the Python community than Ruby, and I think that’s just because Rails was big on testing and has driven so much of the interest in Ruby. The community assumption is that software includes tests.


Comments

  1. The Practice of Programming also spends a chapter on testing, and focuses pretty much exclusively on testing in C – Kernighan and Pike even spend a few sentences discussing how much easier it is in scripting (or more broadly, interpreted languages), thus motivating them to provide more examples of doing it in C.

  2. A lot of the script languages are running in some kind of virtual machine. An upside to virtual machines is that you have some degree of control over them — you can starve them of memory, you can create false error conditions, etc. You can analyze the box (environment) from both the inside and the outside, which makes testing substantially easier.

    In C, there is no box. Put another way, the box is a physical box, and is less amenable to outside tinkering. We’ve already had several debates at work about setting up virtual machines so that we could do respectable error-case testing of our C++ software, but the technology just isn’t there. Never mind just setting up the virtual machines, how do you determine whether you’ve passed the test?

  3. Actually, most dynamic languages don’t manipulate the vm, they manipulate their interface to it. An out-of-memory condition isn’t created by faking it to the language, it’s created by overriding (the equivalent of) malloc() to raise the out-of-memory exception. Same with other false errors conditions.

    I don’t know how much harder it is to override C++’s printf() than it is Ruby’s puts(), but that’s the general strategy.

Leave a Reply

Your email address will not be published.