Versioning: The Next Big Thing «
»


Code: , , ,
7 comments

In the web development world, anyways. So, in the grand scheme of things, maybe not a huge deal to anyone else. Versioning is going to be one of the biggest problems and opportunities there is in web development, and it’s going to take us at least five years to get it right.

Actually, let me admit up front that five years is a shot in the dark, and optimistic to boot. If people keep hanging out with bondage and discipline languages like Java and C# that are still catching up to language and framework developments from the 90’s it’ll take us more like ten years. (Attention Lisp Weenies: Yes, I know you solved every problem forty years ago for certain values of “solved” and “problem” while the rest of us were getting work done.) Not only is versioning a difficult technical problem, it will be difficult to educate programmers in what it is, how it works, and why you’ll wish you used about a year after you decided it was too much work.

I’m writing to get help get the ball rolling on the process of solving this problem and publicizing it. So maybe it’s about time I go into what the problem is instead of just yammering on, eh?

Because your website is served from your machines, you can update it every few days or hours. This is almost pedantically obvious, but it’s a big change from having to press CDs, ship them to a store, and wait for them to go home with customers — it’s even a big change from posting a new version online for customers to download and install.

The web’s faster pace means that updating, say, your database model to add or remove fields is a common occurence rather than something your InstallShield wizard does every 1-2 years. It’s vital that changes be streamlined and safeguarded. What kind of defaults get set when you add fields? How do you save data when you lose fields? How does the programmer make sure the Right Thing happens? If you have the fields Name and Address and want to combine them into a simple MailingAddress field, you’re not just deleting two columns and adding one.

Not only is it complex to change your schema, getting those changes into production are not easy. The script that updates definitions and migrates data (which we tested somewhere other than production, right?) needs to placate the multiple web and database servers used for load-balancing. If you update the database server first, the application code on the web server shouldn’t break because it didn’t get the word about the new schema.

I see too many problems with trying to get multiple servers to update to a new version at the Exact Same Instant, I think schema changes are going to have to be a four-step process with a backwards-compatible first step applied to the database (add columns, loosen restrictions), a code update (with testing!), a backwards-incompatible database change (drop columns, tighten restrictions), and finally another code update to remove outdated usage.

Jakob Kaplan-Moss, one of the creators of Django, just opened the discussion of versioning in Django (with links to prior art). As a web framework with an object-relational mapper, versioning is an important feature.

In addition to databases, web APIs need versioning. Adam Kelsey has opened that discussion, but so far the solutions are pretty rough and I don’t have anything to add to the discussion.

If you’d like to read more about how web development differs from other development, Steve Yegge wrote an excellent article titled It’s Not Software coining the term “servware” to emphasize that web developers need to recognize the ways our code ends up so different. (It’s easy to turn this topic into rah-rah “We’re so special we don’t have to learn from history” bullshit, but nobody wants that.)


Comments

  1. As a sysadmin, I would LOVE to see better versioning of web apps. I’ve deployed code where there are significant questions, even among the developers, exactly what code it is. Even an ever-incrementing build number would be immensely helpful.

    I worked on a project with a web front-end and a VB back-end. They deployed the VB back-end at customer sites. After a while, people forgot what level of code was deployed at the customer site, because the code evolved so fast. All we had was an installation date; there was no way to say “we have build number XXX here.” I kept asking for a versioning system, and all I got in response was “but our code evolves too quickly. We make special builds for customers. We can’t really do it.” Nobody bothered to sit down and think out these problems. It was the messiest project I ever worked on and I’m happy to put it far behind me.

    It just seems to me that webdevs simply want to avoid the extra work of versioning the code. Typically, the management isn’t concerned with “bookkeeping” tasks such as this. They want to get the product out the door, which is fine. But I feel there should be better change control, particularly in smaller organizations, and proper versioning is a huge part of that.

    I mean, I am required by SOX to file change notices for the work I do. Webdevs should be held to the same standard.

  2. I can see how that lack of even basic versioning would be incredibly painful. One of my favorite tricks is to make an install of a web app a checkout directly from source control. Then it’s pretty easy to see what version you have, update to the latest, or switch over to a branch.

  3. One reply: Hello, world!

    Of course, in Zope that takes two magic shell commands (one of which asks several questions), a few lines of Python, an __init__.py, two XML config files, another file of Python for a view class, an XML template…

    I’m teasing a bit here, but I’m not exaggerating the steps. Zope has earned its reputation for being overwhelming, and I think evangelism and education are going to be the hardest part.

    Axiom looks quite nice. The ORM I’m most familiar with is Django’s, and it looks quite similar. I like how you just get to write a function for the upgrade, though registering it looks a little clunky. Just spitballing, it might look nicer as something like:


    class UpgradeShopProduct(axiom.upgrader):
    from = 1
    to = 2

    def upgrade(self, product):
    if 'bacon' in product.name:
    product.description = 'salty'
    else:
    product.description = 'who cares'
    return product

    def downgrade(self, product):
    del(product.description)
    return product

    There’s a little magic naming going on, but I like this style more than registration functions.

  4. Rails already has something called migrations. They update the schema, AND do any data updates. (schema date is not in the models, but in the migrations with rails)

    for example:


    class UpgradeShop 'who cares'
    Product.update_all('description = "salty"', 'name = "bacon"')
    end

    def self.down
    remove_column :product_table, :description
    end
    end

Leave a Reply

Your email address will not be published.