Automatically — err.the

“Automatically”

– PJ on September 17, 2007
Chris made migrations sexy. So sexy, in fact, that DHH committed a variation of his plugin to trunk.

I’m not here to make things sexy. I’m here to make migrations drop-dead gorgeous.

Remember that file that no one uses and normally leaves of subversion?

No, not doc/README_FOR_APP. I’m talking about db/schema.rb of course.

Turns out with this plugin I wrote, it’s the baddest file in your source code.

“How bad, PJ?” you ask.

So bad that when you change your schema.rb from
```
ActiveRecord::Schema.define(:version => 1) do
  create_table :posts do |t|
    t.string  :title
    t.text    :body
  end
end
```
to
```
ActiveRecord::Schema.define(:version => 1) do
  create_table :posts do |t|
    t.string  :title
    t.text    :body
    t.integer :published
  end

  create_table :comments do |t|
    t.string  :name, :url
    t.text    :body
    t.integer :post_id
  end
end
```
and run
```
$ rake db:auto:migrate
```
it’ll execute the following:
```
-- add_column("posts", :published, :integer)
   -> 0.0096s
-- create_table(:comments)
   -> 0.0072s
```
Pretty slick. Run the task again and nothing will happen, just like regular migrations, but change the file and the plugin will do its best to figure out what you’ve done.

What it is Chief

The plugin’s logic is fairly straightforward. It’ll look at the tables & fields you’ve defined in the schema, compare those to the tables & fields in the database, and figure out what it needs to add and drop.

I’ll be adding support for indexes (added, check below) and type changes (also added, last update) shortly.

Limitations

Now, I’m sure you’ve been thinking of all of the cases where this isn’t going to work. Right off the bat, it’s clear you couldn’t use this if you wanted to change the name of an existing column. There’s no way for auto_migrations to know your actual intention, it would drop the original column and create the new one.

But, Can It Scale?!

That having been said, this plugin doesn’t prevent you from running regular migrations if you feel the need. The only thing to keep in mind is that running regular migrations will overwrite schema.rb, so you’ve been forewarned if you’ve spent a long time making the file look pretty.

I’m willing and able to accept patches to make auto_migrations more useful, though. My knowledge of ActiveRecord is seriously lacking, so if there’s room to clean stuff up, please let me know.

Check it Out

Warehouse: http://plugins.require.errtheblog.com/browser/auto_migrations

SVN: svn://errtheblog.com/svn/plugins/auto_migrations

Bugs/Patches: http://err.lighthouseapp.com/projects/466-plugins/tickets

1% Inspiration and 99% Perspiration

The concept for this plugin came when I noticed a migrations branch had been added to DataMapper. Being able to auto-migrate with DM seemed so logical since the fields are defined in the models. It wasn’t until Chris pointed out that schema.rb could be used for the same purpose that I realized auto migrations were possible in Rails as well.

If you’re curious how the auto migration code works for DM, take a look at the rake task I built for vJot. Interestingly enough, due to the way the migrations have been tentatively built for DM, the code is significantly shorter compared to auto_migrations and I know even less about DM than ActiveRecord.

Wait, you mean you aren’t subscribed to Err’s changeset feed? vJot has been powered by DataMapper since the end of July, but I’ll save that for another post.

Breaking News

I’ve just committed support to handle indexes. Using the original example, I’ll add a simple index:
```
ActiveRecord::Schema.define(:version => 1) do
  create_table :posts do |t|
    t.string  :title
    t.text    :body
    t.integer :published
  end

  add_index :posts, :published

  create_table :comments do |t|
    t.string  :name, :url
    t.text    :body
    t.integer :post_id
  end
end
```
Followed by:
```
$ rake db:auto:migrate
-- add_index("posts", ["published"])
   -> 0.0216s
```
Actually, I’m not sure I really need that index, I’ll comment it out for now:
```
ActiveRecord::Schema.define(:version => 1) do
  create_table :posts do |t|
    t.string  :title
    t.text    :body
    t.integer :published
  end

  # add_index :posts, :published

  create_table :comments do |t|
    t.string  :name, :url
    t.text    :body
    t.integer :post_id
  end
end
```
And auto-migrate again:
```
$ rake db:auto:migrate
-- remove_index("posts", {:name=>"index_posts_on_published"})
   -> 0.0187s
```
Dynamite.

Update: Parte Due

I’m here to please, so I’ve just committed a really straightforward AutoMigrations.schema_to_migration method, which can be called via:
```
$ rake db:schema:to_migration
```
If you’re still unsure why you’d use this plugin over regular migrations, you have no excuse now not to try it. It’s really quite fun to use while the schema is still volatile. When things calm down a bit, run the rake task to create the migration and proceed as usual.

Last Time, I Swear

I’ve added change_column support to handle type changes. Assuming the database looks like:
```
ActiveRecord::Schema.define(:version => 1) do
  create_table :posts do |t|
    t.string  :title
    t.text    :body
    t.integer :published
  end
end
```
But, we decide to build the next twitter and the body column only needs to be a varchar:
```
ActiveRecord::Schema.define(:version => 1) do
  create_table :posts do |t|
    t.string  :title, :body
    t.integer :published
  end
end
```
Followed by:
```
$ rake db:auto:migrate
-- change_column("posts", :body, :string)
   -> 0.0309s
```
I’ve really appreciated all of the feedback so far. My next step is to add the :was support and then I think we’ll have something here. Who would’ve thought modifying the database could be fun?!

You errs are sexy code reincarnated! Can you join the Rails core team, please?

this is a very cool piece of code. Isn’t there some logic behind having a distinct, ordered archive of migrations, though? Like say we realize we need to rollback a set of changes we made last week? Does auto_migrations keep a log?

@Austin I haven’t built any facilities to log changes, but it’s certainly possible to add that to the plugin.

August: Unfortunately Rick and I had a knife fight on the hull of a pirate ship 15 years ago and I won, something he still begrudges me to this day.

(That, and we don’t actually patch Rails, just release fancy add-ons.)

Could you handle column renames by having the user comment out the column to be changed in schema.rb and add the changed info on the very next line?

Okay, my question seems silly now that I’ve looked at the code. I’m sure you could parse schema.rb for comment/redef pairs, but that looks to be more trouble than it’s worth.

I must be missing something* because I don’t see what this brings to the party over regular AR migrations.

Probably a clue.

@John this probably isn’t something that you should be using in your production apps (at least not yet), but it dramatically speeds up the development loop for schema changes when you want to quickly prototype something.

For renames, would something like this work?

t.string  :description, :was => :title

adding an option to generate a migration file could be a nice way of migrating down later on.

Crashes out with:

rake aborted! undefined method `attr_accessor_with_default’ for #

Any clues?

Oh, it requires edge rails and active_resource. Froze to edge, installed ActiveResource and works.

Thanks for the heads up Selford.

I’ve removed attr_accessor_with_default from the plugin so running edge is no longer a requirement.

Brilliant work PJ. Now working beautifully on 1.2.3. Thanks a lot.

This plugin would be wonderful if it wrote out a migration file. As it is, it seems like this is roughly equal to just faffing around with ALTER TABLE statements in the database (well, with much better syntax (grin)).

The reason why migrations are utterly fantastic is because they solve the production deployment problem—“my database needs changes for this new version of my app. Rails, please make that happen. kthxbye”.

I think Hobo generates migrations from differences in the database and defined schema: http://hobocentral.net/blog/2007/07/06/so-long-migrations/

This is fantastic! I am one of those that do lots of little changes in schema. This plugin will help me a lot. And +1 for :was=>:title thing and migration generation.

Hearing you guys loud and clear, I’ll add the ability to generate a migration from the schema.rb shortly.

+100 for optionally (on by default) automagically writing a migration file … : )

This is to database-source code synchronization what Rails is to web development. I personally don’t like the migration scheme currently used by Rails—this is much better. The problem I have with the original migration scheme is many-fold: When developing with a team, the numbered files often clash; If you have to pull out a change, you either have to create a new migration to “migrate up” to a previous version, or rename your files so that all later changes are now before the change you want to migrate out; It requires having the files to migrate down. For example, if you want to take a 2 month old (stable) copy of your app from subversion/CVS using a timestamp/tag, and you want to downgrade to it based on current data, your copy doesn’t have the newer migration files needed to downgrade. I’m definitely going to use this in its place. Great job!

Seriously, all I care about is that my expectations for the database (aka db/schema.rb) match the database. I really don’t care what (artificial) version of migration I have to set it to. When developing, why should I have to migrate down, make a change to my migration file, and then migrate back up? Why can’t I just change my schema to what I want and say “make it so”? Need to back out an older change? Just comment it out in the schema. The real versioning I should be tracking is the version of the schema in my Version Control Software. When different team members are working on schema modifications, you simply merge the schema.rb file.

As for the troublesome renaming issue (for columns and tables too), I agree using the :was => :x or :was => [:x, :y, :z] (for developers that constantly change their mind. To solve the downgrading problem (because previous versions didn’t know you’d be changing their names), I recommend utilizing the comment field for columns and tables to record the aliases—so if you can look there if the “was” isn’t specified in the file. You can use a string like “column_name_was=x,y,z”. Only drawback, can’t recycle names (a => b, c => a)

- Just my (very opinionated) 2 cents worth.

Great post, incredibly sexy migration!! :D I just think this can be a little tricky to use in production environment with a lot of programmers, can’t it? I still think it’s safer to use the migrations in its original form, with its versions, upgrades and downgrades. Perhaps it could generate a new migration file when you change the schema.rb file! This way, you could try some different database structures with the db:auto:migrate and, when you find the final one, it could generate a final migration from the old schema to the new defined schema! Anyway, congratulations for the migrations! :D

Felipe Giotto

I think like Matt. The generate of migration file will be amazing.

9 out of 10 hoes agree:

Schema.rb is the hottest customer on the block right now

kudos!

Hey PJ. I waited a bit to try this out as I am swamped at work, and I get harassed for always trying to use new stuff (and getting the team to use em as well) in our project as we race towards our R1.

Well, our migrations have gotten out of hand (numbering-wise, we use err’s remigrate so changes aren’t all over or anything) and I fired up auto migrate and ran one more remigrate to recreate the schema, then did some work on it (like adding indexes) and WOW! BANG! POW! Holy horse dung Batman, I am sold like a Kirby.

Another great plugin from some really clever guys. Thanks a bunch.

Perhaps I am missing something, but whenever I do a db:test:prepare my schema.rb gets all unsexyfied. Am I doing something terribly wrong?

hey – if you like this, go check out the completely automated migrations in the Hobo plugin (hobocentral.net)- no more schema.rb either! just write your models and let Hobo write the migrations automatically for you.

By the way, the ‘sexy migration’ plugin stated above that has found its way into Rails actually originated from the Hobo project.

Go check it out…

I’ve been thinking.. and something like this, but which operates on attributes (columns) defined actually in the models would totally kick ass. Rails has it ass-backwards on this front.. wtf do we need to write migrations and schemas when that info should really be in the model in the first place.

To prevent rake tasks from dumping over your pretty schema (eg, in my case, RSpec), you can add the following kludge to the end of your Rakefile:

Rake.application.instance_variable_get(:@tasks).delete(‘db:schema:dump’) namespace(:db){namespace(:schema){task(:dump){puts “Schema dump disabled”}}}

Great plugin! Thanks. I always found migrations quite unsexy and not very suitable for version control at all.

Peter Cooper really has a point there! I do not want to look in schema.rb or migrations if I can’t remember which attributes my model has, or what I called them. I want to look in the model!

It seems not to work with ActiveRecord’s table name prefix, the plugin tries to change prefix+prefix+tablename instead of prefix+tablename

mm… anyone get it working with :primary_key table option in schema?

—change_column(“movimento_equip”, :movimento_id, :primary_key) rake aborted! Mysql::Error: #42000Multiple primary key defined: ALTER TABLE `movimento_equip` CHANGE `movimento_id` `movimento_id` int(11) DEFAULT NULL auto_increment PRIMARY KEY DEFAULT NULL

Your link: http://plugins.require.errtheblog.com/browser/auto_migrations

is showing a 404

Link http://plugins.require.errtheblog.com/browser/auto_migrations is broken & ruby script\plugin install svn://errtheblog.com/svn/plugins/auto_migrations is also not working

The plugin is now located at:

http://github.com/pjhyett/auto_migrations

Sorry, no more comments :(

What it is Chief

Limitations

But, Can It Scale?!

Check it Out

1% Inspiration and 99% Perspiration

Breaking News

Update: Parte Due

Last Time, I Swear