Running Ruby on Rails Tests in Netbeans

I don't get it. Netbeans is often recommended as an excellent IDE for Ruby on Rails development, not only when targeting the JVM. Nevertheless, even some basic features don't seem to be working with the default setup. You can't even run the tests, which is fundamental to developing using a dynamic language.

What's happening? Suppose you have a simple app and you want to run some tests using the test database. Not sure if this is mandatory when using the built in JRuby but it seems to be normal to use the jdbcmysql adapter. When you try to run the tests you will see something like this:

1) Error:
test_index_is_ok(ContactsControllerTest):
ActiveRecord::StatementInvalid: ActiveRecord::JDBCError: Table 'kontakt_test.contacts' doesn't exist: DELETE FROM `contacts`

followed by the stack trace that isn't really helpful as it's not the root cause. Rails somehow doesn't create the tables in the test database. You'll see a more helpful output when starting the rake task "db:test:prepare" directly in debug mode:

** Invoke db:test:prepare (first_time)
** Invoke db:abort_if_pending_migrations (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:abort_if_pending_migrations
rake aborted!
Task not supported by 'jdbcmysql'
/path/to/netbeans-6.9.1/ruby/jruby-1.5.1/lib/ruby/gems/1.8/gems/rails-2.3.8/lib/tasks/databases.rake:380
/path/to/netbeans/ruby/jruby-1.5.1/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'
/path/to/netbeans/ruby/jruby-1.5.1/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'
/path/to/netbeans/ruby/jruby-1.5.1/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'
[...]
** Execute db:test:prepare
** Invoke db:test:load (first_time)
** Invoke db:test:purge (first_time)
** Invoke environment 
** Execute db:test:purge

The task fails in the database task in the rails lib. You can open up the source code by opening the node Libraries/Built-in-JRuby/rails-2.3.8/lib/tasks/databases.rake in Netbeans.

At line 357 you can see the problem: Rails only expexts some hardcoded adapters, jdbcmysql not being one of them. It skips the task for unknown adapters. Two options to fix it: Insert a regular expression that matches both:

when /mysql/ # instead of when "mysql"

or add the jdbcmysql adapter as a second option:

when "mysql","jdbcmysql"

Now the tests are running and hopefully passing. The same kind of error might occur for other tasks as well as there are some more checks for the mysql adapter in this file. You should be able to fix them the same way.

I wouldn't have expected to have to patch the rails code for using it in Netbeans but this doesn't seem to be uncommon. Using a recent active record version is supposed to fix the problem as you can use mysql as an adapter name then but I didn't find a way to run the jdbc generator from Netbeans. It isn't available in the list of generators and I didn't find a generator gem to download.

What's to be learned for me from this? I got a better understanding of how the build process works using rake. But more importantly: even technologies that have been hyped for a long time might not be that flawless as you would expect.

Refactoring in Git

To me, when using SVN, the most important reason for using an IDE plugin was the refactoring support: SVN doesn't notice when you rename a file, you have to explicitly call svn mv.

I thought this would be a major problem with Git, as a Java refactoring changes the content and the filename in one go. As the content changes the SHA1-checksum also changes and you'd run into problems. Fortunately, that's not the case.

With Git, you don't need a special operation: It detects renames with minor changes automatically.

Time for a test. Suppose you have a simple Java class like this:

public class TestClass {

    public static void main(String [] args) {
        System.out.println("Hello Git");
    }

}

Commit it to the Git repository:

flo@hank:~/git-netbeans$ git add src/TestClass.java
flo@hank:~/git-netbeans$ git commit -m "added test class"
[master 9269c2f] added test class
 1 files changed, 7 insertions(+), 0 deletions(-)
 create mode 100644 src/TestClass.java

Rename the class (either by using an IDE or by executing a manual refactoring by changing the file name and the class name):

public class TestClassWithNewName {

    public static void main(String [] args) {
        System.out.println("Hello Git");
    }

}

git status will tell you something like this:

flo@hank:~/git-netbeans$ git status
# On branch master
# Changed but not updated:
#   (use "git add/rm ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#       deleted:    src/TestClass.java
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#       src/TestClassWithNewName.java
no changes added to commit (use "git add" and/or "git commit -a")

Doesn't look that good yet. It detects an added and a removed file. Next, stage the changes and have another look at the status:

flo@hank:~/git-netbeans$ git rm src/TestClass.java
rm 'src/TestClass.java'
flo@hank:~/git-netbeans$ git add src/TestClassWithNewName.java
flo@hank:~/git-netbeans$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#       renamed:    src/TestClass.java -> src/TestClassWithNewName.java
#

Neat, Git detected a rename. Let's commit and see the log:

flo@hank:~/git-netbeans$ git commit -m "refactored class"
[master 4acd7f1] refactored class
 1 files changed, 1 insertions(+), 1 deletions(-)
 rename src/{TestClass.java => TestClassWithNewName.java} (72%)
flo@hank:~/git-netbeans$ git log src/TestClassWithNewName.java
commit 4acd7f19ccd6cc02816ee7f1293ea5a69d7a4ca7
Author: Florian Hopf 
Date:   Sun Jan 9 14:27:59 2011 +0100

    refactored class

Hmmm, only the last commit? Looks like we have to tell that we want to follow renames:

flo@hank:~/git-netbeans$ git log --follow src/TestClassWithNewName.java
commit 4acd7f19ccd6cc02816ee7f1293ea5a69d7a4ca7
Author: Florian Hopf 
Date:   Sun Jan 9 14:27:59 2011 +0100

    refactored class

commit 9269c2fd194b2bd2b93a18ab88f21fb2180c5870
Author: Florian Hopf 
Date:   Sun Jan 9 13:48:35 2011 +0100

    added test class 

What do I take from this experiment? I guess I won't use the Netbeans Git plugin for now. I still have to get acquainted to the command line and its better to learn the basics first.

Git hook for Redmine messages

At work we are using Redmine with the repository references enabled. When adding special terms like refs #1234 or fixes #1234 to the commit message the commit is automatically assigned to ticket 1234 and shown with the ticket. Only commiting code that references a ticket is considered to be a best practice as all changes are documented with a ticket.

As I'm using the Git SVN bridge now I tend to commit more than using plain SVN. Often I just forget to add the refs marker which is quite annoying. Pro Git introduces a hook that can be used to check your commit message for a special format.

This is the shamelessly copied hook, adjusted to the Redmine keywords:

#!/usr/bin/env ruby
message_file = ARGV[0]
message = File.read(message_file)

$regex = /(refs #(\d+)|fixes #(\d+))/

if !$regex.match(message)
  puts "Your message is not formatted correctly (missing refs #XXX or fixes #XXX)"
  exit 1
end

How to use it? Copy the code to the file .git/hooks/commit-msg in your project and make it executable (chmod +x .git/hooks/commit-msg).

Try to commit without the markers:

flo@hank:~/git-redmine$ git commit -am "commit that doesn't reference a ticket"
Your message is not formatted correctly (missing refs #XXX or fixes #XXX)

And with a marker:

flo@hank:~/git-redmine$ git commit -am "commit that references a ticket, refs #1234"
[master 189b6b1] commit that references a ticket, refs #1234
 1 files changed, 2 insertions(+), 0 deletions(-)

If you want to skip the hook for some reason you can do so using the --no-verify option:

flo@hank:~/git-redmine$ git commit --no-verify -am "special commit that doesn't reference a ticket"
[master d1c0698] special commit that doesn't reference a ticket
 1 files changed, 1 insertions(+), 0 deletions(-)

GoGear title management in Sqllite

While writing this post I noticed that I am just wrong with my assumptions on my GoGear device. Read on to learn why. I'll publish it anyway as the information should still be valid for older versions of GoGear.

I've been looking for a way to enable the bookmarks feature for audio books for my Philips GoGear SA1922. Unfortunately according to this image, which is only displayed on the german site, it's not supported for this version though with the latest firmware there's a special menu option for audio books.

While experimenting I learned a few things along the way that are quite interesting. GoGear uses Sqlite for managing all meta information of the audio files stored. I use golb to transfer music from my Linux machine which does all the magic of extracting ID3 tags and inserting all data in the database. Normally you would use golb -f _system/media/audio/MyDb in the root folder of the mounted storage device to scan all files on the device and write it to the database _system/media/audio/MyDb.

If you want to see or manipulate the data you can use the sqlite client: In the same folder call sqlite _system/media/audio/MyDb. This will open a client console similar to mysql:

flo@hank:/media/PHILIPS$ sqlite _system/media/audio/MyDb
SQLite version 2.8.17
Enter ".help" for instructions
sqlite>

To see the schema information you can issue the .schema command, which display information on all the tables and its indexes:

sqlite> .schema
CREATE TABLE albumTable( iAlbumId INTEGER PRIMARY KEY, cAlbumTitle VARCHAR(100) );
CREATE TABLE artistTable( iArtistId INTEGER PRIMARY KEY, cArtistName VARCHAR(100) );
CREATE TABLE dirTable( iDirId INTEGER PRIMARY KEY, cDirName VARCHAR(260),iParentDirId INTEGER );
CREATE TABLE genreTable( iGenreId INTEGER PRIMARY KEY, cGenreName VARCHAR(50) );
CREATE TABLE playlistTable( iPlaylistId INTEGER PRIMARY KEY,cPlaylistName VARCHAR(100), cFileName VARCHAR(260),iDirId INTEGER );
CREATE TABLE playsongTable( iPlaysongId INTEGER PRIMARY KEY,iPlaylistId INTEGER, iOrderNr INTEGER,iSongId INTEGER );
CREATE TABLE songTable ( iSongId INTEGER PRIMARY KEY,cSongTitle VARCHAR(100),iArtistId INTEGER,iAlbumId INTEGER,iTrackNr INT8,iTrackLength INT16,iNrPlayed INT16,cFileName VARCHAR(260),iDirId INTEGER,iYear INT8,iGenreId INTEGER,iBitRate INTEGER,iSampleRate INTEGER,iFileSize INTEGER,iMediaType INTEGER );
CREATE INDEX album_cAlbumTitle ON albumTable (cAlbumTitle);
CREATE INDEX artist_cArtistName ON artistTable (cArtistName);
CREATE INDEX dir_cDirName ON dirTable (cDirName);
CREATE INDEX dir_iParentDirId ON dirTable (iParentDirId);
CREATE INDEX genre_cGenreName ON genreTable (cGenreName);
CREATE INDEX playlist_cPlaylistName ON playlistTable (cPlaylistName);
CREATE INDEX playsong_iOrderNr ON playsongTable (iOrderNr);
CREATE INDEX playsong_iPlaylistId ON playsongTable (iPlaylistId);
CREATE INDEX playsong_iSongId ON playsongTable (iSongId);
CREATE INDEX song_cFileName ON songTable (cFileName);
CREATE INDEX song_cSongTitle ON songTable (cSongTitle);
CREATE INDEX song_iAlbumId ON songTable (iAlbumId);
CREATE INDEX song_iArtistId ON songTable (iArtistId);
CREATE INDEX song_iDirId ON songTable (iDirId);
CREATE INDEX song_iGenre ON songTable (iGenreId);
CREATE INDEX song_iTrackNr ON songTable (iTrackNr);

To see some of the song information you can query the songTable:

sqlite> select * from songTable limit 3;
1|CRE041 Sprachen|1|1|0|4476|0|chaosradio_express_041.mp3|28|2007|1|128|44100|71709702|1
2|Java Posse #331 - Roundup '10 - Modules|2|2|331|3783|0|JavaPosse331.mp3|28|2010|2|96|44100|45460832|1
3|CRE080 Geschichte der Typographie|1|1|0|7947|0|chaosradio_express_080.mp3|28|2008|3|128|44100|127247455|1

You can use standard sql to update the information:

sqlite> update songTable set cSongTitle = "Sprachen - Chaos Radio Express 41" where iSongId = 1;

At this point I was about to hit the publish button for the post. Luckily I tried if the update happened at all. Turned on my device: Still the old title. Rebootet the device, deleted and recreated all indexes, inspected the Golb sourcecode, found nothing. After a while it struck me: I don't need golb for my GoGear version. This device seems to extract all information from the id3 tags directly, very likely during startup. Classic fail!