Don Dailey has developed an
autotester
for computer go. This allows two Computer Go programs to play against
each other automatically. It is written in ruby, and works on Linux (and
possibly also on Windows).
This program requires that each program speaks GTP 2.0 via stdin and
stdout, and obeys Chinese type rules. The games must be played out so
that no dead stones remain, because scoring is automatically done by
the autotester.
You set up a simple registry file which specifies each program and it
plays multiple round robin tournaments between them all. The results are
stored in a "pgn" file.
The tester can be stopped and restarted, and continues where it
left off.