The 3rd FOST Cup 1997
(Report by David Fotland),
The 3rd FOST cup was held on August 27 and 28, 1997, in Nagoya
Japan, at the Nagoya Congress Center, in conjunction with the IJCAI-97
conference on artificial intelligence.
40 programs entered, up from 17 programs last year, so logistics forced
an unusual tournament system. The programs communicate through
RS-232 cables, and last year there were problems with the long cable
length causing breaks in communication. This year shorter cables were
used, in a setup of 4 rings of tables, with room for 10 programs each.
Pairings within the same group of 10 could be accomplished by moving
cables, but pairings between tables required computers to be moved.
This year FOST provided a DLL and test program for communication, so
almost all programs had the communication protocol. They provided
time the evening before for communication testing as well. In spite of
this there were many communication glitches, but most were resolved
quickly. Go Intellect and Stone both dropped moves against some
programs but not others. In one round, Many Faces was used as a
communication program for Go Intellects opponent, since it retransmits
dropped moves.
The competition schedule was just over a day and a half, to allow time
for the award ceremony and party. So the organizers decided to split the
programs into two groups of twenty for the first days 6 rounds, and use
the results of the first day to form four groups for the second days 4
rounds. The first days groups were seeded based on last years results,
with Hit and Star of Poland added to the strong programs.
At the end of the first day, in the group with Many Faces, there was one
program with 6 wins, one with 5 wins, and 6 programs with 4 wins. The
other first day group had 1 with 6 wins, 2 with 5, and 5 with 4 wins.
Since only 4 programs could be in the top group on the final day, tie
breaking had to be used.
The tie breaker is first, sum of defeated opponents scores (SDS), and
second, sum of opponent scores (SOS), rather than the usual Swiss
system of SOS first, and SDS second. The tie breaking choice led to a
few strange group assignments after the first day. The organizers also
decided to use the results of all 10 rounds for final tie breaking. This led
to some strange final results since the highest programs in the lower
groups had high numbers of wins, which did not reflect their actual
strength, but helped program's SDS who happened to have been paired
with them.
Because of pairing anomalies, Biwako advanced above Many Faces, even
though Many Faces had beaten both Jimmy and Stone, and Biwako had
lost to both Jimmy and Stone. On SOS, Many Faces was 3rd in its
group, but using SDS, it was 5th, and did not make the cut for the top
group in the second day. There was a long discussion among the
organizers about this strange result, but in the end they did not make an
exception to the tie breaking procedure in the rules. They did agree to
change the tournament system for next year.
Handtalk was once again clearly the strongest program, and should have
won all of its games. Its game against Go4++ was close (about 5 points).
It lost the second round to a weak program when it ran out of time while
it was way ahead. This years Handtalk does much more analysis, and
the weak program left many unsettled groups on the board, forcing
Handtalk to do a lot of reading. Professor Chen said he tried over 100
test games, and none were close to losing on time, so he did not
implement time control in his program.
In the last round, Handtalk and Star of Poland were the only programs
with 8 wins, but were not paired with each other. If both had won, there
would have been an additional playoff game. But Star of Poland lost to
Silver Igo, dropping out of the top two places, and Handtalk was playing
Aya, and was way ahead. But Aya is also a weaker program, and left
many unsettled groups on the board. Handtalk was playing slowly and
looked like it would lose on time again. If Handtalk lost, there would be
a 4 way tie for first place, with two of the programs having never played
each other. Tie break would put Handtalk 4th. But before time ran out
for Handtalk, one of the organizers suggested to Hiroshi that he should
resign. About 5 seconds before Handtalk's clock ran out, Hiroshi
resigned for his program. But Handtalk's clock ran out before the
organizers were informed of the resignation. This led to about an hour
of discussion. The rules allow the operator to resign for his program, but
there was a dispute over whether the resignation was before or after the
clock ran out. Some people didnt like the organizers suggesting a
resignation. Some felt that since Handtalk was the strongest program, it
deserved first place. In the end, the resignation was allowed to stand,
and Handtalk finished first, winning 2 million yen.
Handtalk was given a 3 Kyu diploma by the Nihon-Ki-in. After the
tournament, it played an exhibition game against a strong 2 Kyu, and
won by one point with no komi. The 2 kyu was a youngster, with little
experience playing computer programs, and she had to play on the
monitor rather than a go board, but this is still a great achievement for a
go program. The professional judging its strength thought that Handtalk
was about 4 kyu, but felt he had to give the 3 Kyu diploma after
Handtalk won the game. It seems that a dan level program is not as
many years away as we had thought.
Go Intellect and Go 4++ were tied on both tie breakers, but Go Intellect
had beaten Go 4++, so it got second place. In the game between Go 4++
and Go Intellect, Go4++ was ahead until the very end, when Go Intellect
was made a simple atari that Go4++ answered wrong, so Intellect
captured some stones and won the game. The game between Go4++ and
Handtalk was not as close as the one between Go 4++ and
Handtalk, but it was more of a fighting game, so the score may not
indicate the difference in playing strength. G Intellect won 500,000 yen,
and Go 4++ won 200,000 yen. Go Intellect was given a 5 kyu diploma.
About two years ago, the top programs became strong enough to make
interesting products. At about 10 kyu, they could give good games to
most casual players, and beat beginners for about a year. Handtalk,
Go4++, Go Intellect, and Many Faces of Go all had their engines
licensed for sale in Japan and Korea, and altogether have sold about
100,000 copies.
The recent commercial success of computer go products has attracted the
attention of computer game companies. Silver is a new strong Japanese
program being developed by a Japanese game company. 4 people
worked on it for about a year. Hit was developed by a 7 person team
over 18 months, by one of the top Korean computer game companies.
The tournament format used makes it difficult to judge the true strength
of the programs, but we can draw some conclusions. Leaving out
Handtalk's loss on time, Handtalk is clearly stronger than all other
programs again. Entering the tournament, Go4++, Go Intellect, Many
Faces, and Hit all were beating last year's version of Handtalk more than
half the time. But the new Handtalk is stronger.
The next level of program strength includes Go4++, Go Intellect, Star of
Poland, and Silver. Go Intellect beat Go 4++, which beat Star of Poland,
which beat Go Intellect. Silver also beat Star of Poland, and only lost to
Go Intellect by point. So these four programs are very close in
strength and were not defeated by any weaker program. Go4++ looks
stronger than last year. Michael Reiss is automatically collecting
patterns from pro games to suggest moves with good shape, and it
shows. The patterns are used to suggest moves to look at and affect
about 5 percent of the evaluation. Michael was not willing to share the
details of his learning algorithm. His program also has much better
fuseki than it did before. It still plays a very territorial game with little
fighting, and I suspect its fighting strength is lower than the other top
programs.
In the game between Go4++ and Many Faces, the programs each just
surrounded territory. There was no fighting, and Go4++ ended 11 points
ahead. Janusz ran Star of Poland at level 2 (of 4). He says that at the
lower level it looks at fewer moves, so makes fewer strange moves. It is
still good at fighting. Against Many Faces, it killed one of MFs groups,
then lost two of its own, but in the end brought both groups back to life
and killed another of Many Faces.
The next lower strength group includes Gogol, Many Faces, Jimmy, and
Hit. Gogol didnt play any of the other programs in this group, but
Many Faces beat Jimmy. Jimmy beat Hit, and Hit beat Many Faces. Hit
lost a game to Fungo due to a crash, but it was winning. Other than this,
no program in this group was defeated by a weaker program. I only
worked a few weeks last year on Many Faces, and the change in result
shows how much the other programs have improved. Gogol is a new
strong program based on automated learning of patterns, tactics, and
strategy.
Many Faces crashed the first day against Star of Poland and Stone when
the positions became complex, but successfully restarted. I couldn't
reproduce the crash until the first round had started the next day, so in
round 7, against Hit, I used the old, stable, commercial version, which
led to a small loss. This bug did not surface in over 30 test games
against Handtalk. I fixed it for the round 8 game against Stone. Jimmy
is one or two stones stronger than last year and beats Handtalk
sometimes.
The next group is Stone, Goro, Biwako, Aya, Mutsuki, KuruKuru, and
Fungo. Goro beat Mutsuki, which beat Stone, which beat Goro. Stone
beat Biwako, which beat Aya, which beat Mutsuki, which beat Stone.
Biwako beat KuruKuru, which beat Mutsuki. Fungo beat Goro. Fungo
crashed against Explorer, but was winning. Many Faces had a close
game with Goro, a fighting game with Stone, and an easy win over
KuruKuru.
Michael Redmond pro 8 dan, attended, and gave commentary on some of
the games, including the one between Many Faces of Go and Star of
Poland, which had some interesting fighting.
Next Year's FOST cup will be August 28 and 29 in Tokyo. There may
also be a Korean computer go tournament next year. To enter next year's
FOST cup, contact atakahashi@koei.co.jp.
A special technology prize was given to the top amateur program (not
commercialized or university research). I think it went to Masahiro
Tanaka for Biwako, who finished 7th.
Before the Go tournament, there was a 2 day workshop called "Using
Games as a Testbed for AI research". It had many interesting papers,
including two good ones on computer go. Martin Mueller showed how
to extend thermography to positions with ko, and Takuya Kojima
presented an interesting approach to automatic learning of go patterns
from professional games. Michael Buro, the author of Logistello, the
strongest Othello player in the world, gave a talk on automated opening
book learning. Jonathan Schaeffer, author of Chinook, the world's
strongest checkers player, gave a talk on the development of Chinook,
and led a discussion on the effect of smart computer games on society.
We had a panel discussion on the next thing after chess, with opinions
that chess is not dead yet, Shogi is more complex than chess, and Go
may be too difficult.
Tournament results: Final after 6 rounds
Wins SDS SOS Wins SDS SOS
Group A Group Place
1 Chen ZhiXing Handtalk China 9 5A 14 16 3
2 Ken Chen Go Intellect USA 8 46 63 6A 22 22 1
3 Michael Reiss Go 4++ England 8 46 63 6B 18 18 1
4 Janusz Kraszek Star of Poland Poland 8 43 58 5B 17 23 2
5 Naritatsu Yamamoto Silver Igo Japan 7 41 67 5A 17 21 2
6 Tristan Cazenave Gogol France 6 28 59 4A 10 21 4
7 Masahiro Tanaka Biwako Japan 5 21 58 4B 11 18 4
8 Hiroshi Yamashita Aya Japan 4 14 56 4B 11 19 3
Group B
9 Shi-Jim Yan Jimmy Taiwan 8 36 51 4B 10 19 6
10 Park Yong-Goo Fun Go Korea 7 33 53 4A 8 16 8
11 Kuo-Yuan Kao Stone USA 7 31 50 3B 9
12 David Fotland Many Faces USA 7 28 50 4B 10 21 5
13 Ji Il Kim Hit Korea 6 25 53 4A 9 19 7
14 Martin Mueller Explorer Switz. 5 26 58 3A 9
15 Yasuo Ooishi Goro Japan 5 25 59 4A 10 20 5
16 Yasuo Hirooka KuruKuru Japan 5 15 46 4B 7 15 7
17 Tamizou Shiraishi Gosaku Japan 4 19 61 4A 10 19 6
18 Takahisa Yoshida Mutsuki Japan 4 12 45 4B 6 17 8
Group C
19 Tomoyuki Shintaku Tengen Japan 7
20 Noriaki Sanechika Igo1997 Japan 5 20
21 Shinichi Sei Katsunari Japan 5 18
22 Osamu Ushio B/W Japan 5 17
23 HiroFumi Iwasaki Gizumogo Japan 5 12
24 Kenichi Amano Heaven Walk Japan 4 17
25 Mei-Kou Tei Young Leaf Taiwan 4 14
26 Hiroto Yoshii Monkey Jump Japan 4 9
27 Darren Cook Darrens Prog England 3 15
28 Toshikazu Satou Tokyo 97 Japan 3 9
Group D
29 Jee Wonho Go Master Korea 6
30 Tetsuya Wkamatsu Twigo 32 Japan 5 10
31 Takuo Tabuchi Takuchan Japan 5 8
32 Kakuyoshi Nishino Goblin Japan 4
33 Tsuneo Horii Utoro Japan 3
34 Yung Jye Huang Keeping Awk I Taiwan 2 3
35 Masao Maruyama Ranka Japan 2 1
36 Masami Miyagawa Ijokankaku Japan 1 1
37 Hang Joon Kim Rex Korea 1 0
38 Shinji Murakami EVA-01 Japan 0
Full tournament Grid (not in result order):
1 Handtalk +11 -20 +13 +28 +25 +41 +37 +4 +2 +15
2 Go 4++ +39 +35 +22 +27 +3 +18 -4 +37 -1 +12
3 Many Faces +31 +6 +19 -18 -2 +22 -25 +5 +8 +34
4 Intellect +33 +12 +38 +25 +37 +5 +2 -1 -18 +10
5 Goro -41 +28 +11 +29 +8 -4 -21 -3 +7 -22
6 Jimmy +30 -3 -18 +10 +19 +27 +41 +25 +34 +21
7 Mutsuki -18 +40 +26 +19 +22 -15 -34 -8 -5 -25
8 Explorer +21 +17 -37 +38 -5 -25 -22 +7 -3 +41
9 Takuchan -38 -25 -33 -12 +23 +16 -17 +39 +31 Bye
10 Biwako +34 -22 +27 -6 +30 +14 +15 -18 -12 -4
11 Twigo32 -1 -13 -5 +23 -16 -28 +19 +31 +39 +33
12 Gogol +29 -4 +28 +9 +13 -37 -18 +15 +10 -2
13 Katsunari +20 +11 -1 -41 -12 -21 +14 +27 -29 +38
14 Tokyo97 +26 +31 -0 -22 -27 -10 -13 +20 -35 -28
15 Aya -22 +34 +39 +30 -18 +7 -10 -12 -37 -1
16 Utoro -28 -41 -29 -33 +11 -9 +39 -23 -17 +31
17 Go Master -37 -8 +23 -21 -41 +29 +9 +33 +16 bye
18 Star Poland +7 +19 +6 +3 +15 -2 +12 +10 +4 -37
19 Rex +40 -18 -3 -7 -6 -35 -11
20 Darren -13 +1 -21 -37 -28 +23 -26 -14 +30 -35
21 Fun Go -8 -37 +20 +17 +33 +13 +5 +41 +25 -6
22 Stone +15 +10 -2 +14 -7 -3 +8 +34 +41 +5
23 Keeping -25 -38 -17 -11 -9 -20 +40 +16 -33 -39
25 Hit +23 +9 +41 -4 -1 +8 +3 -6 -21 +7
26 Monkey -14 -30 -7 +40 -35 +31 +20 +28 -27 -29
27 B/W +35 +39 -10 -2 +14 -6 -38 -13 +26 +30
28 Young Leaf +16 -5 -12 -1 +20 +11 -29 -26 -38 +14
29 Tengen -12 +33 +16 -5 +38 -17 +28 +35 +13 +26
30 Heaven walk -6 +26 +14 -15 -10 -34 +35 +38 -20 -27
31 Ranka -3 -14 +40 -34 +39 -26 -33 -11 -9 -16
33 Goblin -4 -29 +9 +16 -21 -38 +31 -17 +23 -11
34 KuruKuru -10 -15 +35 +31 +40 +30 +7 -22 -6 -3
35 Gizumogo -27 -2 -34 +39 +26 +19 -30 -29 +14 +20
37 Silver Igo +17 +21 +8 +20 -4 +12 -1 -2 +15 +18
38 Igo97 +9 +23 -4 -8 -29 +33 +27 -30 +28 -13
39 EVA-01 -19 -7 -31 -26 -34 -39 -23
40 Gosaku +5 +16 -25 +13 +17 -14 -6 -21 -22 -8