Arimaa/Arimaa Challenge History

2004

In the 2004 Arimaa Computer Championship, the runaway winner was Bomb, programmed by David Fotland. Fotland had a special incentive to pour time into the creation of Bomb that year; Omar Syed had promised to double the Challenge prize to $20,000 if Fotland could win it in the very first year.

On the human side, Frank Heinemann had won the 2004 Arimaa World Championship, but was reluctant to defend the honor of humanity in the Challenge match with so much money on the line. Therefore Syed, the top-rated human player at the time, defended his own prize.

Early versions of Bomb had seemed to play at least as well as any human in mid-2003. However, humans soon discovered the Camel hostage strategy, which was so effective against Bomb that Fotland had to retune his bot to prevent it from giving up its Camel as a hostage every single game. The version of Bomb that played in the 2004 Challenge was much less aggressive.

There was some suspense as to whether Syed would be able to beat Bomb without recourse to Camel hostages, but it soon emerged that the more-defensive Bomb would gradually concede small strategic weaknesses in face of a lone Elephant attack, allowing Syed to slowly build up a large material advantage before breaking open the game. Bomb appeared to outplay Syed in some open positions, but Syed's accumulated material always held up.

After sweeping five games, Syed became confident enough to attempt to win the final three games under extra constraints. In the sixth game he won without losing a piece, in the seventh game he won without capturing a piece or losing a piece, and in the eighth game he won with a material handicap of a Rabbit.

The final score of 8-0 in favor of Syed was much more lopsided than expected, and belied a few exciting positions in the early games. Fotland opined that he had missed his best chance to win the Challenge, as humans would improve at playing Arimaa more rapidly than he could improve his software.

2005

Bomb faced stiffer competition in the 2005 Arimaa Computer Championship than it had the previous year, and advanced to the challenge match only on tiebreaks. David Fotland had improved Bomb mostly by fixing small bugs and fine-tuning the evaluation function, but he also added one important strategic innovation: Bomb became selectively aggressive with its Camel when the two Elephants were tied up in a hostage situation.

Human understanding of Arimaa had expanded considerably in the intervening year, in particular with initial exploration of the Elephant-Horse attack. Despite these advances, some suspense crept back into the Challenge when the two top-rated human players declined to participate based on time commitments. Omar Syed (then ranked #3) did not want to defend his own prize again, so he persuaded Frank Heinemann, the only other human player rated higher than Bomb, to do the honors.

Heinemann entered the match rated a mere 55 points higher than Bomb, a gap that would imply only a 58% chance of winning each game. The superficially close match eventually turned into a rout, though, as Heinemann launched the Elephant-Horse attack in game after game, and perfected his technique as the match progressed. In the first game Bomb never got much going with its Camel counter-attack, and in the second game it was too little too late. Heinemann engineered an Elephant smother to dominate the third game.

In the fourth game, Heinemann attempted another Elephant smother, which Bomb sidestepped at the last moment to enter an unclear midgame. Without any guiding strategic theme (no Camel hostage, no blockade, no EH attack) Heinemann faltered and lost a double-edged endgame. That was to be Bomb's only victory. In the fifth game Heinemann caught Bomb's counter-attacking Camel in exchange for a Horse, and in the final three games, Heinemann won without losing a piece.

The final score of 7-1 in favor of Heinemann showcased his human adaptability, as his Elephant-Horse attack improved and Bomb's response remained static.

2006

Given his former difficulties persuading top humans to defend the Arimaa Challenge, Omar Syed changed the format beginning in 2006. Instead of a single human playing a two-week, eight-game match against the top computer, three humans would each play a three-week, three-game mini-match, alternating games. The computer would need to win two out of three games in all three mini-matches to win the Challenge. Syed intended to relieve the psychological pressure and time commitment required of volunteer defenders. The changed format also recognized that an increasing number of human players were capable of beating any bot.

The human participants in 2006 Challenge were Karl Juhnke, Greg Magne, and Paul Mertens, ranked first, second, and sixth respectively. The human contingent was supremely confident entering the match, not only based on the results of past years, but also because the year since the previous Challenge had seen the development of half a dozen different strategies for reliably exploiting computer weaknesses. Mertens, in particular, had beaten the 2005 version of Bomb with a variety of enormous material handicaps.

Unbeknownst to the defenders of the Challenge, the computer challenger was the exact version of Bomb from the previous year. Perhaps the humans were fooled by the ease with which Bomb won the 2006 Arimaa Computer Championship, against supposedly improved competition.

In any event, humanity swept the first eight Challenge games, and in six of those games did not lose a piece. Only in the fifth game of the match, against Magne, did Bomb generate winning chances, but Magne rose to the occasion and punched a Rabbit to goal where Bomb had left too thin a defense.

For the ninth and final game, Mertens decided to test the margin of human dominance by offering Bomb a material handicap of a Camel. In theory, playing without a Camel should render all modern strategies ineffective. Mertens was unable to gain control of the game, but did generate a sharp goal race, which Bomb won by a narrow margin.

The final score of 8-1 in favor of humanity reflected the widening gap between humans and computers in Arimaa ability. In all likelihood the Challenge could have been handily defended by the humans ranked eighth through tenth because the techniques required to defeat Bomb with predictable ease had been mastered by all the top players.

2007

Leading up to the 2007 Arimaa Challenge match, everyone was prepared for the possibility that Bomb would remain unimproved from the 2005 version. It was therefore surprising that Bomb won the Computer Championship undefeated, after the same version had lost one game in each of the previous two Computer Championships.

In previous years, Omar Syed had required the bots to play publicly before the Computer Championship so that they could not conceal improvements from the human defenders of the Challenge. However, this was somewhat unsatisfactory, given that developers could change their bots at the last minute after having met the public play requirements. For 2007, in order to guarantee at least some scrutiny of the final version of the Challenge bot, Syed removed the pre-tournament public-play requirement and substituted a qualifying phase after the Computer Championship. Thus Zombie, the second-place finisher from the Computer Championship, advanced along with Bomb to the qualifying phase, even though Zombie lost twice to Bomb in the Computer Championship.

During the qualifying phase, any human (except the bot developers and Challenge defenders) was allowed to play each of the top two bots twice, once with Gold and once with Silver. The bots would not be able to conceal their true strength at this point, since only the bot that won more games would advance to the Challenge. Despite the fact that Zombie was a relative unknown and Bomb had been extensively studied, of the twelve pairs of games humans played against the bots, Bomb won two pairs and Zombie won none. (In the other ten pairs the human player either lost both or won both.) Bomb therefore demonstrated that it was not only able to beat other bots head-to-head, but was also tougher for humans to contend against.

Syed opted to test the growing margin of human dominance by selecting himself as a defender, despite having tumbled to twentieth in the world rankings. For the other defenders he selected Karl Juhnke, ranked first, and Brendan M, ranked nineteenth. Syed's confidence was nearly punished in the first game of the Challenge, when he attempted a double-trap attack, faltered in a complex middlegame, and lost two Horses for a Dog and a Rabbit. Bomb, however, did not properly press its material advantage, and permitted Syed to generate a decisive goal threat on an under-defended wing. It was a bit embarrassing for a brute-force searcher to miss a two-move forced goal against itself, for which there would have been a sound defense, but the branching factor of Arimaa is such that four-ply search depth is impossible even at slow time controls. In Syed's second game, his double-trap attack worked better, resulting in a 24-move rampage. In his third game Syed gave Bomb a Cat handicap in addition to the material sacrifice necessary to lure Bomb out of position so that Syed's double-trap attack could strike home once again.

Juhnke won his three games at handicaps of a Dog, a Horse, and a Camel respectively. The latter two games were tense and complicated, as Bomb uncharacteristically launched a Camel attack in both games, but the slow time control helped Juhnke sort through the tactical complications and gradually overcome his initial material disadvantage. Bomb's loss despite being given a Camel handicap is a humiliation particularly since it didn't hinge on any easily-identifiable evaluation bugs; on the contrary Bomb was outplayed in a tactically complex middlegame, which ought to be a computer's strong suit.

Brendan won his first game in dominating fashion, without losing a piece or allowing any counter-play whatsoever. This win by the nineteenth-ranked human illustrated how deeply strategic knowledge has penetrated the Arimaa community, as Brendan dispensed with tactical tricks, and instead gradually and inexorably tied down Bomb's pieces and took control of the entire board. In Brendan's third game there were no captures until move twenty, when Brendan blundered a Horse. However, a few moves later Brendan set up a dual threat of goal near one trap while threatening Horse capture near another. Bomb, instead of cutting its losses and returning the Horse for a playable position, succumbed to the horizon effect. Bomb accepted a string of smaller losses trying to save the Horse, and ultimately lost the Horse anyway, for a busted position which Brendan converted on move 39.

Bomb's one victory in the Challenge came when Naveed Siddiqui, the 27th-ranked human, had to substitute for Brendan's second game. Bomb shut down Siddiqui's attack, and launched a fierce counter-attack which first won material, and then generated a goal threat. Bomb wrapped up the game with two pretty trick moves. On move 28 Bomb offered an Elephant sacrifice, which Siddiqui wisely declined to defend goal. However, on move 30 when Bomb offered to let its Elephant be blockaded, Siddiqui couldn't resist, and succumbed to a two-move forced goal.

2008

As with 2006 and 2007, an unimproved version of Bomb once again won the Computer Championship. The Computer Championship runner-up, Sharp, participated alongside Bomb in a screening process similar to that of the year before. In the end, Bomb handily won the screening and was chosen once again to face humanity.

Of course, despite Bomb not having improved since 2005, hardware speed continued to march forward. Unfortunately, this slow march was not fast enough to keep pace with the new strategic understanding gained by humans over this time. Thus, the margin between humanity and silicon continued to increase year after year.

By now humanity had mastered all the tricks to destroy Bomb, and the challenge was unsurprisingly a rout, with all three series being swept 3–0 in favour of humanity. The defenders selected were #2 human Jean Daligault, who was the winner of the 2007 World Championship, #3 Greg Magne, and #20 Mark Mistretta.

Magne destroyed Bomb in the first and second games, with the second one being characterized by some as a "whipping", Bomb having lost all but an elephant, horse, and cat within the first 51 moves. Magne also easily won the third game, despite an unusual setup position with a decentralized elephant and poor rabbit placement.

Mistretta achieved a long but definite win against Bomb in his first game. Omar Syed, then #27, substituted for Mistretta's second and third games. In the second game, Syed played an extremely sharp opening and rushed for a goal attack, paying little regard to material. Despite this, Syed's goal attack succeeded and Bomb fell by move 29. In the third game, Syed opted for a more passive opening, and Bomb seemed to have the upper hand briefly. Syed pressed on, however, and eventually won this game too.

Daligault won his first game in convincing fashion. In the second game, Bomb outplayed Daligault in the opening, but the advantage quickly dissipated and Daligault won on move 47. The third game, also a convincing win for Daligault, was the last game of the challenge. Humanity once again dominated silicon, this time sweeping 9–0.

External Links

Up-to-date Challenge information