UCI tournaments

Total results from view of Pirat:

Tournament mode: 40 moves for 20 minutes + 20 moves for 10 minutes + 5 minutes for the remainder of the game; no pondering + no opening libraries

We are using an opening-book which you may find on the page elo-ranking as well for download. The data base consists of 100 openings (inclusive the Nunn libraries I and II) and takes the ECO keys off A00 to E99. Per tournament 200 games (in each case 100 games with white and black) were thus played. A games lasts thus maximally 1 Std. 10 min.

Engine-Version 0.4.6

It took us more than one year to release the new engine version. All evaluation-function which where applied to the root-node of the analysys-tree only were removed and partly integrated in the dynamic evaluation function. Furthermore some evalutation in the positional area were improved significantly.

According to our tournament rules Queen got removed, as the last result from Pirat point of view was more than 70%.

Pirat achieved the following results:

	Version		Leila	Anmon	Pepito	Anmon	Zappa	Aristarch
			0.53h	5.21	1.59	5.60	1.1	4.50
		ELO	2453	2524	2568		2713	2725
05/03	0.2.6	2238	-	16.50%	-	-	-	-
07/03	0.2.7	2246	-	19.25%	-	-	-	-
08/03	0.2.8	2344	-	23.25%	-	-	-	-
09/03	0.2.9	2357	-	25.25%	-	-	-	-
11/03	0.3.0	2363	-	25.50%	-	-	-	-
01/04	0.3.1	2405	41.00%	28.50%	-	-	-	-
07/04	0.3.2	2415	43.25%	40.50%	-	-	-	-
03/05	0.3.3	2442	47.75%	38.25%	29.00%	-	-	-
04/06	0.4.2	2506	62.00%	48.75%	42.75%	-	-	20.75%
09/06	0.4.3	2558	63.00%	48.05%	49.75%	-	33.00%	28.25%
10/07	0.4.6	2630	77.50%	63.75%	61.75%	49.75%	34.00%	36.50%

Against Leila Pirat could improve by 14.5%, against Anmon 5.21 15.75% and against Pepito remarkable 12%.

Against Zappa Pirat could add 1% only. The stronger an engine is, the more difficult it is sometimes to improve percentage-wise. Two runs of 50 games Pirat was able to achieve 42% and 41%, but the runs on the other two computers were almost catastrophic (23% and 30%).

Pirat could improve against the strong engine Aristarch. In 1.5 years remarkable 15.75% and still 8.25% in the last run. Also the run against Aristarch did show large fluctuations. In the first 50 games Pirate won only 22% but in games 51-100 Pirat achieved 47%. Surprisingly we see a strong result of 47% in an area where Pirat had a weak run of 23% against Zappa.

Overall, the two engines Aristarch Zappa uncovered a big weakness of Pirat, namely the king's attack. In many games, where the King of Pirat is under attack, the evaluation of Zappa und Aristarch on one side and Pirat on the other side did show a difference of up to three pawns.

This problem will be corrected in one of the next engine versions. As all static evaluation function have been removed, Pirat will also be able to reuse the analysys-result of the last move.

As the gap between Pepito and Aristarch is too big, we looked for an UCI-Engine , which is about the playing strength of Pirat and thus also closes the gap between Pepito and Aristarch. We used the WEBC-Ratinglist as input. We selected the latest version of Anmon (Anmon5.60). And actually Pirat scored 49.75%, almost 50%.

Game download: Leila, Anmon, Pepito, Anmon56, Zappa, Aristarch

Test-equipment:

A tournament against an UCI-engine is distributed over four PC's and therefore is completed after 2 days. All games are played using 128 MB hashtables and 3,4,5-men Tablebases.

Games 1- 50: AMD Athlon, 3,8 GHz.
Games 51-100: Intel Pentium 4, 3,8 GHz.
Games 101-150: AMD Athlon, 3,2 GHz .
Games 151-200: Intel Pentium 4, 2,0 GHz.

Engine-Version 0.4.3

Our declared goal that the next version will beat Pepito is missed by 0,5%, nevertheless we decided to make a release. Since the Engine-version 0.4.2. achieved more than 70% in the last round against Hagrid and Knightx, both engine were replaced by the much stronger engine Zappa 1.1.

Pirat achieved the following results:

	Version		Queen	Leila	Anmon	Pepito	Zappa	Aristarch
			2.38	0.53h	5.21	1.59	1.1	4.50
		ELO	2416	2453	2524	2568	2713	2725
05/03	0.2.6	2238	-	-	16,50%	-	-	-
07/03	0.2.7	2270	27,50%	-	19,25%	-	-	-
08/03	0.2.8	2344	38,00%	-	23,25%	-	-	-
09/03	0.2.9	2357	40,50%	-	25,25%	-	-	-
11/03	0.3.0	2363	42,50%	-	25,50%	-	-	-
01/04	0.3.1	2405	51,25%	41.00%	28,50%	-	-	-
07/04	0.3.2	2415	53,75%	43,25%	40,50%	-	-	-
03/05	0.3.3	2442	54,50%	47,75%	38,25%	29,00%	-	-
04/06	0.4.2	2506	59,75%	62,00%	48,75%	42,75%	-	20,75%
09/06	0.4.3	2558	70,75%	63,00%	48,00%	49,75%	33,00%	28,25%

As Anmon is a positional strong engine and this release is unchanged from positional point of view the result against Anmon might be explainable.

Game download: Queen, Leila, Anmon, Pepito, Zappa, Aristarch

Test-equipment:

A tournament against an UCI-engine is distributed over four PC's and therefore is completed after 2 days. All games are played using 128 MB hashtables and 3,4,5-men Tablebases.

Games 1- 50: AMD Athlon, 3,2 GHz.
Games 51-100: Intel Pentium 4, 2,8 GHz.
Games 101-150: AMD Athlon, 3,2 GHz .
Games 151-200: Intel Pentium 4, 2,0 GHz.

Engine-Version 0.4.2

For the new version more time was needed than expected. Target was to tie with Anmon5.21 which was met by achieving 48,75%. The engine does have an improved endgame-knowledge, better but still improveable king-safety, updated time-management and better handling of passed pawns.

As the result of Pirat 0.3.3 against Madeleine 0.2 and Amateur was better than 70%, those engines got removed from the benchmark.

Pirat achieved the following results:

	Version		Hagrid	KnightX	Queen	Leila	Anmon	Pepito	Aristarch
			0.7.56	177	2.38	0.53h	5.21	1.59	4.50
		ELO	2317	2340	2416	2453	2524	2568	2725
05/03	0.2.6	2238	43.50%	39.75%	-	-	16.50%	-	-
07/03	0.2.7	2270	46.25%	45.75%	27.50%	-	19.25%	-	-
08/03	0.2.8	2344	60.00%	45.75%	38.00%	-	23.25%	-	-
09/03	0.2.9	2357	56.00%	47.25%	40.50%	-	25.25%	-	-
11/03	0.3.0	2363	54.80%	55.50%	42.50%	-	25.50%	-	-
01/04	0.3.1	2405	60.50%	-	51.25%	41.00%	28.50%	-	-
07/04	0.3.2	2415	58.75%	61.00%	53.75%	43.25%	40.50%	-	-
03/05	0.3.3	2442	64.50%	63.25%	54.50%	47.75%	38.25%	29.00%	-
04/06	0.4.2	2506	72.25%	72.25%	59.75%	62.00%	48.75%	42.75%	20.75%

The next Pirat-version will have some algorithm added discussed in the diploma thesis by Juergen. New target is to tie with Pepito within the next six month. Ruffian was replaced by the slightly stronger UCI-Engine Aristarch4.50, which holds place 21 on CSS-Rating List and which has 101 Elo-points more than Pepito1.59.

Download games: Hagrid, KnightX, Queen, Leila, Anmon, Pepito, Aristarch

Test-equipment:

A tournament against an UCI-engine is distributed over four PC's and therefore is completed after 2 days. All games are played using 128 MB hashtables and 3,4,5-men Tablebases.

Games 1- 50: AMD Athlon, 3,2 GHz.
Games 51-100: Intel Pentium 4, 2,8 GHz.
Games 101-150: AMD Athlon, 3,2 GHz .
Games 151-200: Intel Pentium 4, 2,0 GHz.

Engine-Version 0.3.3

The new engine took more time than anticipated. We realized very late that out teststrategy was no longer sufficient. So far we analyzed lost games and implemented and verified a fix. After implementation of all fixes we rerun our test. This did lead to an improvement of the engine in the past. Now we test a new feature by running 100's of games before adding it to the next release version. This we have to do in order to avoid that a fix solves a problem but creates another one in a won game.

The tournament ended with the following results:

	Version		Madeleine	Hagrid	Amateur	KnightX	Queen	Leila	Anmon	Pepito	Ruffian
			0.2	0.7.56	2.4	177	2.38	0.53h	5.21	1.59	1.0.1
		ELO	2295	2317	2303	2340	2416	2453	2524	2568	2711
05/03	0.2.6	2238	40.00%	43.50%	-	39.75%	-	-	16.50%	-	-
07/03	0.2.7	2270	50.00%	46.25%	46.25%	45.75%	27.50%	-	19.25%	-	-
08/03	0.2.8	2344	58.50%	60,00%	55.25%	45.75%	38.00%	-	23.25%	-	-
09/03	0.2.9	2357	55.25%	56.00%	56.00%	47.25%	40.50%	-	25.25%	-	11.50%
11/03	0.3.0	2363	63.00%	54.80%	56.50%	55.50%	42.50%	-	25.50%	-	-
01/04	0.3.1	2405	63.25%	60.50%	-	-	51.25%	41.00%	28.50%	-	14.50%
07/04	0.3.2	2415	65.50%	58.75%	63.75%	61.00%	53.75%	43.25%	40.50%	-	-
03/05	0.3.3	2442	72.25%	64.50%	75.75%	63.25%	54.50%	47.75%	38.25%	29.00%	-

Beside many improvements in the engine we implemented a missing feature in the GUI. Games lost on time are now stopped. After checking for theoretical draw , 3xrepetition of the final position or mate in one half-move or stale-mate the games is evaluated as win for the other engine. We were forced to implement this feature as some engines start after running out of time dont stay in the panic-mode but spend now significant amount of time for analysis.

Pirat lost for the first time some points against Anmon, however the last improvement of 12% was way over average (2%). We added the engine Pepito 1.59 with can be found in the CSS-Rating List between Anmon5.21 and Ruffian1.0.1.

The designer of pirat where able to improve the strength of their engine by 200 Elo point within less than 2 years.

Download games: Madeleine, Hagrid, Amateur, KnightX, Queen, Leila, Anmon, Pepito.

Test computer:

Engine	Rechner	*Hashtables*	Endspieldatenbanken
Madeleine 0.2	Pentium 2,0 GHz	128 MB	3+4-Steiner
Hagrid 0.7.56	Pentium 2,8 GHz	128 MB	3+4+5-Steiner
Amateur 2.4	Pentium M 1,5 GHz	64 MB	3+4-Steiner
KnightX 177	Pentium 2,0 GHz	128 MB	3+4-Steiner
Queen 2.38	Pentium 2,8 GHz	128 MB	3+4+5-Steiner
Leila 0.53h	Pentium M 1,5 GHz	64 MB	3+4-Steiner
Anmon 5.21	Pentium 2,8 GHz	128 MB	3+4+5-Steiner
Pepito 1.59	Pentium 2,0 GHz	128 MB	3+4-Steiner

Engine-Version 0.3.2

For relocation-reason the work on the engine was on hold from February to June 2004.

In June 2004 the work was resumed. At the end of July the next release was available, where Pirat achieved the following results:

	Version		Madeleine	Hagrid	Amateur	KnightX	Queen	Leila	Anmon	Ruffian
			0.2	0.7.56	2.4	177	2.38	0.53h	5.21	1.0.1
		ELO	2295	2317	2303	2340	2416	2453	2524	2711
05/03	0.2.6	2238	40.00%	43.50%	-	39.75%	-	-	16.50%	-
07/03	0.2.7	2270	50.00%	46.25%	46.25%	45.75%	27.50%	-	19.25%	-
08/03	0.2.8	2344	58.50%	60.00%	55.25%	45.75%	38.00%	-	23.25%	-
09/03	0.2.9	2357	55.25%	56.00%	56.00%	47.25%	40.50%	-	25.25%	11.50%
11/03	0.3.0	2363	63.00%	54.80%	56.50%	55.50%	42.50%	-	25.50%	-
01/04	0.3.1	2405	63.25%	60.50%	-	-	51.25%	41.00%	28.50%	14.50%
07/04	0.3.2	2415	65.50%	58.75%	63.75%	61.00%	53.75%	43.25%	40.50%	-

Pirat could add nearly everywhere. Again Hagrid was not mainstream. Here Pirat lost 1.75%. However a nice result could be obtained against Anmon. Pirat improved around 12% and is here nearly in break-even distance!

Download games: Madeleine, Hagrid, Amateur, KnightX, Queen, Leila, Anmon.

Test computer:

Engine	*Computer*	*Hashtables*	*men table*
Madeleine 0.2	Athlon 500MHz	32 MB	3+4+5-men table
Hagrid 0.7.56	Athlon 500MHz	32 MB	3+4+5-men table
Amateur 2.4	Pentium M 1,5 GHz	64 MB	3+4-men table
KnightX	Athlon 500 MHz	32 MB	3+4+5-men table
Queen 2.38	Celeron 900 MHz	32 MB	3+4-men table
Leila 0.53h	Celeron 900 MHz	32 MB	3+4-men table
Anmon 5.21	Pentium 2,8 GHz	128 MB	3+4+5-men table

Engine-Version 0.3.1

Again two months later we released Pirat-engine 0.3.1:

	Version		Madeleine	Hagrid	Amateur	KnightX	Queen	Leila	Anmon	Ruffian
			0.2	0.7.56	2.4	177	2.38	0.53h	5.21	1.0.1
		ELO	2295	2317	2303	2340	2416	2453	2524	2711
05/03	0.2.6	2238	40.00%	43.50%	-	39.75%	-	-	16.50%	-
07/03	0.2.7	2270	50.00%	46.25%	46.25%	45.75%	27.50%	-	19.25%	-
08/03	0.2.8	2344	58.50%	60.00%	55.25%	45.75%	38.00%	-	23.25%	-
09/03	0.2.9	2357	55.25%	56.00%	56.00%	47.25%	40.50%	-	25.25%	11.50%
11/03	0.3.0	2363	63.00%	54.80%	56.50%	55.50%	42.50%	-	25.50%	-
01/04	0.3.1	2405	63.25%	60.50%	-	-	51.25%	41.00%	28.50%	14.50%

Finally Pirat could add against Hagrid as well. The test run against Amateur and KnightX could not be accomplished, since the test platform was not available. After one had defeated for the first time Queen with 51,25%, we decided add another engine with the play strength between Queen and Anmon. The choice was Leila, the Italian master. With 41% the result was exactly between Queen and Anmon.

Download games: Madeleine, Hagrid, Queen, Leila, Anmon, Ruffian.

Engine-Version 0.3.0

	Version		Madeleine	Hagrid	Amateur	KnightX	Queen	Anmon	Ruffian
			0.2	0.7.56	2.4	177	2.38	5.21	1.0.1
		ELO	2295	2317	2303	2340	2416	2524	2711
05/03	0.2.6	2238	40.00%	43.50%	-	39.75%	-	16.50%	-
07/03	0.2.7	2270	50.00%	46.25%	46.25%	45.75%	27.50%	19.25%	-
08/03	0.2.8	2344	58.50%	60.00%	55.25%	45.75%	38.00%	23.25%	-
09/03	0.2.9	2357	55.25%	56.00%	56.00%	47.25%	40.50%	25.25%	11.50%
11/03	0.3.0	2363	63.00%	54.80%	56.50%	55.50%	42.50%	25.50%	-

Everywhere pirat could add except Hagrid which did better again. Hagrid proved to be an unpleasant engine, which is positional weak, however tactically strong. Thus again 1.2 % were lost. However Pirat added properly against Madeleine and KnightX. The test run against Ruffian was not accomplished, since the difference was simply still too large. Altogether the pirat-engine obtained a result which was ok from our point of view.

Download games: Madeleine, Hagrid, Amateur, KnightX, Queen, Anmon.

Engine-Version 0.2.9

Again one month later the Pirat-engine 0.2.9 was released:

	Version		Madeleine	Hagrid	Amateur	KnightX	Queen	Anmon	Ruffian
			0.2	0.7.56	2.4	177	2.38	5.21	1.0.1
		ELO	2295	2317	2303	2340	2416	2524	2711
05/03	0.2.6	2238	40.00%	43.50%	-	39.75%	-	16.50%	-
07/03	0.2.7	2270	50.00%	46.25%	46.25%	45.75%	27.50%	19.25%	-
08/03	0.2.8	2344	58.50%	60.00%	55.25%	45.75%	38.00%	23.25%	-
09/03	0.2.9	2357	55.25%	56.00%	56.00%	47.25%	40.50%	25.25%	11.50%

Although Pirat added against the stronger Engines, Pirat lost 3.25% against Madeleine and 4% against Hagrid. Nevertheless Pirat improved to 2357. Since Ruffian had approximately the play strength of Fritz6, Ruffian was added, in order to get a first impression about the difference to the strongest engines.

Download games: Madeleine, Hagrid, Amateur, KnightX, Queen, Anmon, Ruffian.

Engine-Version 0.2.8

Already in August 2003 the Pirat-engine 0.2.8 was released:

	Version		Madeleine	Hagrid	Amateur	KnightX	Queen	Anmon
			0.2	0.7.56	2.4	177	2.38	5.21
		ELO	2295	2317	2303	2340	2416	2524
05/03	0.2.6	2238	40.00%	43.50%	-	39.75%	-	16.50%
07/03	0.2.7	2270	50.00%	46.25%	46.25%	45.75%	27.50%	19.25%
08/03	0.2.8	2344	58.50%	60.00%	55.25%	45.75%	38.00%	23.25%

Pirat added properly and obtained 2344 Elo points.

Download games: Madeleine, Hagrid, Amateur, KnightX, Queen, Anmon.

Engine-Version 0.2.7

Many, many errors were found. After the most obvious bugs were eliminated, in June/July 2003 the next run was started:

	Version		Fauce	EngineMax	BigLion	Madeleine	Hagrid	Amateur	KnightX	Queen	Anmon
			0.41c	5.11c	-	0.2	0.7.56	2.4	177	2.38	5.21
		ELO	1924	2014	2079	2295	2317	2303	2340	2416	2524
05/03	0.2.6	2238	86.25%	80.00%	68.50%	40.00%	43.50%	-	39.75%	-	16.50%
08/03	0.2.7	2270	87.75%	80.00%	78.00%	50.00%	46.25%	46.25%	45.75%	27.50%	19.25%

Since the difference between KnightX and Anmon was too large, Amateur and Queen were added to the tournament. We decided to remove all Engines, against pirat got more than 70%.

Download games: Fauce, EngineMax, BigLion, Madeleine, Hagrid, Amateur, KnightX, Queen, Anmon.

Engine-Version 0.2.6

In April/May 2003 the following Engines was tested against Pirat 0.2.6:

	Version		Fauce	EngineMax	BigLion	Madeleine	Hagrid	KnightX	Anmon
			0.41c	5.11c	-	0.2	0.7.56	177	5.21
		ELO	1924	2014	2079	2295	2317	2340	2524
05/03	0.2.6	2238	86.25%	80.00%	68.50%	40.00%	43.50%	39.75%	16.50%

Pirat reached an ELO of 2238.

Download games: Fauce, EngineMax, BigLion, Madeleine, Hagrid, KnightX, Anmon.

Introduction

When the first version of Pirat was released to our beta-tester our problem was that we were not able to estimate the strength of our engine. First tests were however very encouraging.

From time to time it happened that Pirat analyzed scrap. We were thinking about how we could identify the strength and fix bugs. One beta tester therefore asked for the support of the Autoplayer-protocoll.

With the help of the Autoplayer-protocol now first tests were accomplished against other Engines, which were partly very encouraging. In its first game against Fritz6 Pirat could win. Also Crafty with 4:2 was struck. And it was very fast stated nevertheless that the results were not meaningful. As soon as Pirat lost the first game, the other program played the same variant over again and again. Since Pirat currently misses a (book-) learning function, a game was lost after the other one. It must probably hardly be mentioned that the games were often identical. Also we noticed fast that the opening knowledge in form of an opening library played a not insignificant role. Some errors in the pirat-engine were found, however we noticed that we tested often only the learning function of the other engine and its opening repertoire. We than realized that the autoplayer-protocoll was not the tool to improve the strength of the pirat-engine. .

Here now UCI definition came into the discussion. After one stated the fact that UCI standard allowed to suppress usage of opening books, allows to use same opening book for all engines the UCI-standard was implemented..

Since April 2003 we test Pirat around the clock against other Engines (no pondering + no opening libraries). The first Elo number, which was calculated for Pirat, was disappointing, however we finally found a good way to test Engines under same conditions. Thus it was not a surprise that within 24 month Pirat gained more than 200 ELO-points.

[Home]

UCI tournaments

Total results from view of Pirat:

Introduction

Senden Sie E-Mail mit Fragen oder Kommentaren zu dieser Website an: webmaster@go-pirat.de

In case you have any question or comments please do not hesitate to send us an email via webmaster@go-pirat.de

Last update: 03. März 2010