In order to generate these results, we gave each agent an opportunity to negotiate with each opponent (including in self-play) in each domain, and using both profiles. The domains included all those submitted by the participants, plus the Travel domain from ANAC2010, totalling 18 domains. Each negotiation was repeated 10 times, in order to produce statistically significant results. In total, this resulted in 52020 negotiations.";
echo "To calculate the rank of each agent, we used Welch's t test to check for statistically significant differences between the agents' scores. In more detail, we computed this for each pair of agents, to determine, with 95% confidence, which agents beat a specific agent, and which agents are beaten by that agent. For example, an agent which is beaten by two agents, and beats 12 agents (out of a total of 17) is considered to have rank 3-5. That is, with 95% certainty, its true rank lies between 3 and 5 (inclusive).
";
echo "To select the qualifying agents, we identified all agents which have a possibility of a true rank in the top eight positions. Nine agents matched this criteria. The developers of IAMcrazyHaggler2012 (rank 7-9) chose to withdraw their entry from the final, leaving eight qualifying agents, highlighted in green in the table below.
";
echo "By selecting an agent or domain from the tables below, you can view the results in more detail. By navigating through these tables, you can select an agent, an opponent and a domain in order to view the scores achieved in individual negotiations. By clicking the 'show details' link next to a domain, you can view an illustration of the full outcome space and other domain characteristics. All of the outcomes from the qualifying round are also available to download as a CSV file.
";
echo "In the domain tables, the discounting factors (DF) and reservation values (RV) that were used in the qualifying round are shown. Note that for many of the domains which were not submitted with a discounting factor or reservation value, we have added these parameters.
";
}
function finalText()
{
echo "In order to generate these results, we gave each agent an opportunity to negotiate with each opponent (excluding in self-play) in each domain, and using both profiles. The domains included all 17 that were submitted by the participants this year, plus 5 from ANAC2011 and 2 from ANAC2010. Three variants of each of these 24 domains (with different discounting factors and reservation values) were used, totalling 72 domains. Each negotiation was repeated 10 times, in order to produce statistically significant results. In total, this resulted in 40320 negotiations.
";
echo "To calculate the rank of each agent, we used Welch's t test to check for statistically significant differences between the agents' scores. In more detail, we computed this for each pair of agents, to determine, with 95% confidence, which agents beat a specific agent, and which agents are beaten by that agent. For example, an agent which is beaten by two agents, and beats 12 agents (out of a total of 17) is considered to have rank 3-5. That is, with 95% certainty, its true rank lies between 3 and 5 (inclusive).
";
echo "By selecting an agent or domain from the tables below, you can view the results in more detail. By navigating through these tables, you can select an agent, an opponent and a domain in order to view the scores achieved in individual negotiations. By clicking the 'show details' link next to a domain, you can view an illustration of the full outcome space and other domain characteristics. All of the outcomes from the final round are also available to download as a CSV file (note that this file also includes self-play outcomes, but these are not included in the results reported on this site).
";
echo "In the domain tables, the discounting factors (DF) and reservation values (RV) that were used are shown.
";
}
if(!isset($_GET['v']))
{
echo "Presentation
";
echo "In the ANAC2012 Session of the ACAN2012 Workshop, the competition and its results were presented. The presentation slides are available in .pptx or .pdf format.
";
echo "Awards
";
echo "The following awards were presented:
";
echo "";
echo "| Award | Agent | University | Team Members |
|---|
";
echo "| 1st Place | CUHKAgent | The Chinese University of Hong Kong | Jianye Hao, Ho-fung Leung |
";
echo "| 2nd Place | AgentLG | Bar-Ilan University | Luba Golosman |
";
echo "| 3rd Place | OMACagent | Maastricht University | Siqi Chen, Gerhard Weiss |
";
echo "| 3rd Place | TheNegotiator Reloaded | Delft University of Technology | Mark Hendrikx, Alex Dirkzwager |
";
echo "| Best in Discounted Domains | CUHKAgent | The Chinese University of Hong Kong | Jianye Hao, Ho-fung Leung |
";
echo "| Best in Undiscounted Domains | TheNegotiator Reloaded | Delft University of Technology | Mark Hendrikx, Alex Dirkzwager |
";
echo "| Most Social Agent | IAMhaggler2012 | University of Southampton | Colin R. Williams, Valentin Robu,
Enrico H. Gerding, Nicholas R. Jennings |
";
echo "
";
echo "Photos
";
echo "";
echo "
";
echo "
";
echo "
";
echo "
";
echo "
";
echo "
";
echo "
Photographer: Tim Baarslag
";
echo "
Downloads
";
echo "Domain files and agent code (.zip)
";
echo "Detailed Results
";
echo "See the results of each round in more detail:
";
echo "";
}
else if($_GET['v'] == "qualifying" || $_GET['v'] == "final" || (isInternal() && ($_GET['v'] == "test" || $_GET['v'] == "extended" || $_GET['v'] == "risk" || substr($_GET['v'], 0, 5) == "spite" || $_GET['v'] == "thesis")))
{
if($_GET['format'] != 'tex')
{
echo "";
if(!is_null($domain)) {
if(!is_null($agentA)) {
if(!is_null($agentB)) {
echo "
";
echo "| Selected domain: | ".renderDomain($domain)." | Show all domains | ".renderDomainInfoLink($domain)." |
";
echo "| Selected agent: | ".renderAgent($agentA)." | |
";
echo "| Selected opponent: | ".renderAgent($agentB)." | Show all opponents |
";
echo "
";
output(listAll($domain, $agentA, $agentB, 0), "Profile 1");
output(listAll($domain, $agentA, $agentB, 1), "Profile 2");
} else {
echo "
";
echo "| Selected domain: | ".renderDomain($domain)." | Show all domains | ".renderDomainInfoLink($domain)." |
";
echo "| Selected agent: | ".renderAgent($agentA)." | Show all agents |
";
echo "
";
output(listByOpponent($agentA, $domain));
}
} else {
echo "
";
echo "| Selected domain: | ".renderDomain($domain)." | Show all domains | ".renderDomainInfoLink($domain)." |
";
echo "
";
output(listByAgent($domain));
}
//echo "
![](http://anac2012.ecs.soton.ac.uk/domains/2010/12".".png)
";
} else {
if(!is_null($agentA)) {
if(!is_null($agentB)) {
echo "
";
echo "| Selected agent: | ".renderAgent($agentA)." | |
";
echo "| Selected opponent: | ".renderAgent($agentB)." | Show all opponents |
";
echo "
";
output(listByDomain($agentA, $agentB));
} else {
echo "
";
output(listByOpponent($agentA));
output(listByDomain($agentA));
}
} else {
if($_GET['v'] == 'qualifying') {
qualifyingText();
} else {
finalText();
}
output(listByAgent());
output(listByDomain());
}
}
echo "
";
if($title != "")
{
echo "
$title
";
}
echo "
";
echo "";
if(isset($n)) echo "| Rank | ";
foreach($keys as $key)
{
if(is_int($key)) continue;
if($key[0] == '_') continue;
echo "";
if($key == 'DF')
echo "DF";
elseif($key == 'RV')
echo "RV";
elseif($key == 'Agent_Mean')
echo "Mean Score";
elseif($key == 'Agent_Var')
echo "Variance";
//echo "Standard Deviation";
//echo "Standard Error";
else
echo str_replace('_', ' ', $key);
echo " | ";
if($key == "Domain") echo " | ";
}
echo "
";
foreach($table as $row)
{
if(isset($row['Agent']) && qualified($row['Agent']))
{
echo "";
}
else
{
echo "
";
}
if(isset($n)) {
if($above[$row[0]]+1 == $n-$below[$row[0]])
echo "| ".($above[$row[0]]+1)." | ";
else
echo " ".($above[$row[0]]+1)."-".($n-$below[$row[0]])." | ";
}
foreach($keys as $key)
{
if(is_int($key)) continue;
if($key[0] == '_') continue;
echo "";
echo renderCell($row[$key], $key);
echo " | ";
}
echo "
";
}
echo "
";
echo "
".renderDomainInfoLink($s);
case "Agent":
case "Opponent":
return "".renderAgent($s)."";
default:
if(is_numeric($s)) {
if(round($s) == $s)
return $s;
if($s > 0.01)
return sprintf('%.3f', round($s, 3));
else
return sprintf('%.6f', round($s, 6));
//return "X".sprintf('%.10f', 1.96*sqrt(round($s, 10))/sqrt(10))."X";
}
return $s;
}
}
function renderDomain($s) {
$q = 'SELECT name FROM domain WHERE code = ?';
$d = query($q, array($s));
if(count($d) == 0) return "[$s]";
return $d[0]['name'];
}
function renderDomainTex($s) {
if(strlen($s) == 1)
{
$q = 'SELECT name FROM domain WHERE SUBSTR(code, 2, 1) = ? AND owner IS NULL';
}
else
{
$q = 'SELECT name, df, rv FROM domain WHERE code = ?';
}
$d = query($q, array($s));
if(count($d) == 0) return "[$s]";
if(strlen($s) == 1)
{
return $d[0]['name'].' Scenario (averaged over $\delta \in \{0.50, 0.75, 1.00\}, u_{\overline{\alpha}} \in \{0.00, 0.25, 0.50\}$)';
}
else
{
return $d[0]['name'].' Scenario (with $\delta='.sprintf('%.2f', $d[0]['df']).', u_{\overline{\alpha}}='.sprintf('%.2f', $d[0]['rv']).'$)';
}
}
function renderDomainInfoLink($s, $search='code') {
$q = 'SELECT year, id, link FROM domain WHERE '.$search.' = ?';
$d = query($q, array($s));
if(count($d) == 0) return "";
if($d[0]['year'] == "")
{
if($d[0]['link'] > 0)
{
return renderDomainInfoLink($d[0]['link'], 'id');
}
return "";
}
return "Show details";
}
function renderAgent($s, $includeInstitution=true) {
$s = modifyClassname($s);
$q = 'SELECT agentname, institution FROM users WHERE agentclass = ?';
$d = query($q, array($s));
if(count($d) == 0) return "{{$s}}";
if($includeInstitution)
{
return $d[0]['agentname'].', '.$d[0]['institution'];
}
else
{
return $d[0]['agentname'];
}
}
function qualified($s) {
$s = modifyClassname($s);
$q = 'SELECT qualified FROM users WHERE agentclass = ?';
$d = query($q, array($s));
if(count($d) == 0) return false;
return ($d[0]['qualified'] == 1);
}
function modifyClassname($s) {
if($s == "thenegotiatorreloaded.TheNegotiatorReloaded") return "travelingsalesman.TravelingSalesman";
if($s == "agents.OMAC_sp2012b.OMAC_sp2012b") return "agents.OMAC_sp2012.OMAC_sp2012";
if($s == "agents2012.southampton.IAMhaggler2012a0") return "agents2012.southampton.IAMhaggler2012";
return $s;
}
function rlink($s, $key) {
global $domain;
global $agentA;
global $agentB;
$lparts = array($domain, $agentA, $agentB);
if($lparts[0] == null)
$lparts[0] = "*";
switch($key) {
case "Domain":
$lparts[0] = $s; break;
case "Agent":
$lparts[1] = $s; break;
case "Opponent":
$lparts[2] = $s; break;
}
while(count($lparts) > 0 && $lparts[count($lparts)-1] == "") array_pop($lparts);
if(count($lparts) == 1 && $lparts[0] == "*") return "/results/".$_GET['v'].'/';
return "/results/".$_GET['v'].'/'.implode('/', $lparts);
}
function listByDomain($agentA = null, $agentB = null) {
global $cols;
global $table;
global $batchID;
$where = array();
$params = array();
if(!is_null($agentA)) {
$where[] = "agentA = ?";
$params[] = $agentA;
if(!is_null($agentB)) {
$where[] = "agentB = ?";
$params[] = $agentB;
}
}
$where = getWhere($where);
$q = "SELECT domain as Domain, $cols FROM $table $where GROUP BY domain, $batchID";
$q = average($q, 'Domain', ', DF, RV', ' JOIN domain d ON d.code = x.domain');
return query($q, $params);
}
function listByAgent($domain = null) {
global $cols;
global $table;
global $batchID;
$where = array();
$params = array();
if(!is_null($domain)) {
if(strlen($domain) == 1)
{
$where[] = "SUBSTR(domain, 2, 1) = ?";
}
else
{
$where[] = "domain = ?";
}
$params[] = $domain;
}
$where = getWhere($where);
$q = "SELECT agentA as Agent, $cols FROM $table $where GROUP BY agentA, $batchID";
$q = average($q, 'Agent');
return query($q, $params);
}
function listByOpponent($agent, $domain = null) {
global $cols;
global $table;
global $batchID;
$where = array();
$params = array();
$where[] = "agentA = ?";
$params[] = $agent;
if(!is_null($domain)) {
$where[] = "domain = ?";
$params[] = $domain;
}
$where = getWhere($where);
$q = "SELECT agentB as Opponent, $cols FROM $table $where GROUP BY agentB, $batchID";
$q = average($q, 'Opponent');
return query($q, $params);
}
function listAll($domain, $agentA, $agentB, $side) {
global $cols;
global $table;
$fullcols = preg_replace('/avg\(([a-zA-Z]+)\)/', '\1', $cols);
$fullcols = str_replace(', count(*) as _Count', ', SUBSTRING(id, 5, 2) as repetition', $fullcols);
$where = array();
$params = array();
$where[] = "agentA = ?";
$params[] = $agentA;
$where[] = "agentB = ?";
$params[] = $agentB;
$where[] = "domain = ?";
$params[] = $domain;
$where[] = "n = ?";
$params[] = $side;
$where = getWhere($where);
$q = "SELECT $fullcols FROM $table $where ORDER BY SUBSTRING(id, 5, 2)";
$params = array($agentA, $agentB, $domain, $side);
return query($q, $params);
}
function getWhere($where) {
global $excludeSelfPlay;
if($excludeSelfPlay)
{
$where[] = "agentA != agentB";
}
if(count($where) > 0)
{
return "WHERE ".implode(" AND ", $where);
}
else
{
return "";
}
}
/*
function outputDomainChooser($domain) {
echo "\n";
echo domainLink('(all)', $domain == null || $domain == '*');
foreach(array('C1', 'C2', 'C3', 'C4', 'C5', 'C6', 'C7', 'C8') as $d) {
echo domainLink($d, $domain == $d);
}
echo " \n";
}
function outputAgentAChooser($agentA) {
echo "\n";
echo domainLink('(all)', $agentA == null || $agentA == '*');
foreach(array('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H') as $a) {
echo domainLink($a, $agentA == $a);
}
echo " \n";
}
function outputAgentBChooser($agentB) {
echo "\n";
echo domainLink('(all)', $agentB == null || $agentB == '*');
foreach(array('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H') as $a) {
echo domainLink($a, $agentB == $a);
}
echo " \n";
}
function domainLink($str, $current) {
if($current) {
echo ''.$str.'';
} else {
echo ''.$str.'';
}
}
*/
?>
| 
";
echo "
";
echo "
";
echo "
";
echo "
";
echo "
";
echo "