(July 18th, 2020, 20:28)Borsche Wrote: There's also the issue of just how many low peaceweight leaders there are in the game.
This. Very much.
The AIs are of course balanced around the presence of a human player.
High peace weight leaders are supposed to be the player's natural "allies", while low peace weights are the "enemies".
So it stands to reason there should be more "enemies", with the player balancing things out.
Remove the player... and the balance tilts towards the low peace weights.
Which is an issue if we're trying to determine the "best" AI.
Less of an issue if we're looking for the "highest threat" AI.
Hatty played a pretty strong game, fending off both Napoleon and Shaka, and she was in a position to conquer either of them. She just threw her game away by suiciding against the guy who'd been able to grow even stronger while she shielded him.
And was it last year, or the year before, that she held off a 2v1 then 3v1 for ages ?
Not too crummy IMO.
But as a player, if you draw Hatty as a neighbour, you're gonna be pretty happy about it.
So not a "bad" AI, but a low threat one : her position at the bottom of the list then makes perfect sense.
Another thing to take into account is that kills are probably not the greatest indicator of success. Of course all the low peaceweight + high unit building leaders are gonna get more kills thus more points. I mean, I know guys like Monty and Temujin have been struggling for points even with these factors, but the civs that go to war are going to get kills. Even with the random nature of kills set aside (like how Willem was almost lucky to get both his kills in the last game - he was definitely behind Washington in the turn order for the Stalin kill and could've easily lost out on a point) kills might weight somewhat heavily into these rankings. You don't get any points for surviving, but you could also take 20 cities from an AI and snowball into a win only to get 5 points instead of 6 because some bottom feeder sniped the last city.
It honestly might be better to give some points for survival. It might seem counter intuitive, but its all too easy for bad AIs to play a bad game and then die. That's why I like the idea of a 'killed' counter for each AI, since the amount of times you get outright killed seems to be far more indicitive of success than if you're able to claim the last city of someone else. Not that its necessarily counted as negative points or anything, but it definitely serves to display which AIs are consistent (like Kublai or Huyana), which ones are boom or bust (Gandhi, and evidently Willem), and which ones just get killed instantly (Bismark, Sitting Bull).
Maybe like half a point for survival or something. And then maybe grade 'kills' insofar that the conquering AI takes the majority of the cities of the AI getting eliminated. If you're taking 3-4 cities from a neighboring AI, that's pretty much a kill anyway, since you're leaving them so crippled that they basically can't contest the game. The last one would probably take too much time to bother grading - I wouldn't want Sulla to have to bother with it on the fly anyway. It's probably better for a more retrospective approach to the power rankings.
I mean the Survivor contest is what it is though. It's not to find the "best" AI at this point. Remember Justinian was a total shocker and no one had any idea he was a strong AI.
Ya the game is tilted wildly against the 'good' AIs without a human player, but that's kind of the game we're playing now. We're playing AI Survivor. That's why guys like Mao get picked over better AIs like Vicky. Vicky's a monster AI with monster traits, but she sucks at AI survivor.
(July 23rd, 2020, 00:11)Fluffball Wrote: I mean the Survivor contest is what it is though. It's not to find the "best" AI at this point. Remember Justinian was a total shocker and no one had any idea he was a strong AI.
Ya the game is tilted wildly against the 'good' AIs without a human player, but that's kind of the game we're playing now. We're playing AI Survivor. That's why guys like Mao get picked over better AIs like Vicky. Vicky's a monster AI with monster traits, but she sucks at AI survivor.
Even there, she has at least come close to a win once - which of course, scores no points - whereas there are other bottom feeders who could almost have been replaced by ocean tiles for all the effect they have ever had on a game.
It may have looked easy, but that is because it was done correctly - Brian Moore
(July 22nd, 2020, 22:51)Borsche Wrote: It honestly might be better to give some points for survival.
At least for the opening rounds, that's actually the case : wildcard game participation is an extra chance to score points.
I think I'll have to go back and check all five seasons, I'm curious if there's leaders that consistently make the wildcard game. There are certainly more than a few leaders that can't even do that. It might be worth 1 point for survival outright. This would shift winning to 6 points and second place to 3 points - or you could making winning worth 4 points and second place worth 1 (so that the 'survival' bonus boosts both to 5 and 2; the same values that currently exist).
If we're grading leader AI, then the one that consistently survives games, be it in the opening round, wildcard, playoff games or championship is definitely better than the one that consistently gets killed. Tracking deaths along side this would be a good way to contrast the boom or bust leaders, the consistent performers, the ones that only survive but never outright win, and the ones that are fodder for everyone else.
EDIT: Now that I think about it, tracking deaths seems kind of stupid. The only people that aren't going to have 1 death per season are: anyone that doesn't finish top 2 in the wildcard, playoff, or championship games. Points for not dying will already track that.
This is a topic that I've spent a fair amount of time thinking about, and I do think that the current "power ranking" scoring system is a solid metric at evaluating the AI leaders. (For AI Survivor purposes, obviously, not making any wider claims since AI performance is so heavily tied to the many different settings that can be picked for a game of Civ4.) Awarding points for top finishes seems like a no-brainer since the goal is to win each match (or come as close as possible to winning). And yes, kills seem to be a good proxy for testing how strong an AI happened to be in each game. Generally speaking, AI leaders become strong by attacking other leaders and consuming their territory, just as our human players do in our Multiplayer games. Defeating another leader and taking over their cities also correlates very strongly with having one of the top finishes. For example, in Season Four the 30 leaders that fell in the opening round only managed 4 kills between them while the 6 leaders that made the Championship collected 18 kill between them. In Season Three, the 6 leaders in the Championship had a staggering 26 kills between them! That's not a coincidence: Stalin and Caesar and Gilgamesh made the finals specifically because they were smashing people left and right. Yes, you can sometimes have a leader swiping a kill at the last second in undeserving fashion, but that's actually pretty rare in practice. The dominant powers are typically the ones scoring the eliminations and riding them to eventual wins.
By way of contrast, the minor achievement of "not dying" doesn't seem to correspond terribly well with a strong performance. We've had far too many cases of an inert leader sitting in the corner of the map doing nothing interesting, and that's not something that I think should be rewarded with points. The old scoring system from the first few seasons was based on "not getting eliminated" and it ended up valuing a bunch of non-entities like Brennus and Qin simply because they had managed to avoid dying. I came up with my current scoring system because it values action and decisiveness over passivity. As I wrote at the time, this system rewards AI leaders for finishing highly and taking out other competitors as opposed to simply avoiding death. Should you really score points for limping into the Wildcard game and then again for not dying against a crummy field of leaders there? I personally don't think so.
Let's take a quick look at the opening round games from this season and use them as an example of how the leader scoring has played out thus far. Starting with Game One:
Mehmed and Tokugawa take out three leaders between them while Pacal techs his way into an unbeatable state. Mehmed finishes second because Pacal fought two lategame wars against Tokugawa and was stomping Japan. This seems like a fair scoring breakdown to me.
Suryavarman is dominant in this game and that's reflected in him scoring by far the most points. Brennus is the second-strongest AI but gets attacked by Khmer late in the game to get knocked out of second place. The solid game played by Brennus is recognized under our scoring system by him getting the same points as Louis despite ending up in the Wildcard match.
Gandhi plays the perfect game and runs away with a Spaceship victory. Caesar ends the game #1 in score and territory but can't catch up to Gandhi's superior teching. They end up scoring almost the same amount of points and both move on to the playoffs - feels like a very fair outcome here. Hammurabi does exactly nothing throughout the game and deservedly gets zero points.
Saladin takes advantage of a weak showing from Ragnar and Kublai Khan to (very slowly) reach dominant status and win an extremely late victory. Hannibal also takes a long time to defeat Pericles/Ramesses without quite being able to catch up to Saladin. Kublai Khan scores one point before dying and the rest of the leaders couldn't manage that much.
Asoka takes advantage of a disastrous Hatshepsut attack to push for an early Cultural victory. Charlemagne absorbs the Zulus and most of France, finishing the game with the top score and land area. Bismarck is nowhere close to the leaders but does score a single point by eliminating Napoleon. Again, this feels like a roughly accurate scoring for how this game played out.
Huayna Capac: 7 points (Winner, 2 kills)
Augustus Caesar: 2 points (Second, 0 kills)
Mao Zedong: 1 point (1 kill)
Gilgamesh: 0 points (0 kills)
Huayna Capac is a god and runs away with this game despite a cramped starting position. Augustus should have 3 points for this game but was somehow beat to the final Celtic city by Mao's crippled Chinese civ. This is one situation where the scoring system is less than perfect but Augustus still ends up with more points than any of the non-Huayna leaders by virtue of finishing second (and gets another chance to score further points in the playoffs). Gilgamesh does nothing after his fast start and deservedly gets no points.
Willem researches Rifling tech and therefore wins the game (with the Dutch leading in tech for essentially the whole thing). Genghis Khan is the clear #2 power in this world and gets 2 points for eliminating both Persian leaders. However, his stupidity in attacking Willem knocks him out of second place and causes him to tie with Washington instead of scoring more points.
Overall, this system isn't perfect but I think it works pretty well. The leaders who were dominant in their opening round games scored tons of points, and the games that had strong second-place leaders (Caesar, Hannibal, Charlemagne) came close to matching the winner in score. Games that were total blowouts (Suryavarman, Huayna Capac, Alexander) had runner up leaders that were far behind in scoring points. I'm not really seeing leaders who scored few or zero points that have a strong case as to why they should have received more. Maybe Brennus and Augustus Caesar, I guess? But Brennus was in the process of dying when his game ended and Augustus wasn't competent enough to finish off a much weaker neighbor so I don't feel much sympathy for either one. I really don't think mediocre performances like Hammurabi or Gilgamesh have any serious argument for deserving points - what did either of them do in their respective games? Sitting in the corner and not dying doesn't seem too impressive to me.
Anyway, I like the current system because it's reasonably accurate and it's also really simple to understand. I'm not convinced that adding anything else would improve accuracy and I'd prefer to avoid the complexity.
It is really interesting to compare the data for all techs vs correct techs. All techs seemed to have really made the higher tier but less consistent AIs like Mansa or Cathy perform much better when compared to the past few years where we've had the correct starting techs.
In Soviet Russia, Civilization Micros You!
"Right, as the world goes, is only in question between equals in power, while the strong do what they can and the weak suffer what they must."
“I have never understood why it is "greed" to want to keep the money you have earned but not greed to want to take somebody else's money.”
(July 24th, 2020, 00:16)antisocialmunky Wrote: It is really interesting to compare the data for all techs vs correct techs. All techs seemed to have really made the higher tier but less consistent AIs like Mansa or Cathy perform much better when compared to the past few years where we've had the correct starting techs.
I was thinking something similar; this is probably the biggest change in between seasons to affect which leaders do well or not. Almost to the point that it feels like we should have separate power rankings for the two. Mansa, like you said, is a great example of an AI suddenly doing FAR worse since the change was implemented. I should say I think it's a great change, it just does have a huge impact.
Edit: there are also arguments to say that Mansa was just unlucky the last two seasons, I suppose.
Personally, the one recommendation I would make is that I think a bonus point for winning without a kill or something would be good to somwhat equalize for leaders such as Gandhi, Mansa Musa and the like whose ideal win involves no wars at all. Odds are that if you win a game without a single kill, you were incredibly dominant in some area (teching or culture) to beat those who were. I am guessing this wouldn't happen since it adds a bit of overall needless complexity given it is so minor, but I do think it would help state, for example, Pacal's tech dominance and Gandhi's peaceful dominance this season where they ran away with the game without a single conquest.
I had meant to get in predictions and what I am about to post in earlier, but real life prevented me from doing so. And so here I am at 3:20 AM (as of the start of writing this) making this post haha.
First spoiler tag here: I had felt like it seemed this season had a surprising number of points scored by leaders outside the normal high end, so I did a very quick count of everyone who scored points this season and how many points they had in the spoiler below. This is very simple and not, like, advanced statistical looks or anything, but people may find it interesting nonetheless. I wanted to do this before the Wildcard round so nobody had a chance to score in more than one game.
Of the 23 leaders who scored points in the opening round, a full 12 of them were leaders at or below the median of 7 points scored, which given how top heavy AI Survivor has historically been (IE the top leaders dragging the median UP) is pretty impressive. Instead using the average, 10 points, gives us Willem in this group as well.
Perhaps as notable, though, is how many of the top leaders FAILED to score points: 3 of the 8 Pool One leaders failed to score a single point, and of those 3 ALL of them were in the top 3 highest scoring AI (Justinian, Mansa Musa, Stalin). Kublai was also eliminated, but managed to score a point. similarly, FIVE of the 8 Pool Two leaders were eliminated without scoring a single point, meaning half of all pool leaders failed to score at all (which, in turn, makes up 8/52 of overall leaders). The end result is that even outside the median, leaders with more mediocre scors such as Tokugawa and Hannibal managed to boost their profile.
Obviously, one level of this could be simple randomness: Civ IV is balanced enough we would expect any lesder to, in theory, be capable of a win / points if you played enough games. Certainly this would fit for someone like Washington, whose scoring circumstance was rather fluky (tho he didn't play all THAT badly imo). Another possibility is that this is a course correction: AI Survivor has traditionally been ridiculously top heavy but with only 4 seasons of play there is still large amounts of variance, so it is very possible that the top end is overly represented, especially since the playoffs have a snowball effect (winning allows you to potentially score more points, so a single good run can do a lot). More serious game-by-game analysis would need to be done to go more beyond this.
In addition, rule changes and the map can be factors. For example, less free Deity techs likely makes early aggression easier because neighbors are more likely to be easy pickings compared to AI Production + Turn 1 Archers. Similarly, this year's map has been notable for lacking copper in various areas, allowing true warmongers like Alexander and Genghis Khan to thrive. Copper doesn't just mean you can whack an army without it, but it allows the AI's army to be built up stronger than the othr before war which can make all the difference since taking down cities with Walls is high casuality. In fact, Hannibal, Genghis Khan and Alexander all started off their games attacking a neighbor without copper and being able to ride that that to superior points. This suggests the common theory that these are circumstantial leaders, mostly Alexander/Genghis, are true.
Due to the distribution of finishes for Pool One/Two leaders, each playoff game has two Pool Leaders in it, although if Mao finishes in the top two of the Wildcard round he will give one game three. This also means unless Game 1 finishes Pacal + Charlemagne, Game 2 finishes Suryavarman + Julius Caeser and Game 3 finishes Huanya + Gandhi, we WILL see non-Pool leaders score at least one more 2nd or higher (pending Mao as mentioned). In addition, at least one non-pool leader will score from the wildcard as Mao is the only pool leader from there. This means that we'll be seeing a further rise of lower end leaders.
Hannibal, barring results changing to boot him bac out, has made it into Pool Two at 16 points, booting out Shaka. Willem going to 16 points also has him in Pool Two, booting out either Pericles or Zara Yaqob who are tied. Other leaders who could in one game move from un-Pooled to Pool Two OR from Pool Two to Pool One inclue: Tokugawa (2 points behind Pool Two entry), Alexander of all people (11 points puts him 4 behind Pool Two, so a single win gets him there), Asoka (Asoka's 11 poins ditto), Pacal II (Two points behind Charlemagne, 4 behind Suryavarman), Gandhi (1 point behind Charlemagne, 3 behind Suryavarman). As a random note, it is actually not impossible Kublai Khan loses his Pool One slot, but it would require at least 3 of Charlemagne, Pacal, Gandhi and Suryavarman to win or get 2nd and then score in the championship game.
I was gonna add more but I forgot since I'm so tired so w/e.
And after that, I'm also going to as usual post my Wildcard Game predictions:
This map is brutal for Brennus and Bismarck, neither of whom have easy copper early and so are going to be destroyed by barbarians. This is the main reason I pick Bismarck first to die over Hammurabi, along with Hammurabi's Bowmen actually being useful.
The southern tundra is going to spawn lots of barbarians, but Gilgamesh and Tokugawa both have great traits to deal with that: Gilgamesh gets quick border pops to defog and get cultural bonus and both are Protective which in a Raging Barbarian AI game is actually very good. In addition, Gilgamesh has capital copper with a sick capital while Tokugawa has copper so close he CAN'T avoid it, riverside grassland gems and some excellent nearby city possibilities. Sick capital, too. He is also neighbor to Bismack, meaning he's likely to get land by eating Bismarck (gee, that sounds familiar!). Note that Gilgamesh also starts with great techs here. I also feel like both of these AI in general are underestimated / underrated, Tokugawa can actually become a Pool Two leader if he wins!
The wildcard Ai-wise this game is Genghis Khan, who could roll over Bismarck or Hammurabi (moreso Bismarck) early and wn, or could war dec an "ally" and sink their game. I only predict 12 wars since I feel like most will drag on, but Tokugawa and Gilgamesh will not play nice and eventually one conquers the world. T366 end date as raging barbs will make it last longer and there's like no great techer this field. Similarly why I am picking Domination.
I see two other main ways this goes: Brennus founds a religion and spreads it to key allies, likely leading an alliance against Hammurabi (who likes culture) and Mao (who essentially doesn't care for it). Brennus then cruises to a win as a religious bruiser, not too unlikely.
If the raging barbs truly keep the aggressive AI down or Genghis Khan otherwise causes a lot of trouble for them, I can also see a world where Bismarck + Hammurabi as natural allies tech until too high of a lead to be conquered and win.
![[Image: survivor5-1-21.jpg]](http://www.sullla.com/Civ4/Survivor/S5/survivor5-1-21.jpg)
![[Image: survivor5-2-22.jpg]](http://www.sullla.com/Civ4/Survivor/S5/survivor5-2-22.jpg)
![[Image: survivor5-3-24.jpg]](http://www.sullla.com/Civ4/Survivor/S5/survivor5-3-24.jpg)
![[Image: survivor5-4-28.jpg]](http://www.sullla.com/Civ4/Survivor/S5/survivor5-4-28.jpg)
![[Image: survivor5-5-23.jpg]](http://www.sullla.com/Civ4/Survivor/S5/survivor5-5-23.jpg)
![[Image: survivor5-6-22.jpg]](http://www.sullla.com/Civ4/Survivor/S5/survivor5-6-22.jpg)
![[Image: survivor5-7-20.jpg]](http://www.sullla.com/Civ4/Survivor/S5/survivor5-7-20.jpg)
![[Image: survivor5-8-28.jpg]](http://www.sullla.com/Civ4/Survivor/S5/survivor5-8-28.jpg)
![[Image: smile.gif]](http://www.sullla.com/Smilies/smile.gif)
