2) AI objectives
This is a different case and a different situation. It should be decided/talked through before our first contest [AI research].
How current AI objectives work: a) During the game setup, every AI player chooses a game objective, out from 6 possible (Diplomat, Militarist, Expansionist, Technologist, Industrialist, Ecologist). Note that races have strong objective preferences (usually a primary at about 40% and a secondary at about 25%); but, with a few exceptions, every race can acquire every objective. b) What objectives do behind the scenes:
- planetary spending ratios
- research spending ratios
- [minor influence] there are a few occurences for Expansionist and Militarist inside AI decision making c) there is an important extra WAR objective.
Once any AI enters in a war with any other player, it stops using any of the above settings and follows this one "war" pattern.
Since MoO is more or less a war game, the AI players are almost incessantly at war with someone else, after they make contact with several other races. The primal AI objectives thus become more and more irrelevant with time passing. We may think about them as "starting objectives": they are important in the early game, but later they fade away.
The interesting questions behind this:
Should we reduce all objectives to one "optimal" for the sake of AI testing?
Should we use the objectives for different strategies that "fit" most for some races, and reduce the RND spread of choice for starting obectives instead, similar to how they are used in 1.31 (like: Darloks are always Technologists, because they play best as Technologists)?
Do we really conserve the extremely simple model of universal WAR objective? Shouldn't there be more of war strategies - at least a "War_advantage" one and a "War_defense" one?
(January 25th, 2014, 20:38)kyrub Wrote: I can see where you are going with MoM and Civ but ...is Moo more complex than chess? Estimating positions in chess is incomparably harder than in MoO, tactical moves computing, positional play... Ships cannot move to several places in a turn, they all move the same, unlike chess figures... I have my doubts.
This topic is quite complex, but here are a few points to consider when we determine difficulty of strong AI development:
1. The average branching factor in chess is 35. In MOO it is >100,000 (number of possible moves per turn). The majority of possible moves can be heuristically eliminated in both chess and MOO due to being "probably bad moves".
2. Average number of moves (decisions) in chess is about 40 for a full game on high skill level. In MOO it is estimated >240 (between equal and high skill players).
3. Chess has no random outcomes for actions. MOO has many of those.
4. In chess, player has full knowledge of game state at all times. In MOO, this is never the case.
5. Chess is a 2-player game. MOO is a multiplayer game, which makes collaboration an additional requirement to consider. Therefore, estimate and belief on the other players' intention and skill is far more important in MOO.
6. In chess, on the average, the first ~20 moves can be made from lookup tables at high skill level and deviating from it is almost certainly undesirable for a strong AI. In MOO, this is <15 moves on the average.
7. Chess AI coding can be effectively approached by search algorithms combined with well-tuned, well-tested weight functions. This allows utilization of computing speed and power. The more hardware you put in, the better results the AI will achieve. No performance limit, "no plateau" is known as of today. MOO cannot be reduced to a search problem. Strong AI requires other, more complex methods. These approaches utilize computing speed much less. More hardware does not lead to better AI (at least not in the foreseeable future.)
8. The value of a chess position is relatively straightforward. Material value is very important, with a few additional constraints to consider on top of that. Estimation of the value of a position is typically reliable, and there are only occasional exceptions. Even if an exception is found, the coding can follow immediately and it can be solved for next game, so AI will not make same mistake. In MOO, the estimation of the value of a position is more complex and depends on many more variables than in chess and again involves (true) uncertainty (not just uncertainty due to insufficient computing power like in current chess AI). If loss is earned by AI, it is difficult to pinpoint which move/decision was responsible for that and harder to fix it.
9. Chess has a well defined, frozen rule set. In MOO, the expectation and likelihood is that the rules will change in the future, even after a so-called strong AI is ready. This significantly hinders AI development.
10. Chess has a small set of rules. MOO has a larger set of rules. Effective AI development requires optimal tradeoff estimation and exploitation. With more rules, these tradeoffs are more abundant and harder to estimate and tune. Situational value of a small part of the game state is harder to estimate. In chess, this is less of a problem due to the existence of fewer parts of the game state.
11. There was significant worldwide interest and funding for the creation of a strong chess AI (much less for a so called "fun AI"). Funding for MOO type game AI up to today is minimal at best. This led to sufficient efforts in chess to reach breakthrough, but insufficient efforts for MOO type games so far, which is likely to continue in the foreseeable future.
12. The lack of a large pool of expert AI coders with expert game knowledge combined with lack of motivation & efforts hinders the AI development process, since there is insufficient exchange of ideas and code segments between professional coders and expert players. Also there were many AI tournaments in chess, but other than C-Evo (which is also very minimal) I am not aware of AI tournaments for complex TBS.
13. (my) Conclusion: coding world champion chess AI is far easier than coding world champion MOO AI (even after AI liberation, when AI plays by exact same rules as human). That is in spite of having far more chess players than MOO players.
kyrub Wrote:EDIT: instead of rant, here is your offset (for the "code" I showed you).
0x8563B As I said, I guess it won't help you...
Thanks, it helped. Definitely that is a bug. Beyond doubt.
Logic of function is to return the number of existing colony ships even if no new ship is created.
(So you had shown me wrong logic: "IF no colony ships created goto SKIP".)
Function returns zero only in several exceptions when the game is not able to create/add a new ship: no fleets, no colony ship designs, zero production, fleet picked does not exist.
The last case (if I realized it correctly) results in the same bug as '4th cs'. But this case is not to occur normally, I suppose.
OK. So the issue of targeting a strong AI development is settled. My future comments will aim this as the primary target. All other objectives are secondary to this from now on until stated otherwise.
First of all, your issue falls under the umbrella of "AI not playing by the same rules as humans". Humans are not required to follow and state their rigid objectives. AI liberation would not require AI stating what it is doing. Clearly a so called "fun AI" objective which is purely for human entertainment and actually hurts the AI. By removing or making this optimal, flexible and complex, AI will get stronger, so I am OK with fiddling with this.
Could you please define little better what these translate to?
(January 27th, 2014, 10:24)kyrub Wrote: usually a primary at about 40% and a secondary at about 25%
So only the primary is displayed and there is a secondary that is also from this list of 6?
What does 40% mean? AI allocates 40% of overall planetary resources to that objective in each turn until war?
(January 27th, 2014, 10:24)kyrub Wrote: What objectives do behind the scenes:
- planetary spending ratios
So an Ecologist spends 40% on eco since turn 1? Or 40% of tech research on planetology?
(January 27th, 2014, 10:24)kyrub Wrote: - research spending ratios
So a Technologist spends 40% on technology since turn 1?
(January 27th, 2014, 10:24)kyrub Wrote: Should we reduce all objectives to one "optimal" for the sake of AI testing?
That's a good next step to follow at this point, but do not delete the old ones. Instead, add a 7th objective "Perfectionist". If you can make 100 round, AI only tournaments, then that would give data about which works better. Always 1 AI plays with the newest objective. 5 with older objectives. In end you only record objective, winner, race. Best would be to fix races and objectives for 100 rounds then rotate. This way you only record winner. Can you code this? Fitness function (for objective) is most games won out of 100.
(January 27th, 2014, 10:24)kyrub Wrote: Should we use the objectives for different strategies that "fit" most for some races, and reduce the RND spread of choice for starting obectives instead, similar to how they are used in 1.31 (like: Darloks are always Technologists, because they play best as Technologists)?
Too early for this. Maybe after a "fit" (~optimal) AI is known, but even then keeping AI's options open is a good idea. Part of AI weakness is predictability.
(January 27th, 2014, 10:24)kyrub Wrote: Do we really conserve the extremely simple model of universal WAR objective? Shouldn't there be more of war strategies - at least a "War_advantage" one and a "War_defense" one?
The more diverse the options for the AI are, the better. However, this means more work for us and harder to tune the AI. We do it, but go gradually. So going from 1 to 2 is OK for next step. More discussion is needed.
Judging from your reaction, my post was incredibly confusing. Sorry. I'll try again.
How 1.31 works
1. AI players play with so called "objectives" (specific game plans). Objectives are visible to the player, displayed in the Diplomatic screen (or Report screen, I am not sure) - that is really poor.
2. Objectives are set for the entire game, unless event Diplomatic murder happens (I think). Also "Alt+P" can probably mix objectives randomly.
2. These objectives are not constant for every race (e.g. Klackons are not Ecologists in every game), but they are randomized at the beginning of each game.
3. The table in the first attachment shows the odds of race => objective choice:
4. How is the objective used in the 1.31 AI? It determine research % distribution and planetary % distribution.
5. How exactly? Let's look at the table in second attachment (pdf). The numbers in the research part of the table represent odds that this particular tech field will be chosen as a priority tech field (will receive 75%) for next 10 years. Example: for the Diplomat objective, there is number 2 in Planetology column. This means that a race with Diplomat objective has a 2 in 16 chance (12,5%) to use 75% of its research budget for planetology.
6. Planetary bar ratios work in a similar way, with the exception that not all remaining percents are used, but d8 percents are added repeatedly, until the whole 100% are filled. Example: A planet of a Militarist race has another 12% non-allocated percents. In first round, the RNG chooses SHIPS (4 in 13 chance) and adds random 5% (from max 8). In second round, the RNG chooses TECH (5 in 13 chance) and rolls full 8%. Since 5+8 = 13% which is too much, the result is rounded to 7%, which is added to the TECH bar. The planet is done.
7. There is the extra hidden objective "being at war". This overrides any starting AI objective.
What to do with objectives during Create_AI campaign?
1. Work with just one objective ("optimal"), to simplify things.
2. Make objectives entirely race specific and use them to strengthen the racial AI (this means: to change race table to always attribute Industrialist to Meklars, and make Industrialist to suit Meklar perfectly). - [EDIT "but we've 6 objectives and 10 races."]
3. Get rid of the "objective" system entirely --- [EDIT "and use the racial dependence only - but later this removes the possibility to change the game plans during a game".]
What to do with objectives in the long term?
1. They cannot be visible to the human player, we easily agree.
2. I would like to conserve them and use them. AI working on some larger game plan is certainly better than no game plan at all.
3. I think the objectives should not be fixed, but flexibly changed throughout the game. That is quite easy to do.
4. I think that "at war" game plan is not a bad instrument, but very rude, and it should be more polished (defensive war, offensive war etc).
5. I think one or two game plans can be very race specific (Silicoids, maybe Darloks).
I hope it's clearer now.
(I will respond to the interesting chess post once I get some time as well.)
(January 28th, 2014, 16:23)Andrew Wrote: Logic of function is to return the number of existing colony ships even if no new ship is created.
(So you had shown me wrong logic: "IF no colony ships created goto SKIP".)
Sorry, it's wrong. Read again carefully.
If you don't believe me, check the difference between unpatched and patched AI expansion, if it has been only a problem of non-existent fleet, it would have occured exceptionnally. Which is not the case.
For the sake of clarity, I guess the Objective_Odds.pdf file shows probabilities and not odds, right? So for example, human 42% dipl mean that in a new game humans will play diplomat 42% of the time. Odds=42% would be about P=0.3, so Humans would play Diplomat 30% of the time. (Odds is defined as favorable cases divided by unfavorable cases.)
(January 30th, 2014, 16:42)kyrub Wrote: The numbers in the research part of the table
To clarify for others: the research part of the table is the columns from comp through weapon.
(January 30th, 2014, 16:42)kyrub Wrote: The numbers in the research part of the table represent odds
Those are neither odds nor probabilities. Those can be called "weights". Dividing a weight by the sum of the corresponding row of the research part of the table determines probability.
(January 30th, 2014, 16:42)kyrub Wrote: will receive 75%
to clarify: "75% of research spending"
(January 30th, 2014, 16:42)kyrub Wrote: (will receive 75%) for next 10 years
You did not state, but I guess you also mean
a) the other 5 research areas each get 5% of research spending
b) research spending is probabilistically reallocated by the same formula every 10 years
Your point 6 is not clear. Overall, use of left side of table is not clear. Also allocate 75% to first roll? Where does your 12% come from? Also, reallocate every 10 years?
(January 28th, 2014, 16:23)Andrew Wrote: Logic of function is to return the number of existing colony ships even if no new ship is created.
(So you had shown me wrong logic: "IF no colony ships created goto SKIP".)
Sorry, it's wrong. Read again carefully.
If you don't believe me, check the difference between unpatched and patched AI expansion, if it has been only a problem of non-existent fleet, it would have occured exceptionnally. Which is not the case.
Maybe you have not understood me. I think we are talking about the same. Read again carefully...
Unlike assembler language my English is far from perfect. But you must know what is a function (subroutine that return a value). I said about function "Create colony ship" and it is to (must) return the amount of existing colony ships. Even if there is no newly created one. It's absolutely true and your patch code confirms this. Therefore subsequent checking the result of this function for zero is interpreted as "IF no colony ships exist SKIP". The word "created" is unnecessary here and confuses only.
Quote:[but this thread is not exactly about this...]
Partly offtop. That's why I hide this under spoiler.
(January 30th, 2014, 21:50)WhiteMage Wrote: (Odds is defined as favorable cases divided by unfavorable cases.)
Blimey. Until today I lived in a world where "odds" were synonymous to "probability", in English. I used "odds" everywhere as it was a shorter word, easier to type, hehe. As for "weight", I actually knew there was a word that can exprime that kind of situation... but I couldn't remember or find it.
Quote:Your point 6 is not clear. Overall, use of left side of table is not clear. Also allocate 75% to first roll? Where does your 12% come from? Also, reallocate every 10 years?
I wrote a shorter version, hoping that the example would clarify the process, mainly because planetary ratios are not currently on the table in the Create_AI contest. I thought we could reach some consensuss on the concept of Objectives before we start messing with research part.
1. allocation based on AI objective is never 75% like in research, but always RND (1..8)
2. but allocation happens after first applying a few fixed rules (Eco allocation, fixed Tech allocation, fixed Ind allocation etc.)
3. I took 12% just as a small and simple-to-show amount of remaining non-allocated percents. In my estimation, an AI planet has usually about 50-70% of non-allocated production after t[/quote]he application of fixed rules (see 2.) in 1.31. With the exception of under-developped planets with few factories, these have a large amount of fixed IND %.
@Andrew
Let's not argue about words. I am happy we can agree on it being a bug. (I have had endless debates with another resident here on RBO about half of the MoM patch being wrong, so I tend to avoid this type of discussion. Sorry for my previous outburst again.)
