Login

Seravy

Currently working on AI diplomacy, declaration of war and overland movement changes.
This is complicated and will most likely take a long time, maybe even a full week or more.

So far I managed to
-Make the AI consider the actual relative military strength of the players for threatening, proposing treaties, declaring wars, breaking treaties. Also to add a difficulty level based penalty to all of the above.
-Increase the diplomatic bonus for Peaceful and Lawful wizards, decrease it for Chaotic.
-Fix some bug that caused the AI to declare war or random nonplayer targets.
-Achieve that Peaceful and Lawful wizards never attack player towns without war until their units are attacked at least once. They still attack nodes, though. Of course only if there is no treaty to stop it.
-That towns of wizards holding a pact or alliance are not considered as attack targets, instead of trying then retracting the movement and wasting the unit's turn repeatedly.
-That Astrologer now displays the real strength of player's armies instead of just a sum of maintenance costs
-And that Astrologer displays a shorter bar for power because it was near full right at the beginning, the scope was too small. It also now adds casting skill to this, with a higher weight than power base.
-Make militarist and expansionist wizards declare war automatically if they have military superiority (and/or bad relations) with another.
-Make all wizards declare war based on the difference in military strength and relations, but this is somewhat random and works the opposite of the above : The bigger the difference in strength the less likely war is, in both directions. Weak wizards are not worth wasting time on, while overly strong ones would be suicide. Relations work the normal way ofc.
-Remove the insecticide change that dumbed down the AI to consider all player units 4 times weaker than they really are to force suicidal aggression.
-fixed a whole ton of bugs with AI item creation.
-Peaceful and lawful wizards skip the "Aggression over time" procedure that makes them attack towns without being attacked first and without a war declaration based purely on random chance. Other wizards can still do it.
-That significant negative diplomatic actions that result in treaties broken when they existed no longer cause a hidden, permanent, diplomatic penalty with ALL other unrelated wizards.
-And increased the "safety" period after forming a new treaty before the AI can consider cancelling it or declare war. This should help making peace treaties actually matter and last some time, as they are also included.

Look forward to 0.7, but as I said, it will take time.

Seravy · October 25th, 2015, 17:42

Today's main progress :
Alliance allows walking near towns.
Allies will reward the player for killing units of shared enemies.
Myrran default wizards will unpick Myrran before proceeding with picks because always having the same wizard(s) there is not fun.

Seravy · October 26th, 2015, 04:40

0.7 is up!
I'm really tired from all the work so I haven't done more than the minimum necessary testing. If you discover a bug, please report it.

This update is huge. It's probably the biggest one in the mod's history, and I could even say, the game's history as well, since it unlocks an entire implemented but unfortunately not working feature : The AI can initiate an audience with the player and ask for treaties, spells, etc. Yes, this also means in the original game when the AI was trying to complain about something, in most cases it didn't go through and you got a war on you without ever knowing the reason.
Forming an alliance will now have 2 new benefits as well : You'll be allowed to move through their territory, and you will get gold rewards if you kill units belonging to your ally's enemy. The latter might need adjustment on the amount of gold, but for now I'm happy I managed to even implement it.
Astrologer army strength display and power display is updated to have better representations of the player's true power. Army strength, in particular, is now based on the true calculated military value of units, which the AI is using for decisions, and which has also been significantly improved.
Calculated astrologer army strength is now used in all diplomatic decisions where military strength was checked, instead of the original, rather silly checks that only considered units standing on their home continent, and rated them based on costs.
Additionally, all such diplomatic checks now have a difficulty level modifier.
War declaration has been altered too. Militarist and Expansionist wizards will declare war on anyone weaker than them unless they are also peaceful or lawful. All wizards declare war on other wizards similarly powerful as them, but of course relations and personality is a modifier to this, so you can avoid the war if you have a good relation and/or they are peaceful/lawful. Difficulty level has double the normal modifier in this check.

Quite a few other bugs have been also fixed, including AI item creation.

Strategic combat has been altered to divide damage on troops more evenly, and consider the relative value of units when dealing the damage, better units will take less damage usually. The original was coded to maximalize causalities on the winner's side, and although Kryub tried to make it weight the damage based on unit costs, this was halfway done and had no effect, as well as removing the original weights, so overall made it worse.

I've also reversed Kryub's change that made the AI undervalue player units and due to this, go on suicide attacks for no reason at all.

Additionally, the player is now allowed to have 1 more city per land size before the AI starts to consider them overextending than before, the number is now 4*(land size setting).

Another major change : Myrran wizards will first unpick their Myrran trait before proceeding with spending picks. This ensures that you'll have a greater variety of wizards on Myrror instead of always the only one (or four) that have the trait by default.

MrBiscuits · October 26th, 2015, 06:31

I have to say these changes sound absolutely amazing, thanks for putting the massive amount of work in.

I have to ask how you are able to understand the code and mod the changes, could you give a very brief outline of how you do it? Also I am wondering whether you would consider making a repository for your code so others can see each change and where it occurs in the code.

Seravy

(October 26th, 2015, 06:31)MrBiscuits Wrote: I have to say these changes sound absolutely amazing, thanks for putting the massive amount of work in.

I have to ask how you are able to understand the code and mod the changes, could you give a very brief outline of how you do it? Also I am wondering whether you would consider making a repository for your code so others can see each change and where it occurs in the code.

Basically it's 6 steps

1. Disassemble the code with a disassembler (IDA is recommended)
2. Figure out what the referenced data structures contain (This one is hard but you only need to do it once per data structure)
3. You need to be able to read and understand 8086 assembly code and write new code for some of the more advanced tasks
4. You need to find where the code you want to change is within the exe and what is the problem with it in case of fixing a bug.
5. If the change needed isn't something as simple as replacing "add 3" with "subtract 2" you need to find useless parts of the code that do nothing and remove them, then rearrange code so that the empty space is where new code is to be inserted, all of this using just the hex editor. Including recalculating all offsets for all jumps pointing to the moved code.
6. After step 5 is done, you use an assembler to compile the code you want to insert and add it into the space you made.

Then you run the game and see if it works, and use the disassembler to make sure all offsets and code was properly inserted. Miscalculating the offset even by 1 makes the code unstable and do totally random things.
It usually takes many times of doing step 5 and 6 until the result works as intended, as it's quite easy to overlook some detail.

I'm not going to make a repository because it would increase the amount of work many many times.
Instead, after the mod is complete (or at least reaches 1.0), I will make a differential file of this and 1.40n and split up the changes into individual files addressing various fixes, then gather the general purpose ones and apply them to 1.40n and release it as a "Raid 1.50" bugfix release. The individual files will also be available for public.

Unfortunately a lot of the AI logic won't be compatible because some changes depend on others.
For example the change of Lionheart uses the space gained through the removal of Black Channels. The new AI logic for battle spell choices is inserted to where the choice for Word of Recall was, as that spell is now gone. A lot of AI changes are related to spells having new effect, and it would be a huge work trying to separate all of these from changes involving the unchanged spells. However, where it's obvious which change the code belongs to, will all be separated, eventually. I want to make sure everything works perfectly before I start doing that, there is no point separating individual changes when there still is a chance they need to be further altered.

Also, although I name the identified variables and such to make the disassembled code readable, those names are only saved for the original wizards.exe. The changed exe will not have the names inserted because it is a different, just similar exe file that has to be disassembler from zero every time I open it after a change. I don't think the disassembler has the function to export names from one exe to another, changed copy of the same file, although it would be nice if it could do that.

For reference, the two hardest things to do so far were
-Making the AI use the "grant gold for killing enemies" event which was completely unused. This required the making of over 160 bytes of space, and writing new code of that size (roughly 50 lines). Doesn't sound that much but the truth is 160 bytes are a lot when a single instruction is 2-6 bytes and you have to remove existing code to make the space. Fortunately compiled code such as wizards.exe often contains redundant calculations, but not always and not everywhere.
-Making Spell Ward counter combat spells of the same realm. This needed about 80 new bytes in a much tighter area than the previous, and it had to use data I wasn't yet familiar with, the "combat copy of city enchantments" one. I found that out the hard way, after seeing the normal city data is swapped out during combat from the memory.

Compared to this, changing alliance to ignore your units near towns was a single byte change, had to make the jump related to it go to the "end of procedure" instead of the "give the warning" and that's it. Of course, figuring out all the diplomacy variables to the point that I was able to recognize this code is giving the warning took weeks of reading the code and identifying parts and effects.

Below is the code I inserted for the gift gold event, and it had to be separated to 2 parts because that's how I was able to get enough free space.

As you can see, the code tells nothing about where it goes or how the space is made for inserting, so it's not really useful for the public.

MrBiscuits

Off topic post moved to here:

http://realmsbeyond.net/forums/showthread.php?tid=7856

(October 26th, 2015, 07:05)Seravy Wrote:
(October 26th, 2015, 06:31)MrBiscuits Wrote: I have to say these changes sound absolutely amazing, thanks for putting the massive amount of work in.

I have to ask how you are able to understand the code and mod the changes, could you give a very brief outline of how you do it? Also I am wondering whether you would consider making a repository for your code so others can see each change and where it occurs in the code.

Basically it's 6 steps

1. Disassemble the code with a disassembler (IDA is recommended)
2. Figure out what the referenced data structures contain (This one is hard but you only need to do it once per data structure)
3. You need to be able to read and understand 8086 assembly code and write new code for some of the more advanced tasks
4. You need to find where the code you want to change is within the exe and what is the problem with it in case of fixing a bug.
5. If the change needed isn't something as simple as replacing "add 3" with "subtract 2" you need to find useless parts of the code that do nothing and remove them, then rearrange code so that the empty space is where new code is to be inserted, all of this using just the hex editor. Including recalculating all offsets for all jumps pointing to the moved code.
6. After step 5 is done, you use an assembler to compile the code you want to insert and add it into the space you made.

Then you run the game and see if it works, and use the disassembler to make sure all offsets and code was properly inserted. Miscalculating the offset even by 1 makes the code unstable and do totally random things.
It usually takes many times of doing step 5 and 6 until the result works as intended, as it's quite easy to overlook some detail.

I'm not going to make a repository because it would increase the amount of work many many times.
Instead, after the mod is complete (or at least reaches 1.0), I will make a differential file of this and 1.40n and split up the changes into individual files addressing various fixes, then gather the general purpose ones and apply them to 1.40n and release it as a "Raid 1.50" bugfix release. The individual files will also be available for public.

Unfortunately a lot of the AI logic won't be compatible because some changes depend on others.
For example the change of Lionheart uses the space gained through the removal of Black Channels. The new AI logic for battle spell choices is inserted to where the choice for Word of Recall was, as that spell is now gone. A lot of AI changes are related to spells having new effect, and it would be a huge work trying to separate all of these from changes involving the unchanged spells. However, where it's obvious which change the code belongs to, will all be separated, eventually. I want to make sure everything works perfectly before I start doing that, there is no point separating individual changes when there still is a chance they need to be further altered.

Also, although I name the identified variables and such to make the disassembled code readable, those names are only saved for the original wizards.exe. The changed exe will not have the names inserted because it is a different, just similar exe file that has to be disassembler from zero every time I open it after a change. I don't think the disassembler has the function to export names from one exe to another, changed copy of the same file, although it would be nice if it could do that.

For reference, the two hardest things to do so far were
-Making the AI use the "grant gold for killing enemies" event which was completely unused. This required the making of over 160 bytes of space, and writing new code of that size (roughly 50 lines). Doesn't sound that much but the truth is 160 bytes are a lot when a single instruction is 2-6 bytes and you have to remove existing code to make the space. Fortunately compiled code such as wizards.exe often contains redundant calculations, but not always and not everywhere.
-Making Spell Ward counter combat spells of the same realm. This needed about 80 new bytes in a much tighter area than the previous, and it had to use data I wasn't yet familiar with, the "combat copy of city enchantments" one. I found that out the hard way, after seeing the normal city data is swapped out during combat from the memory.

Compared to this, changing alliance to ignore your units near towns was a single byte change, had to make the jump related to it go to the "end of procedure" instead of the "give the warning" and that's it. Of course, figuring out all the diplomacy variables to the point that I was able to recognize this code is giving the warning took weeks of reading the code and identifying parts and effects.

Below is the code I inserted for the gift gold event, and it had to be separated to 2 parts because that's how I was able to get enough free space.

cmp byte ptr [bp+0Ch],8
jnz skipall ; Action type=killed units in combat
cmp di,0
jnz skipall ; Performed by the player
mov cx,1
allwizardsloop:
cmp cx,si
jz nextwizard ; Skip the wizard being hurt
mov ax,cx
mov dx, 4C8h
imul dx
add ax, si
mov bx, ax
cmp byte ptr [bx-5FDEh], 3 ; 'üE
jnz nextwizard ; Skip if they aren't at war with the victim
mov ax,[bp-2] ;Use base diplamatic value not charismatic one
neg ax ; as a positive value
sar ax,1 ; Effect/8
cmp ax,1
jl nextwizard ; No effect if it's too weak
mov bx,cx
cmp byte ptr[bx-5FCCh],0 ; Skip if wizard already wanting to talk about something else
jnz nextwizard
mov byte ptr[bx-5FCCh],6 ; Event type 6 thanks for attack
mov bx,cx
add [bx-5FE4h],ax ; add the points
shl bx,1
mov [bx-5FD8h],20 ; counts as though it was a +20 effect to avoid overwriting with trivial things
mov [bx-5FC6h],si ; set victim as referred wizard
push ax
mov ax,cx
mov dx, 4C8h
imul dx
mov bx, ax
pop ax
add [bx-5FE4h],ax ; add the points to other side too
cmp byte prt [bx-5FDEh],2
jnz nextwizard ; done if no alliance, otherwise
mov dx, [bx-5DE0h] ; wizard's gold
sar dx,1 ; halve it, not giving more than half owned
cmp dx,0
jz nextwizard ; can't give gold if not having any
call alliance

nextwizard:
inc cx
cmp cx,5
jnz allwizardsloop
skipall:

nop
nop
nop
nop
nop
nop
nop
nop
nop

alliance:
push bx
mov bl,64h ; Multiply effect strength by 100
mul bl
pop bx
cmp ax,dx
jle ok
mov ax,dx ; max gold given is what we have/2
ok:
sub [bx-5DE0h],ax ; deduce spent gold
mov bx,cx
mov byte ptr[bx-5FCCh],2 ; Event type 2 pay gold instead of just saying thanks
shl bx, 1
mov [bx-5EFAh], ax ; Offer this much gold*100
add [-5DE0h],ax ; player gain the gold
mov [bx-5FA2h],si ; set victim as referred wizard
retn

As you can see, the code tells nothing about where it goes or how the space is made for inserting, so it's not really useful for the public.

Thanks for your swift and comprehensive reply. It's more complicated than I originally thought!

I downloaded IDA and got it to dissemble the Wizard.exe, but I'm struggling to navigate to even start looking at the code. I have a tab called "Hex view A" that just contains lines like these and none of the sort of machine code that you wrote above:

seg038:0CC0 82 00 74 13 83 3E AA 82 00 7E 0C 0E E8 10 FF C7 é.tâ>¬é.~Þ Ã

How for example would I look at the machine code for where the program "starts" ? And then how would you follow the code from that point?

I don't understand why you need to create space before adding new code. What would happen if you added code without freeing up the space first? Is there no way of adding things without removing other things first and just change the reference to point at the new space?

Point 5 in your list looks the most difficult to me, I hadn't quite realised how restricted you were when making changes. I wonder if some custom tools could be written to make this stage much easier and less painstaking. Or somehow find an alternative solution to stage 5 in your list.

Sorry for all the question, I've done a fair bit of coding but never anything quite like this before. I'm going from "maybe I could help out" to "how on earth is that even possible". If this is cluttering up your mod thread too much maybe we could move this discussion to a new topic instead.

Also I am curious, do you think it would be possible to rewrite the code from scratch using the disassembled code? Not that I think it would be a good idea and it is obviously a huge amount of work, it's just something that I'm wondering about. It might be easier in the long run than messing around with hex code for each tiny change.

And I'm glad that you are going to split up the changes and add them as a bugfix release. I had visions of you arriving in a whirlwind, make this amazing mod and then disappearing without anyone knowing how you managed to do any of it.

Seravy

You'll need these views :

Hex view : This just shows you the bytes in the file. So if you want to find the machine code for some instructions, you can look here.
Ida view : This is the disassembled code. If it's not open by default then try View-subviews-disassembly.
Names : A list of stuff you have named after you identified them. To name a procedure or data reference, right click it and select rename.
Functions : A list of the functions/procedures in the file.
Strings : these are just the string constants found in the exe. Not all of them are actually here, but most are. This view is useful if you know the text the game displays and want to find where it gets displayed. For example you want to allow building when there are already 9 units in town then you search for that message.

There are more views but they aren't very important for now.

Quote:I don't understand why you need to create space before adding new code. What would happen if you added code without freeing up the space first?

You can't. If you add even a single extra byte, everything in the file after that will be at a different address than where the loader looks for it and the game would not even load, as well as all jumps between the two parts being off by one byte.

Quote:Is there no way of adding things without removing other things first and just change the reference to point at the new space?

You can jump to a new space, yes, but you need to MAKE the new space first. There aren't miraclous unused large spaces in the file everywhere for us, sadly. And, it has to be in the same segment, otherwise the process is ultra-difficult. Also if the jump you modify is a byte value then it has to be within +-127 bytes, otherwise you need to make +1 byte space for a larger jump instruction.

Quote: I hadn't quite realised how restricted you were when making changes. I wonder if some custom tools could be written to make this stage much easier and less painstaking. Or somehow find an alternative solution to stage 5 in your list.

Not really. You literally have to look for "adds 2 to the value, then throws the result away and overwrite it with 5." style of coding where you can remove the "adds 2" part without changing the overall effect. In other words, use your brain, so it cannot be done automatically, only humans are able to do this part.
In easy cases one such place can yield 9-15 bytes of space and you can find 6-8 of them near together.
In hard cases, you have to actually re-code stuff to make it use less space through code optimalization.
For example to add the "Turns = x" display in combat, I had to recode it from

set position X=45
set position Y=60
set text "mana"
call text display
set position X=55
set position Y=60
convert amount of mana to text
call text display
set position X=45
set position Y=60
set text "skill"
call text display
set position X=55
set position Y=60
convert amount of skill to text
call text display

into this

set a=55, b=60, c="mana", amount = amount of mana
call "displaytext"
set b=70, c="skill", amount = amount of skill
call "displaytext"
set b=80, c="turns", amount = amount of turns
call "displaytext"

displaytext:
set position X = a, position Y=b
set text "c"
call text display
set position X= a+20
convert amount to text
call text display

as you can see "displaytext" is the original second half which displayed the "skill" line, except it uses the passed parameters instead of the original contants. The first half to display the "Mana" line is removed, and calls the second half 3 times, with 3 sets of parameters to display 3 lines that are no longer limited. To further restrict the options, "Turns" had to exist as a text constant available and ready to use in the data segment, fortunately it did, as it is displayed elsewhere. You can see it uses the text belonging to another game part because "Mana:" and "Skill:" has a ":" but "Turn" does not, it was only available without it. Otherwise I would have also needed free room in the data segment to add new text (I guess there are some unused texts like "as swiftness spell" which kryub made not to appear").

In this particular case there was no space that could be made, and as far calls are unmovable, I had to re-use the same one to be ran once more than originally to display more lines of text. Took about 4 hours to get it work.

Assuming you have a large block of free space somewhere (I actually do because I replaced a huge AI routine somewhere so the original is not called anymore) you could in theory get away with only making 3 bytes of space, and using a near call to it IF you are within the same segment. There are about 160 segments so guess what the chances are for that? Yeah... my nice huge empty space will not be useful, it's in a segment where I don't need to add stuff at least at the moment.

By the way, if you are looking for a variable or a procedure in particular, it might be faster to ask than to try find it amongst the 3500 procedures the game has yourself. Even I only identified about one third of them so far.

I feel like we are getting really offtopic here, though, I think you should make a thread for this instead if you have more questions to ask.

MrBiscuits · October 26th, 2015, 09:17

(October 26th, 2015, 08:26)Seravy Wrote: I feel like we are getting really offtopic here, though, I think you should make a thread for this instead if you have more questions to ask.

Agreed. I made a new topic here:

http://realmsbeyond.net/forums/showthread.php?tid=7856

Hadriex · October 27th, 2015, 04:06

Hey Serevy, would it be possible for you to make a short guide on how to alter the wizard/hero portraits, names, and spellbooks/retorts.

Something clear and easy for a layman to follow (preferably with images).

I ask because when you 'finish' CoM I have a few places I'd like to post links.

Let me explain with an example: I know a webcomic that's really into fantasy wargaming, and will (probably) give space on the front page for a week or so. If I include a little guide from the mod's author that would allow a fan to insert characters from the comic into MoM, then I would be CERTAIN that they'd post it.

Seravy · October 27th, 2015, 04:42

(October 27th, 2015, 04:06)Hadriex Wrote: Hey Serevy, would it be possible for you to make a short guide on how to alter the wizard/hero portraits, names, and spellbooks/retorts.

Something clear and easy for a layman to follow (preferably with images).

I ask because when you 'finish' CoM I have a few places I'd like to post links.

Let me explain with an example: I know a webcomic that's really into fantasy wargaming, and will (probably) give space on the front page for a week or so. If I include a little guide from the mod's author that would allow a fan to insert characters from the comic into MoM, then I would be CERTAIN that they'd post it.

Cool, let's see then. Will something like this do?

Names, books and retorts are in Magic.exe.
2AFE0 : Name of the wizard
Following that, there are 5 bytes for the 5 types of books, word values so there will be a 0 between the actual numbers of each type of book.
Following that an ID code for the retort, unfortunately I don't have my list anymore but it goes from 0 to 11h as there are 17 retorts total. Also don't remember if 00 or FF is the "no retort" option because I never used it. There are exactly 14 wizard slots.

To insert the images, you need to use the tweaker.
Use open LBX, select the LBX you want to change.
It'll show the list of entries in the file, you can go through them and see if any are wizard images.
If yes, you check the size, make an equal size png, and import it.
To import, you need to place the file into the folder before opening the LBX, as the tweaker does not update the list of files real time.
I wasn't able to change animation due to a tweaker bug yet, but I did manage to hack the LBX so that the animations are replaced by single images, and from that point the rest is the same. Use the already altered LBX files that have the Touhou wizards and you are good to go.
The game uses 256 colors. The tweaker will apply the palette for you, but it isn't perfect, it generally yields a better result to take palette from one of the original images in that LBX file, and apply it to your new image. Some of the color codes actually are used for system functions like flags, pixels that end up with that color code can appear the wrong color in the game. If you identify such a color, your best bet is to remove it from the palette and make the image not use it at all.
There are a large number of files that contain wizard portraits or animations of some sort, including : HALOFAM.LBX, MAGIC.LBX, WIN.LBX, LOSE.LBX, SPLMASTER.LBX, SPELLOSE.LBX, WIZLAB.LBX, CONQUEST.LBX, DIPLOMAC.LBX, MOODWIZ.LBX, LILWIZ.LBX and WIZARDS.LBX.
To change hero portraits, you modify PORTRAITS.LBX. The default names used for them are in NAMES.LBX.

Finally, the HELP.LBX contains all help texts for the game. You need to modify this if you want right clicking on the wizard to pop up information about your new wizard(s).

Navigation

Extra Menu

About us