Sometime during Civ4 original a player named Blake decide to make a Mod called Better AI. This was a huge success and Firaxis - realising the difference on AI-quality - hired Blake and incorporated large parts of his AI into Civ4 - Warlords(IIRC). The drop in easily beaten difficulty is around the same all the players experienced when the AI-update was officially released.
As I remember he changed the way the AI managed its cities, which tiles it improves with what improvement, the handling of fleets/air power and naval invasion, increased the AI-urge to win and so on.
Sadly it happened such a long time ago that there are scarce information about that time and all the changes. Blake himself took a sabbatical after the work and dropped from the face of the planet.
He was a member on civfanatics and here but with all the server-changes, software-updates this history is most likely lost.
...
Found something from old patchnotes (
Warlords 2.08):
AI city placement improvement as per Blake
Improved AI use of ’emphasize’ buttons as per Blake
Pop-rush and draft AI improvement as per Blake
Worker AI improvement as per Blake
Improved civics AI as per Blake
Tweaked AI building priorities as per Blake