Pitboss hosting

[sunrise089]

Since we're discussing the technical side of pitboss and this thread may have a wider audience, let me throw two things out there:

1) Since I've been using my 'temporary' server I'm noticing a reoccurring issue where civ->router will go from <.01% of my gigabit ethernet bandwidth to ~.2%, aka almost all of my ISP-provided upload bandwidth. Unfortunately there are many variables that are difficult for me to isolate: I didn't ever notice this issue at my old location, but that was on much faster internet so if the 'bandwidth leak' uses something like a fixed size and not just everything available it could have gone unnoticed; I didn't notice it at the new location on the old PC but that PC was dying and so was restarted pretty frequently; it seems to happen pretty randomly, but I don't have it plotted, I only know it has been several days apart each time it has occurred; it could be something malicious (paging Kylearan?) considering the PC is pretty open to the internet, but I am running up-to-date MS Security Essentials; and...

2) My Win7 civ install is crazy jury-rigged. In order to run multiple instances of the pitboss client I need to have multiple BtS directories that contain the Civ.ini file. In XP and with the original retail BtS release this was no problem, as the directory is Firaxis->....->BtS. In Win7 + Civ4 Complete (again I've not checked my original disks on Win7 or the new disk on XP) though BtS is installed in the 2k Games directory but Civ.ini is installed in My Games. In the current server I just manually copied the Firaxis directory over and changed the drive letters in the shortcuts and it worked, but this isn't an ideal fix especially if it's contributing to the upload issue.

Anyone have any ideas other than "spend a ton of time testing different combinations?"

[spacetyrantxenu]

Since we're discussing the technical side of pitboss and this thread may have a wider audience, let me throw two things out there:

1) Since I've been using my 'temporary' server I'm noticing a reoccurring issue where civ->router will go from <.01% of my gigabit ethernet bandwidth to ~.2%, aka almost all of my ISP-provided upload bandwidth. Unfortunately there are many variables that are difficult for me to isolate: I didn't ever notice this issue at my old location, but that was on much faster internet so if the 'bandwidth leak' uses something like a fixed size and not just everything available it could have gone unnoticed; I didn't notice it at the new location on the old PC but that PC was dying and so was restarted pretty frequently; it seems to happen pretty randomly, but I don't have it plotted, I only know it has been several days apart each time it has occurred; it could be something malicious (paging Kylearan?) considering the PC is pretty open to the internet, but I am running up-to-date MS Security Essentials; and...

2) My Win7 civ install is crazy jury-rigged. In order to run multiple instances of the pitboss client I need to have multiple BtS directories that contain the Civ.ini file. In XP and with the original retail BtS release this was no problem, as the directory is Firaxis->....->BtS. In Win7 + Civ4 Complete (again I've not checked my original disks on Win7 or the new disk on XP) though BtS is installed in the 2k Games directory but Civ.ini is installed in My Games. In the current server I just manually copied the Firaxis directory over and changed the drive letters in the shortcuts and it worked, but this isn't an ideal fix especially if it's contributing to the upload issue.

Anyone have any ideas other than "spend a ton of time testing different combinations?"

Do you have any kind of bandwidth monitoring or logging set up to detect the upload issue, or is it possible that it's happening more frequently than you think and you just don't notice it? I would expect that uploading game saves should use as much of your upload bandwidth as is available at the time whenever someone tries to connect to the game (assuming their available download bandwidth exceeds your upload capacity, which should usually be the case). You want your upload capacity to be maxed out when someone is downloading the save, that's what it's there for. Those kind of bursty connections are what having high bandwidth available is good for. You probably never noticed before because your old connection (FIOS?) probably had much greater upload capacity -- if you're on DSL now (and I don't remember you saying what the numbers are now, or what service you're on now vs. what you had before the move) I can see your upload capacity being as low as 1/10th what you had before. That would definitely be noticeable, and when I was logging in to PB8 after you moved to the new location I could definitely tell a difference when logging into the game. The game saves where what, 500 KB in size by then? It took a lot longer, it was definitely noticeable as a player then. I wonder if that's the problem, just lower available bandwidth, and that you're not detecting that it happens a lot more frequently than you think.

I don't have as much of a theory on the second item, I've only recently dabbled with pitboss hosting at all and only spent one night setting up a multi-host environment. What are you worried about being jury-rigged about your setup? When I set it up I used the ALTROOT method in the post you linked and created a new directory for Pitboss games at C:\Pitboss and then created subfolders under that folder for the individual games that I could host (C:\Pitboss\2057 , C:\Pitboss\2058 , etc.) Other than the stupid bug/feature where it copied all my game saves into the first ALTROOT directory I had created (2057) it worked fine. Are you worried about security or what is the specific issue you're trying to address?

I'm not sure this setup has anything to do with your first issue, bandwidth usage, and I'm not yet sure that's an actual problem that needs solving, rather that it could be a mathematical outcome of having less upload bandwidth than you're used to. If you're certain, though, that the occurrence is NOT the same as players frequently logging in and out of your server grabbing game saves, well that pretty much torpedoes my bandwidth theory and maybe there is a problem to resolve.

[T-hawk]

In the current server I just manually copied the Firaxis directory over and changed the drive letters in the shortcuts and it worked, but this isn't an ideal fix especially if it's contributing to the upload issue.

I don't have much to add, just to note that manually copying over the entire Civ 4 directory seems to work fine for pretty much any purposes. My current Civ 4/BTS installation has been manually copied across three Windows instances now, and the game itself has never needed reinstalling, which means it must not be dependent on any registry entries or anything else. Pitboss works too at least for my occasional mad scientist lab hacking; haven't tried multi-hosting.

[sunrise089]

Thanks for the great feedback space and T-Hawk!

Do you have any kind of bandwidth monitoring or logging set up to detect the upload issue, or is it possible that it's happening more frequently than you think and you just don't notice it?

No monitoring and it certainly could be happening more often. I don't think it is though, since it's happened four times so far and each time the pattern has been [days of no issues]->[hours of steady issue]->[restart all civ instances (not the PC)]->cured for several more days. The 'days' length seems to vary, but I'm on various networked devices pretty often and it definitely seems to be an all-or-nothing thing.

I would expect that uploading game saves should use as much of your upload bandwidth as is available at the time whenever someone tries to connect to the game (assuming their available download bandwidth exceeds your upload capacity, which should usually be the case). You want your upload capacity to be maxed out when someone is downloading the save, that's what it's there for. Those kind of bursty connections are what having high bandwidth available is good for. You probably never noticed before because your old connection (FIOS?) probably had much greater upload capacity -- if you're on DSL now (and I don't remember you saying what the numbers are now, or what service you're on now vs. what you had before the move) I can see your upload capacity being as low as 1/10th what you had before. That would definitely be noticeable, and when I was logging in to PB8 after you moved to the new location I could definitely tell a difference when logging into the game. The game saves where what, 500 KB in size by then? It took a lot longer, it was definitely noticeable as a player then. I wonder if that's the problem, just lower available bandwidth, and that you're not detecting that it happens a lot more frequently than you think.

A lot to digest here. First, you're correct that I went from a glorious 25mbps upload to a sad 2mbps. However game saves are in the 250k range, which is still just 1-2 seconds of full-bandwidth upload time. I can't speak to the difference in loading games across my locations, but keep in mind the move was also accompanied by adding two more games and the possibly related final death of the old server. In any case, civ transferring a game save is not simply a matter of transferring the file. It takes much too long for that on even modest connections, so there must be some sort of decoding and/or processing happening on the receiving end.

I don't have as much of a theory on the second item, I've only recently dabbled with pitboss hosting at all and only spent one night setting up a multi-host environment. What are you worried about being jury-rigged about your setup? When I set it up I used the ALTROOT method in the post you linked and created a new directory for Pitboss games at C:\Pitboss and then created subfolders under that folder for the individual games that I could host (C:\Pitboss\2057 , C:\Pitboss\2058 , etc.) Other than the stupid bug/feature where it copied all my game saves into the first ALTROOT directory I had created (2057) it worked fine. Are you worried about security or what is the specific issue you're trying to address?

I'm only worried about security as it relates to a possible cause for the first issue. Otherwise I only care in that it's an easier variable for me to isolate than accelerating the move to the (tentative Mac Pro) new server or moving physical locations.

I'm not sure this setup has anything to do with your first issue, bandwidth usage, and I'm not yet sure that's an actual problem that needs solving, rather that it could be a mathematical outcome of having less upload bandwidth than you're used to. If you're certain, though, that the occurrence is NOT the same as players frequently logging in and out of your server grabbing game saves, well that pretty much torpedoes my bandwidth theory and maybe there is a problem to resolve.

Probably should have said this from the beginning, but while I can't speak to whether any of these issues are CAUSED by in-game actions, the upload 'leak' continues to occur when no one is connected to any of the games.

[sunrise089]

The plot thickens. For the first time today the server was sending out between 0% (so less than .01% which is the lowest able to be indicated) and .3% (max upload). Sure enough Speakeasy reported 1.8mbps upload (so most of my 2mpbs but not all), and Teamspeak reported about 10% upload packet loss. As usual restarting fixed everything.

I'm going to really try to get my Mac Pro up and running this week and see how that works. Added evidence to the 'this is some sort of security issue' theory is that the server PC, but not my main desktop, is Win7 SP0.

[Caledorn]

The plot thickens. For the first time today the server was sending out between 0% (so less than .01% which is the lowest able to be indicated) and .3% (max upload). Sure enough Speakeasy reported 1.8mbps upload (so most of my 2mpbs but not all), and Teamspeak reported about 10% upload packet loss. As usual restarting fixed everything.

I'm going to really try to get my Mac Pro up and running this week and see how that works. Added evidence to the 'this is some sort of security issue' theory is that the server PC, but not my main desktop, is Win7 SP0.

I keep running into this very same problem with the ISDG on a regular basis. I've yet to figure out a reason for it, but I noticed that it has a higher probability of occurring if a player logs in from a location with a bad Internet connection (as in a long route with many sketchy routing points). At one point there was a turnplayer who was on vacation somewhere in South America, and this "phantom package" problem occurred every single time he had logged in to the game.

I'm running the Pitbosses from the same Civ4 installation, but with separate copies of the PB executable file for each game. Further, my altroot folders contain separate ini files for each game. If you want more info about how I do that, poke me and I'll try to explain better.