Pitboss Redundancy Discussion

[Krill]

Yeah, I want to echo Ref, good to hear from him, but this doesn't sound great from a health perspective. I hope I'm wrong, and if not I hope for a good recovery if not.

scooter, I'll send you a PM shortly (posting here to make sure I actually follow through)

[Brian Shanahan]

Does the PB client require a graphics card be present, or integrated graphics of some description?

Edit: Wait, I just checked the Rmak/Zulan tech thread, it's been 10 years since they shut down Gamespy? That can't be right, it feels ilke it was only 3 years ago? Wait that was still in Covid lock down times. WTF. Is this what dementia feels like?

It's covidmentia.

[Krill]

Zulan, thank you for keeping the Civ 4 scene going. If you hear anything about Ramk please can you keep us informed?

I should ask, what is the challenge with hosting the Natneg servers, and the side loading? What sort of bandwidth do we need to make it work? I'm mindful that I can get a 1.6Gbps down/115Mbps up just by upgrading internet, and it is just as trivial for me to upgrade the last leg of my home network cable to 10GbE. If it's other hardware that is the problem, I will be using an Epyc 7532 (32 core, 64 thread) with 256GB of RAM so I can dedicate quite a bit of resource to it if needed. how does this compare to what is currently in place?

Ref, scooter, I was hesitent to be more forthcoming until I built and tested the used hardware I've bought, but it's working now. I can confirm that I should be able to run a Debian or Mint VM with a graphical interface dedicated as a pitboss host, and keep this separate from my own home network. This means that the pb-watchdog should function without alteration, and this is limited by my capabilities as an admin (that's a pretty big shortcoming).

I will keep you updated on Ramk if I hear anything.

For NatNeg you need a public IPv4 address dedicated to the host (vm) running the server, not only the local router. Well, I can't know for sure, but I highly doubt it would work stable behind nat, no matter the port forwarding. NatNeg is too flaky as is... That is very rare to have in a home network. The address should also be stable given that many players prefer to use a hosts file rather than the patched exe. That's the reason I am rarely updating the server despite the outdated hardware, because I can't keep the IPv4 address.

The hardware and bandwidth is not really the issue. I use a vm with 6 threads of a i7-7700 and 12 GiB memory for all the current PBs, web, lobby, natneg. It is on a 1 Gb symmetrical connection. I have about 10 GiB outgoing 2 GiB incoming in total per month when the watchdog is running properly. PB should also run fine behind proper port forwarding (I do that in docker anyway). There have been some network issues with fragmented UDP packets, mostly for large PBs where the player name list gets very long.

The trouble with running the web server for save-sideloading is just that the setup of properly exposing the right directories is not smoothly automated.

Zulan, thanks for the explanation.

Am I right to assume then that there is only about 10 GiB of space needed for additional saves per month? And is this for all of the log in and log out saves? I'm just thinking about longer term storage needs and automating the duplication of saves from the VM to the longer term storage for archival purposes.

You said earlier that there may not be anything stopping the game server being hosting on another network, and still using the nat neg on your server. Is there anything stoping us setting up a redundant nat neg server on a third persons home network behind a DDNS?

Thanks again for your advice on this.

[Agrippina]

Crazy idea: how about a pre-made sdcard image for a Raspberry Pi3B (Raspbian Stretch, 32-bit) that includes a working CIV4 Pitboss setup?
This could decentralize CIV4 Pitboss Hosting.

Something similar was proposed in 2017 on civfanatics, titled: "Headless Servier":
https://forums.civfanatics.com/threads/h...er.612866/

[scooter]

Crazy idea: how about a pre-made sdcard image for a Raspberry Pi3B (Raspbian Stretch, 32-bit) that includes a working CIV4 Pitboss setup?
This could decentralize CIV4 Pitboss Hosting.

Something similar was proposed in 2017 on civfanatics, titled: "Headless Servier":
https://forums.civfanatics.com/threads/h...er.612866/

https://github.com/civ4-mp
https://github.com/civ4-mp/docker-civ4-pitboss-server

This is pretty much what exists in a different form, but it's just not been refined to be super easy to plug-and-play. Zulan/Ramk and co basically built tools to run a headless PB instance that can run as a docker container. It's just not super resilient and requires a lot of knowledge of various gotchas to get it fully up to speed, and currently nobody active has a total grasp on every piece of it. I have a decent handle on most of it, but some parts of getting it to run modded and headlessly without spawning errors is still a bit of a mystery to me. Unfortunately, the most knowledgeable person (Ramk) is inactive these days.

[Zulan]

Crazy idea: how about a pre-made sdcard image for a Raspberry Pi3B (Raspbian Stretch, 32-bit) that includes a working CIV4 Pitboss setup?
This could decentralize CIV4 Pitboss Hosting.

Something similar was proposed in 2017 on civfanatics, titled: "Headless Servier":
https://forums.civfanatics.com/threads/h...er.612866/

Sometimes it's easy to forget we're talking about a 19 year old proprietary Windows application. Civ4 is a x86 binary, the arm processor in Raspberry Pi systems cannot run that natively, but it's not powerful enough to emulate it. And it's still commercial software, that is still being sold, it would be risky at best to package and distribute that freely (even if only for PB).

As mentioned by scooter, the Dockerfile repository is the closest thing. However, it is designed to work with the PB mod, which is kinda a requirement for headless hosting anyway.

[Zulan]

Zulan, thanks for the explanation.

Am I right to assume then that there is only about 10 GiB of space needed for additional saves per month? And is this for all of the log in and log out saves? I'm just thinking about longer term storage needs and automating the duplication of saves from the VM to the longer term storage for archival purposes.
Sorry that answer got lost. FWIW here's the info.

The 10 GiB is network traffic, not space. The saves are pretty small, I think I have about 100 MB for 8k saves (login/logoff/autosave) total, you can prune them if needed.

You said earlier that there may not be anything stopping the game server being hosting on another network, and still using the nat neg on your server. Is there anything stoping us setting up a redundant nat neg server on a third persons home network behind a DDNS?

Thanks again for your advice on this.

NatNeg won't work on a home network because. I mean, I don't know that for a fact, but if there's one thing that probably doesn't work behind a Nat. It's NatNeg.

Technically, there are three different domain names for Nat so that could probably run on different servers. Whether that actually works in a positively redundant way, i.e., the system works fine if one server goes down, we don't know. It may just be multiple dns names for load balance, but not fault tolerenace. It may also be that the servers would have to share state, which is not implemented in our NatNeg server.