[NO PLAYERS] Seeding and ranking discussion

[AdrienIer]

Small modification of my idea : why not use the system used to select papers for science conferences ? They give a grade out of 5 for what they think of the paper and another grade out of 5 for their knowldedge of the subject. The opinion of people who know more about the subject is weighted more of course. Usually people get to review papers in their field but sometimes mistakes happen and they grade themselves with a 2 or 3 to signify that they're not really reliable.

I'm suggesting that because even if you have only a vague idea of someone's worth you could want to rate him just for kicks (like Mardoc did with REM and Dantski). You just have to be honest in how well you think you know someone's level.

How would people feel about grading their own knowledge of a player's skill out of 5 and the player out of 10 ?

[Commodore]

Player Roster:
- Old Harry (with occasional chirping from Fintourist)
- Molach
- Ipecac
- Gavagai
- Dantski and V8mark
- TheWannabe
- Greywolf
- Alhazard
- ReallyEvilMuffin
- 2metraninja

I like this much. I'm going to rank by odds of winning this:

- Old Harry (with occasional chirping from Fintourist): 100%

Damnit. Well, that was a wash.

Okay, score 'em out of ten:

10/10 Old Harry (with occasional chirping from Fintourist)
8/10 Gavagai
6/10 2metraninja
5/10 ReallyEvilMuffin
4/10 Alhazard
4/10 Molach
4/10 Ipecac
2/10 Greywolf
2/10 Dantski and V8mark
1/10 TheWannabe

Despite some tied ranking, those are also my ordering from best to worst.

[AdrienIer]

I'm going to rank by odds of winning this:

- Old Harry (with occasional chirping from Fintourist): 100%

Damnit. Well, that was a wash.

The whole point of this thread is to make sure this isn't true

[GermanJoey]

I'll do one of these for REM and for ah, 2metra or Ipecac. Would be great if someone could cover a few of the others.

Gavagai

Strengths: Extraordinarily resiliant, always scheming how to achieve his goals. Very strong all-round player, very experienced in pitbosses. One of RB's best tacticians. Capable of running a huge empire.
Weaknesses: Sometimes overly tunnel visions on a goal, especially when that goal is to destroy someone - even if to the detriment to himself. Not very good at reading opponents' intentions sometimes. Not the best mid-game economy management; often starts falling behind the tech leaders and then has to scramble to catch up.

Notable recent performances:

* PB16 - http://realmsbeyond.net/forums/showthread.php?tid=6497
Gav got off to a good start but then couldn't help but snipe a pair of workers from Gaspar/NH with a chariot, which drew their vengence for reasons Gavagai didn't understand. (see Weakness #1). He eventually started to recover by conquering his neighbor Bantams but by that point SevenSpirits had a gigantic lead and the game was called.

* PB17 - http://realmsbeyond.net/forums/showthread.php?tid=6617
In this game, Gavagai partnerned with Zanth. He doesn't post much early but a writes bunch of great war reports later. They eventually win the game.

* PB21 - http://realmsbeyond.net/forums/showthread.php?tid=7056
Executes a nice mids rush with Gandhi. This game was eventually called due to inactivity.

* PB22 - http://realmsbeyond.net/forums/showthread.php?tid=7173
The epic modern era spacerace that is close to my heart. Gavagai claws out of a tough map situation to eventually conquer HAK and DonovanZoi (replacement for mackoti). He eventually gets dogpiled, still survives, but is edged out to space by TBS. (replacing Gawdzak)

[Bacchus]

It seems I cannot edit the thread title, which is a shame, added in a NO PLAYERS bit into the body though.

My first ideas on mechanism design are these:

Upon initial discussion we will get a feel for how many tiers of players there are. Once we have the map, we can pre-assign starlocs to "danger" tiers, which in turn will let us rank the starlocs, as we will have a general idea of how strong/aggressive the neighbourhood is. I do think that the for the purposes of this ranking we should take aggressiveness seriously, somebody like Jowy raises difficulty almost as much as, say, Krill. This makes it independent of the quality assessment.

Starlocs will be edited to make sure that there are as many of them in a particular tier as there players.

The tiered approach can also help us out on cutting unnecessary discussion on where exactly to place each player. Say there are 3 tier 1 players and 3 tier one locations -- we can just assign within these randomly. Of course we can then edit for FUN.

On the proposed 1-10 scoring to be averaged across voters

There are a couple of issues with this. The first, it cuts against the tiering system. We are likely to end up with something like:

Code:

Pisces - 9.2
Leo - 8.9
Taurus - 8.8
Gemini - 8.6
...

There will probably be no clear cut-offs, which would mean that each starloc would have to be linked to each player in a very rigid way.

Then there are some really weird biases coming from taking the mean:

Gemini (10, 4, 4, 3) would be ranked the same as Leo (6, 5, 5, 5). This mechanism really motivates to give all players either 0s or 10s, as doing so maximizes the effect of your vote. The advantage of a close-knit community built on mutual trust, though, is that we can expect people to give a genuine ranking, not play the system.

I also feel somewhat uncomfortable giving such a precise rating — it's assigning too much certainty to a subjective assessment, can I really differentiate between a 2 and a 3?

My idea at this stage would be to start with a 4-tiered set-up: ("Most trouble for neighbours", "Material concern for neighbours", "Manageable neighbour", "Least trouble for neighbours") for the purposes of adjusting starloc quality and a similar 4-tier set-up for actually assigning starlocs: ("Most handicapped", "Fairly handicapped", "Fairly boosted", "Most boosted"). For this, a 1-5 score would be sufficient. All people with an average between 1 and 2 get assigned to the bottom tier, between 2 and 3 to the next tier, and so on.

How do we deal with lack of info/abstentions?

Accompany your vote with a qualification of "Confident", "Fairly confident", "Not confident". If we have at least three confidents, we take their vote, and in need of a tie-break consult the "Fairly confident" votes. If we don't have at least three confident votes or at least five "Fairly confident" votes, we assign "Manageable neighbour" and "Fairly boosted" tiers.

[GermanJoey]

you need to do a full edit (not a quick edit) on the first post to edit the thread title.

[Grimace]

you need to do a full edit (not a quick edit) on the first post to edit the thread title.

You can also do a click-hold from the pitboss 30 forum list on the thread title for this thread.

[Grimace]

My $.02

Ranking players is ALWAYS going to be subjective, and every single lurker is likely to rank them differently if they operate in a vaccuum (seeing other people's rankings may influence your choices). Starting locations are a much more objective measure, so that's easier to do, but let's be honest -- it's going to be a "do our best" situation no matter what method we use.

Also, civ choice is going to have a significant impact on the quality of start locations, so I personally would wait to rank the players until after civs have been chosen/assigned -- in this game, knowing the start location shouldn't matter since it'll need to be customized based on their civ choice and "handicap" anyway.

To be honest, I would just have each lurker who wants to be involved rank each player (secretly -- PM rankings to whomever is going to manage it) from 0 - 5 on both their skill and their civ and average it across all of the submissions. Will it be perfect? No. But it's easy to do and should be "good enough".

[The Black Sword]

To be honest, I would just have each lurker who wants to be involved rank each player (secretly -- PM rankings to whomever is going to manage it) from 0 - 5 on both their skill and their civ and average it across all of the submissions.

I thought we have to come up with some player rankings and then use the civ pick/starting positions to even things out? I think player skill carries more weight than civ/leader pick in any case.

My rough thoughts:

Tier 1: OH
Tier 2: Gav
Tier 3: 2metra, Alhazard, ipecac, Molach
Tier 4: Greywolf, Wannabee

My knowledge of Dantski is vague memories of old games where he wasn't in contention. So I'm not that confident ranking him but I guess I wouldn't go above Tier 3.

In general I think anyone in the same tier has a decent chance of beating others in that tier(in a FFA though, not a duel). I would bet on someone in a higher tier to win the majority of their games against someone in a lower tier. I'd have to think a bit more to differentiate between tiers.

I'm dueling Greywolf in FFH btw and he didn't go worker first. Maybe that was just due to the FFH setting though. Since then I've been choking him with invisible Sidar units and he hasn't had many opportunities to make moves, good or bad.

[Mardoc]

in this game, knowing the start location shouldn't matter since it'll need to be customized based on their civ choice and "handicap" anyway.

I think you have a different mental model for the process than they actually want us to use. The process I think they're asking for is:
1. Commodore rolls the map, puts start pictures in here
2. We assign each player a start location, and three civs to choose from
3. We buff/nerf the starts as we think appropriate.
4. Post in everyone's thread their starting screenshot and civ/leader options
5. They pick which option they want
6. Commodore edits in picks to the final map and starts the game.

In particular, they want to know what their starting spot looks like before they choose their civ, which means we can't change the start after they've picked civs. But I think we're allowed to assign civ choices as part of the handicap - we can make Old Harry pick from America, Japan, and Russia, while TheWannabe picks from India, Inca, and Egypt.

rank each player (secretly

I don't see the value in secret rankings, myself - not unless you would self-censor to avoid hurting feelings after the game but be honest in secret. I feel like discussing people will bring us to accuracy a lot more than averaging our ignorance. A lot better to share knowledge.