PDA

View Full Version : ANGBAND: Challenge Leaderboard


Qaj the Fuzzy Love Worm
19-03-2002, 22:15:38
Player/Round AVG AVG-A AVG-S Rank Rank-A Rank-S

Sir Penguin 31.5 31.5 31.5 1 1 1
Qaj 21.5 21.5 20.5 2 2 2
MDA 21.5 21.5 20.5 2 2 2
Darkstar 14.5 14.5 14.5 4 4 4
GregW 4.5 4.5 3.5 6 5 5
DaShi 14.5 0 0 4 6 6

Current Official Scoring Rules (subject to change)
The Angband Challenge scoring system is a rank-based system that is based on averaging of subjectivly modified Character Level (CLevel-S) as depicted in the final Character Dump (CD) of Challenge entrants.

Pure CLevel is the base of the subjective score, and is, as of writing, modified in the following ways:


Dead characters will have two CLevel-S points removed
Obviously superior characters will earn additional CLevel-S points
Late/disqualified dumps will be worth 0 CLevel-S points


There is no such thing as a tie-breaker in the Angband Challenges. Any equal scores will result in a tie.

Overall ranking is determined by averaging the CLevel-S scores from all rounds thus far played. The higher the average, the higher your rank.

Qaj the Fuzzy Love Worm
19-03-2002, 22:17:26
Anyone have any improvements or ideas as to how the leaderboard can be "pepped up" a little? Any information that's missing? As of the very first post, there's not a whole lot of information to display. Maybe I should stick my stats spreadsheet up on the web or something...

Sean
19-03-2002, 22:42:58
How is the rank calculated?

Darkstar
19-03-2002, 23:48:51
That's a damn good question, Sean.

And Where is number 13? Just got my taxes to finalize, so I've got time to kick all your people's lily white candy asses! :D

Qaj the Fuzzy Love Worm
20-03-2002, 17:48:55
The rank is calculated like this:

Each player is given an individual rank for each challenge. In Challenge 11, rank was from 1 to 5.

With round 12 it becomes a little trickier, since the number of total participants becomes 6. What I've done to calculate rank here is to add DaShi to the list, assign him rank 6 for round 11, and assign the no-shows (and DaShi, since he was disqualified) as rank 6 for round 12. So we end up with:


Player Avg. Rank 11 12

Sir Penguin 1 1 1 1
Qaj 3 2 3 3
MDA 3 2 4 2
Darkstar 4 4 2 6
GregW 5.5 5 5 6
DaShi 6 6 6 6


Average is just a simple average of ranks over the two rounds, and Rank dervies directly from average.

Any new players will automatica lly be assigned the lowest possible rank in each of the past Challenges. So, if next round the number goes up to nine, Persons A, B and C will be ranked 9 for Challenge 11 and 12. I've done this so that newcomers can't jump to the top of the ranking system by winning their first challenge (1/1 = 1, wheras SP would be 2nd, with 1, 1 and, say, 2 => 4/3 = 1 and a third). They have to work their way up the table. But, a high score in Challenge 13 by person C would net them 19/3=6 and a third, which could leave them somewhere in the middle. Further good performances would carry them up the board.

I couldn't think of a fairer way to do it at short notice yesterday (I was busy) so this is the tentative ranking system so far. Criticisms and comments welcome.

Sean
20-03-2002, 18:21:08
I can see that being a little unfair if someone just loses, or, alternatively, if someone beats the rest of you by a country mile.

Also, I like the way MDA is ahead of you (not in rank, just in order), because he has a higher finish. Good performances should be rewarded more than finishing 3rd all the time.

Qaj the Fuzzy Love Worm
20-03-2002, 20:35:27
Re: the MDA thing, that's true, but right now I've got only my own subjective judgement to rely on, which could be faulty sometimes :) I'll look at your points and try to incorporate them into some kind of mathematical formula to use instead.

But then, I'm also wondering if a difficulty rating should be assigned individual characters. But I wouldn't want to have to produce a chart of all combinations to draw on, and that would be a requirement to determine minimum and maximum difficulties prior to any contest.

Sean
20-03-2002, 20:38:13
You could just ask the challenger to rate them on a scale of 1 to 5 (or 4, or 3).

Lady_of_Chicken
21-03-2002, 06:27:54
[Qaj posts]

Sure, but then that doesn't necessarily mean that the creator will get the difficulty right.

But now that I think about difficulty, I realise it doesn't matter, since everyone's scores for that round are already automatically scaled by difficulty - no-one operates a character any more or less difficult than any other. The total and average will be affected, but will likewise be affected the same for everyone. Thus there's no real reason to include difficulty as a factor.


I created a fresh spreadsheet of results today at work, and checked out some averages and how it affected ranking. Perhaps I'll post it tomorrow. But what I did was created two new columns for each Challenge. In the first was the raw scores, based on CLevel only. The next column was actual scores - adjusted for disqualifications. The third was subjective scores, adding or subtracting (n) CLevels depending on non-CLevel modifiers (such as DaShi's wonderful progress with Volray in round #12, which I unfortunately couldn't test out properly since he was disqualified :) )

The third way came out with the better results, though it left MDA and myself with exactly the same average, and thus the same rank with no difference at all (but MDA would be first on the ranking since it's alphabetical :) ). The only part I don't like is that it's a subjective modification again. Unless I can work out some hard and fast modifiers based on stats, max recall level (not every variant displays this info.), equipment, etc. it's open for abuse. Contestants will either have to abide by my decisions, or I could throw it open to a vote on score modification for everyone. The last option seems like too much hard work. The former option means working out mods for every variant as we play it.

So again I'm stuck with thinking about it :D


Oh, and Darkstar, go bug MDA :) It's due next Wednesday.

Darkstar
21-03-2002, 06:51:51
Questions, questions, I have questions...

What is the point of the challenge? Highest CLevel, period? Highest Clevel and still alive?

We should set this.

Doing the ranks would be easy. Give the Winner of the round 1 point per player he beats. Go down the list like that. Generates a POINT total.

Do the same for #12.

But it needs to be based on the objective.
Obviously, if you beat the game, you win the challenge (and if more then one does it, they tie as winners).

#11 seemed to be about pure CLevel to me. Living or dead. Isn't that what #12 was? Most advanced CLevel?

MDA
21-03-2002, 17:08:08
I'm rolling around the idea of a Klackon Ranger (MOO!) for the next challenge. I don't know how their racial speed bonus works yet, but they have nature magic, which I like, and bow attack rate bonuses after passing certain levels, also cool. Carrying around loads of arrows to take advantage of that isn't quite as cool. A ranger shouldn't be quite as much trouble as a vampire chaos warrior, most of the winner dumps I've seen on the newsgroups are rangers (admittedly ELVEN rangers) so I know they're viable.

Rest assured I'm thinking/experimenting with class/race combinations for the next challenge. It will almost certainly be a [Z] challenge, and I see no reason to make everyone download another version than the one they already have.
If anyone has any complaints, I might be convinced to make a golem mage.:)



Prepositions are bad things to finish sentences with.

Qaj the Fuzzy Love Worm
21-03-2002, 18:57:58
#11 was pure CLevel, since the characters that were dumped weren't qualitatively different (at least, weren't significantly qualitatively different) from each other to necessitate fiddling with the pseudo-score.

In other words, all the characters were around the expected sophistication for their CLevel.

(In fact, the previous 10 rounds, which were held at various places on the Web such as KittyLand and the Angband forum I set up on EZBoard) were much the same.

It all changed with #12 when, after DaShi's post, I came across a character qualitatively different from the others. DaShi's Volray had superior stats and equipment for such a low CLevel character.


The criteria I've been semi-consciously applying to the characters has been ability to reach the end of the game and defeat the end of game boss (depending on variant). Now, in #12, DaShi was clearly in a better position (all other things like strategy, game play style etc. aside) to win than SP at that point in time. SP had one CLevel up, but DaShi was better able to survive. So, DaShi earns some subjective points on that one, or would have.

Dead characters are, obviously, in no position to win. However, allowances are made to chart the progress of the character also. I've been toying with the idea of subtracting 2 subjective CLevels as a penalty for a dead character.

So, in the end, the round-level ranking is based on both expected future, and demonstrated past, performances.


Overall ranking is a little different. I like the average of all subjective scores being the criteria for ranking. Though, the method you've suggested (1 point for each player you beat) gets me exactly the same total ranking as I've generated using my methods so far :) However, this doesn't take into account, as Sean points out, cases where one player trounces all his opponents by a huge margin. If SP had gotten CLevel 49 when all others were stuck on CLevel 5, he deserves to have that boost propagate through the entire contest for being 10 times better.

As it stands, with my subjective scoring, SP is already doing about 33% better than MDA or myself, and 66% better than Darkstar. Consistency counts for something, too. It'll take a few good efforts by the rest of us to catch up. With the 1point-per-player system, MDA only needs to win #13 and #14 to be on an even footing, and only needs to do it by the skin on his teeth. But, if he does really well (say, he finishes the game twice, while SP only gets CLevel 3) he still ranks the same, ie. equal, even though two of his performances have been outstanding.

And now, to post the Current Official Scoring Rules (subject to change) into the top post :)

Darkstar
21-03-2002, 19:16:06
I don't think subjectivity is any good.

So what if you are in a superior stat/survivability position? Won't help a Chaos Warrior if that patron decides 'You so funny! Get in my court now!' (You are dead.)

Or get a lethal mutation...

Or...

Just a point.

I've had CLevel 24 characters just fall over dead walking to the next city. Great AC, Great Stats. Very nice EQ. A quick one time message 'It hits.' and the character is dead in ZAngBand already. (I don't care what you guys say... the wilderness is WAY more dangerous then the dungeon in ZAngBand.)

It's your score board, but unless they WIN, it's just a wild guess on your part on whether they are really in a better position to win or not. Unless it's played to win, there is no way of telling. The Random Number God is bound to get unkind, after all. ;)

And us working stiffs are not going to win the game in two weeks. Maybe on a 2 week vacation, but not in a 'play for 2 to 6 hours a night' deal. ;) But for a win, *double* the points of that competition then. That effectively ranks them #1 for that sized match, twice, as a reward. Large enough? Consistancy will pay off purely in the point total.

And if 2 more people join into the next competition, your system is screwed. Point total makes it easy, versus what you are doing. You are having to average in players that didn't play that round, for whatever reason. So, you'd end up doing a lot of back averaging, wouldn't you?

Say Shining1 and Lady Rachel decide to join in the next match... skip the next 4, and then come back and play the following 4. That's a whole lot of tracking going on for you... Point total makes it easy.

Look at the K Hunts. People come and go. Consistancy is the rule for the top of the board though...

Just some things for you to consider...

Sean
21-03-2002, 19:23:37
DS, that is bull. Someone with better stats is in a better position to win, if the numbers are random, because although everyone has the same chance of being killed by their Deity, the person with the best stats has the best chance of winning if that does not happen.

Qaj the Fuzzy Love Worm
21-03-2002, 20:10:54
And don't forget that only chaos warriors are subject to the whims of their patrons. Other characters (the other 10 or so classes) aren't. Not not mention other variants. The problems of chaos warriors are insignificant in this tournament (unless the next ten characters are CWs, which will just plain suck :) )

But to fully counter your assertion, I could be playing a Golem Warrior, and come face to face with a horribly out-of-depth cyberdemon on DLevel 60. One rocket, and you're meat, even with 1000+ HP, especially if you don't have the right resistance. CWs are a little more random than most, but they're equally random across all players.


Next point. Your point system won't help any new players, or reduce backtracking. As it stands, all new players (and players who skip a round) automatically get a rank of total_players. All prior scores are set to 0, for base CLevel, Actual CLevel (CLevel after disqualifications), and CLevel-S (CLevel-A after modification). The average of CLevel-S determines the rank.

Back averaging and back scoring isn't a problem with a spreadsheet. Copy and paste is quick and effective. It took about 30 seconds to update my table to incorporate DaShi. I had bigger problems making the thing readable (which I've done to nice effect now) and working out someof the subjective score modifications (which I'm still evaluating).

An averaging system is superior to a system of assigning serial integers as points, becuase it rewards consistently better play. The more you play and the better you play, the better your average, and thus your rank. An integer system "flattens" the differences between players and rounds, and doesn't allow for an uneven distribution.

Neither system benefits no-shows, or latecomers, and that's the wy it should be. No-one should join the tournament late and immediately be positioned in the middle or top. Only hard work and good play will elevate you. In other words, if you want to lead the board, you need to be serious about it. Just like the KHunt leaderboard. You don't play, you don't score, you move down. You come in late, you start at the bottom and work your way up.


Thirdly, it IS possible for working stiffs (I'm one!) to win the game. In fact, a while back the larger community of Angbanders had a challenge where they attempted to beat the game in under a million turns. Surprisingly, some people were successful, and there was even one, I believe, who won in under 500,000. Boggles the mind. Of course, you have to be really good, or really lucky to do that. It also means you have to minimize your rest periods, and rely heavily on ranged attacks. Not so difficult when you realise that there are a few good strategies for just those kinds of play styles, and Angband won't penalise you for it. Except to make battles a bit more tedious with regard to keystrokes.

Sir Penguin
21-03-2002, 21:47:52
Originally posted by Darkstar
(I don't care what you guys say... the wilderness is WAY more dangerous then the dungeon in ZAngBand.)


From rec.games.roguelike.angband:


Target Selected.
The Giant skeleton troll is hit hard.
The Giant skeleton troll is unaffected!
The Spectral Wyrm breathes nexus.
(No, it didn't. I just imagined it. There's no Spectral Wyrm there
anyway. So I can safely carry on zapping this troll.)
The Giant skeleton troll stumbles.
Your body starts to scramble...
Target Selected.
The Giant skeleton troll is hit hard.
The Spectral Wyrm breathes nether.
The Giant skeleton troll is immune.
(Lucky Giant skeleton troll.)

Nice tombstone: Killed by a Spectral Wyrm in the wilderness.


:)

That's for 2.7.0, though.

SP

MDA
21-03-2002, 22:09:11
Just a thought:

Don't most of these *bands have their own internal scoring system that appears on a dead character's headstone and in the morgue listing of best characters?

Or is the in-game scoring broken?

Qaj the Fuzzy Love Worm
22-03-2002, 01:09:10
You're right, it does. We could use score as an indication. It'd mean an extra few steps to submit at the end of the Challenge, however I'm sure it isn't terribly difficult.

The only problem I see with it is that gold is taken as a factor. Now, gold doesn't necessarily mean much except that you've found some nice stuff in the dungeon and sold it, or that you've been extraordinarily lucky in the casino. And after a certain point in the game, you can't do anything with it - all the stuff you find in the dungeon is better than you could buy anywhere but the Black Market (where it's far too rare to worry about anyway).

What I might do for the next Challenge is post the CLevel-S scores with the ranking as I see it, and let you guys squabble over who should be scored what (with the caveat that you can't vote on your own entry). It'll make things more complex and slow to finalise, but we might end up with something most of us can live with.

AzNmYsTiKaLdEmOn
22-03-2002, 05:09:41
Hey Qaj, remember me?

- ajwa@home

Qaj the Fuzzy Love Worm
22-03-2002, 23:40:09
I love the Ignore function :)

And to return to topic, the Angband Leaderboard (the FULL version) is available online at the Angband Challenge (http://www.geocities.com/qaj_stars/chal.html) webpage.

Darkstar
23-03-2002, 07:57:17
The legend's *title* bar in the leader board is messed up on my display. Everything else is good.

Sir Penguin
23-03-2002, 10:21:38
Just about everything's messed up in my browser :).

SP

Sir Penguin
24-03-2002, 04:47:00
Just a stray thought: are we allowed to post two dumps for one challenge, if they're from different games, or are we only supposed to post our best?

SP

Darkstar
24-03-2002, 05:42:34
It's your board. I disagree, but cannot go into details thanks to CG not accepting LP from my current location.

Whatever floats your boat...

DaShi
24-03-2002, 05:53:31
Qaj, looks fine to on mine. My name's a little low, but otherwise fine.

Darkstar
24-03-2002, 06:24:53
Doesn't really matter to me. I just think he's doing a lot of work for nothing, as there is a simpler system that will yield the same info that most people would be interested in. If it makes him happy though, then it's no skin off my chin.

MDA
24-03-2002, 15:25:46
I need an address to send this challenge to :)

DaShi
25-03-2002, 00:02:51
Originally posted by Qaj the Fuzzy Love Worm
It should be :)

SP, send it to bogusantispam_ac@ausi.com

Without the antispam, of course :)

I'll put the official thread up sometime this afternoon, if I get it in time. But right now I've got some stupid IT meeting to go to. Bleah.

From the #11 thread.

MDA
25-03-2002, 18:00:36
Ah, then I should have something for Qaj in a few hours, once I'm home.

Qaj the Fuzzy Love Worm
25-03-2002, 22:15:32
Hmmm, I thought I posted similar info. in the #12 thread too. Now, if only my mail site would let me in...

MDA
26-03-2002, 02:25:21
A few hours, indeed. :nervous:

Qaj the Fuzzy Love Worm
26-03-2002, 20:44:56
Got it. New thread will appear TODAY.

Darkstar
26-03-2002, 22:12:38
Where? Where? :D