Defeating cheaters - is the solution not to care?

Networking & Multiplayer
#1
The consensus seems to be that trying to detect cheats on the client side is, in the long term, futile. In addition, the base problem is that cheaters are [i]assholes[/i]. Even with perfect cheat detection, assholes are still assholes, and will find some other way to disrupt the game. So now the problem of identifying cheaters is subsumed into the broader problem of identifying assholes.

Identifying cheaters is largely a technical challenge: you scan their machine for hacked graphics libraries, memory change tools, etc. It’s difficult to do in a non-trivial way, and at the end of the day the client can’t be trusted.
Identifying assholes is something that players can do really, really easily. Everytime you see someone speed-hacking their way around a counterstrike map, your first thought is unlikely to be “Hey, look at that packet-injection attack!” but instead “Hey, look at that asshole!”

An approach that I’ve been thinking about is to simply ignore the whole cheating issue, and instead have a reputation system so that you are more likely to find an enjoyable game
It’s basically the same as the Credence system for combatting peer-to-peer spam.

How it breaks down:
Everyone has a cryptographic identity, and can give ratings to other players:

  • thumbs up - good guy
  • thumbs down - asshole

These ratings are persistent and are shared between clients, allowing one client to make a judgement on the assholeness of an unknown client based on the ratings of others. If client A’s ratings are broadly similar to client B, they are likely to have similar opinions on who is an asshole, and so A can trust B’s judgements on clients unknown to A.

Entry to a server would be based on the judgement of the server owner, or perhaps even the average judgement of the current inhabitants.

And now for the inevitable strained analogy: The server is a nightclub with a bouncer that permits or denies entry based on the opinions of the owner/people already inside:
“I don’t know you, but my good friends X and Y, whose ratings I generally agree with, think you’re a good guy, so I’ll give you the benefit of the doubt. Welcome aboard!”
“Nobody I know has ever met you before, but you’ve rated I, J and K as good guys, while I know for a fact that they are assholes. This has cast you in a bad light, so for now you’ll have to leave. I suggest that you mix with a better crowd in future”
“While it is true that A, B and C have rated you up, I do not generally agree with the other ratings that A, B and C have made, and so you’re not getting in”

The strengths of this approach:

  • It’s decentralised
  • It doesn’t matter if assholes rate each other up, as no-one cares about the opinion of an asshole
  • Everyone gets a say. No more people whining to the admin: they simply make their own rating, and if their opinion starts to diverge from that of the server, maybe it’s time to move on…
  • It’s not just applicable for finding cheaters, rate people on if you enjoyed playing with them or not, and watch as all of the foulmouthed spawn-camping chatspammers are forced to converge on playing solely with each other

Some problems:

  • Adoption - getting a system like this to a useful state will take time
  • People not voting on each other- It might be worthwhile to have a default positive rating as people are much more likely to bother themselves to vote an asshole down than a good guy up
  • There’d have to be some servers with permissive entry policies, so that new players can prove themselves not to be assholes before they can move on to servers with higher standards
  • On a busy server, there may be a lot of people coming and going, and so the saved ratings may grow very large. Some kind of time-based pruning would probably help here
  • People can still cheat as long as they’re not an asshole, so this probably isn’t suitable for competitive play

Any thoughts?

#2

Peer-to-Peer policing can be a good model I believe. I play in a web-based “strategy” game that has a very strong community mentality. It’s message boards and chat are very, very active with everyone joining up with an Aliance and Aliances with good reputations getting support from other Aliances when needed…

A key point on this is that “everyone knows everyone”. When new players come into the game they usually develop some reputation ( good, bad or indifferent ) based on thier play style and how they behave in the public forums. Players who are serious jerks could find themselves being booted from thier aliance, not allowed to join aliances ( no one wants them ), attacked into oblivion or even banned from the game. If they rejoin with a new ID they often show their true colors again and wind up in the same situation. Actual cheaters or really rude players will have thier IP banned and some pretty successful / active policing which keeps them from sneaking back in.

Some of that is obviously peer pressure while other parts ( game & forum bans ) are controlled by some moderators. I have been giving thought to a model that would automate some of the moderator-style tasks based on obvious cheating criteria ( comparing known values and statistics to expected ) and a peer rating system. Your suggestion is very similar ( and far more detailed in the peer-ranking side ).

There is always the concern that a player that just “plays good” and wins a lot may find themselves ranked negatively because other players will think they had to cheat ( 'i’m too good to loose that bad!" ) or simply are pissed off for getting wooped. So if you are controlling things at a server level there should be some method of appealing a ranking.

Another concern would be a player that finds themselves racking up the thumbs-down and hacks out a new unique ID. Whatever method you use to generate, assign and validate that ID would be open to the same hacks as any registration code. Of course, that player would then loose all of their accumulated score and rankings and start fresh if he did that.

#3

It could be argued that ranking down out of spite is the behaviour of an asshole, and so being ranked in such a fashion is not the problem it first appears.
If the spiteful party is alone in ranking you down, all that is achieved is that he distances his ratings profile from that of the server, and so his opinions will carry less weight on that server. If the whole server responds spitefully, or the spiteful party controls the server: well, being denied access to a server full of assholes isn’t a huge concern.

Indeed, players need to have some value associated with their ID.
For example, if you use cheats in CounterStrike and are detected, Valve will close your account, and you’ve lost access to the games you paid for.
For free games this incentive is not possible, and so the value must lie in your accrued reputation. Cheaters and other assholes will want to disrupt servers of good gamers, but they will need to accrue enough ratings from good gamers in order to gain access. They are free to keep generating new IDs, but they will have to put in some work, thus providing the value that they lose if they then revert to type. If they act as assholes with a newly-created ID, they will quickly find that everyone they play against are also assholes.

This does highlight the need for proving ground servers that new players can access, and that are also used by more highly-rated players, in order that the new players can gain some valuable rankings.
Perhaps it could become a rite of passage that the ascending generation of gamers could spend time playing against the next, sorting the assholes from the good guys, before leaving for a higher plane of asshole-free gaming in the highly rated servers :slight_smile:

#4

The accumulated rankings would definetly drive people to wanting to maintain thier IDs. Sort of like the pride regular ebay users have in their rankings. If you cheat people a lot and keep creating new IDs then you just won’t get the feedback scores that some people require before doing business with you.

Thinking about this, if you developed your peer-rating-system in some form of open, pluggable fashion then it may be even more powerful. Often players don’t stick with just one game but rather pick up a few. If there was a central storage of peer ratings that games could plug into ( similar to game lobbys but purely for exchanging ratings ) then it may carry more weight for the players. This would also help reduce the need for “proving ground” servers since players will start accumulating rankings across multiple games.

If you behave well in one set of games it gives you a “leg up” with your ranking in any other game that uses the same repository. This would also mean if you acted like an @$$ in one game it could hurt your rankings elsewhere. That brings more of a real-world set of social norms into play. If you’re an @$$ at work or school you are not very likely to be invited over to a co-worker’s cookout asked to join some classmates going out for pizza.

So…when will the specs be ready for us??? ;D

#5

Sharing ratings across games is interesting, but might introduce a few problems. For instance: sprinting around at full speed going hog-wild with a boomstick is the norm in games like Quake3, but is likely to be frowned upon in a slower-paced, more tactical FPS. Thus a player may be denied access to a Quake3 server where his behaviour would be perfectly acceptable simply because his style of play annoyed people in the slower game.

[quote]So…when will the specs be ready for us?Huh? Grin
[/quote]
As soon as I find someone else to write my thesis I suppose ;D

#6

This is a really interesting idea. Definitely worthy of a java.net project… or some sort of DarkStar plugin :slight_smile:

#7

It may be something to consider using a ratio vs. flat number-based score system…or a combination of the two. If you have only played ten games but all ten gave you a good rating, that may be better than a player that has played thirty games and recieved thumbs-down on twenty of them. They each have ten thumbs-up, but one is pretty erratic. Maybe showing it as:
10/10 ( 100% )
10/30 ( 33% )

Also, to avoid the problem of game-specific behaviour it may be good to have more than just a single thmb-up / down option. Or perhaps to keep it more simple for the user ( who we rely on to provide the rankings ) the games could register as certain styles and there could be a couple rating options within those.

Some sample game caterogies:
Game Category - Fast Paced FPS
Game Category - Tactical FPS
Game Category - Fast Paced RTS
Game Category - Strategic-paced RTS

Each would have these rank categories
Rank Category - Agression
Rank Category - Fair Play / Sportsmanship
Rank Category - Teamwork

So, a player that ranks high in agression and teamwork may do great in either of the Fast-Paced games but would probably not get easy access to a server running a Strategic-Paced RTS that has a cap on those values. However, a player with a low Fair Play score would find that hurts them across all game genres.

Game designers would specify the general category of their game when they have it register with the ranking repository, and whoever is running the game server would set the min / max ratios for thier particular server. So if you want to only allow players with at leatst a 75% Fiar Play ranking to help ensure people know you’re the place to go for a game without much in-game rudeness.

#8

I strongly suspect that the system needs to be completely brain-dead simple in order to get significant adoption. As soon as you start asking the player anything more complex than “Do you want to play with this guy again?”, I think you’ll see people just not bothering to vote. It’s a cardinal rule of user interface design - “Don’t make the user think!”

Looking back at my first post I see I can clear up some details:

When one client is asked to make a judgement on another, it first looks in it’s own ratings and sees if he has encountered it before, if so, the stored rating is returned and all is well. If not, A will then consult his peers (Which may be other clients on the same server, ratings files from other players that are cached locally, or perhaps even just a random selection of other clients playing the same game, discovered using some kind of p2p overlay) and ask them “What do you think of this guy?”. The peers will then make their own judgements using the same method (obviously there’ll have to be some limits to the depth of recursion), and return them along with their ratings files. The original judging client then takes each response, and weighs it according to how similar the corresponding ratings file is to his own, takes an average, and the judgement is done. Each server has a threshold value that applying clients must be judged to exceed for entry to be granted.
Example: A is asked to judge B, and so asks C, D and E for their opinions. C has played against B before, and rates him thumbs up, or 1. D hasn’t played B before, and so goes off to consult with his peers, eventually coming back with a rating of 0.8. E has also played B before, and took a dislike to him, rating him at -1.
A then compares the ratings files of C, D and E to his own. In the subset of clients that A and C have both directly rated, they agree 90% of the time, and so the final rating from C is 1 * 0.9 = 0.9. A and D agree 100% of the time, for a rating of 0.8. E is somewhat petulant in his rankings however, and so only agrees with A 10% of the time, for a final ranking of -0.1.
So, A’s final opinion of B is ( 0.9 + 0.8 + -0.1 ) / 3 = 0.53
Obviously you can jiggle the numbers somewhat so that there is a threshold of like-mindedness before another clients opinion is valued, and that comparing ratings must have at least x valid points on which to compare, etc

Writing that out has just revealed to me that a dishonest client can fool another into taking his opinion more seriously by returning a fake ratings file that is similar to the requesting client’s. Hmmm, some thought needed…

Also from the first post:

[quote]“Nobody I know has ever met you before, but you’ve rated I, J and K as good guys, while I know for a fact that they are assholes. This has cast you in a bad light, so for now you’ll have to leave. I suggest that you mix with a better crowd in future”
[/quote]
Denying entry on the grounds of bad judgement is perhaps a little harsh. After all, maybe I, J and K were well behaved when the client encountered them. A clients ratings differing from that of the server shouldn’t be used as grounds for denying entry, but might serve as a useful indicator that they might not get on so well in that server…

At any rate, until someone build the thing, and incorporates it into a successful game with a large community, it’s all just talk :-\

#9

May I suggest implementing only positive reinforcement, that is, that you may not vote someone to be an asshole, you may only vote them to be good.

Cas :slight_smile:

#10

Positive reinforcement sounds good. Also, allow people to rate others only if they won.

#11

Using only positive reinforcement would mean that each client would have to remember everyone that they’ve played against, whether thay rated them or not. Otherwise everyone’s ratings profile would compare as being identical.
If the clients do remember everyone they’ve played against, the lack of a positive rating would become the de facto asshole rating, and ratings comparisons would suffer. For example: Client A witnesses client B cheating, and so gives him the worst possible rating, a zero. Client C also plays against B, but does not witness any untoward behaviour. So, in an effort not to tar this apparently perfectly good player with the same brush used for blatant cheaters, he rates B up. A and C’s ratings are now at odds, due to the fact that there is no effective option for registering indifference.
Writing this out has made me realise that having a default positive rating, as mentioned in the first post, is maybe not such a good idea.

Comparing ratings is a way to measure the extent to which two people are like-minded. Being able to compare likes and dislikes, as opposed to simply likes (and maybe indifference) will, AFAICS, give a more valid measurement.

As to only allowing the victors to make a vote, there are a number of problems:

  • Imagine that you were beaten by someone using cheats. You’d very much like to vote them down, but would be unable to. It would be an odd situation in which you were only allowed to say whether or not you enjoyed the game if you won it.
  • Rating someone is a personal choice. If someone wants to rate “wrongly” (ie: in a way that differs from your own), that’s their choice to make. All that will happen is that their opinion will carry less weight with you in future
  • At the end of the day, rating is done on the client, and so there’s no way to stop it

With regards to the exploit

[quote]a dishonest client can fool another into taking his opinion more seriously by returning a fake ratings file that is similar to the requesting client’s
[/quote]
I haven’t thought of a foolproof way to combat this, but the risk can be mitigated by

  • Only asking people whom you have rated up for opinions. They are hopefully less likely to be dishonest
  • Asking for a larger number of opinions, so that the dishonest opinion does not affect the final answer so much
  • Perhaps some sanity checks to determine if a particular client’s ratings file is always suspiciously similar, despite changes in between opinion gathering
#12

How easy is it to create accounts?

If I can easily create many accounts the whole scheme falls apart because I can…

(a) create a bunch of dummy acounts just to “vote myself up”

or
(b) create a new account in order to avoid all the down votes I’ve collected.

#13

Agreed, but a client A will only value the opinion of the dummy account if:

  • The dummy accounts ratings correspond closely with it’s A’s. This admittedly can be spoofed.
  • The client has encountered the dummy account before, and rated it positively. This cannot be spoofed

So, in order for a dishonest client to get a good rating, he would have to have to create enough dummy accounts in order to overpower the opinions of the honest clients, and then also spend enough time playing each dummy account honestly, in order that they are rated up. This is a significant effort.
Assuming this is done, you only need to spot him cheating once and the effort is wasted. You will no longer care that the dummy accounts rate him up, as you have first-hand knowledge that he is a cheater. Also, the opinion of the dummy accounts is now tainted in your eyes, and so you will value it less in future.

Hey, even assholes deserve the chance to make a fresh start. This will mean that new players will have to put up with some amount of asshole behaviour on the proving ground servers, but assuming they weather this, they should be able to look forward to asshole-free gaming on higher-rated servers.

#14

I guess my concerns with sucha system come down to
(a) Manipulation. As others poinetd out players will often weild such a systemas just another ‘weapon’ against those they dislike for personal reasons.
(b) Overhead. If you make it too easy to play with othersm this systemj is worthless. if you make it too hard it discourages community building which is job #1 in an RPG.

How will people ever get decent rating if people are afraid to play with them until they already have them?

#15

One last BIG question is, what kidn fo gakme is this?

In a game where gameplay ash no lasting consqeunces (ie an FPS) you cna afford some ocassional cheating. In games where a cheated player will have to live with the results of that cheat for any length of time… its a lot bigger deal.

#16

But if they do vote maliciously, they only really hurt themselves by distancing their rating profile from that of the majority. This means that not only will their opinion carry less weight with the majority in future, but they will take more notice of the opinions of others who vote dishonestly, and the accuracy of their estimations will suffer. It is in every client’s interests to vote honestly.

This is a concern, that higher-rated servers will become very “cliquey”. But again, it will be in the interests of the clique to take a chance on new players in order to get some new blood. If they are happy in their clique, that’s their lookout.
I suppose that the system can be thought of as an automatic clan-builder. Like-minded players who get along will automatically gravitate towards each other, without having to organise clan tryouts and the like.

[quote]One last BIG question is, what kind of game is this?

In a game where gameplay has no lasting consquences (ie an FPS) you can afford some ocassional cheating. In games where a cheated player will have to live with the results of that cheat for any length of time… its a lot bigger deal.
[/quote]
I totally agree on this point. I had only really envisaged using this kind of system in a game without consequences lasting between sessions. If you’re doing a persistent RPG though, you’ve probably already got a honking great central authority that can do this kind of thing in a less fuzzy way. In any case, no system for identifying cheaters is foolproof.

This system isn’t so much for identifying cheaters, as for identifying assholes.
Saying whether someone is using technical hacks is an objective decision that very few players are able to make dependably without colouring their opinion with their own personal feellings, vendettas etc.
On the other hand, everyone can spot an asshole. It’s a subjective decision that is always 100% accurate for that person. All this system provides is a way to find like-minded people whose opinions you can probably trust, in order that you can make judgements on other players that you have not personally encountered before.

On a side note, I’ve thought of a more robust way to defeat the exploit:

[quote]a dishonest client can fool another into taking his opinion more seriously by returning a fake ratings file that is similar to the requesting client’s
[/quote]
Instead of requesting a ratings file directly, the request is routed through several intermediaries. To whit:

A wants to see the ratings history of B, so he generates an encryption key E and encrypts it with B’s public key. This is then sent through a series of random nodes, with an increasing probability that the next hop will be to B. Eventually, B receives the message, encrypts his ratings file with the decrypted key E, and returns it along the path that the request took. A then decrypts the ratings file with E, and we’re done.
The important thing to note is that B did not know who the ratings file was destined for, and so did not know which set of ratings would result in the highest weighting for his opinion. Also, as E is encrypted with B’s public key, only B can reply, and so the intermediate nodes cannot hijack the ratings file and change it to their liking.
Even if all of the intermediate nodes are dishonest, and in cahoots with B, they cannot determine who A is.

#17

I’m really intrigued by this idea, and want to get it into code so we can test the idea behind it.

Some thoughts:

Some of you mentioned that this was a decentralized, p2p system. Would relying on JXTA, a Sun-released p2p framework be advisable?

In addition, would it be advisable to combine this type of “social ranking” with the Glicko2 project currently underway?

If anyone posts a project proposal for this or a volunteer project for this, I’ll gladly join.

#18

Mostly depends on your performance needs. JXTA has some overhead as it is doing network vritualization. OTOH it handles msot of the hard P2P things liek briding sub nets over fire-walls and such for you.

Dunno. Im still; not entirely “groking” this system.

Abusers often group together and collude so Im still not sure it can’t be badly abused. You have a number of assumptions like this in your logic that Im not yet convinced will hold out in the real world…

#19

But it also easily achieves the “intermediary routing” idea posted by someone else a little ways up in this thread. Ah, here it is.