QUOTE (Bob Lord of Evil @ Jun 15 2009, 12:42 PM)

Ummm...
*raises his hand in the back of the hall*
Excuse me. 54 out of 10,000...qualifies as most???
Most of the respondents...yes.
Yeah, I think I indicated that. To quote myself, "With the 54 respondents so far..." I'm under no illusions that I have a representative sample yet. It might get there, it might not. Even if it did, to be sure that I had a representative sample I'd have to bust out a statistics prog and run the numbers. And I a) don't have one and b) don't remember how to do it. I was just ruminating about how the results so far conformed with my hypotheses, not that they constituted anything like proof.
QUOTE
Your comparison is between SR4 and -----other----- RPGs. Does anyone else see the pink elephant in the room?
No, I don't. Care to elaborate?
My own reasoning is thus: in order to have a valuable result, you must poll some kind of objective standard. If you ask "how good is SR4?" you have an invalid poll, because it's completely amorphous. Two people might like the game exactly as much, but because terms like "good" and "excellent" and "average" are relative, there's a good chance they wouldn't answer the same way. One person might call their enjoyment "excellent" and another might call it "good," even though in reality their enjoyment is pretty much equivalent.
The objective standard gives people a point of reference. I'm using relative language like "more," "the same," and "less" in order to ensure that everyone is using approximately the same reference point. That's not to say that they all have the same breadth of experience of RPGs, but at the very least they're all answering the same question.
One way I can see why you'd think my poll was skewed is if you started with the preconception that everyone on Dumpshock definitely liked SR4 more than other games. Then, you might jump to the conclusion that a favorable response was a foregone conclusion and that this poll was not genuine. I did not start with that preconception -- I genuinely want to know if that's the case. Is Dumpshock full of people who like Shadowrun, or is there a large population that actually dislikes it because of whatever flaws it might have?
And overall, this poll lacks the serious flaws that a really skewed poll would have. Over and over, I see polls on Dumpshock that make the answer that the poster doesn't like sound really unattractive. This poll would be skewed towards the answer I wanted if, say, one option had been "Shadowrun rules!" and another had been "Duh, me no likey." Each answer is based on the same point of reference, and each one is phrased with equally neutral language. That is the exact opposite of a skewed poll. You might think I asked the wrong question, but there's no way this poll gets a skewed response to the question I actually asked.
QUOTE (Muspellsheimr @ Jun 15 2009, 12:51 PM)

Which is precisely why I always include an "Other (Please Explain)" option.
Such an option is not needed for the first two, because they are essentially a Yes-No question (the only thing Other is not needed for), but should have been included in Question 3.
Other is not a very useful response. Especially because it's likely to start a flame war and derail the post, which is supposed to be about the poll.
I did just figure out how to use the data I collected, though -- at first glance, it's not useful because we can't know whether the responses are based on a small group of people who selected all 9 or close to it, or a larger group who all selected 2-5 answers each. However, looking at each category compared to the total number of respondents, we can see how many people who took the poll selected a given answer. Right now, with 54/67 people saying that the fluff in general is one of the game's best qualities, we know that the fluff is an overwhelming reason why the respondents like the game. With only 8/67 selecting the matrix rules, we know that most respondents do not think they're a best feature. So there is some interesting stuff that will come out of that third question.