IPB

Welcome Guest ( Log In | Register )

4 Pages V   1 2 3 > »   
Reply to this topicStart new topic
> Probability and Statistics, Are the developers checking them?
Jrayjoker
post May 12 2005, 06:19 PM
Post #1


Neophyte Runner
*****

Group: Members
Posts: 2,453
Joined: 17-September 04
From: St. Paul
Member No.: 6,675



Hi all you play testers, developers, and all other assorted opinionated SOBs who hang out on these here forums. ;)

I have seen some of the code for probability of x number of dice having y number of successes against target number z bandied about in these forums. I find them somewhat fun and helpful, but my question is this:

Is anyone on the dev team actually applying Probability & Statistics to the new system and making sure it is reasonable, or are the play testers expected to do the number crunching if they feel like it?

I guess a way to paraphrase my question would be, how mathematically rigorous are the developers choosing to be at this time?

Anyone in the know care to comment at this time?

Also, does anyone have any idea how common or uncommon it is for a game developer to apply p&s to the game mechanic?
Go to the top of the page
 
+Quote Post
Nikoli
post May 12 2005, 06:26 PM
Post #2


Chicago Survivor
*********

Group: Dumpshocked
Posts: 5,079
Joined: 28-January 04
From: Canton, GA
Member No.: 6,033



well, probability and statistical analysys should he high on the design end, but chances are they are just plugging in numbers and guestimating impact basedon what they see. Nothing wrong with this as it's faithfully served P&P rpg's since their inception.
It's difficult to imagine the true impact of what .05% increase in probability actually means ove rthe lifetime of a character. what they likely look at is, how likely does it seem over the course of a gaming session that this will happen, not over the next 1000000 sessions.
Go to the top of the page
 
+Quote Post
mfb
post May 12 2005, 06:28 PM
Post #3


Immortal Elf
**********

Group: Members
Posts: 11,410
Joined: 1-October 03
From: Pittsburgh
Member No.: 5,670



my hope would be that they're actually checking probabilities and stuff like that, and not just making up die rolls for their examples. unless you play a game on a near-daily basis, it's hard to do that and come up with realistic results, because you're simply not familiar enough with the mechanics to be able to reliably guestimate levels of success.
Go to the top of the page
 
+Quote Post
Jrayjoker
post May 12 2005, 06:31 PM
Post #4


Neophyte Runner
*****

Group: Members
Posts: 2,453
Joined: 17-September 04
From: St. Paul
Member No.: 6,675



True, true. But you can calculate your probabilities based on a smaller samples and get valid (and perhaps sound) results IIRC, it just ups the chance for anomolies.

There are fields of mathematical study that look for trends in small populations, etc. that would apply quite well to a night's worth of dice rolls. I am in no way versed in those maths, but I know they exist.
Go to the top of the page
 
+Quote Post
Kagetenshi
post May 12 2005, 07:01 PM
Post #5


Manus Celer Dei
**********

Group: Dumpshocked
Posts: 17,006
Joined: 30-December 02
From: Boston
Member No.: 3,802



Given how relatively easy it is to calculate a lot of the probabilities (and, more importantly, how easy it is to find or write small programs to calculate the probabilities of the more iffy sections), there's no excuse whatsoever for using "samples". The actual probabilities are easy to find, unless they're doing something really weird with the mechanics.

~J
Go to the top of the page
 
+Quote Post
Charon
post May 12 2005, 07:02 PM
Post #6


Running Target
***

Group: Members
Posts: 1,011
Joined: 15-February 05
From: Montréal, QC, Canada
Member No.: 7,087



QUOTE (Jrayjoker)
True, true. But you can calculate your probabilities based on a smaller samples and get valid (and perhaps sound) results IIRC, it just ups the chance for anomolies.

What? This ain't a survey. No need for extrapolation. We are talking dice rolls here, not "How many people intend to vote republicans in the next election".

The exact probabilities of success for any given dice roll can be easily calculated and considering that these are pretty basic maths, I'm certain that at least a few people on the design team are intimately familiar with the ins and out of the system.
Go to the top of the page
 
+Quote Post
blakkie
post May 12 2005, 07:05 PM
Post #7


Dragon
********

Group: Members
Posts: 4,718
Joined: 14-September 02
Member No.: 3,263



Working out the probabilities explicitly is important because gamers do it. Not just consciously such as by keen people here, but subconciously gamers learn over time tend to learn what is roughly more/less probable, and what is generally more "profitable". Knowing these the odds up front the developer can make a better guess at how the game will be eventually be played without actual playtesting. The playtesting is still required to proof the system.

Note: You have to be careful with playtesting. Learning the probabilities in a system emperically can take time, so it can takes some time for the playtesting to find the groove where the game will normally get played.
Go to the top of the page
 
+Quote Post
mfb
post May 12 2005, 07:10 PM
Post #8


Immortal Elf
**********

Group: Members
Posts: 11,410
Joined: 1-October 03
From: Pittsburgh
Member No.: 5,670



which is why you never guestimate during playtest or development. ever.
Go to the top of the page
 
+Quote Post
Jrayjoker
post May 12 2005, 07:12 PM
Post #9


Neophyte Runner
*****

Group: Members
Posts: 2,453
Joined: 17-September 04
From: St. Paul
Member No.: 6,675



QUOTE (Charon)
QUOTE (Jrayjoker @ May 12 2005, 01:31 PM)
True, true. But you can calculate your probabilities based on a smaller samples and get valid (and perhaps sound) results IIRC, it just ups the chance for anomolies.

What? This ain't a survey. No need for extrapolation. We are talking dice rolls here, not "How many people intend to vote republicans in the next election".

The exact probabilities of success for any given dice roll can be easily calculated and considering that these are pretty basic maths, I'm certain that at least a few people on the design team are intimately familiar with the ins and out of the system.

So, are you suggesting that a rigorous approach isn't necessary? I am not sure that it is necessary to write a thesis on the mechanics, but with a system that has been highly dependant on making things harder or easier based on conditional modifiers I think it should be considered. If we like the feel of the game because we can play with the odds, then the odds need to be considered in the new system.
Go to the top of the page
 
+Quote Post
mfb
post May 12 2005, 07:18 PM
Post #10


Immortal Elf
**********

Group: Members
Posts: 11,410
Joined: 1-October 03
From: Pittsburgh
Member No.: 5,670



indeed. a rigorous approach is very necessary, especially when you're playing with a new system.
Go to the top of the page
 
+Quote Post
Charon
post May 12 2005, 07:27 PM
Post #11


Running Target
***

Group: Members
Posts: 1,011
Joined: 15-February 05
From: Montréal, QC, Canada
Member No.: 7,087



QUOTE (Jrayjoker @ May 12 2005, 02:12 PM)
QUOTE (Charon @ May 12 2005, 01:02 PM)
QUOTE (Jrayjoker @ May 12 2005, 01:31 PM)
True, true. But you can calculate your probabilities based on a smaller samples and get valid (and perhaps sound) results IIRC, it just ups the chance for anomolies.

What? This ain't a survey. No need for extrapolation. We are talking dice rolls here, not "How many people intend to vote republicans in the next election".

The exact probabilities of success for any given dice roll can be easily calculated and considering that these are pretty basic maths, I'm certain that at least a few people on the design team are intimately familiar with the ins and out of the system.

So, are you suggesting that a rigorous approach isn't necessary?

Err, no. The contrary.

When dealing with dice rolls, it's relying on samples and extrapolation that isn't rigorous. You can calculate the exact value with few basic formulas. I can give you the exact probabilities of X successes with Y dice in less than a minute with Excel or a calculator.

Samples is what you do when you can't build a mathematical model. Like determining who is gonna win the elections with a survey.
Go to the top of the page
 
+Quote Post
Kagetenshi
post May 12 2005, 07:31 PM
Post #12


Manus Celer Dei
**********

Group: Dumpshocked
Posts: 17,006
Joined: 30-December 02
From: Boston
Member No.: 3,802



QUOTE (Jrayjoker)
So, are you suggesting that a rigorous approach isn't necessary? I am not sure that it is necessary to write a thesis on the mechanics, but with a system that has been highly dependant on making things harder or easier based on conditional modifiers I think it should be considered. If we like the feel of the game because we can play with the odds, then the odds need to be considered in the new system.

As others have already said, using a sample when the actual probability is available through simple calculation is the antithesis of a rigorous approach.

~J
Go to the top of the page
 
+Quote Post
blakkie
post May 12 2005, 07:32 PM
Post #13


Dragon
********

Group: Members
Posts: 4,718
Joined: 14-September 02
Member No.: 3,263



QUOTE (mfb @ May 12 2005, 01:10 PM)
which is why you never guestimate during playtest or development. ever.

You mean you -shouldn't-, ever. :)

I remember about 3 months or so after Ultima Online went retail (fall '97). A new member was brought onto the development team, his handle was Evil John i believe. He mentioned that he was doing damage over time calculations to try balance out the combat system. These were calculations that had never been done before! :eek: They had just winged it....and if you had played UO in the first few months you would have seen just how much it showed. It was really, really bad.
Go to the top of the page
 
+Quote Post
mfb
post May 12 2005, 07:56 PM
Post #14


Immortal Elf
**********

Group: Members
Posts: 11,410
Joined: 1-October 03
From: Pittsburgh
Member No.: 5,670



indeed. i hope the same thing doesn't happen with SR4.
Go to the top of the page
 
+Quote Post
Jrayjoker
post May 12 2005, 08:17 PM
Post #15


Neophyte Runner
*****

Group: Members
Posts: 2,453
Joined: 17-September 04
From: St. Paul
Member No.: 6,675



QUOTE (Charon)
Err, no. The contrary.

When dealing with dice rolls, it's relying on samples and extrapolation that isn't rigorous. You can calculate the exact value with few basic formulas. I can give you the exact probabilities of X successes with Y dice in less than a minute with Excel or a calculator.

Samples is what you do when you can't build a mathematical model. Like determining who is gonna win the elections with a survey.

Thanks for the clarification.
Go to the top of the page
 
+Quote Post
Lady Anaka
post May 12 2005, 08:36 PM
Post #16


Target
*

Group: Members
Posts: 76
Joined: 26-February 02
Member No.: 873



Numbers have been run. Probabilities have been calculated. Contrary to popular belief, dartboards and random throws were not involved.

:)
Go to the top of the page
 
+Quote Post
blakkie
post May 12 2005, 08:42 PM
Post #17


Dragon
********

Group: Members
Posts: 4,718
Joined: 14-September 02
Member No.: 3,263



QUOTE (Lady Anaka)
Numbers have been run. Probabilities have been calculated. Contrary to popular belief, dartboards and random throws were not involved.

:)

Can you comment on the use or non-use of halucinatory fauna?
Go to the top of the page
 
+Quote Post
Lady Anaka
post May 12 2005, 08:49 PM
Post #18


Target
*

Group: Members
Posts: 76
Joined: 26-February 02
Member No.: 873



To my knowledge, no bits or pieces of animals were ingested, smoked, or otherwise rubbed onto anyone in an effort to aid in game design and or cause halucinations during this process.

Oh, and EW. Just saying.
Go to the top of the page
 
+Quote Post
Pistons
post May 12 2005, 08:54 PM
Post #19


Not Cameron Diaz
**

Group: Retired Admins
Posts: 472
Joined: 26-February 02
From: Newark, Delaware
Member No.: 188



No animals were harmed in the creation of this game system. It is entirely possible that plant life may have been abused, but none of them are commenting and thus it is difficult to verify. :P
Go to the top of the page
 
+Quote Post
Kagetenshi
post May 12 2005, 09:01 PM
Post #20


Manus Celer Dei
**********

Group: Dumpshocked
Posts: 17,006
Joined: 30-December 02
From: Boston
Member No.: 3,802



For the purposes of that statement, are babies considered animals or plant life?

~J
Go to the top of the page
 
+Quote Post
Phantom Runner
post May 12 2005, 09:05 PM
Post #21


Target
*

Group: Members
Posts: 81
Joined: 12-April 02
From: the shadows....
Member No.: 2,548



QUOTE (Lady Anaka)
To my knowledge, no bits or pieces of animals were ingested, smoked, or otherwise rubbed onto anyone in an effort to aid in game design and or cause halucinations during this process.

Oh, and EW. Just saying.

But that does not rule out ritual enemas ...

:eek:
Go to the top of the page
 
+Quote Post
Guest_Crimsondude 2.0_*
post May 12 2005, 09:09 PM
Post #22





Guests






QUOTE (Pistons @ May 12 2005, 02:54 PM)
No animals were harmed in the creation of this game system. It is entirely possible that plant life may have been abused, but none of them are commenting and thus it is difficult to verify.  :P

Well, duh. If that wasn't true, the devs wouldn't have been able to hit the freelancer crack pipe.

That is unless you've switched to freelancer crank.
Go to the top of the page
 
+Quote Post
Ellery
post May 13 2005, 03:53 AM
Post #23


Moving Target
**

Group: Members
Posts: 778
Joined: 6-April 05
Member No.: 7,298



Does anyone have solid evidence that any game designers actually rigorously analyze and understand the probability and statistics of the game they're creating? Game designers often have a literary background rather than an analytic one. Publication schedules are tight, leaving little time for anything viewed as extra. Players don't typically have time to analyze rules statistically before they decide to purchase a book. And so on. All the short-term pressures are against performing any analysis at all (aside from what can be inferred from experience while playtesting), let alone a thorough one.

Of course, from a long-term perspective, a little extra work early on can avoid enormous, ongoing, systematic problems that arise from a system that just doesn't work probabilistically. (It can also help avoid lots of smaller glitches.)

I've played enough MMORPGs to be very skeptical that they really bother to analyze anything there--and this is with teams of programmers who are mostly from a CS background, and thus have at least been exposed to a lot of math, including in most places a required course that covers probability among other things (usually called "discrete math" or somesuch). The design teams there are much larger, and the people are more suited to perform the analysis, and I still see people altering powers so they become completely useless or absurdly overpowered to a level that would be caught by a few dozen hours of playtesting let alone a careful analysis.

So, yes, I think the analysis is very important to do. But unless I'm missing something, SR would be the exception rather than the rule if an analysis were done carefully here. I'd encourage the developers to perform the analysis themselves, or ask their playtesters to, or both! But until I hear official word that they are doing this, I don't see much justification for thinking that they are or will based on my perception of the industry standard.
Go to the top of the page
 
+Quote Post
Phantom Runner
post May 13 2005, 04:15 AM
Post #24


Target
*

Group: Members
Posts: 81
Joined: 12-April 02
From: the shadows....
Member No.: 2,548



QUOTE (Ellery)
Does anyone have solid evidence that any game designers actually rigorously analyze and understand the probability and statistics of the game they're creating?... SR would be the exception rather than the rule if an analysis were done carefully here.  I'd encourage the developers to perform the analysis themselves, or ask their playtesters to, or both!  But until I hear official word that they are doing this, I don't see much justification for thinking that they are or will based on my perception of the industry standard.

I know from personal experience both playtesting several games and from discussion with a few developers that some (only the ones I have personal experience with) are created with an eye toward the statistics, in so much as that statistics are taken into consideration. Some games companies do employ people with advanced mathematics degree(s).

Saddly I have also partaken in playtesting of games where there is almost no statistical support and we playtesters had to do all the number crunching ourselves (not very fun I assure you)...
Go to the top of the page
 
+Quote Post
Ellery
post May 13 2005, 04:30 AM
Post #25


Moving Target
**

Group: Members
Posts: 778
Joined: 6-April 05
Member No.: 7,298



We can count the major contributors to SR on our fingers. Do we know anything about their affinity for statistics? (I don't know many of them.)
Go to the top of the page
 
+Quote Post

4 Pages V   1 2 3 > » 
Reply to this topicStart new topic

 



RSS Lo-Fi Version Time is now: 21st May 2024 - 07:46 AM

Topps, Inc has sole ownership of the names, logo, artwork, marks, photographs, sounds, audio, video and/or any proprietary material used in connection with the game Shadowrun. Topps, Inc has granted permission to the Dumpshock Forums to use such names, logos, artwork, marks and/or any proprietary materials for promotional and informational purposes on its website but does not endorse, and is not affiliated with the Dumpshock Forums in any official capacity whatsoever.