Yokai

04-11-2013, 09:20 AM

TLDR: As of the date shown in the chart below, the best tier to buy appears to be T4 if you want oranges, and T3 if you want purples. This analysis uses the validated collected data from Noupoi's excellent community data collection project here: http://forums.defiance.com/showthread.php?20963-The-Lockbox-Data-Collection-Project

UPDATE: I won't bother with a second screenshot, but here are the same probabilities for a purchase of THIRTEEN T4 lockboxes. (As opposed to the metrics below, which are based on THREE T4 lockboxes.) At the current 30% discount for buying $50 USD of bits, that is the number of T4 lockboxes you can open for spending $50 cash. If you buy 13 T4s and open them all in a row, your probability of getting AT LEAST ONE orange is 95%, AT LEAST TWO oranges is 80%, and AT LEAST THREE oranges is 57%. In real world terms, that means if you spend $50 USD on a lockbox spree, you have a 20% chance of scoring AT MOST only one orange, and you have a 43% chance of scoring AT MOST only two oranges. Think about it.

POST DLC1 Update (23-Aug): I've seen a lot of threads since the release of DLC1 claiming that the drop rates for T4 are either better or worse than before. This is incorrect. Dahanese said on 12-Aug that "I have a reply from the devs! We should change that to 'will now also drop new charged weapons' as that's the change.". So early release notes for the DLC implied that drops rates for T4 would be better, but in fact that turned out to be wrong and the only difference was that the new charge weaps were added to the T4 loot tables. Similarly, adding new weapons to loot tables does not change drop probability for purples or oranges or blues, etc. That's not how loot tables work. Point of all this is that not much has changed since the last time I updated the chart below, so these probabilities are still accurate.

http://i.imgur.com/4KzrZFG.png

----- Keep reading if you care to see the details behind the TLDR -----

First, you should look at Noupoi's excellent community project here, and look at his spreadsheet. Ignore the "cost per item" data, because the percentages you see there are telling you a different measure than the actual cumulative probability of scoring oranges or purples. Just focus on the "drop rate" numbers there. http://forums.defiance.com/showthread.php?20963-The-Lockbox-Data-Collection-Project. Also, be sure to look ONLY at the "Validated" worksheet tab. The chart above is based only on the validated data collected so far.

Now, with those baseline drop rates, we're going to ask the question: "What is my cumulative probability of getting AT LEAST 1 of each color if I spend 192 keys?" For example, you can spend those 192 keys on 24 T2 lockboxes and open all of them. What is your chance of scoring AT LEAST 1 orange from those 48 "item rolls"? What is your chance of scoring AT LEAST 2 or AT LEAST 3 purples from those 48 item rolls? What if you decide to buy eight T3 lockboxes instead with those 192 keys? What is your chance of AT LEAST 1 orange from those 24 item rolls (with slightly higher drop rates than the T2 boxes)? And so on.

This type of probability question is identical to questions like "If I flip a coin 20 times, what is the probability of getting AT LEAST 8 "heads" as a result? Or of getting EXACTLY 12 tails as a result?" And so on. Or with dice rolls; "If I roll this 6-sided die 10 times, what is the probability of getting at least two 6s?" And so on.

This type of probability is easily calculated using a notion called "cumulative probability", which is based on a "binomial distribution". Look it up on Wikipedia. If that makes your head hurt, know that there's an easier way! You can simply plug three simple numbers into an online binomial distribution calculator such as at http://stattrek.com/online-calculator/binomial.aspx . Such calculators will typically require inputs called "trials" and "successes". The trial is the number of flips or rolls. In our case of using 192 keys, the number of trials is 48 for T2, 24 for T3, and 12 for T4. The success probability for each trial is, in our case, the "drop rate" for a given color from each type of lock box. The "Number of successes" is, in our case, the number of oranges or purples you're looking for.

So, for example, if you want to know the cumulative probability for AT LEAST 1 orange from spending 192 keys on eight T3 lockboxes, the three numbers you'd plug into the calculator are:

"Probability of success on a single trial": .05

"Number of trials": 24

"Number of successes": 1

The result would be 71%, which is your odds of actually scoring at least 1 orange from those 192 keys spent this way.

------ A Note About Sample Size and Accuracy -------

Edit: I'm adding this section after seeing several spurious comments challenging accuracy or sampling techniques, hopefully to stave off further well-intentioned but incorrect arguments.

In this type of problem, it is a well-established, universal technique to use sampling techniques to make reasonable predictions about a total population. This revolves around concepts of normal distribution, confidence levels, confidence intervals, and standard error. Without going into all the statistical jargon, the short TLDR is very very simple:

Any sample of 20 data points or more can yield strongly significant results. Any sample of more than 377 data points is usually waaaaaayyyyyy more than necessary to yield high confidence results. Specifically:

A sample size of 30 data points yields results that we can be 95% confident will match the actual population results all but 17.8% of the time (if we could measure every single member of the population accurately) if we tested 100 such different samples.

A sample size of 100 data points yields results that we can be 95% confident will match the actual population results all but 9.8% of the time (if we could measure every single member of the population accurately) if we tested 100 such different samples.

A sample size of 377 data points yields results that we can be 95% confident will match the actual population results all but 5% of the time (if we could measure every single member of the population accurately) if we tested 100 such different samples.

And so on.

Noupoi's sample size is now over 2000 data points. That equates to 95% confidence that the collected data already matches the actual population (aka the "real" drop rates put in by Trion) all but 2% of the time if we were to wipe out Noupoi's results right now and start over with 2000+ data points. And then wipe and re-do another 100 times.

In other words, the observed results we have RIGHT NOW are "close enough for rock and roll". You can trust them. At least until Trion makes a stealth change to the actual drop rates.

Finally, because the base drop rates we're working with have such a VERY HIGH confidence to be accurate, the calculated cumulative probabilities are effectively bullet proof. If you argue otherwise, you're simply wasting everyone's time. These numbers will remain bullet proof until the day that Trion decides to tweak the underlying drop rates (whether they bother to tell us that fact or not.)

Now for a final caveat: The above rules of thumb about sample size are true only if you have truly random sampling in your sampling methodology. In the case of Noupoi's collection technique, it is not truly random. So yes, the confidence level cannot truly claim to be a 95% confidence level within 2% error of the actual population. But regardless, the sheer number of data point still makes the confidence very high overall.

UPDATE: I won't bother with a second screenshot, but here are the same probabilities for a purchase of THIRTEEN T4 lockboxes. (As opposed to the metrics below, which are based on THREE T4 lockboxes.) At the current 30% discount for buying $50 USD of bits, that is the number of T4 lockboxes you can open for spending $50 cash. If you buy 13 T4s and open them all in a row, your probability of getting AT LEAST ONE orange is 95%, AT LEAST TWO oranges is 80%, and AT LEAST THREE oranges is 57%. In real world terms, that means if you spend $50 USD on a lockbox spree, you have a 20% chance of scoring AT MOST only one orange, and you have a 43% chance of scoring AT MOST only two oranges. Think about it.

POST DLC1 Update (23-Aug): I've seen a lot of threads since the release of DLC1 claiming that the drop rates for T4 are either better or worse than before. This is incorrect. Dahanese said on 12-Aug that "I have a reply from the devs! We should change that to 'will now also drop new charged weapons' as that's the change.". So early release notes for the DLC implied that drops rates for T4 would be better, but in fact that turned out to be wrong and the only difference was that the new charge weaps were added to the T4 loot tables. Similarly, adding new weapons to loot tables does not change drop probability for purples or oranges or blues, etc. That's not how loot tables work. Point of all this is that not much has changed since the last time I updated the chart below, so these probabilities are still accurate.

http://i.imgur.com/4KzrZFG.png

----- Keep reading if you care to see the details behind the TLDR -----

First, you should look at Noupoi's excellent community project here, and look at his spreadsheet. Ignore the "cost per item" data, because the percentages you see there are telling you a different measure than the actual cumulative probability of scoring oranges or purples. Just focus on the "drop rate" numbers there. http://forums.defiance.com/showthread.php?20963-The-Lockbox-Data-Collection-Project. Also, be sure to look ONLY at the "Validated" worksheet tab. The chart above is based only on the validated data collected so far.

Now, with those baseline drop rates, we're going to ask the question: "What is my cumulative probability of getting AT LEAST 1 of each color if I spend 192 keys?" For example, you can spend those 192 keys on 24 T2 lockboxes and open all of them. What is your chance of scoring AT LEAST 1 orange from those 48 "item rolls"? What is your chance of scoring AT LEAST 2 or AT LEAST 3 purples from those 48 item rolls? What if you decide to buy eight T3 lockboxes instead with those 192 keys? What is your chance of AT LEAST 1 orange from those 24 item rolls (with slightly higher drop rates than the T2 boxes)? And so on.

This type of probability question is identical to questions like "If I flip a coin 20 times, what is the probability of getting AT LEAST 8 "heads" as a result? Or of getting EXACTLY 12 tails as a result?" And so on. Or with dice rolls; "If I roll this 6-sided die 10 times, what is the probability of getting at least two 6s?" And so on.

This type of probability is easily calculated using a notion called "cumulative probability", which is based on a "binomial distribution". Look it up on Wikipedia. If that makes your head hurt, know that there's an easier way! You can simply plug three simple numbers into an online binomial distribution calculator such as at http://stattrek.com/online-calculator/binomial.aspx . Such calculators will typically require inputs called "trials" and "successes". The trial is the number of flips or rolls. In our case of using 192 keys, the number of trials is 48 for T2, 24 for T3, and 12 for T4. The success probability for each trial is, in our case, the "drop rate" for a given color from each type of lock box. The "Number of successes" is, in our case, the number of oranges or purples you're looking for.

So, for example, if you want to know the cumulative probability for AT LEAST 1 orange from spending 192 keys on eight T3 lockboxes, the three numbers you'd plug into the calculator are:

"Probability of success on a single trial": .05

"Number of trials": 24

"Number of successes": 1

The result would be 71%, which is your odds of actually scoring at least 1 orange from those 192 keys spent this way.

------ A Note About Sample Size and Accuracy -------

Edit: I'm adding this section after seeing several spurious comments challenging accuracy or sampling techniques, hopefully to stave off further well-intentioned but incorrect arguments.

In this type of problem, it is a well-established, universal technique to use sampling techniques to make reasonable predictions about a total population. This revolves around concepts of normal distribution, confidence levels, confidence intervals, and standard error. Without going into all the statistical jargon, the short TLDR is very very simple:

Any sample of 20 data points or more can yield strongly significant results. Any sample of more than 377 data points is usually waaaaaayyyyyy more than necessary to yield high confidence results. Specifically:

A sample size of 30 data points yields results that we can be 95% confident will match the actual population results all but 17.8% of the time (if we could measure every single member of the population accurately) if we tested 100 such different samples.

A sample size of 100 data points yields results that we can be 95% confident will match the actual population results all but 9.8% of the time (if we could measure every single member of the population accurately) if we tested 100 such different samples.

A sample size of 377 data points yields results that we can be 95% confident will match the actual population results all but 5% of the time (if we could measure every single member of the population accurately) if we tested 100 such different samples.

And so on.

Noupoi's sample size is now over 2000 data points. That equates to 95% confidence that the collected data already matches the actual population (aka the "real" drop rates put in by Trion) all but 2% of the time if we were to wipe out Noupoi's results right now and start over with 2000+ data points. And then wipe and re-do another 100 times.

In other words, the observed results we have RIGHT NOW are "close enough for rock and roll". You can trust them. At least until Trion makes a stealth change to the actual drop rates.

Finally, because the base drop rates we're working with have such a VERY HIGH confidence to be accurate, the calculated cumulative probabilities are effectively bullet proof. If you argue otherwise, you're simply wasting everyone's time. These numbers will remain bullet proof until the day that Trion decides to tweak the underlying drop rates (whether they bother to tell us that fact or not.)

Now for a final caveat: The above rules of thumb about sample size are true only if you have truly random sampling in your sampling methodology. In the case of Noupoi's collection technique, it is not truly random. So yes, the confidence level cannot truly claim to be a 95% confidence level within 2% error of the actual population. But regardless, the sheer number of data point still makes the confidence very high overall.