The Lockbox Data Collection Project

[Snaloe]

Opened 3 xT2 lockbox after reading this thread.

Got this



Been getting crap off all of the T3's I have been opening not one orange.

[Yokai]

This is true, but the point still stands - leaving leftover keycodes in the analysis will lead to a biased analysis (so long as those leftovers can be used later).

Sticky, I really feel you're simply barking up the wrong tree.

The 72 keycodes is simply to enable an even comparison rate among the three tiers. Look, it's really really really simple:

You can't accumulate more than 75 keys at a time. You -must- make a purchase. Which do you purchase? If you purchase a T4, that's it. You have four "item rolls" from that purchase.

To compare probabilities against that purchase decision, you simply ask "how many T3 lockboxes can I purchase with the same 72 keycodes", which translates to getting nine "item rolls" from those 72 keycodes (at different T3 drop rates). And you ask "how many T2 lockboxes can I purchase with the same 72 keycodes? Which yields 18 "item rolls" from T2 lockboxes.

Sure, yes, you have 8 key codes left over if you decide to take a chance on the T4 lockbox, but that is irrelevant. If you save up those 8 and eventually get 72 again and make the same decision to purchase a T4 again, your odds DON'T CHANGE AT ALL. They're still the same crappy T4 odds.

Here's another way to look at it: whether you buy three T3 boxes or 500 T3 boxes, the cumulative probability for any 9 consecutive item rolls will never vary. Likewise for T2 boxes: whether you buy nine T2 boxes or 2000 T2 boxes, the cumulative probability for any 18 consecutive rolls will never vary.

What the preceding paragraph means is that if you buy 100 T4 boxes or 300 T3 boxes or 900 T4 boxes (which would cost exactly the same: 7500 key codes), the PROPORTIONAL odds for all three tiers will remain exactly the same: T2 will STILL have a better cumulative probability to net you MORE oranges and MORE purples than T3 or T4. See how it works?

[Stickasylum]

What the preceding paragraph means is that if you buy 100 T4 boxes or 300 T3 boxes or 900 T2 boxes (which would cost exactly the same: 7500 key codes), the PROPORTIONAL odds for all three tiers will remain exactly the same: T2 will STILL have a better cumulative probability to net you MORE oranges and MORE purples than T3 or T4. See how it works?

The problem is, if you have enough keycodes for 900 T2 boxes (7500 keycodes), you can buy 112 T4 lockboxes (T4 are 64 keycodes a pop). This is your leftovers at play, and means that your answer for 7500 keycodes may be different than your answer for 72 keycodes. With the current numbers, the answer won't be different, but the method itself not generalizable. Without equal-cost ratios, the analysis will only be valid if you have the given number of keycodes, and no chance of obtaining more.

[Armanewb]

You can't accumulate more than 75 keys at a time. You -must- make a purchase. Which do you purchase? If you purchase a T4, that's it. You have four "item rolls" from that purchase.

To compare probabilities against that purchase decision, you simply ask "how many T3 lockboxes can I purchase with the same 72 keycodes", which translates to getting nine "item rolls" from those 72 keycodes (at different T3 drop rates). And you ask "how many T2 lockboxes can I purchase with the same 72 keycodes? Which yields 18 "item rolls" from T2 lockboxes.

Sure, yes, you have 8 key codes left over if you decide to take a chance on the T4 lockbox, but that is irrelevant. If you save up those 8 and eventually get 72 again and make the same decision to purchase a T4 again, your odds DON'T CHANGE AT ALL. They're still the same crappy T4 odds.

Your cumulative probability at the end should use 24 T2s, 8 T3s and 3 T4s, as that is the correct ratio of keycodes/lockboxes. While it is correct to say that the cumulative probability for 9 tries does not vary, the cost of those 9 tries does whether you use T2 lockboxes (36 codes for 9 items), T3 (72 codes for 9 items) or T4 (144 codes for 9 items).

[Jaws]

Bought 14 Tier 2 lockboxes. 3 epic items and 25 green.

EGO of 234

[Yokai]

Your cumulative probability at the end should use 24 T2s, 8 T3s and 3 T4s, as that is the correct ratio of keycodes/lockboxes. While it is correct to say that the cumulative probability for 9 tries does not vary, the cost of those 9 tries does whether you use T2 lockboxes (36 codes for 9 items), T3 (72 codes for 9 items) or T4 (144 codes for 9 items).

You guys really don't get it. I know you mean well, but you don't get it. The overall pattern won't change, nor will the answer. What's worse, the cumulative probability you're asking me to calculate is not actually possible to replicate in game. You can never buy 3 T4 lockboxes in one sitting and "roll the dice" on them for 12 pulls in a row. Putting out cumulative probabilities like that just confuses the issue.

Look I'll prove it by humoring you this one time, using your specific numbers:


T2 (48 trials)

B: 100%.........100%.........100%
P: ..97%...........86%..........66%
O: ..77%..........42%...........17%


T3 (24 trials)

B: 100%.........100%.........100%
P: ..96%...........84%..........62%
O: ..71%..........34%...........12%


T4 (12 trials)

B: 100%.........100%.........100%
P: ..86%...........56%...........26%
O: ..63%..........25%.............7%

See? The pattern is exactly the same. The ratio is exactly the same. T2 is still the best choice for purples and oranges. Period.

[Armanewb]

You guys really don't get it. I know you mean well, but you don't get it. The overall pattern won't change, nor will the answer. What's worse, the cumulative probability you're asking me to calculate is not actually possible to replicate in game. You can never buy 3 T4 lockboxes in one sitting and "roll the dice" on them for 12 pulls in a row. Putting out cumulative probabilities like that just confuses the issue.

Ok, you are being a bit of a condescending git right here. Who cares that you can't buy 3 T4s in one go? Are you restricted to having only 75 keycodes in your entire Defiance life? If the answer is no, then your argument is irrelevant. Of course the cumulative probability we asked is possible in game, because you can earn enough keycodes to go back multiple times to the vendor. Why you perceive that as something you can't accept is bizarre.

At the end of the day, I think your metric doesn't tell any more of a story than the ones listed. If something has a higher return per 100 keycodes (as discussed earlier), mathematically it has a higher probability at any specified interval than its peers. That's all.


@Noupoi, I would also add in the margin of error at a=0.05 and a=0.01. The formulas are 1.96/(2*SQRT(SUM(Number of Items in the Tier))) for a=0.05 and 2.54/(2*SQRT(SUM(Number of Items in the Tier))) for a=0.01.

At the current count, with exclusion via validation, the margin of error is greater than the probability for oranges for Tier 2 and (barely) Tier 3.

[Stickasylum]

What's worse, the cumulative probability you're asking me to calculate is not actually possible to replicate in game. You can never buy 3 T4 lockboxes in one sitting and "roll the dice" on them for 12 pulls in a row. Putting out cumulative probabilities like that just confuses the issue.

Buying 3 T4 lockboxes simply means saving up 64 keycodes 3 times. The fact that there is some game playing in between is irrelevant. The probability of at least one orange given 3 T4 lockboxes means the probability of at getting at least one orange if you save up 64 keycodes 3 times.

The important question (from your probability perspective) is not "What should I do with my current 72 keycodes to maximize my probability right now?" it is "Which boxes should I spend keycodes on to maximize my potential for an orange drop in the long run?" These questions can have different answers.

Here's another example where your method can give bad advice, with theoretical numbers you might find more tenable:

T2 orange drop rate of 2%, T4 drop rate of 8%. Using your method (72 keycodes to spend on 1 T4 or 9 T2):

Probability of at least one orange in 1 T4 box = 1 - (1-.08)^4 = 0.284
Probability of at least one orange in 9 T2 boxes = 1 - (1-.02)^18 = 0.305

Suppose instead we look at what happens over the course of 3 T4 purchases (194 keycodes, with some playing in between drops). This would have bought 24 T2 boxes:

Probability of at least one orange in 3 T4 boxes = 1 - (1-.08)^12 = 0.632
Probability of at least one orange in 24 T2 boxes = 1 - (1-.02)^48 = 0.621

Note that your method would recommend buying T2 boxes. Ignoring that left over 8 keycodes while buying T4 boxes means that your missing out on the long-term advantage of buying T4 boxes. with these made-up drop rates, with 72 keycodes in hand, you are more likely to get an orange drop right now buying T2 boxes, but you'll be more likely to get an orange in the long-term buying T4 boxes.

The fact that I made up those numbers not relevant. It shows that the recommendations from your method can depend on the number of keycodes spent, a problem that the equal-value-ratio method does not have. Just because it is not inconsistent with the current drop rates does not make the method valid. Even a broken clock is right twice a day!

Look I'll prove it by humoring you this one time, using your specific numbers:

What do the different columns mean?

T2 is still the best choice for purples and oranges. Period.

Yes, that's not in question (with the current values from the table). We're just saying that using 1:3:9 for the T4:T3:T2 is not valid for the types of conclusions you wish to draw.

At the end of the day, I think your metric doesn't tell any more of a story than the ones listed. If something has a higher return per 100 keycodes (as discussed earlier), mathematically it has a higher probability at any specified interval than its peers. That's all.

Unfortuntately, this isn't always true (I thought the same thing at first, too). Here's an example where the expected number of oranges per keycode is higher for T2 boxes, but the probability of obtaining at least one orange item from T4 lockbox is greater than from 4 T2 boxes. Basically, because expectation is weighted by the number of oranges obtained, but the probability of at least one orange doesn't encapsulate how many oranges are obtained, under specific circumstances the values can be different. That's why I've been arguing that the expected number of drops is the better statistic to use to compare efficiencies (since most people want to get lots of oranges, rather than minimize their chance of not getting an orange).

@Noupoi, I would also add in the margin of error at a=0.05 and a=0.01. The formulas are 1.96/(2*SQRT(SUM(Number of Items in the Tier))) for a=0.05 and 2.54/(2*SQRT(SUM(Number of Items in the Tier))) for a=0.01.

At the current count, with exclusion via validation, the margin of error is greater than the probability for oranges for Tier 2 and (barely) Tier 3.

Adding the margin of error is a great idea! However, this margin of error will overestimate the error for small drop rates. I'd recommend computing a 95% margin of error separately for each drop rate using

1.96*sqrt(p*(1-p)/Number of Items for Tier),

where p is the drop rate as a decimal. Use 2.54 instead of 1.96 for a 99% margin of error and multiply by 100 to make the margin of error a percentage. For example, the 95% margin of error for a 4% T2 orange drop rate estimated from 1000 items would be

1.96*sqrt(0.04*(1-0.04)/1000) = 0.0121, or 1.21%

[Yokai]

Buying 3 T4 lockboxes simply means saving up 64 keycodes 3 times. The fact that there is some game playing in between is irrelevant. The probability of at least one orange given 3 T4 lockboxes means the probability of at getting at least one orange if you save up 64 keycodes 3 times.

The important question (from your probability perspective) is not "What should I do with my current 72 keycodes to maximize my probability right now?" it is "Which boxes should I spend keycodes on to maximize my potential for an orange drop in the long run?" These questions can have different answers.

Here's another example where your method can give bad advice, with theoretical numbers you might find more tenable:

T2 orange drop rate of 2%, T4 drop rate of 8%. Using your method (72 keycodes to spend on 1 T4 or 9 T2):

Probability of at least one orange in 1 T4 box = 1 - (1-.08)^4 = 0.284
Probability of at least one orange in 9 T2 boxes = 1 - (1-.02)^18 = 0.305

Suppose instead we look at what happens over the course of 3 T4 purchases (194 keycodes, with some playing in between drops). This would have bought 24 T2 boxes:

Probability of at least one orange in 3 T4 boxes = 1 - (1-.08)^12 = 0.632
Probability of at least one orange in 24 T2 boxes = 1 - (1-.02)^48 = 0.621

Note that your method would recommend buying T2 boxes. Ignoring that left over 8 keycodes while buying T4 boxes means that your missing out on the long-term advantage of buying T4 boxes. with these made-up drop rates, with 72 keycodes in hand, you are more likely to get an orange drop right now buying T2 boxes, but you'll be more likely to get an orange in the long-term buying T4 boxes.

The fact that I made up those numbers not relevant. It shows that the recommendations from your method can depend on the number of keycodes spent, a problem that the equal-value-ratio method does not have. Just because it is not inconsistent with the current drop rates does not make the method valid. Even a broken clock is right twice a day!


What do the different columns mean?


Yes, that's not in question (with the current values from the table). We're just saying that using 1:3:9 for the T4:T3:T2 is not valid for the types of conclusions you wish to draw.



Unfortuntately, this isn't always true (I thought the same thing at first, too). Here's an example where the expected number of oranges per keycode is higher for T2 boxes, but the probability of obtaining at least one orange item from T4 lockbox is greater than from 4 T2 boxes. Basically, because expectation is weighted by the number of oranges obtained, but the probability of at least one orange doesn't encapsulate how many oranges are obtained, under specific circumstances the values can be different. That's why I've been arguing that the expected number of drops is the better statistic to use to compare efficiencies (since most people want to get lots of oranges, rather than minimize their chance of not getting an orange).



Adding the margin of error is a great idea! However, this margin of error will overestimate the error for small drop rates. I'd recommend computing a 95% margin of error separately for each drop rate using

1.96*sqrt(p*(1-p)/Number of Items for Tier),

where p is the drop rate as a decimal. Use 2.54 instead of 1.96 for a 99% margin of error and multiply by 100 to make the margin of error a percentage. For example, the 95% margin of error for a 4% T2 orange drop rate estimated from 1000 items would be

1.96*sqrt(0.04*(1-0.04)/1000) = 0.0121, or 1.21%

I give up. Sorry but you're both obstinate and wrong. Take a took at your T2 numbers that you made up. Im not even going to bother describing your major gaffe here. You just like to argue.

The actual cumulative probability for at least one orange from three T4 bosses is 63%. The actual cumulative probability for at least one orange from 24 T4 boxes is 73%. You're stuck on the wrong formula and the wrong approach for the job.

[GhostTLT]

by the way

THANKS for doing this, it's very interesting!!!!