The Lockbox Data Collection Project

[Stickasylum]

The cost info in the spreadsheet is NOT the same as the cumulative probablity. Two different measures.

Originally, my intuition told me that both the expected number of drops per X keycodes and the probability of at least one drop given X keycodes would give the same ordering of tiers, and thus be equivalent in terms of an efficiency measure. I did the math, however, and it turns out that they usually give the same answer, but not always! Here's an example where we get different answers (it will look a bit contrived because there is a very small window where the answers are different):

Let p_2 = 0.05 and p_4 = 0.19 be the probabilities an orange item for one item of a T2 and T4 lockbox, respectively. Suppose we spend 64 keycodes on lockboxes (1 T4 or 8 T2 boxes). Then the probabilities of at least on orange are:

P(at least one orange from 1 T4 box) = 1 - (1 - 0.19)^(1*4) = 0.5695
P(at least one orange from 8 T2 boxes) = 1 - (1 - 0.05)^(8*2) = 0.5599

So it looks like T4 boxes are they way to go! But we get a lot more items out of 8 T2 boxes (16 vs 4), so how many oranges do we expect to get on average?

Average number of oranges from 1 T4 box = 0.19 * (1*4) = 0.76
Average number of oranges from 8 T2 boxes = 0.05 * (8*2) = 0.8

Now it looks like T2 boxes are better! So which strategy should we choose? If what we care about is getting getting one orange, as opposed to none, we should go with the T4 box. On the other hand, if we care about getting as many oranges as possible in the long run, we should stick to buying T2 boxes! I believe that most people probably fall in the second camp. Since you'll likely be buying many boxes over your time in Defiance, the average number of orange items per keycode spent seems like the more relevant measure of efficiency of boxes.

That said, both numbers are useful, and for the probabilities the dropboxes seem to have, both methods will lead to the same buying strategies. However, the efficiency measure currently used is better for getting the more oranges on average. Also, it is much easier to generalize. If the expected number of oranges per 100 keycodes spent is 2 and you spend 200 keycodes, you expect 4 oranges. Probabilities need to be recomputed according to formula:

Probability of at least one orange = 1 - (1-p)^n,
where p is the p of an orange for one item and n the number of items purchased.

Not as easy to do in one's head!

The cost ratio is irrelevant for cumulative probability. Given that you have 72 keys sitting in your hand, you have the BEST probability of scoring an orange by spending all of them on T2 lockboxes. It doesn't matter that 8 keys will be left over if you buy the T4 lockbox.

It is relevant in the sense of the number of boxes purchasable. Suppose p_2 = 0.05 and p_4 = 0.2. With 72 keycodes, 1 T4 box gets you a probability of at least one orange of 1-(1-0.2)^4 = 0.59 of at least one orange. Nine T2 boxes gives you a probability of at least one orange of 1-(1-0.05)^18 = 0.60. Your best strategy isn't to blow it all on T2 boxes, though! If you just bought one T4 box, then saved up for another, the probabilities for 2 T4 and 16 T2 boxes are, respectively 1-(1-0.2)^8 = 0.83 and 1-(1-0.05)^32 = 0.81. In this case, buying T4 boxes is the better strategy, but ignoring those left-over keycodes biased your analysis.

BTW I should clarify one thing too. While you have a better overall cumulative probability of scoring an orange by spending only on T2 lockboxes, you will have far fewer blues and purples show up in your attempts. With T3 and T4 purchases, you'll at get a lot more purples and blues while you wait for an orange to show up.

So on the whole, T3 and T4 are probably still better choices to buy. Because even though you have a lower cumulative probability to score an orange, you have a much higher cumulative probability to score purples and blues.

I'll try to run the numbers later for the best cumulative probability of purples and for blues.

This is a good point, but remember that you get a lot more items from T2 boxes! You get twice as many per keycode as T3 boxes, and four times as many items than T4 boxes. I suspect that this probably evens out the blues and purples in favor of T2s as well (though I still suspect that the T2 orange drop rate is 1 or 2%, and not sufficient to be more efficient than T3s).

[Drall]

Again, could a dev just post the #s? If your math is terrible/wrong that is fine, just own it and let us know while you fix the problem. Keeping the players in the dark is unacceptable and there is clearly the perception that T4 boxes are terrible.

(Also 3b 1g x2 from last two T4 boxes at 1XXX ego rating)

[Stickasylum]

Since I was doing the math already, I went ahead and computed the probability cutoffs determining when one tier is better than another.

Let p2, p3, p4 be the probability of a given rank of item for each item in a T2, T3, and T4 lockbox, respectively. From the perspective of maximizing the expected number of drops of that rank per keycode spent, then

T2 is better than T3 if 2*p2 > p3
T3 is better than T4 if 2*p3 > p4
T2 is better than T4 if 4*p2 > p4

For example, if the orange drop rate from T4 boxes is 8%, then T3 boxes are a better investment if their orange drop rate is greater than 4% and T2 boxes are better if their orange drop rate is greater than 2%.

This also applies to combined groups. The Blue/Purple/Orange drop rate from T4 boxes looks like it is about 75% now. Then T3s are a better investment if the B/P/O drop rate is greater than 37.5% and T2s are better with a rate greater than 18.75%. Both of these look like they are true! T3s look to have a B/P/O rate of about 60%, meaning T2s are only better with a B/P/O rate over 30%. Right now, they look like they have about that (though my personal data puts them closer to 25%). So from our current data, it looks like T2 and T3 and neck-and-neck for BPO efficiency, both beating T4s handily.


We can get similar bounds for the probability of at least one item of a given rank for given number of keycodes spent, but the inequalities are bit more complicated:

T2 has better probability than T3 if p2 > 1-(1-p3)^(1/2)
T3 has better probability than T4 if p3 > 1-(1-p4)^(1/2)
T2 has better probability than T4 if p2 > 1-(1-p4)^(1/4)

For example, if T4 has a 8% orange drop rate, then the probability of at least one orange is better for T3 boxes if p3 > 1-(1-0.08)^(1/2) = 0.041 (i.e. 4.1%). The probability of at least one orange is greater for T2 boxes if the T2 orange drop rate is greater than 2.1%.

Again, categories can be combined. The current T4 B/P/O drop rate is about 75%, so T3 needs better than 50% and T2 better than 29% to have better probability of at least on rare or better, both of which appear to be true (though T2 is awfully close). For T2 to have better probability than T3, the T2 drop rate needs to be at least 37%, which doesn't appear to be true. Thus T3 appears to have the best probability of getting at least one rare or better item (though T2 will give roughly the same number of rare or better items, on average!)

Again, could a dev just post the #s? If your math is terrible/wrong that is fine, just own it and let us know while you fix the problem. Keeping the players in the dark is unacceptable and there is clearly the perception that T4 boxes are terrible.

(Also 3b 1g x2 from last two T4 boxes at 1XXX ego rating)

That would be awesome. By all means, post a thread and link it here! In the meantime, we're having a philosophical discussion about statistics, because statistics are cool.

Also, there isn't anything wrong with the math, we're mostly talking about which statistics are most appropriate for determining buying strategy. Distinctions like expected value vs. probability are important regardless of whether the numbers come from Trion Worlds or observation - we still need to decide how turn them into a strategy!

[Montrovont]

I saw this the other day and just got around to posting my info into the form. I've been tracking my T4 purchases since launch (that's all I've bought). My luck seems to be horrible with these, as I'm way below the percentages in the results. Only 1 orange and 4 purples out of 11 T4 lockboxes so far:

(4/2/2013, Shondu's Consulate, EGO 287, Resources) - Blue VOT Spanner Protector, Green Pyroblast Std-23W, Green VOT Outbreaker, Blue FRC Big Boomer

(4/4/2013, Happy Pow Farms, EGO 376, Resources) - Blue VBI TACC Assault Rifle, Blue Ironclad Blastproof II D, Blue VOT Swarm Cannon, Blue VOT Fragger

(4/5/2013, Bug 'n' Chug, EGO 449, Resources) - Blue VOT Infector, Green FRC heavy Scattergun, Green VBI LM-43 Thunder, Blue VBI GL-1 Ground Pounder

(4/6/2013, Headlands Transit Depot, Resources) - Blue VOT Auto Lobber, Purple VOT Blaster, Blue FRC Birdshot Pump, Blue FRC Assault Carbine

(4/6/2013, Iron Demon Ranch, EGO 550, Resources) - Blue VOT Rebounder Cannon, Green FRC Heavy Sawed-Off Scattergun, Blue VBI LM-12 Rocker, Purple Rebel Blastproof III DX

(4/6/2013, Earth Republic Camp, EGO 618, Resources) - Orange FRC Bull Rush 45, Purple VBI Guided Launcher, Blue VBI Short-Barrel Shotgun, Blue FRC Heavy Assault Carbine

(4/7/2013, Last Chance, EGO 662, Resources) - Blue VBI TACC Assault Rifle, Green FRC Auto-Scattergun, Green VOT Pulser, Blue Hurricane Reloader II D

(4/7/2013, Muir Processing, EGO 785, Resources) - Blue VOT Swarm Cannon, Blue FRC Heavy Scattergun, Blue FRC Saw, Purple VOT Tachmag Pulser

(4/7/2013, Bloodbath Gorge, EGO 824, Resources) - Blue VBI BM-4 Stingray, Green VOT Infector, Blue FRC Sub-Carbine, Blue VBI INF-27 Immunizer

(4/10/2013, Last Chance, EGO 862, Reources) - Blue FRC Big Boomer, Green VOT Pulser, Blue VBI BAS-7 Derailer, Blue VOT Nano Fragger

(4/10/2013, Headlands Transit Depot, EGO 900, Resources) - Blue VOT Spanner, Blue VBI LM-43 Thunder, Green VBI CS-X Cluster Shot, Blue FRC Birdshot Pump

[Yokai]

@stickasylum

Your theory crafting is fun for some, but also clouds the issue for many. Your first long example used T4 base numbers that were unrealistic (19% drop rate? No.)

In your next post, you came to essentially the same conclusion that running the numbers through a simple binomial probability calculator would yield.

I favor the simple approach. This type of decision requires applying binomial distribution. There are easy to use calculators for this. Explain what variables to plug in and which type of cumulative result to look at.

I'm not trying to rag on your contribution, but merely to clear up something I feel will confuse other readers.

I'll summarize as simply as possible: the best chance at scoring an orange will come from buying T2 lockboxes, but you'll get a ton of greens and very few blues and purples before you finally score an orange. If you buy T3 or T4 lockboxes, you'll have less chance at scoring an orange given the same number of key codes spent, but you will find a LOT more blues and purples in the process. Name your poison. Trion actually balanced the cost/risk/reward fairly well, IMO.

Again, the exact numbers are as follows, if the drop rate for an orange per tier is 4, 6, and 8 percent respectively for T2, T3, and T4. This assumes you have 72 key codes to spend and are trying to decide how to best spend them. With those 72 keys, you could buy one T4 box, 3 T3 boxes, or 9 T2 boxes.

1x T4 lockbox = 4 trials at 8% per trial = 28.3% cumulative probability to get at least one legendary from the box

3x T3 lockboxes = 9 trials at 6% per trial = 42.7% cumulative probability to get at least one legendary from the 3 boxes

9x T2 lockboxes = 18 trials at 4% per trial = 52.0% cumulative probability to get at least one legendary from the 9 boxes

[Drall]

Since I was doing the math already, I went ahead and computed the probability cutoffs determining when one tier is better than another.

Let p2, p3, p4 be the probability of a given rank of item for each item in a T2, T3, and T4 lockbox, respectively. From the perspective of maximizing the expected number of drops of that rank per keycode spent, then

T2 is better than T3 if 2*p2 > p3
T3 is better than T4 if 2*p3 > p4
T2 is better than T4 if 4*p2 > p4

For example, if the orange drop rate from T4 boxes is 8%, then T3 boxes are a better investment if their orange drop rate is greater than 4% and T2 boxes are better if their orange drop rate is greater than 2%.

This also applies to combined groups. The Blue/Purple/Orange drop rate from T4 boxes looks like it is about 75% now. Then T3s are a better investment if the B/P/O drop rate is greater than 37.5% and T2s are better with a rate greater than 18.75%. Both of these look like they are true! T3s look to have a B/P/O rate of about 60%, meaning T2s are only better with a B/P/O rate over 30%. Right now, they look like they have about that (though my personal data puts them closer to 25%). So from our current data, it looks like T2 and T3 and neck-and-neck for BPO efficiency, both beating T4s handily.


We can get similar bounds for the probability of at least one item of a given rank for given number of keycodes spent, but the inequalities are bit more complicated:

T2 has better probability than T3 if p2 > 1-(1-p3)^(1/2)
T3 has better probability than T4 if p3 > 1-(1-p4)^(1/2)
T2 has better probability than T4 if p2 > 1-(1-p4)^(1/4)

For example, if T4 has a 8% orange drop rate, then the probability of at least one orange is better for T3 boxes if p3 > 1-(1-0.08)^(1/2) = 0.041 (i.e. 4.1%). The probability of at least one orange is greater for T2 boxes if the T2 orange drop rate is greater than 2.1%.

Again, categories can be combined. The current T4 B/P/O drop rate is about 75%, so T3 needs better than 50% and T2 better than 29% to have better probability of at least on rare or better, both of which appear to be true (though T2 is awfully close). For T2 to have better probability than T3, the T2 drop rate needs to be at least 37%, which doesn't appear to be true. Thus T3 appears to have the best probability of getting at least one rare or better item (though T2 will give roughly the same number of rare or better items, on average!)



That would be awesome. By all means, post a thread and link it here! In the meantime, we're having a philosophical discussion about statistics, because statistics are cool.

Also, there isn't anything wrong with the math, we're mostly talking about which statistics are most appropriate for determining buying strategy. Distinctions like expected value vs. probability are important regardless of whether the numbers come from Trion Worlds or observation - we still need to decide how turn them into a strategy!

Didn't mean any math/statistics you guys are looking at, I just mean the actual numbers behind each tier. Regardless of everything being done here you are all still just speculating since you don't have the actual information or a large enough amount of information to draw your data from.

By wrong I meant their assumption that t3/4 > t2 when they designed it which doesn't exactly seem to be the case.

[Mattwi]

So it would be best to buy Tier 2 to get the best rarity? Also will they fix this because it seems like a no brainer to buy tier 2s?

Im just wondering which is best for me to get good stuff at the least cost. I bought a tier 4 and got nothing good. So which should i buy for the best probability to get good stuff. (Including the cost of course)

[nnja303]

In my personal opinion you cant just take each tear box into consideration when figuring this out... From my experience so far I have better luck at different vendors, I think this may play into it. Every piece of orange gear ive gotten is from muir. Ive bought numerous boxes from san fran as well and those seem to have horrid drop rates, those usually end up more greens than any other.

[Stickasylum]

@stickasylum

Your theory crafting is fun for some, but also clouds the issue for many. Your first long example used T4 base numbers that were unrealistic (19% drop rate? No.)

In your next post, you came to essentially the same conclusion that running the numbers through a simple binomial probability calculator would yield.

I favor the simple approach. This type of decision requires applying binomial distribution. There are easy to use calculators for this. Explain what variables to plug in and which type of cumulative result to look at.

I'm not trying to rag on your contribution, but merely to clear up something I feel will confuse other readers.

The math was an example to show that using the average number of orange drops per 100 keycodes spent (labeled "Drop rate / cost per keycode" in the table) vs. using the probability of at least one orange drop can lead to different conclusions about which lockbox to buy. The numbers were contrived, because most of the time, the two methods agree. This does mean, however, that there is a very real difference between the two methods. I argued that using average number of drops per 100 keycodes is what most people would actually want, as most people are interested in maximizing their volume of oranges, rather than minimizing the probability of not seeing any (you'll quickly get at least one orange using any method). The average number of drops per 100 keycodes is also easier to compute, and easier to generalize (just multiply by the number of keycodes you spend / 100, no binomial calculator necessary!).

The second example was to show you that, if you are computing the probability of at least one orange drop, in order for the probabilities to be comparable you need to use a lockbox ratio with equal values. Using different keycode amounts worth of lockboxes for each tier can lead to incorrect conclusions about which lockboxes are best to buy.

Did you not find these arguments persuasive? Basically, we both believe that the simplest method should be used. I just think that the current method (average number of drops per 100 kecodes, labelled "Drop rate / cost per item") is both the simplest method, and the measure to use if you want to maximize volume (the probability measure is more suited to one-off trials, like game shows, lotteries, or personal insurance).

So it would be best to buy Tier 2 to get the best rarity? Also will they fix this because it seems like a no brainer to buy tier 2s?

Im just wondering which is best for me to get good stuff at the least cost. I bought a tier 4 and got nothing good. So which should i buy for the best probability to get good stuff. (Including the cost of course)

From the current data, it looks that way. If the numbers are correct, you'll get a better orange and purple drop rate per keycode spent than either T3 or T4 boxes, and nearly the same blue drop rate per keycode spent as T3 boxes.

I'm skeptical of the listed orange drop rate for T2 boxes, however, because it seems to be heavily skewed by a single data entry. I suspect that the orange drop rate for T2 boxes is closer to 1-2%, in which case T3 boxes are the clear winner for both orange drops and blue or better drops. T2 boxes would still be better than T4 for blue and purple drops, but not for oranges (but that's neither here nor there, since you should be buying T3s )

Didn't mean any math/statistics you guys are looking at, I just mean the actual numbers behind each tier. Regardless of everything being done here you are all still just speculating since you don't have the actual information or a large enough amount of information to draw your data from.

By wrong I meant their assumption that t3/4 > t2 when they designed it which doesn't exactly seem to be the case.

Pretty much. There are enough data, at least, to say that T4 seems to be a bad investment!

[Drall]

Feel a little insulted with the current drop rates. Lost count of how many I've opened of each type but still only have 1 orange and 1 purple to show for it. (1329 ego so w/e playtime/crazy amount of keys that is)

Also another T4 just now with 1 green, 3 blues. =/

If nothing else it seems like they should remove greens from T4 and maybe up the purple rate? Orange being ultra rare seems fine but its annoying when 4/4 is trash.