PDA

View Full Version : The Lockbox Data Collection Project



Noupoi
04-05-2013, 06:07 AM
Following the discussions in the lockboxes threads I’ve been reading, I decided to throw a quick form together where everyone can submit what they’ve been getting from lockboxes. The idea is to eventually build up enough data to work out the drop rates and which tier is most worth getting.

If you are planning to take part in this project, please try and submit the results for ALL lockboxes you buy, not just the ones that were exceptionally good/bad- this will skew the results.

Submit your results here (https://docs.google.com/forms/d/1gN-E70tdU65mrl2oEX1C2rzN364motJ_9t1wMDLfHSg/viewform)

View all collected data here (https://docs.google.com/spreadsheet/ccc?key=0AhDKIMOCd2bkdFlSdmNocG1HSHU4VVBXeXdoWDY3Q mc&usp=sharing)

The spreadsheet of results is quite basic at the moment, but I’ll work on it as we get more data. If anyone wants to help with this, feel free to send me a PM, and I’ll give you edit permissions on the spreadsheet.

Any feedback is welcome!

Help spread the word by adding this to your sig (http://forums.defiance.com/profile.php?do=editsignature)!

Come join the Lockbox Data collection Project!
Official Thread (http://forums.defiance.com/showthread.php?20963-The-Lockbox-Data-Collection-Project)|Submit results! (https://docs.google.com/forms/d/1gN-E70tdU65mrl2oEX1C2rzN364motJ_9t1wMDLfHSg/viewform)|View collected data! (https://docs.google.com/spreadsheet/ccc?key=0AhDKIMOCd2bkdFlSdmNocG1HSHU4VVBXeXdoWDY3Q mc&usp=sharing)

Leslannatrix
04-05-2013, 06:09 AM
great idea, bookmarked

Werb
04-05-2013, 07:02 AM
this is a really nice idea!

push

Cable
04-05-2013, 10:16 AM
Great idea.
Submitted data (bump)

Adair
04-05-2013, 10:45 AM
Does the loot EGO boost affect lock boxes?

Gammeck
04-05-2013, 10:55 AM
About to get my tier 4, will report in!

Krahn
04-05-2013, 10:59 AM
Nice, I'll post results as I get them.

Thanks!

Werb
04-05-2013, 11:06 AM
again :) puush

Noupoi
04-05-2013, 12:45 PM
Results are starting to look promising. :D Keep sending them in!


Does the loot EGO boost affect lock boxes?
I'm not sure, it's not something I've considered.

cassell
04-05-2013, 12:47 PM
Great job. Well done.

Alshadow
04-05-2013, 12:49 PM
I Pulled a Orange VBI Tacc Auto Shotgun a purple pistol and 2 purple shields from my first Tier four lockbox

square
04-05-2013, 01:28 PM
I Pulled a Orange VBI Tacc Auto Shotgun a purple pistol and 2 purple shields from my first Tier four lockbox

http://www.dodge-dart.org/forum/attachments/chrysler-discussion/7739d1361749553-2014-jeep-cherokee-no-not-grand-20921.jpg

Gammeck
04-05-2013, 01:33 PM
What I got from my T4 lockbox:

-Bio grenade
-Rocket launcher
-Shield
-Shotgun

NalkorRN
04-05-2013, 01:35 PM
Okay, this is ****ed up.

What I got from my T4 lockbox:

-Rocket launcher
-Shield
-Shotgun

And nothing else. Why? I only had 3 inventory slots left. I instantly ran to a vendor to sell some junk, hoping that I would then receive my 4th item. Nothing. :mad:

Oh well. Lesson learnt I guess...
Open up the Defiance store and check the Claim Items tab, it should act as an overflow bag for looted inventory items.

Gammeck
04-05-2013, 01:43 PM
Open up the Defiance store and check the Claim Items tab, it should act as an overflow bag for looted inventory items.

Wow.. Much, much love to you my friend.

Received an orange bio grenade.

Never really liked those bio grenades... but its ORANGE!!

Xtheoneandonly
04-05-2013, 01:58 PM
You got my support!

NalkorRN
04-05-2013, 02:39 PM
Wow.. Much, much love to you my friend.

Received an orange bio grenade.

Never really liked those bio grenades... but its ORANGE!!

I found that out purely on a guess. I saw after an Arkfall that my inventory was full, turned a few duplicate grenades into ark salvage and decided on a whim to check the defiance store and sure enough on Claim Items it had the blue bio-grenade ready to be claimed.

Gammeck
04-05-2013, 03:31 PM
Bumping for this awesome idea!

Valethar
04-05-2013, 03:34 PM
Does the loot EGO boost affect lock boxes?

Does not appear to, which is a good thing because the loot boost is broken. If it affected lockboxes, you'd be getting white shields and grenades instead of green/blue weapons, etc..

Valethar
04-05-2013, 03:37 PM
The spreadsheet of results is quite basic at the moment, but Iíll work on it as we get more data.

I see quite a few greens popping from those Tier 4's, further proof that they're not working as intended. If I'm not mistaken, a Dev post stated Tier 4 boxes would never give an Uncommon item. Is that not what a green rarity item is?

Gammeck
04-05-2013, 03:41 PM
Dev post stated Tier 4 boxes would never give an Uncommon item. Is that not what a green rarity item is?

Yup, green is uncommon. I thought the odds were the same in either tier?

changedMind
04-05-2013, 03:49 PM
Thanks for putting this together! One piece of feedback: is there a particular reason you have frequency rather than % used on the bar graph? It's been a while since I took any stats, but a % would allow us to compare the rates between lock box tiers as opposed to a general distribution for individual tiers.

Drall
04-05-2013, 04:00 PM
This is a cool idea, but could the devs just give us the info...? Don't see why they would keep us in the dark on this especially when some people are paying cash.

Noupoi
04-05-2013, 04:28 PM
Thanks for putting this together! One piece of feedback: is there a particular reason you have frequency rather than % used on the bar graph? It's been a while since I took any stats, but a % would allow us to compare the rates between lock box tiers as opposed to a general distribution for individual tiers.

The ratio between each colour is the same within each tier is the same, but I can see comparisons between tiers can be difficult, and the current graph could also be an issue if get a lot more results in one tier than another. To this end, I've added a seperate graph for drop rates. :D


This is a cool idea, but could the devs just give us the info...? Don't see why they would keep us in the dark on this especially when some people are paying cash.
I'm not sure why they haven't. In the dev posts I've read, all we know is that higher tier boxes have better chances of dropping better rarity items.

Valethar
04-05-2013, 04:33 PM
Yup, green is uncommon. I thought the odds were the same in either tier?

Nope. What we were told is that Tier 3 has better odds than a Tier 2, in addition to having the extra item. Tier 4 is supposed to have higher odds than Tier 3, and not ever contain an uncommon. Seems like maybe there's a bug or two in there somewhere they've got to step on. :)

Gammeck
04-05-2013, 04:56 PM
Nope. What we were told is that Tier 3 has better odds than a Tier 2, in addition to having the extra item. Tier 4 is supposed to have higher odds than Tier 3, and not ever contain an uncommon. Seems like maybe there's a bug or two in there somewhere they've got to step on. :)

That's a shame. I wasted quite some time to get those keys. :(

Do you have a source to where we were told that T4's don't drop uncommons? I'm interested in checking it out in full detail.

Valethar
04-05-2013, 06:11 PM
That's a shame. I wasted quite some time to get those keys. :(

Do you have a source to where we were told that T4's don't drop uncommons? I'm interested in checking it out in full detail.

It's here on the forums somewhere. I'll see if I can get that idiot search 'feature' to actually cooperate with me for a change LOL

Edit: It appears, after being caught by many players proving him wrong, he changed his post. However, it doesn't change the quotes in the thread, which I managed to find and post below.


http://cdn.defiance.com/defiance/img/forum/misc/quote_icon.png Originally Posted by Athelan http://cdn.defiance.com/defiance/img/forum/buttons/lastpost-right.png (http://forums.defiance.com/showthread.php?p=185039#post185039)
The highest tier lockboxes will never give uncommon items. This is one of the reason to go for them over the middle tier.Now they're saying that the above information is incorrect. We already knew that because we've been getting greens from T4 boxes. Now you're left wondering if perhaps there's a bug that they've just not found, which is why the greens appear in T4 boxes, or if they changed their minds about the rarity. After all, if they can sell you a box of greens for $5, why would they want to sell you a box of blues and purples instead?

Instead of giving details, we just got an edit stating that the quote above was incorrect.


Just a correction, this comment is incorrect. Please see my post below for the correct info.

Thanks
Sledgehammer70


Tier 4 has a much better chance of getting epics and a slightly increased chance of getting legendaries.

Even this is suspect, based on information found here (https://docs.google.com/spreadsheet/ccc?key=0AhDKIMOCd2bkdFlSdmNocG1HSHU4VVBXeXdoWDY3Q %20mc#gid=1). Looking at the numbers so far, the percentages for T3 and T4 boxes are almost identical for 'epics (purple)', though the numbers for rares (blue) does sharply increase. The numbers for legendaries (orange) actually drops instead of increasing.

Bear in mind that the samples sizes don't make this a reliably scientific data set, but as I pointed out in another thread, players couldn't care less about the science, they're interested in the gameplay, and the perception is that T4's are overpriced and underwhelming. Given the numbers thus far, people aren't going to be inclined to try their luck with those boxes, especially at the overinflated rates on the store, if the perception is that they're just going to get a box full of greens.

Instead of saying 'oops, that info was wrong', they need to step up and explain why players should be encouraged to pick up those T4's, and give us some statistics, etc.. to back it up.

usrevenge2
04-05-2013, 06:33 PM
in the beta for ps3 tier 1 was uncommon items 2 was rare and 3 was epic/legendary but that was the beta....

I have almost 70 keys so might as well get a tier 4 then cry when i get junk

IIFatalshockII
04-05-2013, 06:42 PM
Great idea for a thread. Hopefully we pick up solid statistics from this.

SuliX
04-05-2013, 06:58 PM
Tier3: Green Green Blue
Tier3: Green Blue Blue
Tier3: Green Blue Blue

Kurze
04-05-2013, 08:19 PM
Tier 4: 4 blues
Tier 4: 3 blues 1 purple

Drall
04-05-2013, 08:26 PM
Sample size is still too small to tell, but it does seem like tier 4 likely isn't worth it. Wish I could remember how many I've opened but only found 1 purple and 1 orange in T4 so far. =/

shortname
04-05-2013, 08:46 PM
tier 3 green pistol blue assault rifle orange grenade ( my first ) think that was my 5th or so tier 3

Forex
04-05-2013, 09:02 PM
The data in your chart doesn't add up properly. Tier 3 lock boxes total item count is 227. You get 3 items per lock box. According to your chart, there have been a total of 75.7 tier 3 lock boxes opened. Sorry to be a downer but I think the chart is being tampered with.

Sanguinesun
04-05-2013, 09:26 PM
I knew it was going to be purposely skewed by those campaigning to state tier 3's give more oranges. You cant trust people to give honest information in this regard.

Valethar
04-06-2013, 03:32 AM
I knew it was going to be purposely skewed by those campaigning to state tier 3's give more oranges. You cant trust people to give honest information in this regard.

Very true. People are screwing with the data hardcore, entering bogus numbers like 7000+ EGO rating and 10's for each rarity in the box.

CharmCitysKing
04-06-2013, 03:39 AM
This needs to be stickyd!!!!

obside
04-06-2013, 03:42 AM
Hi
Opened a t4 and got 4 blues.
Previously opened 2 box t3 got 2 green one purple and other all blues.
Hope will help

Noupoi
04-06-2013, 04:12 AM
To everyone that's been posting results, thanks, but please also submit them through the form so the spreadsheet is updated!


The data in your chart doesn't add up properly. Tier 3 lock boxes total item count is 227. You get 3 items per lock box. According to your chart, there have been a total of 75.7 tier 3 lock boxes opened. Sorry to be a downer but I think the chart is being tampered with.

If the issue is that the number is not an integer, chances are that an error was made by someone entering data, and not entering all the items obtained from each lockbox, though it's possible that people are also entering false data.


I knew it was going to be purposely skewed by those campaigning to state tier 3's give more oranges. You cant trust people to give honest information in this regard.

There's that, and the possibility that some people subconsciously being selective with the data being submitted. One possible example would be someone getting a great drop from a lockbox, and as a result, coming on the forums to look for threads about lockboxes to post in. This basically means that we may have a good proportion of people submitting extreme data, inflating the droprates for greens and oranges.

I'm hoping that with enough good data, small issues like that would become insignificant.

Xyr3s
04-06-2013, 06:10 AM
after launch i bought 1 tf and i got a couple blues some green and an orange autofragger combat shotgun :) i danced the victory dance ^^

blurps
04-06-2013, 06:18 AM
Pretty much the same here: 1st t3 box on live - one green something, a purple pistol thingy and an orange FRC Auto-Scattergun. Jackpot :)

CookingWithGuns
04-06-2013, 06:50 AM
Is it just me or does there seem to be an a bias towards dropping shotguns.

Noupoi
04-06-2013, 01:08 PM
Quick update: I've added some graphs to the spreadsheet showing what the most efficient lockbox to buy is based on the data. Also just realised that in Defiance, it's 'lock box' instead of 'lockbox' >.<'


Is it just me or does there seem to be an a bias towards dropping shotguns.

It's not something I've noticed. Must just be the ones you've been getting.

Gammeck
04-06-2013, 01:22 PM
Another T4 here:

VOT Blast Rife (YEAH BABY!!)
VOT Blast Rife (lol...)
VBI INF-27 Immunizer
VOT Blaster - Once again, green in T4...

Ashnon
04-06-2013, 02:57 PM
3 blues and a green on a t4...just not worth it.

Gammeck
04-06-2013, 08:10 PM
$$$$ Yet another T4 $$$$














... 4 blues

Leathernek1
04-06-2013, 08:30 PM
I just submitted my data, but thought I'd mention here that eight Tier 3 boxes, and three tier 4 boxes yielded no orange items.

Maelkyan
04-07-2013, 03:32 AM
... though it's possible that people are also entering false data.
True! Just look at the line where they put 10 items for each rarity... So dumb!
Should need a little spring cleaning :cool: .


To everyone that's been posting results, thanks, but please also submit them through the form so the spreadsheet is updated!
Here's my input:
4/6/2013 18:23:46 / 260 / Tier 3 / PC / 1 2 0 0
4/6/2013 18:55:51 / 260 / Tier 3 / PC / 1 1 1 0

4/7/2013 11:38:07 / 329 / Tier 4 / PC / 1 3 0 0

4/9/2013 10:41:23 / 439 / Tier 2 / PC / 2 0 0 0
4/9/2013 10:44:39 / 439 / Tier 2 / PC / 2 0 0 0
4/9/2013 10:44:53 / 439 / Tier 2 / PC / 1 1 0 0

4/11/2013 0:50:44 / 512 / Tier 2 / PC / 1 1 0 0
4/11/2013 0:50:59 / 512 / Tier 2 / PC / 1 1 0 0
4/11/2013 0:51:13 / 512 / Tier 2 / PC / 2 0 0 0
4/11/2013 0:51:26 / 512 / Tier 2 / PC / 2 0 0 0
4/11/2013 0:51:38 / 512 / Tier 2 / PC / 2 0 0 0
4/11/2013 0:51:50 / 512 / Tier 2 / PC / 2 0 0 0
4/11/2013 0:52:42 / 512 / Tier 2 / PC / 2 0 0 0
4/11/2013 0:52:54 / 512 / Tier 2 / PC / 2 0 0 0

4/13/2013 11:04:27 / 628 / Tier 2 / PC / 1 1 0 0
4/13/2013 11:12:19 / 628 / Tier 3 / PC / 1 2 0 0
4/13/2013 11:12:42 / 628 / Tier 3 / PC / 1 2 0 0
4/13/2013 11:12:54 / 628 / Tier 3 / PC / 2 1 0 0

4/17/2013 10:58:53 / 758 / Tier 4 / PC / 1 3 0 0

Avenger
04-07-2013, 08:54 AM
I saw this thread yesterday, didn't post in it as I went off to play a bit and get some results, and when I came back I couldn't find this again - atleast google drive had a bit of a memory so found it again via that :)

Anyway - submitted results for 8 lockboxes (6xT3, 2xT2) to add to the data collection.

Looking at the data there are some clearly false results - people posting more items per category than a lock box can drop in total, but atleast the bar charts make those fairly clear to spot. I don't know if it is possible to identify and exclude certain data entries?

the overall summary doc is really handy and show a clear trend with greens and blues. I think purple and orange may tay a bit longer to determine due to people being more inclined to report really good findings that average/poor ones, but it will all come with time.

good work and I will keep note of my future lock box results for this.

Gammeck
04-07-2013, 02:34 PM
Here's one more T4. Gonna put all of my data into the chart soon.

Sniper
Rocket Launcher
Auto Fragger
Pistol

fishboy11
04-07-2013, 02:38 PM
wow so tier 4 are actually the worst, is it just me or does this scream greed in that they want you to spend money on bits to get the tier 4's since people will assume they are the best but really arent and will have to keep buying them to get an orange, not thinking that someone will compile the data and figure it out?

ten4
04-07-2013, 02:39 PM
Last 3 t4s I bought had 4 shotguns each and not one of them was purple + 10 them were green. I hate shotguns, but evidently the game only wants to give me those in these boxes AND loot drops.

Someone tell the Defiance loot gods to stop giving me #%#% shotguns. THANKS!

Evilution
04-07-2013, 02:42 PM
I give up on tier 3, I know people say they are better and have had luck with them. But I have probably had around 25-30 and only 1 purple and no orange. I've had 4 tier-4 and had 1 orange and 4 purple.

fishboy11
04-07-2013, 02:43 PM
I give up on tier 3, I know people say they are better and have had luck with them. But I have probably had around 25-30 and only 1 purple and no orange. I've had 4 tier-4 and had 1 orange and 4 purple.


i have bought 5 tier 4's and only gotten greens and blues

Werb
04-08-2013, 12:05 AM
tier 2 2 greens...
tier 3 2 greens 1 blue

Tigress
04-08-2013, 07:21 AM
I only buy tier 4 and i guess im lucky. So far i have 3 orange weapons 2 rocket launchers and 1 shotgun. If anyone have a orange sniper or assault rifle they want to trade for something i have send a message.

Arramakaian
04-08-2013, 08:14 AM
Excellent idea and execution!

One thing to note: there should be decay of results. What I mean by that is that more recent results should carry more weight than those further in the past. This is because Trion will certainly tweak drop rates in patches, so results entered during a prior build are not applicable to current build.

Alternatively drop rates could be reported on a timescale, maybe grouping each week's submissions. This would tell even more about the development of rates.

Krazee DD
04-08-2013, 08:25 AM
Thank you for this thread. NEEDED.

http://assets0.ordienetworks.com/images/GifGuide/clapping/riker.gif

Yokai
04-08-2013, 08:44 AM
It's not clear whether a larger "cost per try" value is better or worse. Could you please clarify?

CookingWithGuns
04-08-2013, 09:04 AM
These stats obviously aren't part of a big enough sample to determine just how accurate they are yet, but I think we can safely say that there was probably a mistake in implementing the drop rate for the boxes. Probably not a major one, but something still seems off.

I actually think a large increase to Tier 4's chances on purple, while consuming max keys (what is it, 72 or something?) would be a good move.

Vuule
04-08-2013, 09:56 AM
Are you ****ing kidding me.. Kept buying Tier 4 lockboxes, purchased 5 in total or so. No legendary.

Just a minute ago..
"Let's try a Tier 3 for once"
Green sniper, Orange Infector, Green Infector..
"YOU'VE GOT TO BE F'ING KIDDING MEH!"

-.-;

wise00
04-08-2013, 10:45 AM
One of the most important threads here. Bravo!!!!!!!!!!!!;););)

Zhaocore
04-08-2013, 11:52 AM
T4 Lockbox #1 : 2 blues weapons, 1 green grenade, 1 green weapon
T4 Lockbox #2 : 1 purple weapon, 1 blue grenade, 2 green shields
T3 Lockbox #1 : 1Green grenade, 1 blue weapon, 1 blue shield
T3 Lockbox #2 : 1 Orange weapon, 1 blue weapon, 1 green shield

edit: ok so my orange weapon is gone. I tried to add a mod slot to it and now..its gone...is that a known bug?

Vuule
04-08-2013, 12:18 PM
T4 Lockbox #1 : 2 blues weapons, 1 green grenade, 1 green weapon
T4 Lockbox #2 : 1 purple weapon, 1 blue grenade, 2 green shields
T3 Lockbox #1 : 1Green grenade, 1 blue weapon, 1 blue shield
T3 Lockbox #2 : 1 Orange weapon, 1 blue weapon, 1 green shield

edit: ok so my orange weapon is gone. I tried to add a mod slot to it and now..its gone...is that a known bug?

Yes. Has something to do with the matrix salvage. If you use it while entering PvP, or disconnecting or w/e, there's a chance it will disappear. It's a known issue and they're working on it.

Arctic Fox
04-08-2013, 12:27 PM
This is SUPER. Thanks man. I'll post up every lock box I buy starting today

Noupoi
04-08-2013, 12:37 PM
Great to see the positive response to this thead!



One thing to note: there should be decay of results. What I mean by that is that more recent results should carry more weight than those further in the past. This is because Trion will certainly tweak drop rates in patches, so results entered during a prior build are not applicable to current build.

Alternatively drop rates could be reported on a timescale, maybe grouping each week's submissions. This would tell even more about the development of rates.

That's a valid point, though I don't think Trion will make any significant changes to the drop rates without mentioning it, perhaps in the patch notes somewhere.
With more data, this does seem worth doing, but I also need to figure out a good way to set it up first. :p


It's not clear whether a larger "cost per try" value is better or worse. Could you please clarify?

Cost per try is how many keycodes you are paying for each item. It's not so much the cost per try that matters, but Drop rate / Cost per try shown in the Efficiency graph, where higher is better. This is a measure of the likelihood of obtaining a orange/purple/blue for each keycode you spend.

Arramakaian
04-08-2013, 12:47 PM
That's a valid point, though I don't think Trion will make any significant changes to the drop rates without mentioning it, perhaps in the patch notes somewhere.
With more data, this does seem worth doing, but I also need to figure out a good way to set it up first. :p

Given the wacky results, it is safe to assume they will be tweaked to make the 4-unit boxes more attractive somehow. But yeah, you need tens of observations to make statistically meaningful conclusions.

A quick and dirty way of doing the decay would be to weigh everything posted in the last 7 days so that they total 50% of the final average, and all prior entries would be the other 50%. Figures are just suggestions.

Millby
04-08-2013, 04:20 PM
Given the wacky results, it is safe to assume they will be tweaked to make the 4-unit boxes more attractive somehow. But yeah, you need tens of observations to make statistically meaningful conclusions.

A quick and dirty way of doing the decay would be to weigh everything posted in the last 7 days so that they total 50% of the final average, and all prior entries would be the other 50%. Figures are just suggestions.

Great thought. Another quick/dirty method would simply be to only return results on the X most recent submissions. Where X is a number high enough to return statistically significant results balanced against frequency and number of submissions. Your result wouldn't likely be too different from Arramakaian's method, but might be a bit easier to implement.

nack516
04-08-2013, 08:41 PM
Very nice, I will be adding my data as it comes.

Atlas TKD
04-08-2013, 08:47 PM
My bits were taken and I didn't bother trying out again with keys and for the most part got nothing worth while so far on my new character. My first character where I thought origin meant something with synergy when everyone was to stupid to just say they are separate managed to get a berserker shotgun.

Arctic Fox
04-09-2013, 07:30 AM
I told the Clan about this one. I'm grabbing data from all of them to submit

Stickasylum
04-09-2013, 10:39 AM
Great to see the positive response to this thead!
Cost per try is how many keycodes you are paying for each item. It's not so much the cost per try that matters, but Drop rate / Cost per try shown in the Efficiency graph, where higher is better. This is a measure of the likelihood of obtaining a orange/purple/blue for each keycode you spend.

As it is currently computed, the "Drop rate / Cost per try" statistic is actually the expected number of drops per 100 keycodes spent. It would probably be more clear to drop the % sign (but keep the multiplication by 100) and change the name to "Average drops / 100 keycodes" or "Expected drops / 100 keycodes".

Also, maybe take out "tries" altogether and just use "cost per item".

Thanks for putting this together! It should really help make the lockbox system more transparent!

Mustaine
04-09-2013, 10:52 AM
Reporting in all my T4 boxes.

Enepttastic
04-09-2013, 12:10 PM
Just saw this thread. Submitted results from lockboxes I just opened and will try to remember to keep reporting.

Noupoi
04-09-2013, 01:02 PM
Glad to see how many submissions we're getting!


As it is currently computed, the "Drop rate / Cost per try" statistic is actually the expected number of drops per 100 keycodes spent. It would probably be more clear to drop the % sign (but keep the multiplication by 100) and change the name to "Average drops / 100 keycodes" or "Expected drops / 100 keycodes".

Also, maybe take out "tries" altogether and just use "cost per item".

I can see how 'try' can be confusing to someone skimming through the spreadsheet. I've changed it back to item instead.

However, I don't agree with your conclusion on 'Drop rate / Cost per try'. 'Drop rate / Cost per try' is calculated as the drop rate divided by the number of keycodes you are spending, which is the drop rate for each keycode you spend. As an example, the current drop rate for a T2 Orange item is 2.96%, and you spent 4 key codes on that item. This means that for each keycode you spent, you got a 0.74% chance of getting an Orange. It's perfectly valid to express the likelihood of a drop, a probability, as a percentage.

Converting from probabilty to actual numbers, if each keycode out of the 100 keycodes gave you an item, you would expect 0.74% (the chance per keycode) of those items to be Orange, which does work out to 0.74 Orange items. Only, the statistic displayed is one of probability (again, chance per keycode), not the expected number of drops (out of 100) as you suggest, so the the percentage sign should be kept. Q.E.D.? :p

Armanewb
04-09-2013, 02:20 PM
Not trying to mess up your data, but you currently have several suspicious entries. Rows 27, 29, 30, 91, 185, 290 all have some ridiculous data statistically (e.g. 10/10/10/10, 0/0/0/10, 3/4/2/9) and may be someone trying to screw with your numbers.

Also, some of the data doesn't add up (may want to implement a validation) e.g. row 27, you'd expect to have a multiple of 3 with a Tier 3 lockbox, but there are 40 items there.

Edit: Did a quick check, 28 entries fail the basic validation. If you remove them, the results look more like:

http://i.imgur.com/EMxPeIX.jpg

Problem is, Tier 2 boxes have a low sample so we don't know if they are actually any good.

dreadburn
04-09-2013, 02:26 PM
Just added data for my 6 tier 3's and 2 tier 2's. Very nice thread so once I had read it I started collecting data today from my boxes.

Still haven't seen a Legendary yet :(

Warjack
04-09-2013, 02:29 PM
I always buy T3..i think its the best way for keycodes/good loot. Got already two oranges...a Saw machine gun and a bolt action sniper...exactly the weapons i needed!

Kerry
04-09-2013, 03:07 PM
Just know that it is far better to get the Tier 2 boxes over the others. As stated on the lock boxes themselves the item drop chances are only uncommon and above.

It would be different if the tier 4 gave a minimum of purple items only.

That is how it works now and I suspect it may get changed as there is absolutely no reason anyone should buy a lockbox above tier 2.

Going on 10 oranges from tier two boxes so far, including shields and grenades. Good luck!

Noupoi
04-09-2013, 03:41 PM
Not trying to mess up your data, but you currently have several suspicious entries. Rows 27, 29, 30, 91, 185, 290 all have some ridiculous data statistically (e.g. 10/10/10/10, 0/0/0/10, 3/4/2/9) and may be someone trying to screw with your numbers.

Also, some of the data doesn't add up (may want to implement a validation) e.g. row 27, you'd expect to have a multiple of 3 with a Tier 3 lockbox, but there are 40 items there.

Edit: Did a quick check, 28 entries fail the basic validation. If you remove them, the results look more like:

http://i.imgur.com/EMxPeIX.jpg

Problem is, Tier 2 boxes have a low sample so we don't know if they are actually any good.

There are some data entries that look suspect, but I'm not sure if I want to manually delete data, as this would mean that we are being selective we the data.

It would be a good idea to implement basic validation to check that the submitted data is a multiple of the number of items from that lockbox. Being totally honest, I'm not familiar with Google Sheets nor the Apps Scripting language, and am not sure how to best go about doing validation on ranges.

Having said that, this is one of the formulae I hacked together to perform said validation:
=ARRAYFORMULA(SUM(IF(ISBLANK('Form Responses'!E2:E)=FALSE,IF(MOD(('Form Responses'!E2:E+'Form Responses'!F2:F+'Form Responses'!G2:G+'Form Responses'!H2:H),2)=0, 'Form Responses'!E2:E, 0),0)))

There's a sizeable difference between the new 'validated frequency' column and the originals, so I'd rather someone check over my formulae to before I changed the main frequency column in case I screwed something up. :D

This is my attempt at explaining wth the formula does: Checks if row is for submission of specified tier (in this case, T2), and then checks that the SUM of all the results from that row is a multiple of the number of items in the lockbox. If any of these checks are failed, the total for that row is returned as 0.

Stickasylum
04-09-2013, 03:48 PM
Glad to see how many submissions we're getting!
However, I don't agree with your conclusion on 'Drop rate / Cost per try'. 'Drop rate / Cost per try' is calculated as the drop rate divided by the number of keycodes you are spending, which is the drop rate for each keycode you spend. As an example, the current drop rate for a T2 Orange item is 2.96%, and you spent 4 key codes on that item. This means that for each keycode you spent, you got a 0.74% chance of getting an Orange. It's perfectly valid to express the likelihood of a drop, a probability, as a percentage.

Converting from probabilty to actual numbers, if each keycode out of the 100 keycodes gave you an item, you would expect 0.74% (the chance per keycode) of those items to be Orange, which does work out to 0.74 Orange items. Only, the statistic displayed is one of probability (again, chance per keycode), not the expected number of drops (out of 100) as you suggest, so the the percentage sign should be kept. Q.E.D.? :p

The "Drop rate / cost per item" is a correct title, since that is how it's calculated. The value isn't a probability, however! Adding up the the Drop Rate / cost per item values for, say, Tier 2 gives you 25, not 100 as would be necessary to describe a probability distribution with percentages (and incidentally, 25 is number of items you can buy with 100 keycodes!) Tier 3 gives you 12.5 and Tier 4 gives you 6.25, again the number items 100 keycodes will buy you in those respective tiers.

The drop rate % for, say, tier 2 orange, gives us the number of orange items we would expect to get, on average, if we bought 100 items. Say the drop rate is 2%. Then the Drop Rate / cost per item computation looks like (with unit cancelling):


(2 orange items / 100 items) / (4 keycodes / item)

= (2 orange items / 100 items) * (1 item / 4 keycodes)
= 0.5 orange items / 100 keycodes

That is, for every 100 keycodes we spend, we expect to get 0.5 orange items, on average. :)

Noupoi
04-09-2013, 03:58 PM
Just know that it is far better to get the Tier 2 boxes over the others. As stated on the lock boxes themselves the item drop chances are only uncommon and above.

It would be different if the tier 4 gave a minimum of purple items only.

That is how it works now and I suspect it may get changed as there is absolutely no reason anyone should buy a lockbox above tier 2.

Going on 10 oranges from tier two boxes so far, including shields and grenades. Good luck!

Like Armanewb mentioned, the sample size for T2 lockboxes is far smaller than that for T3 and T4, so I'm not very confident in those results. The sample size is too small to rule out factors mentioned earlier in the thread like people with amazing drops (such as yours :P) looking for somewhere to discuss that, getting this thread more attention from people with lots of legendarys than than the majority who are getting the 'meh' results that I'm expecting.

But really, you must be really lucky- lady RNG is on your side. Out of all the lockboxes I've brought so far, I've not gotten one orange. >.>

Noupoi
04-09-2013, 04:13 PM
The "Drop rate / cost per item" is a correct title, since that is how it's calculated. The value isn't a probability, however! Adding up the the Drop Rate / cost per item values for, say, Tier 2 gives you 25, not 100 as would be necessary to describe a probability distribution with percentages (and incidentally, 25 is number of items you can buy with 100 keycodes!) Tier 3 gives you 12.5 and Tier 4 gives you 6.25, again the number items 100 keycodes will buy you in those respective tiers.

The drop rate % for, say, tier 2 orange, gives us the number of orange items we would expect to get, on average, if we bought 100 items. Say the drop rate is 2%. Then the Drop Rate / cost per item computation looks like (with unit cancelling):


(2 orange items / 100 items) / (4 keycodes / item)

= (2 orange items / 100 items) * (1 item / 4 keycodes)
= 0.5 orange items / 100 keycodes

That is, for every 100 keycodes we spend, we expect to get 0.5 orange items, on average. :)

That's an interesting point with cancelling units. They've never really been my strong suit. :P
I'm now not entirely sure about the % following the 'cost per item' which has now confused me.

Surely by calling the statistic 'per item', we are removing 1/item from the unit, essentially bringing the unit of cost per item to 'Keycodes'.

This means that (2 orange items / 100 items) / (4 keycodes) goes to 0.02 (probability) / 4 (keycodes), so the the y-axis should really is something like probabilty/keycode, which should have the unit /Keycode when probabilty is expressed as a number between 0 and 1, or %/Keycode when expressed as a percentage?

Edit: After typing all this out, I've just realised we're really just arguing semantics, and what really matters is which tier has the highest bar. ^^




˙sı ʇıun ʇɔǝɹɹoɔ ǝɥʇ ʇɐɥʍ ʍouʞ oʇ ʇuɐʍ llıʇs I

Stickasylum
04-09-2013, 04:18 PM
Like Armanewb mentioned, the sample size for T2 lockboxes is far smaller than that for T3 and T4, so I'm not very confident in those results. The sample size is too small to rule out factors mentioned earlier in the thread like people with amazing drops (such as yours :P) looking for somewhere to discuss that, getting this thread more attention from people with lots of legendarys than than the majority who are getting the 'meh' results that I'm expecting.

But really, you must be really lucky- lady RNG is on your side. Out of all the lockboxes I've brought so far, I've not gotten one orange. >.>


I'm going on 70 T2s without a single orange. Assuming the Orange drop rate is really around 5%, we'd need around 2000 T2 item drops to get a 95% confidence margin of error of + or - 1%. If the drop rate is lower, we'd need an even larger sample size! Also, I suspect that there will be some positive selection bias in the orange drop rates - people will likely remember to report drops more often when they pull oranges than when they pull greens! A large sample size will help combat this bias somewhat, so long as it comes in the form of people submitting large batches of drops.


That's an interesting point with cancelling units. They've never really been my strong suit. :P
I'm now not entirely sure about the % following the 'cost per item' which has now confused me.

Surely by calling the statistic 'per item', we are removing 1/item from the unit, essentially bringing us to the unit of cost per item to 'Keycodes'.

This means that (2 orange items / 100 items) / (4 keycodes) goes to 0.02 (probability) / 4 (keycodes), so the the y-axis should really is something like probabilty/keycode, which should have the unit /Keycode when probabilty is expressed as a number between 0 and 1, or %/Keycode when expressed as a percentage?

Whenever we have a "per" in the name of a statistic, we're going to have a unit on the top and on the bottom. Cost per item has units keycodes / item. In the computation, the item unit cancels with the bottom item unit of (2 orange items / 100 items). There isn't a strikeout font in this forum, but I'll use to indicate the cancelling:

(2 orange items / 100 items) / (4 keycodes / item)

= (2 orange items / 100 items) * (1 item / 4 keycodes)
= 0.5 orange items / 100 keycodes



Edit: After typing all this out, I've just realised we're really just arguing semantics, and what really matters is which tier has the highest bar. ^^

Yep! The statistic definitely works as a measure of drop efficiency. I just think
"Expected number of drops per 100 keycodes spent" (or something similar) makes the actual value easier to interpret, and gives the reader an idea of how much they would have to spend for an orange. And it shouldn't have a % sign :)

Noupoi
04-09-2013, 04:51 PM
Whenever we have a "per" in the name of a statistic, we're going to have a unit on the top and on the bottom. Cost per item has units keycodes / item.

Hehe, I agree with how you've cancelled the units, but running the original unit for 'Cost per item' through my head doesn't make logical sense to me.

As an example, if a shopkeeper told you something costed £5 per bottle, and later someone else asked you how much they needed to pay per bottle, you would answer £5, not £5 per bottle, which when taken in context to the question, is £5 per bottle per bottle. But if they asked you 'how much does it cost?', you would answer £5 per bottle. o.O

Edit: As another example, but without the per bit: if box was 50cm wide, and someone asked out how wide the box was, you would answer 50cm. But if you were asked how many cm wide the box was, you would answer 50. So if the unit was stated in the quantity, there would be no need to state it again in the answer.

Avenger
04-09-2013, 05:15 PM
Good to see lots more entries being added - just added a few more T3 and T2 boxes of my own - still yet to get an orange.

Data validation is a good idea, and if a formula can apply it even better. A few clearly disruptive entries are in there - 10 of the 16 oranges reported in T2 boxes come from one person's entry.

hugdealer
04-09-2013, 05:25 PM
opened a tier 3, got 2 orange! no joke!

the nade was a huge upgrade
http://i45.tinypic.com/34s58hh.jpg

http://i50.tinypic.com/sm3m6g.jpg

knightofthebroho
04-09-2013, 05:26 PM
I think I've opened 4 (maybe 5) T4 boxes now and haven't gotten anything better than a blue. I think the RNG hates me.

Stickasylum
04-09-2013, 05:52 PM
Hehe, I agree with how you've cancelled the units, but running the original unit for 'Cost per item' through my head doesn't make logical sense to me.

As an example, if a shopkeeper told you something costed £5 per bottle, and later someone else asked you how much they needed to pay per bottle, you would answer £5, not £5 per bottle, which when taken in context to the question, is £5 per bottle per bottle. But if they asked you 'how much does it cost?', you would answer £5 per bottle. o.O

Edit: As another example, but without the per bit: if box was 50cm wide, and someone asked out how wide the box was, you would answer 50cm. But if you were asked how many cm wide the box was, you would answer 50. So if the unit was stated in the quantity, there would be no need to state it again in the answer.

Ah, I see where your confusion is coming from now! Your second example is actually the solution to this problem. When you say "50" in answer to how many cm wide the box is, there is an implied unit of centimeters (from the question). Similarly, when you say the cost per bottle is £5, the "per bottle" unit is implied, since it is included in the preceding phrase. In fact, with unit prices, the "per unit" part is often implied, since it is obvious. You could just say bottles cost £5, and most people would assume you mean "per bottle". However, if the quantity doesn't have an obvious unit, then it is easy to see how the "per quanity" unit must be part of the dimension. For instance, saying "flour costs £5" is ambiguous without also specifying how much you get for £5!

Basically, all cost measures have a "per unit" part. Even if that part is simple enough that we usually ignore it and still be understood, it is still important for dimensional analysis!

Edit: adding in my T2 data!

Edit2: I think I see where some of the invalid data might be coming from. I had 108 values to submit, so I had to use multiple forms, some of which probably didn't add up to a multiple of 2. I haven't made google doc forms before, but if you could change the selection to a number entry, that would probably fix a lot a of the problem with invalid entries (and make bulk entry easier).

Armanewb
04-09-2013, 06:06 PM
There are some data entries that look suspect, but I'm not sure if I want to manually delete data, as this would mean that we are being selective we the data.

It would be a good idea to implement basic validation to check that the submitted data is a multiple of the number of items from that lockbox. Being totally honest, I'm not familiar with Google Sheets nor the Apps Scripting language, and am not sure how to best go about doing validation on ranges.

Having said that, this is one of the formulae I hacked together to perform said validation:
=ARRAYFORMULA(SUM(IF(ISBLANK('Form Responses'!E2:E)=FALSE,IF(MOD(('Form Responses'!E2:E+'Form Responses'!F2:F+'Form Responses'!G2:G+'Form Responses'!H2:H),2)=0, 'Form Responses'!E2:E, 0),0)))

There's a sizeable difference between the new 'validated frequency' column and the originals, so I'd rather someone check over my formulae to before I changed the main frequency column in case I screwed something up. :D

This is my attempt at explaining wth the formula does: Checks if row is for submission of specified tier (in this case, T2), and then checks that the SUM of all the results from that row is a multiple of the number of items in the lockbox. If any of these checks are failed, the total for that row is returned as 0.

I uploaded my file to let you see how I did it.

https://docs.google.com/spreadsheet/ccc?key=0AkapJ4wrMmSxdEMyN3NpelpVZUZzYWt0aE80WWdRS FE&usp=sharing

It uses a modulus and a vlookup to check whether the sum is appropriate.

Maetrik
04-09-2013, 07:01 PM
I just opened a T4 lockbox and got:
-Orange
-Purple
-Purple
-Green

RawrKitty
04-09-2013, 07:26 PM
OP i hope you or some other person is clearing out the obvious troll submissions.

i see several submissions that have gotten '10 legendaries" from one box.

its really ****ing up the statistics.

Manzikeen
04-09-2013, 08:26 PM
3 Tier 2 Lockboxes (already added the data)

Green-Green
Green-Purple
Green-Purple

Noupoi
04-10-2013, 03:36 AM
Edit2: I think I see where some of the invalid data might be coming from. I had 108 values to submit, so I had to use multiple forms, some of which probably didn't add up to a multiple of 2. I haven't made google doc forms before, but if you could change the selection to a number entry, that would probably fix a lot a of the problem with invalid entries (and make bulk entry easier).

This is something I could do, though this opens up the possbility for people to also enter large amounts of fake data easily, having a greater effect on the results. This would require a lot more validation rules and perhaps even verification of large sets of data- something I'm not sure if I would want to do.

Stickasylum
04-10-2013, 07:58 AM
That makes sense!

Doctor love
04-10-2013, 11:42 AM
Why don't we just start a petition for Trion to give us the drop rates of orange,purple etc of the lock boxes like they should have done........

Brian
04-10-2013, 11:48 AM
Wow! This thread is superb. Great job Noupoi and those who contributed.

Thorbrand
04-10-2013, 11:50 AM
You need a over 10 option for Greens, Blues, and Purples. Can't wait until I have over 10 Oranges.

Arctic Fox
04-10-2013, 11:52 AM
It appears upon checking right now, that tier 2 has taken the lead in efficiencies overall

Thorbrand
04-10-2013, 11:53 AM
I haven't bought that many lock boxes and T2 and T3 or useless to purchase. I know people say otherwise on these forums but all my good gear at my Ego level have always come from T4 lockboxes and you never actually get a useless lockbox on T4 which isn't the case with T2 and T3. I have 2 oranges now and haven't really had to work that hard to get them. To easy if you was me.

Brian
04-10-2013, 12:00 PM
I haven't bought that many lock boxes and T2 and T3 or useless to purchase. I know people say otherwise on these forums but all my good gear at my Ego level have always come from T4 lockboxes and you never actually get a useless lockbox on T4 which isn't the case with T2 and T3. I have 2 oranges now and haven't really had to work that hard to get them. To easy if you was me.Two oranges? Ah man! I have the worst of luck. Opened about 11 T4 boxes; perhaps a little bit more?

Yokai
04-10-2013, 12:25 PM
I figured I'd point this out if nobody else has already. I haven't read all pages of this thread, but one thing the spreadsheet lacks is the cumulative binomial probability. Which is the real number to look at when deciding the best tier to buy.

Let's use legendary (orange) as an example. Say you have 72 key codes, and you want to know what the best chance of scoring AT LEAST 1 legendary from those 72 key codes. Based on the collected data so far, it seems like T2 = 4% chance, T3 = 6% chance, and T4 = 8% chance.

With cumulative binomial probablity, you have the following odds for spending those 72 key codes:

1x T4 lockbox = 4 trials at 8% per trial = 28.3% cumulative probability to get at least one legendary from the box

3x T3 lockboxes = 9 trials at 6% per trial = 42.7% cumulative probability to get at least one legendary from the 3 boxes

9x T2 lockboxes = 18 trials at 4% per trial = 52.0% cumulative probability to get at least one legendary from the 9 boxes

----

As you can see, your best cumulative probability for scoring a legendary per X number of keys comes from T2 lockboxes, pure and simple. This is basic probability math.

Now of course, if Trion changes the drop rates on us, we'd have to collect more data again to reverse engineer the drop rates. Regardless, once we know the drop rates for each tier, you can go find a binomial statistics calculator and figure out the odds exactly.

Here's a good online probabilty calculator that I use all the time for questions like these, and also for building card decks for CCGs (wherein you want to use hypergeometric distribution to figure out your deck builds).

http://stattrek.com/online-calculator/binomial.aspx

BTW I'm pretty sure that Google spreadsheets has a binomial probability function built in, but you'll have to create some tables of prebuilt probabilities and then write some gscript to pick the right numbers out of the tables. It's a PITA but I did it once for a Google-docs based deck builder for Shadow Era decks.

Diedel
04-10-2013, 01:09 PM
I figured I'd point this out if nobody else has already. I haven't read all pages of this thread, but one thing the spreadsheet lacks is the cumulative binomial probability. Which is the real number to look at when deciding the best tier to buy.

Let's use legendary (orange) as an example. Say you have 72 key codes, and you want to know what the best chance of scoring AT LEAST 1 legendary from those 72 key codes. Based on the collected data so far, it seems like T2 = 4% chance, T3 = 6% chance, and T4 = 8% chance.

With cumulative binomial probablity, you have the following odds for spending those 72 key codes:

1x T4 lockbox = 4 trials at 8% per trial = 28.3% cumulative probability to get at least one legendary from the box

3x T3 lockboxes = 9 trials at 6% per trial = 42.7% cumulative probability to get at least one legendary from the 3 boxes

9x T2 lockboxes = 18 trials at 4% per trial = 52.0% cumulative probability to get at least one legendary from the 9 boxes

----

As you can see, your best cumulative probability for scoring a legendary per X number of keys comes from T2 lockboxes, pure and simple. This is basic probability math.

Now of course, if Trion changes the drop rates on us, we'd have to collect more data again to reverse engineer the drop rates. Regardless, once we know the drop rates for each tier, you can go find a binomial statistics calculator and figure out the odds exactly.

Here's a good online probabilty calculator that I use all the time for questions like these, and also for building card decks for CCGs (wherein you want to use hypergeometric distribution to figure out your deck builds).

http://stattrek.com/online-calculator/binomial.aspx

BTW I'm pretty sure that Google spreadsheets has a binomial probability function built in, but you'll have to create some tables of prebuilt probabilities and then write some gscript to pick the right numbers out of the tables. It's a PITA but I did it once for a Google-docs based deck builder for Shadow Era decks.

I really hope that the current percentages are wrong, because that would basically mean that the higher boxes are a rip-off and i would hope that somebody at TRION would be better at basic math. I assumed that the cahnces for T4 would be so high that it is more likely to get legendaries/purples even with the reduced amount of items, you already get far less loot, so the Chance is higher that you won't be using it already.

Stickasylum
04-10-2013, 01:58 PM
That same information is encapsulated in the "Drop rate / cost per item" column, which is actually the expected number of items at each level per 100 keycodes spent. While this doesn't tell you the probability of at least one legendary, it does tell you how many you would expect to get (and the efficiency ordering will be the same, so long as the # of lockboxes have the same cost). Since most people probably care more about the number of legendaries they get than the probability of getting at least one legendary, it is probably the more useful metric of efficiency (though it wouldn't hurt to have both!)

Note, however, that the # of lockboxes per keycode amount don't have a 1:3:9 ratio, so comparing those probabilities isn't so useful in terms of resources. The simplest ratio that actually leads to equivalent keycode pricing is 3:8:24 - your approximation overweights T2 and underweights T3 slightly.

I'm skeptical of the T2 legendary drop rate as it is currently estimated. My own estimate (from fewer data points) is closer to 1%, and most of legendaries in this table came from a single 10,10,10,10 entry.

Bluegobln
04-10-2013, 02:05 PM
This pisses me off. Seriously, why the F would you design it that way, the BASIC math involved to make sure this was at WORST equally cost effective at each tier of lockbox is so easy to do I can do it on napkin right now...

How they decided to do it and make higher tier lockboxes WORSE is just stunning. Are you kidding? And if it it WAS intentional, once again, what the hell? Why? Are you kidding?

MaritimeMage
04-10-2013, 02:09 PM
Thanks for this. Keep up the good work.

cfStatic
04-10-2013, 02:25 PM
This thread has some good information in it. However, the one thing that has put me off so far is that a dev has said outright that you're not supposed to be getting uncommon(green) items in tier 4 lock boxes. But because we are, there is therefore a possibility that they're somehow bugged with the loot tables that they use to reward players with. Legendaries could be a bit more common in T4 if that was the case.

Yokai
04-10-2013, 03:08 PM
That same information is encapsulated in the "Drop rate / cost per item" column, which is actually the expected number of items at each level per 100 keycodes spent. While this doesn't tell you the probability of at least one legendary, it does tell you how many you would expect to get (and the efficiency ordering will be the same, so long as the # of lockboxes have the same cost). Since most people probably care more about the number of legendaries they get than the probability of getting at least one legendary, it is probably the more useful metric of efficiency (though it wouldn't hurt to have both!)

Note, however, that the # of lockboxes per keycode amount don't have a 1:3:9 ratio, so comparing those probabilities isn't so useful in terms of resources. The simplest ratio that actually leads to equivalent keycode pricing is 3:8:24 - your approximation overweights T2 and underweights T3 slightly.

I'm skeptical of the T2 legendary drop rate as it is currently estimated. My own estimate (from fewer data points) is closer to 1%, and most of legendaries in this table came from a single 10,10,10,10 entry.

The cost info in the spreadsheet is NOT the same as the cumulative probablity. Two different measures.

The cost ratio is irrelevant for cumulative probability. Given that you have 72 keys sitting in your hand, you have the BEST probability of scoring an orange by spending all of them on T2 lockboxes. It doesn't matter that 8 keys will be left over if you buy the T4 lockbox.

Yokai
04-10-2013, 04:42 PM
BTW I should clarify one thing too. While you have a better overall cumulative probability of scoring an orange by spending only on T2 lockboxes, you will have far fewer blues and purples show up in your attempts. With T3 and T4 purchases, you'll at get a lot more purples and blues while you wait for an orange to show up.

So on the whole, T3 and T4 are probably still better choices to buy. Because even though you have a lower cumulative probability to score an orange, you have a much higher cumulative probability to score purples and blues.

I'll try to run the numbers later for the best cumulative probability of purples and for blues.

Doctor love
04-10-2013, 05:13 PM
Everything in real life that you spend money on with a chance of gaining something out of it like the lotto or scratch and win all provide us with a the chances of getting what we want. With that said, i believe we should focus our energy on getting trion to just tell us the percentages of drop rates for each tier of lockboxes. At the end of the day you can spend real money on these boxes. Would you spend money on a tier 4 knowing thats it % drop made it worst then a tier 2?Lets start a thread to get the real percentages from the Trion company not some made up faulty percentage from people who lie.... sorry starter of this thread.....

Stickasylum
04-10-2013, 05:49 PM
The cost info in the spreadsheet is NOT the same as the cumulative probablity. Two different measures.


Originally, my intuition told me that both the expected number of drops per X keycodes and the probability of at least one drop given X keycodes would give the same ordering of tiers, and thus be equivalent in terms of an efficiency measure. I did the math, however, and it turns out that they usually give the same answer, but not always! Here's an example where we get different answers (it will look a bit contrived because there is a very small window where the answers are different):

Let p_2 = 0.05 and p_4 = 0.19 be the probabilities an orange item for one item of a T2 and T4 lockbox, respectively. Suppose we spend 64 keycodes on lockboxes (1 T4 or 8 T2 boxes). Then the probabilities of at least on orange are:

P(at least one orange from 1 T4 box) = 1 - (1 - 0.19)^(1*4) = 0.5695
P(at least one orange from 8 T2 boxes) = 1 - (1 - 0.05)^(8*2) = 0.5599

So it looks like T4 boxes are they way to go! But we get a lot more items out of 8 T2 boxes (16 vs 4), so how many oranges do we expect to get on average?

Average number of oranges from 1 T4 box = 0.19 * (1*4) = 0.76
Average number of oranges from 8 T2 boxes = 0.05 * (8*2) = 0.8

Now it looks like T2 boxes are better! So which strategy should we choose? If what we care about is getting getting one orange, as opposed to none, we should go with the T4 box. On the other hand, if we care about getting as many oranges as possible in the long run, we should stick to buying T2 boxes! I believe that most people probably fall in the second camp. Since you'll likely be buying many boxes over your time in Defiance, the average number of orange items per keycode spent seems like the more relevant measure of efficiency of boxes.

That said, both numbers are useful, and for the probabilities the dropboxes seem to have, both methods will lead to the same buying strategies. However, the efficiency measure currently used is better for getting the more oranges on average. Also, it is much easier to generalize. If the expected number of oranges per 100 keycodes spent is 2 and you spend 200 keycodes, you expect 4 oranges. Probabilities need to be recomputed according to formula:

Probability of at least one orange = 1 - (1-p)^n,
where p is the p of an orange for one item and n the number of items purchased.

Not as easy to do in one's head!



The cost ratio is irrelevant for cumulative probability. Given that you have 72 keys sitting in your hand, you have the BEST probability of scoring an orange by spending all of them on T2 lockboxes. It doesn't matter that 8 keys will be left over if you buy the T4 lockbox.

It is relevant in the sense of the number of boxes purchasable. Suppose p_2 = 0.05 and p_4 = 0.2. With 72 keycodes, 1 T4 box gets you a probability of at least one orange of 1-(1-0.2)^4 = 0.59 of at least one orange. Nine T2 boxes gives you a probability of at least one orange of 1-(1-0.05)^18 = 0.60. Your best strategy isn't to blow it all on T2 boxes, though! If you just bought one T4 box, then saved up for another, the probabilities for 2 T4 and 16 T2 boxes are, respectively 1-(1-0.2)^8 = 0.83 and 1-(1-0.05)^32 = 0.81. In this case, buying T4 boxes is the better strategy, but ignoring those left-over keycodes biased your analysis.


BTW I should clarify one thing too. While you have a better overall cumulative probability of scoring an orange by spending only on T2 lockboxes, you will have far fewer blues and purples show up in your attempts. With T3 and T4 purchases, you'll at get a lot more purples and blues while you wait for an orange to show up.

So on the whole, T3 and T4 are probably still better choices to buy. Because even though you have a lower cumulative probability to score an orange, you have a much higher cumulative probability to score purples and blues.

I'll try to run the numbers later for the best cumulative probability of purples and for blues.

This is a good point, but remember that you get a lot more items from T2 boxes! You get twice as many per keycode as T3 boxes, and four times as many items than T4 boxes. I suspect that this probably evens out the blues and purples in favor of T2s as well (though I still suspect that the T2 orange drop rate is 1 or 2%, and not sufficient to be more efficient than T3s).

Drall
04-10-2013, 06:18 PM
Again, could a dev just post the #s? If your math is terrible/wrong that is fine, just own it and let us know while you fix the problem. Keeping the players in the dark is unacceptable and there is clearly the perception that T4 boxes are terrible.

(Also 3b 1g x2 from last two T4 boxes at 1XXX ego rating)

Stickasylum
04-10-2013, 06:29 PM
Since I was doing the math already, I went ahead and computed the probability cutoffs determining when one tier is better than another.

Let p2, p3, p4 be the probability of a given rank of item for each item in a T2, T3, and T4 lockbox, respectively. From the perspective of maximizing the expected number of drops of that rank per keycode spent, then

T2 is better than T3 if 2*p2 > p3
T3 is better than T4 if 2*p3 > p4
T2 is better than T4 if 4*p2 > p4

For example, if the orange drop rate from T4 boxes is 8%, then T3 boxes are a better investment if their orange drop rate is greater than 4% and T2 boxes are better if their orange drop rate is greater than 2%.

This also applies to combined groups. The Blue/Purple/Orange drop rate from T4 boxes looks like it is about 75% now. Then T3s are a better investment if the B/P/O drop rate is greater than 37.5% and T2s are better with a rate greater than 18.75%. Both of these look like they are true! T3s look to have a B/P/O rate of about 60%, meaning T2s are only better with a B/P/O rate over 30%. Right now, they look like they have about that (though my personal data puts them closer to 25%). So from our current data, it looks like T2 and T3 and neck-and-neck for BPO efficiency, both beating T4s handily.


We can get similar bounds for the probability of at least one item of a given rank for given number of keycodes spent, but the inequalities are bit more complicated:

T2 has better probability than T3 if p2 > 1-(1-p3)^(1/2)
T3 has better probability than T4 if p3 > 1-(1-p4)^(1/2)
T2 has better probability than T4 if p2 > 1-(1-p4)^(1/4)

For example, if T4 has a 8% orange drop rate, then the probability of at least one orange is better for T3 boxes if p3 > 1-(1-0.08)^(1/2) = 0.041 (i.e. 4.1%). The probability of at least one orange is greater for T2 boxes if the T2 orange drop rate is greater than 2.1%.

Again, categories can be combined. The current T4 B/P/O drop rate is about 75%, so T3 needs better than 50% and T2 better than 29% to have better probability of at least on rare or better, both of which appear to be true (though T2 is awfully close). For T2 to have better probability than T3, the T2 drop rate needs to be at least 37%, which doesn't appear to be true. Thus T3 appears to have the best probability of getting at least one rare or better item (though T2 will give roughly the same number of rare or better items, on average!)


Again, could a dev just post the #s? If your math is terrible/wrong that is fine, just own it and let us know while you fix the problem. Keeping the players in the dark is unacceptable and there is clearly the perception that T4 boxes are terrible.

(Also 3b 1g x2 from last two T4 boxes at 1XXX ego rating)

That would be awesome. By all means, post a thread and link it here! In the meantime, we're having a philosophical discussion about statistics, because statistics are cool.

Also, there isn't anything wrong with the math, we're mostly talking about which statistics are most appropriate for determining buying strategy. Distinctions like expected value vs. probability are important regardless of whether the numbers come from Trion Worlds or observation - we still need to decide how turn them into a strategy!

Montrovont
04-10-2013, 09:54 PM
I saw this the other day and just got around to posting my info into the form. I've been tracking my T4 purchases since launch (that's all I've bought). My luck seems to be horrible with these, as I'm way below the percentages in the results. Only 1 orange and 4 purples out of 11 T4 lockboxes so far:

(4/2/2013, Shondu's Consulate, EGO 287, Resources) - Blue VOT Spanner Protector, Green Pyroblast Std-23W, Green VOT Outbreaker, Blue FRC Big Boomer

(4/4/2013, Happy Pow Farms, EGO 376, Resources) - Blue VBI TACC Assault Rifle, Blue Ironclad Blastproof II D, Blue VOT Swarm Cannon, Blue VOT Fragger

(4/5/2013, Bug 'n' Chug, EGO 449, Resources) - Blue VOT Infector, Green FRC heavy Scattergun, Green VBI LM-43 Thunder, Blue VBI GL-1 Ground Pounder

(4/6/2013, Headlands Transit Depot, Resources) - Blue VOT Auto Lobber, Purple VOT Blaster, Blue FRC Birdshot Pump, Blue FRC Assault Carbine

(4/6/2013, Iron Demon Ranch, EGO 550, Resources) - Blue VOT Rebounder Cannon, Green FRC Heavy Sawed-Off Scattergun, Blue VBI LM-12 Rocker, Purple Rebel Blastproof III DX

(4/6/2013, Earth Republic Camp, EGO 618, Resources) - Orange FRC Bull Rush 45, Purple VBI Guided Launcher, Blue VBI Short-Barrel Shotgun, Blue FRC Heavy Assault Carbine

(4/7/2013, Last Chance, EGO 662, Resources) - Blue VBI TACC Assault Rifle, Green FRC Auto-Scattergun, Green VOT Pulser, Blue Hurricane Reloader II D

(4/7/2013, Muir Processing, EGO 785, Resources) - Blue VOT Swarm Cannon, Blue FRC Heavy Scattergun, Blue FRC Saw, Purple VOT Tachmag Pulser

(4/7/2013, Bloodbath Gorge, EGO 824, Resources) - Blue VBI BM-4 Stingray, Green VOT Infector, Blue FRC Sub-Carbine, Blue VBI INF-27 Immunizer

(4/10/2013, Last Chance, EGO 862, Reources) - Blue FRC Big Boomer, Green VOT Pulser, Blue VBI BAS-7 Derailer, Blue VOT Nano Fragger

(4/10/2013, Headlands Transit Depot, EGO 900, Resources) - Blue VOT Spanner, Blue VBI LM-43 Thunder, Green VBI CS-X Cluster Shot, Blue FRC Birdshot Pump

Yokai
04-10-2013, 10:11 PM
@stickasylum

Your theory crafting is fun for some, but also clouds the issue for many. Your first long example used T4 base numbers that were unrealistic (19% drop rate? No.)

In your next post, you came to essentially the same conclusion that running the numbers through a simple binomial probability calculator would yield.

I favor the simple approach. This type of decision requires applying binomial distribution. There are easy to use calculators for this. Explain what variables to plug in and which type of cumulative result to look at.

I'm not trying to rag on your contribution, but merely to clear up something I feel will confuse other readers.

I'll summarize as simply as possible: the best chance at scoring an orange will come from buying T2 lockboxes, but you'll get a ton of greens and very few blues and purples before you finally score an orange. If you buy T3 or T4 lockboxes, you'll have less chance at scoring an orange given the same number of key codes spent, but you will find a LOT more blues and purples in the process. Name your poison. Trion actually balanced the cost/risk/reward fairly well, IMO.

Again, the exact numbers are as follows, if the drop rate for an orange per tier is 4, 6, and 8 percent respectively for T2, T3, and T4. This assumes you have 72 key codes to spend and are trying to decide how to best spend them. With those 72 keys, you could buy one T4 box, 3 T3 boxes, or 9 T2 boxes.

1x T4 lockbox = 4 trials at 8% per trial = 28.3% cumulative probability to get at least one legendary from the box

3x T3 lockboxes = 9 trials at 6% per trial = 42.7% cumulative probability to get at least one legendary from the 3 boxes

9x T2 lockboxes = 18 trials at 4% per trial = 52.0% cumulative probability to get at least one legendary from the 9 boxes

Drall
04-10-2013, 10:26 PM
Since I was doing the math already, I went ahead and computed the probability cutoffs determining when one tier is better than another.

Let p2, p3, p4 be the probability of a given rank of item for each item in a T2, T3, and T4 lockbox, respectively. From the perspective of maximizing the expected number of drops of that rank per keycode spent, then

T2 is better than T3 if 2*p2 > p3
T3 is better than T4 if 2*p3 > p4
T2 is better than T4 if 4*p2 > p4

For example, if the orange drop rate from T4 boxes is 8%, then T3 boxes are a better investment if their orange drop rate is greater than 4% and T2 boxes are better if their orange drop rate is greater than 2%.

This also applies to combined groups. The Blue/Purple/Orange drop rate from T4 boxes looks like it is about 75% now. Then T3s are a better investment if the B/P/O drop rate is greater than 37.5% and T2s are better with a rate greater than 18.75%. Both of these look like they are true! T3s look to have a B/P/O rate of about 60%, meaning T2s are only better with a B/P/O rate over 30%. Right now, they look like they have about that (though my personal data puts them closer to 25%). So from our current data, it looks like T2 and T3 and neck-and-neck for BPO efficiency, both beating T4s handily.


We can get similar bounds for the probability of at least one item of a given rank for given number of keycodes spent, but the inequalities are bit more complicated:

T2 has better probability than T3 if p2 > 1-(1-p3)^(1/2)
T3 has better probability than T4 if p3 > 1-(1-p4)^(1/2)
T2 has better probability than T4 if p2 > 1-(1-p4)^(1/4)

For example, if T4 has a 8% orange drop rate, then the probability of at least one orange is better for T3 boxes if p3 > 1-(1-0.08)^(1/2) = 0.041 (i.e. 4.1%). The probability of at least one orange is greater for T2 boxes if the T2 orange drop rate is greater than 2.1%.

Again, categories can be combined. The current T4 B/P/O drop rate is about 75%, so T3 needs better than 50% and T2 better than 29% to have better probability of at least on rare or better, both of which appear to be true (though T2 is awfully close). For T2 to have better probability than T3, the T2 drop rate needs to be at least 37%, which doesn't appear to be true. Thus T3 appears to have the best probability of getting at least one rare or better item (though T2 will give roughly the same number of rare or better items, on average!)



That would be awesome. By all means, post a thread and link it here! In the meantime, we're having a philosophical discussion about statistics, because statistics are cool.

Also, there isn't anything wrong with the math, we're mostly talking about which statistics are most appropriate for determining buying strategy. Distinctions like expected value vs. probability are important regardless of whether the numbers come from Trion Worlds or observation - we still need to decide how turn them into a strategy!

Didn't mean any math/statistics you guys are looking at, I just mean the actual numbers behind each tier. Regardless of everything being done here you are all still just speculating since you don't have the actual information or a large enough amount of information to draw your data from.

By wrong I meant their assumption that t3/4 > t2 when they designed it which doesn't exactly seem to be the case.

Mattwi
04-10-2013, 10:29 PM
So it would be best to buy Tier 2 to get the best rarity? Also will they fix this because it seems like a no brainer to buy tier 2s?

Im just wondering which is best for me to get good stuff at the least cost. I bought a tier 4 and got nothing good. So which should i buy for the best probability to get good stuff. (Including the cost of course)

nnja303
04-10-2013, 10:39 PM
In my personal opinion you cant just take each tear box into consideration when figuring this out... From my experience so far I have better luck at different vendors, I think this may play into it. Every piece of orange gear ive gotten is from muir. Ive bought numerous boxes from san fran as well and those seem to have horrid drop rates, those usually end up more greens than any other.

Stickasylum
04-10-2013, 10:46 PM
@stickasylum

Your theory crafting is fun for some, but also clouds the issue for many. Your first long example used T4 base numbers that were unrealistic (19% drop rate? No.)

In your next post, you came to essentially the same conclusion that running the numbers through a simple binomial probability calculator would yield.

I favor the simple approach. This type of decision requires applying binomial distribution. There are easy to use calculators for this. Explain what variables to plug in and which type of cumulative result to look at.

I'm not trying to rag on your contribution, but merely to clear up something I feel will confuse other readers.


The math was an example to show that using the average number of orange drops per 100 keycodes spent (labeled "Drop rate / cost per keycode" in the table) vs. using the probability of at least one orange drop can lead to different conclusions about which lockbox to buy. The numbers were contrived, because most of the time, the two methods agree. This does mean, however, that there is a very real difference between the two methods. I argued that using average number of drops per 100 keycodes is what most people would actually want, as most people are interested in maximizing their volume of oranges, rather than minimizing the probability of not seeing any (you'll quickly get at least one orange using any method). The average number of drops per 100 keycodes is also easier to compute, and easier to generalize (just multiply by the number of keycodes you spend / 100, no binomial calculator necessary!).

The second example was to show you that, if you are computing the probability of at least one orange drop, in order for the probabilities to be comparable you need to use a lockbox ratio with equal values. Using different keycode amounts worth of lockboxes for each tier can lead to incorrect conclusions about which lockboxes are best to buy.

Did you not find these arguments persuasive? Basically, we both believe that the simplest method should be used. I just think that the current method (average number of drops per 100 kecodes, labelled "Drop rate / cost per item") is both the simplest method, and the measure to use if you want to maximize volume (the probability measure is more suited to one-off trials, like game shows, lotteries, or personal insurance).


So it would be best to buy Tier 2 to get the best rarity? Also will they fix this because it seems like a no brainer to buy tier 2s?

Im just wondering which is best for me to get good stuff at the least cost. I bought a tier 4 and got nothing good. So which should i buy for the best probability to get good stuff. (Including the cost of course)

From the current data, it looks that way. If the numbers are correct, you'll get a better orange and purple drop rate per keycode spent than either T3 or T4 boxes, and nearly the same blue drop rate per keycode spent as T3 boxes.

I'm skeptical of the listed orange drop rate for T2 boxes, however, because it seems to be heavily skewed by a single data entry. I suspect that the orange drop rate for T2 boxes is closer to 1-2%, in which case T3 boxes are the clear winner for both orange drops and blue or better drops. T2 boxes would still be better than T4 for blue and purple drops, but not for oranges (but that's neither here nor there, since you should be buying T3s :D )


Didn't mean any math/statistics you guys are looking at, I just mean the actual numbers behind each tier. Regardless of everything being done here you are all still just speculating since you don't have the actual information or a large enough amount of information to draw your data from.

By wrong I meant their assumption that t3/4 > t2 when they designed it which doesn't exactly seem to be the case.

Pretty much. There are enough data, at least, to say that T4 seems to be a bad investment!

Drall
04-10-2013, 11:11 PM
Feel a little insulted with the current drop rates. Lost count of how many I've opened of each type but still only have 1 orange and 1 purple to show for it. (1329 ego so w/e playtime/crazy amount of keys that is)

Also another T4 just now with 1 green, 3 blues. =/

If nothing else it seems like they should remove greens from T4 and maybe up the purple rate? Orange being ultra rare seems fine but its annoying when 4/4 is trash.

Gabbro1000
04-10-2013, 11:26 PM
I don't think it needs it, but BUMP!

Awsome idea, finally getting some clarification on the drop rates of the boxes. I'll start noting what I get from mine.

Jens
04-11-2013, 12:25 AM
ive opened a few tier 2 dont remember what i got but also got 12 tier 4 lockboxes. with all blue. no green, no purple, no roange. just blue :)

Yokai
04-11-2013, 04:10 AM
The second example was to show you that, if you are computing the probability of at least one orange drop, in order for the probabilities to be comparable you need to use a lockbox ratio with equal values. Using different keycode amounts worth of lockboxes for each tier can lead to incorrect conclusions about which lockboxes are best to buy.



This is the part of your argument that I disagree with. A T4 give you exactly 4 trials at 8% each trial. A T3 gives you exactly 3 trials at 6% per trial. A T2 gives you exactly 2 trials at 4% per trial.

72 Keycodes will therefore yield 4 trials at T4, 9 trials at T3, and 18 trials at T2.

We have all the exact numbers we need to calculate exact cumulative probability in each scenario per 72 available keys to spend, and that tells a very clear and accurate story: you have the best chance to score at least one orange by buying only T2 lockboxes, but you'll see very few purples and greens in the process. You have a lower chance to score oranges with either T3 or T4, but you'll see more purples and blues in the process.

IMO, the "cost per" measure does not tell nearly as clear of a story.

Moat people see plenty of good purples show up on the vendor Sale slots or as guaranteed rep items. Same for blues, which also show up frequently as drops. However, nobody sees oranges on the vendors or as drops. Therefore, most people buy boxes specifically to get oranges.

BTW, -IF- the drop rate for oranges from a T2 is only 2%, then the cumulative probability for 72 Keycodes is 30.5%. In which case T3 would in fact be the best probability even if the true drop rate for T3 is also only 5% (which drops the *** prob for T3 to 36.5%).

but so far the collected data indicates 4, 6, and 8 percent drop rates, making T2 the best bet for oranges.

Nomad N7
04-11-2013, 06:27 AM
Wow! great idea. I just bought 2 Tier 3 boxes and with drastically different results.
#1: 2 Green - 1 Blue - 0 Purple - 0 Orange
#2: 0 Green - 3 Blue - 0 Purple - 0 Orange

Haha. I've submitted in the form too. Good job for making this statistics study. Keep it up!

Armanewb
04-11-2013, 08:03 AM
IMO, the "cost per" measure does not tell nearly as clear of a story.


I think this metric should be renamed "Expected Items per 100 keys" and the values multiplied by 100. That should solve any confusion around it. It's not a percentage (they don't add to 100% anyway), it's just a benchmark metric.

Also, can you add a column with number of observations? Just sum up total items for that tier (G + B + P + O) divided by number of items per box. That might help put the statistical significance into perspective.

Yokai
04-11-2013, 09:40 AM
I think this metric should be renamed "Expected Items per 100 keys" and the values multiplied by 100. That should solve any confusion around it. It's not a percentage (they don't add to 100% anyway), it's just a benchmark metric.

Also, can you add a column with number of observations? Just sum up total items for that tier (G + B + P + O) divided by number of items per box. That might help put the statistical significance into perspective.


My numbers aren't supposed to add up to 100%. Cumulative probability doesn't work that way.

May I suggest that you look at the link in my sig block for a more comprehensive explanation of what my numbers mean and what types of questions the notion of cumulative probability answers?

Armanewb
04-11-2013, 10:35 AM
My numbers aren't supposed to add up to 100%. Cumulative probability doesn't work that way.

May I suggest that you look at the link in my sig block for a more comprehensive explanation of what my numbers mean and what types of questions the notion of cumulative probability answers?

Wasn't talking about your numbers, rather the presentation in the spreadsheet. I've edited the quoted text to be more accurate.

Stickasylum
04-11-2013, 10:55 AM
I'm pretty sure he was referring to the "Drop rate / cost per item" column in the table. He seems to be addressing the OP. (Edit: beaten on that one!)


This is the part of your argument that I disagree with. A T4 give you exactly 4 trials at 8% each trial. A T3 gives you exactly 3 trials at 6% per trial. A T2 gives you exactly 2 trials at 4% per trial.

72 Keycodes will therefore yield 4 trials at T4, 9 trials at T3, and 18 trials at T2.


But where does the 72 keycode value come from? This analysis would be ideal if we had 72 keycodes to spend and that was it, ever, but that's not how keycodes work. The player can always accumulate more - left-over keycodes get spent later! Ignoring them can lead to your analysis giving different answers on which tier to buy depending the number of keycodes to spend (see my previous example). Should we tell a player with 8 keycodes to spend that they should buy a T2 lockbox because they have a 0% probability of an orange from T3 and T4 lockboxes (because they can't buy any)? T2 may be the correct choice, but we want to be able to say that buying a T2 is better than saving for a T3 or T4.

To get a consistent answer on which lockboxes to buy to maximize the long-term probability of at least one orange, we thus need to use equal-keycode-value ratios of lockboxes to avoid leftover. The simplest such ratio is 3:8:24. This can be simplified slightly, because there is no need to use an integral number of lockboxes, we just need an integral number of items. Hence, to compare the efficiency of the lockboxes at producing at least on orange, compare 3 T4 items, 8 T3 items, and 24 T4 items.



We have all the exact numbers we need to calculate exact cumulative probability in each scenario per 72 available keys to spend, and that tells a very clear and accurate story: you have the best chance to score at least one orange by buying only T2 lockboxes, but you'll see very few purples and greens in the process. You have a lower chance to score oranges with either T3 or T4, but you'll see more purples and blues in the process.


With the current values in the table, you'll actually expect to get 6.7 Blues/Purples per 100 keycodes spent on T2 boxes, 6.8 per 100 keycodes spent on T3s, and only 4 per 100 keycodes spent on T4s. T2 is better than T4 even for blue/purple accumulation (though slightly worse than T3), you just get a metric ton of greens to go with them!


Another statistic that might be useful to add to the table is "Expected keycodes spent to first orange drop". This would give similar information to the probability of at least one orange (though not equivalent), but be easy to compute and guaranteed to be consistent with the "Drop rate / cost per item" statistic. Computation is just (cost per item) / (orange drop rate as decimal).

For example, the current T4 orange drop rate is 7.68%, so on average we expect to spend 16 / 0.0768 = 203.3 keycodes on T4 boxes before our first orange. The T3 drop rate is currently 4.43%, so we would expect to spend 180.6 keycodes before our first orange. With the T2 drop rate at 2.72%, we expect to spend 142.1 keycodes before our first orange.

fishboy11
04-11-2013, 10:58 AM
he probably chose 72 because 75 is the maximum you can hold at anytime but 75 does not divide into the number of keys it takes to open a box

Stickasylum
04-11-2013, 11:48 AM
This is true, but the point still stands - leaving leftover keycodes in the analysis will lead to a biased analysis (so long as those leftovers can be used later). Suppose T4 really did have the best chance at producing oranges efficiently. With 72 keycodes you could get a better probability of at least one orange by buying one T4 and one T2, spending all 72 keycodes. But this is a bad long-term recommendation! Instead you'd want to buy one T4 and save the leftovers for the next T4. This is the same problem with comparing probabilities obtained while leaving leftovers - you're ignoring the fact that those leftovers can be used later! It might give the same answer as a valid analysis, but that doesn't make the analysis valid (except in the case where you only have 72 keycodes with no possibility of obtaining more).

Snaloe
04-11-2013, 11:52 AM
Opened 3 xT2 lockbox after reading this thread.

Got this

http://s11.postimg.org/k1ufmf9v6/FRC_Heavy_Assault_Carbine.jpg

Been getting crap off all of the T3's I have been opening not one orange.

Yokai
04-11-2013, 12:08 PM
This is true, but the point still stands - leaving leftover keycodes in the analysis will lead to a biased analysis (so long as those leftovers can be used later).

Sticky, I really feel you're simply barking up the wrong tree.

The 72 keycodes is simply to enable an even comparison rate among the three tiers. Look, it's really really really simple:

You can't accumulate more than 75 keys at a time. You -must- make a purchase. Which do you purchase? If you purchase a T4, that's it. You have four "item rolls" from that purchase.

To compare probabilities against that purchase decision, you simply ask "how many T3 lockboxes can I purchase with the same 72 keycodes", which translates to getting nine "item rolls" from those 72 keycodes (at different T3 drop rates). And you ask "how many T2 lockboxes can I purchase with the same 72 keycodes? Which yields 18 "item rolls" from T2 lockboxes.

Sure, yes, you have 8 key codes left over if you decide to take a chance on the T4 lockbox, but that is irrelevant. If you save up those 8 and eventually get 72 again and make the same decision to purchase a T4 again, your odds DON'T CHANGE AT ALL. They're still the same crappy T4 odds.

Here's another way to look at it: whether you buy three T3 boxes or 500 T3 boxes, the cumulative probability for any 9 consecutive item rolls will never vary. Likewise for T2 boxes: whether you buy nine T2 boxes or 2000 T2 boxes, the cumulative probability for any 18 consecutive rolls will never vary.

What the preceding paragraph means is that if you buy 100 T4 boxes or 300 T3 boxes or 900 T4 boxes (which would cost exactly the same: 7500 key codes), the PROPORTIONAL odds for all three tiers will remain exactly the same: T2 will STILL have a better cumulative probability to net you MORE oranges and MORE purples than T3 or T4. See how it works?

Stickasylum
04-11-2013, 12:21 PM
What the preceding paragraph means is that if you buy 100 T4 boxes or 300 T3 boxes or 900 T2 boxes (which would cost exactly the same: 7500 key codes), the PROPORTIONAL odds for all three tiers will remain exactly the same: T2 will STILL have a better cumulative probability to net you MORE oranges and MORE purples than T3 or T4. See how it works?

The problem is, if you have enough keycodes for 900 T2 boxes (7500 keycodes), you can buy 112 T4 lockboxes (T4 are 64 keycodes a pop). This is your leftovers at play, and means that your answer for 7500 keycodes may be different than your answer for 72 keycodes. With the current numbers, the answer won't be different, but the method itself not generalizable. Without equal-cost ratios, the analysis will only be valid if you have the given number of keycodes, and no chance of obtaining more.

Armanewb
04-11-2013, 12:23 PM
You can't accumulate more than 75 keys at a time. You -must- make a purchase. Which do you purchase? If you purchase a T4, that's it. You have four "item rolls" from that purchase.

To compare probabilities against that purchase decision, you simply ask "how many T3 lockboxes can I purchase with the same 72 keycodes", which translates to getting nine "item rolls" from those 72 keycodes (at different T3 drop rates). And you ask "how many T2 lockboxes can I purchase with the same 72 keycodes? Which yields 18 "item rolls" from T2 lockboxes.

Sure, yes, you have 8 key codes left over if you decide to take a chance on the T4 lockbox, but that is irrelevant. If you save up those 8 and eventually get 72 again and make the same decision to purchase a T4 again, your odds DON'T CHANGE AT ALL. They're still the same crappy T4 odds.

Your cumulative probability at the end should use 24 T2s, 8 T3s and 3 T4s, as that is the correct ratio of keycodes/lockboxes. While it is correct to say that the cumulative probability for 9 tries does not vary, the cost of those 9 tries does whether you use T2 lockboxes (36 codes for 9 items), T3 (72 codes for 9 items) or T4 (144 codes for 9 items).

Jaws
04-11-2013, 12:34 PM
Bought 14 Tier 2 lockboxes. 3 epic items and 25 green.

EGO of 234

Yokai
04-11-2013, 01:08 PM
Your cumulative probability at the end should use 24 T2s, 8 T3s and 3 T4s, as that is the correct ratio of keycodes/lockboxes. While it is correct to say that the cumulative probability for 9 tries does not vary, the cost of those 9 tries does whether you use T2 lockboxes (36 codes for 9 items), T3 (72 codes for 9 items) or T4 (144 codes for 9 items).

You guys really don't get it. I know you mean well, but you don't get it. The overall pattern won't change, nor will the answer. What's worse, the cumulative probability you're asking me to calculate is not actually possible to replicate in game. You can never buy 3 T4 lockboxes in one sitting and "roll the dice" on them for 12 pulls in a row. Putting out cumulative probabilities like that just confuses the issue.

Look I'll prove it by humoring you this one time, using your specific numbers:


T2 (48 trials)

B: 100%.........100%.........100%
P: ..97%...........86%..........66%
O: ..77%..........42%...........17%


T3 (24 trials)

B: 100%.........100%.........100%
P: ..96%...........84%..........62%
O: ..71%..........34%...........12%


T4 (12 trials)

B: 100%.........100%.........100%
P: ..86%...........56%...........26%
O: ..63%..........25%.............7%

See? The pattern is exactly the same. The ratio is exactly the same. T2 is still the best choice for purples and oranges. Period.

Armanewb
04-11-2013, 01:22 PM
You guys really don't get it. I know you mean well, but you don't get it. The overall pattern won't change, nor will the answer. What's worse, the cumulative probability you're asking me to calculate is not actually possible to replicate in game. You can never buy 3 T4 lockboxes in one sitting and "roll the dice" on them for 12 pulls in a row. Putting out cumulative probabilities like that just confuses the issue.

Ok, you are being a bit of a condescending git right here. Who cares that you can't buy 3 T4s in one go? Are you restricted to having only 75 keycodes in your entire Defiance life? If the answer is no, then your argument is irrelevant. Of course the cumulative probability we asked is possible in game, because you can earn enough keycodes to go back multiple times to the vendor. Why you perceive that as something you can't accept is bizarre.

At the end of the day, I think your metric doesn't tell any more of a story than the ones listed. If something has a higher return per 100 keycodes (as discussed earlier), mathematically it has a higher probability at any specified interval than its peers. That's all.


@Noupoi, I would also add in the margin of error at a=0.05 and a=0.01. The formulas are 1.96/(2*SQRT(SUM(Number of Items in the Tier))) for a=0.05 and 2.54/(2*SQRT(SUM(Number of Items in the Tier))) for a=0.01.

At the current count, with exclusion via validation, the margin of error is greater than the probability for oranges for Tier 2 and (barely) Tier 3.

Stickasylum
04-11-2013, 01:54 PM
What's worse, the cumulative probability you're asking me to calculate is not actually possible to replicate in game. You can never buy 3 T4 lockboxes in one sitting and "roll the dice" on them for 12 pulls in a row. Putting out cumulative probabilities like that just confuses the issue.

Buying 3 T4 lockboxes simply means saving up 64 keycodes 3 times. The fact that there is some game playing in between is irrelevant. The probability of at least one orange given 3 T4 lockboxes means the probability of at getting at least one orange if you save up 64 keycodes 3 times.

The important question (from your probability perspective) is not "What should I do with my current 72 keycodes to maximize my probability right now?" it is "Which boxes should I spend keycodes on to maximize my potential for an orange drop in the long run?" These questions can have different answers.

Here's another example where your method can give bad advice, with theoretical numbers you might find more tenable:

T2 orange drop rate of 2%, T4 drop rate of 8%. Using your method (72 keycodes to spend on 1 T4 or 9 T2):

Probability of at least one orange in 1 T4 box = 1 - (1-.08)^4 = 0.284
Probability of at least one orange in 9 T2 boxes = 1 - (1-.02)^18 = 0.305

Suppose instead we look at what happens over the course of 3 T4 purchases (194 keycodes, with some playing in between drops). This would have bought 24 T2 boxes:

Probability of at least one orange in 3 T4 boxes = 1 - (1-.08)^12 = 0.632
Probability of at least one orange in 24 T2 boxes = 1 - (1-.02)^48 = 0.621

Note that your method would recommend buying T2 boxes. Ignoring that left over 8 keycodes while buying T4 boxes means that your missing out on the long-term advantage of buying T4 boxes. with these made-up drop rates, with 72 keycodes in hand, you are more likely to get an orange drop right now buying T2 boxes, but you'll be more likely to get an orange in the long-term buying T4 boxes.

The fact that I made up those numbers not relevant. It shows that the recommendations from your method can depend on the number of keycodes spent, a problem that the equal-value-ratio method does not have. Just because it is not inconsistent with the current drop rates does not make the method valid. Even a broken clock is right twice a day!



Look I'll prove it by humoring you this one time, using your specific numbers:

What do the different columns mean?


T2 is still the best choice for purples and oranges. Period.
Yes, that's not in question (with the current values from the table). We're just saying that using 1:3:9 for the T4:T3:T2 is not valid for the types of conclusions you wish to draw.



At the end of the day, I think your metric doesn't tell any more of a story than the ones listed. If something has a higher return per 100 keycodes (as discussed earlier), mathematically it has a higher probability at any specified interval than its peers. That's all.


Unfortuntately, this isn't always true (I thought the same thing at first, too). Here's an example where the expected number of oranges per keycode is higher for T2 boxes, but the probability of obtaining at least one orange item from T4 lockbox is greater than from 4 T2 boxes. (http://forums.defiance.com/showthread.php?20963-The-Lockbox-Data-Collection-Project&p=284436&viewfull=1#post284436) Basically, because expectation is weighted by the number of oranges obtained, but the probability of at least one orange doesn't encapsulate how many oranges are obtained, under specific circumstances the values can be different. That's why I've been arguing that the expected number of drops is the better statistic to use to compare efficiencies (since most people want to get lots of oranges, rather than minimize their chance of not getting an orange).



@Noupoi, I would also add in the margin of error at a=0.05 and a=0.01. The formulas are 1.96/(2*SQRT(SUM(Number of Items in the Tier))) for a=0.05 and 2.54/(2*SQRT(SUM(Number of Items in the Tier))) for a=0.01.

At the current count, with exclusion via validation, the margin of error is greater than the probability for oranges for Tier 2 and (barely) Tier 3.

Adding the margin of error is a great idea! However, this margin of error will overestimate the error for small drop rates. I'd recommend computing a 95% margin of error separately for each drop rate using

1.96*sqrt(p*(1-p)/Number of Items for Tier),

where p is the drop rate as a decimal. Use 2.54 instead of 1.96 for a 99% margin of error and multiply by 100 to make the margin of error a percentage. For example, the 95% margin of error for a 4% T2 orange drop rate estimated from 1000 items would be

1.96*sqrt(0.04*(1-0.04)/1000) = 0.0121, or 1.21%

Yokai
04-11-2013, 02:24 PM
Buying 3 T4 lockboxes simply means saving up 64 keycodes 3 times. The fact that there is some game playing in between is irrelevant. The probability of at least one orange given 3 T4 lockboxes means the probability of at getting at least one orange if you save up 64 keycodes 3 times.

The important question (from your probability perspective) is not "What should I do with my current 72 keycodes to maximize my probability right now?" it is "Which boxes should I spend keycodes on to maximize my potential for an orange drop in the long run?" These questions can have different answers.

Here's another example where your method can give bad advice, with theoretical numbers you might find more tenable:

T2 orange drop rate of 2%, T4 drop rate of 8%. Using your method (72 keycodes to spend on 1 T4 or 9 T2):

Probability of at least one orange in 1 T4 box = 1 - (1-.08)^4 = 0.284
Probability of at least one orange in 9 T2 boxes = 1 - (1-.02)^18 = 0.305

Suppose instead we look at what happens over the course of 3 T4 purchases (194 keycodes, with some playing in between drops). This would have bought 24 T2 boxes:

Probability of at least one orange in 3 T4 boxes = 1 - (1-.08)^12 = 0.632
Probability of at least one orange in 24 T2 boxes = 1 - (1-.02)^48 = 0.621

Note that your method would recommend buying T2 boxes. Ignoring that left over 8 keycodes while buying T4 boxes means that your missing out on the long-term advantage of buying T4 boxes. with these made-up drop rates, with 72 keycodes in hand, you are more likely to get an orange drop right now buying T2 boxes, but you'll be more likely to get an orange in the long-term buying T4 boxes.

The fact that I made up those numbers not relevant. It shows that the recommendations from your method can depend on the number of keycodes spent, a problem that the equal-value-ratio method does not have. Just because it is not inconsistent with the current drop rates does not make the method valid. Even a broken clock is right twice a day!


What do the different columns mean?


Yes, that's not in question (with the current values from the table). We're just saying that using 1:3:9 for the T4:T3:T2 is not valid for the types of conclusions you wish to draw.



Unfortuntately, this isn't always true (I thought the same thing at first, too). Here's an example where the expected number of oranges per keycode is higher for T2 boxes, but the probability of obtaining at least one orange item from T4 lockbox is greater than from 4 T2 boxes. (http://forums.defiance.com/showthread.php?20963-The-Lockbox-Data-Collection-Project&p=284436&viewfull=1#post284436) Basically, because expectation is weighted by the number of oranges obtained, but the probability of at least one orange doesn't encapsulate how many oranges are obtained, under specific circumstances the values can be different. That's why I've been arguing that the expected number of drops is the better statistic to use to compare efficiencies (since most people want to get lots of oranges, rather than minimize their chance of not getting an orange).



Adding the margin of error is a great idea! However, this margin of error will overestimate the error for small drop rates. I'd recommend computing a 95% margin of error separately for each drop rate using

1.96*sqrt(p*(1-p)/Number of Items for Tier),

where p is the drop rate as a decimal. Use 2.54 instead of 1.96 for a 99% margin of error and multiply by 100 to make the margin of error a percentage. For example, the 95% margin of error for a 4% T2 orange drop rate estimated from 1000 items would be

1.96*sqrt(0.04*(1-0.04)/1000) = 0.0121, or 1.21%

I give up. Sorry but you're both obstinate and wrong. Take a took at your T2 numbers that you made up. Im not even going to bother describing your major gaffe here. You just like to argue.

The actual cumulative probability for at least one orange from three T4 bosses is 63%. The actual cumulative probability for at least one orange from 24 T4 boxes is 73%. You're stuck on the wrong formula and the wrong approach for the job.

GhostTLT
04-11-2013, 02:30 PM
by the way

THANKS for doing this, it's very interesting!!!!

Stickasylum
04-11-2013, 02:31 PM
I give up. Sorry but you're both obstinate and wrong. Take a took at your T2 numbers that you made up. Im not even going to bother describing your major gaffe here. You just like to argue.

If you wouldn't mind posting one more time, what do you think is wrong with that value? Its just an illustration of where your method can fail.

I'm actually a graduate student in statistics, so these sorts of questions interest me. I'm also interested in finding (relatively) simple ways to explain statistical intricacies (like the one we're discussing), so I appreciate you input, if perhaps not your tone.

taskun56
04-11-2013, 03:08 PM
I hate to be a **** here but you need to cull your data and start over with a better system. Anything past 4 in the options menu is creating false data from a bunch of trolls who have nothing better to do.

UglyCoyote
04-11-2013, 03:11 PM
Thumbs up for the idea and the work done ... keep up ....

Exes
04-11-2013, 03:12 PM
The sources for those samples are really not trustworthy, making statistics in a non regulated environment is always fail.

Noupoi
04-11-2013, 03:19 PM
This thread's really blown up- far bigger than I expected when starting it, but great to see... And the discussion has gotten much deeper than my understanding of statistics goes. :S

So from reading the responses, are these the changes you guys want to be made?

Use validated frequency as basis for new calculations
Add estimate for margin of error. (with forumla 1.96*sqrt(p*(1-p)/Number of Items for Tier) ?)
Change 'Drop rate / Cost per try' to 'Expected drops / 100 keycodes'


Again, I wouldn't mind giving you guys access to modify the spreadsheet if you want to help, though I'll admit I don't really understand sure what the disagreement with calculations is about.


I hate to be a **** here but you need to cull your data and start over with a better system. Anything past 4 in the options menu is creating false data from a bunch of trolls who have nothing better to do.

I opted to have the option to input more than 4 items of a type in one go as a compromise between saving time for actual users submitting data for multiple lockboxes, and preventing trolls from entering large amounts of false data easily. I'm open to suggestions for ways to do this better.

RawrKitty
04-11-2013, 03:26 PM
This thread's really blown up- far bigger than I expected when starting it, but great to see... And the discussion has gotten much deeper than my understanding of statistics goes. :S

So from reading the responses, are these the changes you guys want to be made?

Use validated frequency as basis for new calculations
Add estimate for margin of error. (with forumla 1.96*sqrt(p*(1-p)/Number of Items for Tier) ?)
Change 'Drop rate / Cost per try' to 'Expected drops / 100 keycodes'


Again, I wouldn't mind giving you guys access to modify the spreadsheet if you want to help, though I'll admit I don't really understand sure what the disagreement with calculations is about.



I opted to have the option to input more than 4 items of a type in one go as a compromise between saving time for actual users submitting data for multiple lockboxes, and preventing trolls from entering large amounts of false data easily. I'm open to suggestions for ways to do this better.

Sadly, as i said in a previous post the last time i checked this thread, you will get people adding in troll data regardless of what you do. the only way to ensure it 100% is to validate, most people will not do this, but i'd gladly submit proof (videos) of my data because i want a truely accurate data, as the jerkish dev's refuse to give us probabilities for the lockboxes...

Yokai
04-11-2013, 03:27 PM
If you wouldn't mind posting one more time, what do you think is wrong with that value? Its just an illustration of where your method can fail.

I'm actually a graduate student in statistics, so these sorts of questions interest me. I'm also interested in finding (relatively) simple ways to explain statistical intricacies (like the one we're discussing), so I appreciate you input, if perhaps not your tone.

Okay, first off, I apologize for the prickly tone. Second, I apologize for being wrong, lol. I read your most recent example in a rushed state and misinterpreted something.

So, permit me to grovel for a moment and savor the taste of shoe leather. Okay, that was supposed to be funny but just sounded weird, lol (Jon Stewart face) :)

Okay, that being done now, you are correct. If the percentages were different from their current observed values, then indeed in the "over time scenario" the cumulative probabilities for 12 T4 versus 24 T3 versus 48 T2 would be a more accurate comparison.

I salute your patience. I will also revise the numbers in my sig block link below to reflect this more accurate comparison.

That all said, in this particular case with these particular numbers, with an observed 3% drop rate on T2 boxes, with a 2000+ sample size, it still holds that even in the 12 - 24 - 48 comparison, T2 is still the clear winner. BTW in that short hand table a few posts above with the 12 - 24 - 48 numbers? The three columns are (left to right): AT LEAST 1......AT LEAST 2.....AT LEAST 3. Similar to the more verbose layout in my sig block link below. Make sense?

Yes, it can be arguable whether to trust the sample collection method in Noupoi's spreadsheet, but let's assume people were mostly honest and didn't try to grief the collection data). Perhaps it _would_ be beneficial to all to wipe the spreadsheet, strengthen the form input against possible (or accidental) impossible results input, and start over. I think our current sample size in the current run of the spreadsheet is high confidence, so wiping and sampling again wouldn't hurt anything.

In fact, as I suggested before, it might be worth wiping and restarting every month just in case the Trion devs decide to tweak the real drop rate numbers and not tell us they've done so.

THEST8TMENT
04-11-2013, 03:33 PM
I'm glad there are other people on here who know how to use statistics. It looks like to me that the efficiency of keycodes to orange will eventually become closer and closer since the statistics of tier 2 lock boxes is more common and more accessible. I think that it doesn't matter what box you buy you will get the same amount of orange over a long enough period. But because it is not basing the orange directly off of each person, I would suggest you buy tier 2 boxes if you think you are lucky enough to catch those rare oranges, but if not go for the tier 4 you will be let down a lot less. My two cents. If you guys need any more help with the statistics on any other part of the game hit me up.

Razor
04-12-2013, 03:07 AM
What about bits?

Costs for lockboxes in bits are different from rescources.

T2 120 bits
T3 240 bits
T4 400 bits

So what would be the most effective way to get orange using bits?

jetah
04-12-2013, 08:10 AM
What about bits?

Costs for lockboxes in bits are different from rescources.

T2 120 bits = 1.20$
T3 240 bits = 2.40$
T4 400 bits = 4.00$

So what would be the most effective way to get orange using bits?

I added a rough cost in dollar terms to your quote. Personally, if you're spending bits, don't buy lockboxes. Instead buy inventory or buy Loot/Scrip Boost. Keys and Salvage are too easy to obtain.

I'm 606 EGO with 120k Salvage, 39k Scrip and spent my Keys but I do have 272 inventory. I haven't salvaged any items since 500.

Yokai
04-12-2013, 08:15 AM
Noupoi: What is the new set of "Validated Frequency" stats in the upper right of the spreadsheet? You've now got two sets of drop rates. Even more confusing is that the "validated" drop rate for oranges is exactly the same for T2 and T3. Is this a new work in progress?

On a different angle, your original "Frequency" and "Drop Rate" columns for T2 seem to be moving the T2 drop rate for oranges much closer to 2% than the 3% it appeared to be yesterday. PSA for everyone: if the true drop rate for T2 oranges is only 2%, then the "best value" for your keys spent becomes T3 lockboxes. Just saying. I'm about to go update my page in the sig block link below accordingly.

Noupoi, on a different angle, would you be willing to improve the data collection form to minimize the ability to enter impossible result sets, then wipe the data and start over? Anything we can do to improve sampling accuracy will be beneficial to all. :)

jetah
04-12-2013, 09:11 AM
Noupoi: What is the new set of "Validated Frequency" stats in the upper right of the spreadsheet? You've now got two sets of drop rates. Even more confusing is that the "validated" drop rate for oranges is exactly the same for T2 and T3. Is this a new work in progress?

On a different angle, your original "Frequency" and "Drop Rate" columns for T2 seem to be moving the T2 drop rate for oranges much closer to 2% than the 3% it appeared to be yesterday. PSA for everyone: if the true drop rate for T2 oranges is only 2%, then the "best value" for your keys spent becomes T3 lockboxes. Just saying. I'm about to go update my page in the sig block link below accordingly.

Noupoi, on a different angle, would you be willing to improve the data collection form to minimize the ability to enter impossible result sets, then wipe the data and start over? Anything we can do to improve sampling accuracy will be beneficial to all. :)

I think the idea was that you could open multiple boxes and input it. ie you could open 2 T4 boxes and just have 1 entry for 8 items.

Noupoi
04-12-2013, 12:12 PM
Noupoi: What is the new set of "Validated Frequency" stats in the upper right of the spreadsheet? You've now got two sets of drop rates. Even more confusing is that the "validated" drop rate for oranges is exactly the same for T2 and T3. Is this a new work in progress?

On a different angle, your original "Frequency" and "Drop Rate" columns for T2 seem to be moving the T2 drop rate for oranges much closer to 2% than the 3% it appeared to be yesterday. PSA for everyone: if the true drop rate for T2 oranges is only 2%, then the "best value" for your keys spent becomes T3 lockboxes. Just saying. I'm about to go update my page in the sig block link below accordingly.

Noupoi, on a different angle, would you be willing to improve the data collection form to minimize the ability to enter impossible result sets, then wipe the data and start over? Anything we can do to improve sampling accuracy will be beneficial to all. :)

Validated frequency only includes entries that pass the basic validation which checks if the number of items in that entry is a multiple of the number of items you could get from a lockbox of that tier- this means that impossible data is ignored. The idea was that you could input the results for multiple lockboxes at a time, but also to make it difficult for trolls to enter large amounts of false data at once.

I'm open for suggestions on how to improve the data entry form. If it doesn't actually involve changing the questions, but how they're inputted, there's no need to wipe the data. We could just redo/add a results page for data entered after a certain time - so not including data before a certain point, and move over to that when we have enough data for that to become reliable.

Edit: I've also given edit access to the people who've asked me for it. Hopefully you'll be seeing changes they make soon!

Yokai
04-12-2013, 12:39 PM
Validated frequency only includes entries that pass the basic validation which checks if the number of items in that entry is a multiple of the number of items you could get from a lockbox of that tier- this means that impossible data is ignored. The idea was that you could input the results for multiple lockboxes at a time, but also to make it difficult for trolls to enter large amounts of false data at once.

I'm open for suggestions on how to improve the data entry form. If it doesn't actually involve changing the questions, but how they're inputted, there's no need to wipe the data. We could just redo/add a results page for data entered after a certain time - so not including data before a certain point, and move over to that when we have enough data for that to become reliable.

Okay, then the validated data is showing an interesting trend/pattern:

http://i.imgur.com/VCSb2LU.png

At first I was surprised at how closely these validated results matched the non-validated results in nearly every respect except for the steep drop in probability for oranges from T3 boxes. But then I looked more closely at the pure drop rates (not the cumulative probability). See the pattern? Here's what I see, translated to verbal narrative:

T2 has a very low drop rate for blues and higher, with nearly 75% of drops being green.
T3 keeps THE SAME LOW drop rate for orange as a T2, but greatly pushes up the drop rate for blues and purples.
T4 slightly ups the drop rate for blue, keeps purple the same as T3, and finally gives you a 250% improvement in drop rate for orange.

Honestly, this pattern makes sense for for the value proposition among the three tiers from the standpoint of drop rates alone. I could see a designer/dev picking numbers like these because they "look good and make sense, and will encourage players to hold out for T4s if they really want an orange, but to settle for T3 if what they're really looking for are purples and blues. We'll give them so many greens at T2 that they'll at least be tempted to move up to T3".

But in this case, look at the whacky, non-linear value propostion when you view the cumulative probability per X number of keys spent!

T2 has the best cumulative odds for getting an orange
But T3 has the best cumulative odds for getting a purple
T4 is still on the bottom of the heap, though.

TomahawkJackson
04-12-2013, 05:33 PM
Edit: I've also given edit access to the people who've asked me for it. Hopefully you'll be seeing changes they make soon!I'm hard at it already...

CHANGELOG
Moved Validated Results to it's own tab, where they will grow their own charts, based on the Validated data.
Corrected a raw data entry where the user input EGO 'one hundred and twenty', exactly how it says not to do it on the form. Joker. It's now 120.
Removed a raw data entry where the user input EGO 7410.
Removed several repeated raw data entries where the user said they got 10 of everything from a T2 box. 7 times in a row.

TODO
Make charts replicating the raw charts on the first tab, using the Validated data.
Create additional charts showing the likelihood of receiving multiple items based on the number of items received/the number of Lockboxes submitted (i.e. the data actually shows that the users got 686 Green Items from 241 T2 Lockboxes. You are likely to get TWO green items from a T2 box. That should be represented by a 200% chance to get a Green Item.)

Armanewb
04-12-2013, 06:52 PM
Whoever changed the title of the Expected Items per 100 Keystones to lockboxes, I've changed the title back. It's not per 100 lockboxes, as they cost different amounts of keystones and thus it would not be a comparable measure.

Stickasylum
04-12-2013, 08:13 PM
Whoever changed the title of the Expected Items per 100 Keystones to lockboxes, I've changed the title back. It's not per 100 lockboxes, as they cost different amounts of keystones and thus it would not be a comparable measure.

It also shouldn't be a percentage, since it is a count! If you reformat it, you'll probably need to multiply by 100 to keep the number the same.


Okay, first off, I apologize for the prickly tone. Second, I apologize for being wrong, lol. I read your most recent example in a rushed state and misinterpreted something.



No worries! Glad to know I'm not crazy :)

Thunderclap
04-12-2013, 11:51 PM
Sledge is on live stream now. When someone asked the question stating about the different tiers he said it is supposed to be random and that you would have to open "tens of thousands" of boxes to prove that another tier is better than 4. So the offical answer is there is nothing wrong. (he's on his twitch channel if you wonder 4/12/13 10 ish to 11 ish starting. stream if he records it has no sound other than him to begin with

Yokai
04-13-2013, 06:28 AM
Sledge is on live stream now. When someone asked the question stating about the different tiers he said it is supposed to be random and that you would have to open "tens of thousands" of boxes to prove that another tier is better than 4. So the offical answer is there is nothing wrong. (he's on his twitch channel if you wonder 4/12/13 10 ish to 11 ish starting. stream if he records it has no sound other than him to begin with

Not everyone understands the notion of cumulative probability very well. T4 certainly has better inherent drop rates per box. We can already see that from the observed results so far in the data collection--validated or not.

The problem is whether the relative pricing for each box supports Sledge's claim based on the notion of cumulative probability. So far the observed results indicate that other tiers are better than T4 when you look at things from the notion of cumulative probability. Put simply: for any given amount of keys, you get more T2 or T3 "item rolls" than T4 "item rolls", and the cumulative probability for those increased rolls can actually work out to a BETTER chance at scoring an orange or purple.

Unless Trion balanced the three tier prices and inherent drop rates with a specific eye towards cumulative probability, to ensure that T4 also offers the BEST cumulative probability, and T3 offers the NEXT BEST cumulative probability, then Sledge might well be wrong in his statement.

Finally, the "you'd need to open tens of thousands of boxes to prove" is a well intended and innocent statement, but also simply wrong. If we had accurate reporting to the spreadsheet collection form, actually all we'd need is 377 data points per teir to feel pretty damn confident.

Stickasylum
04-13-2013, 09:50 AM
Well, to be fair, as the table gets more data (and as the data gets validated), the orange drop rates seem to be getting closer and closer to the cutoffs where the efficiencies are equivalent. From the expected drops per keycode perspective, the T3 drop rate needs to be more than half the T4 rate for T3 to better, and the T2 needs to be more than a quarter. Right now, the orange drop rate for T3 is awfully close to this cutoff, so its looking like we would need a lot more data to know either way (and the effect would likely be very small). Similarly, to show T3 is better from the cumulative probability perspective, you would need to test the hypothesis that p_3 > 1-(1-p_4)^(1/2). Again, the current numbers are awfully close to this cutoff!

It's even entirely possible that the real drop rates have a 1:2 ratio, giving T3 and T4 equivalent efficiencies for orange drops. If this is the case, however, T3 is still clearly better than T4 as it clearly has better blue and purple efficiency (and you run less risk of accidentally maxing your keycodes and wasting incoming keycode drops). The only downside to T3 would be the greater scrip cost, but scrip has never been a problem for me, even buying T2s!

kkhendy
04-13-2013, 09:57 AM
Amazing idea, submitted and will continue to submit.

Captain Rimjaw
04-13-2013, 03:02 PM
posted my latest

DeMoted
04-13-2013, 03:04 PM
That sucks I just bought like 20 T4 boxes and didn't see this till today.

Victim Eyes
04-13-2013, 03:08 PM
so you're suggesting it's better off to purchase tier 3 in the long run? I've had a terrible string of luck with the past four or five tier 4 that I've purchased. No orange and maybe one or two purples total...

Yokai
04-13-2013, 03:55 PM
According to the -validated- numbers earlier this morning, T4 pulled ahead of the pack for oranges, and T3 sits at the front of the pack for purples. See my sig block for a chart as of 9am CST this morning.

Rat2man
04-13-2013, 05:05 PM
So just to clarify... for data to be counted as validated can I still do multiple boxes per entry? (Ie 6 items for 2 T3, or 8 items for 4 T2) or do I need to be doing each box seperate?

Avenger
04-13-2013, 06:10 PM
I'm hard at it already...

CHANGELOG
Moved Validated Results to it's own tab, where they will grow their own charts, based on the Validated data.

Thanks for your input on this.
Not wanting to jump on errors as you said the page is still work in progress, but a couple of the calculated figures seem odd.

"Calculated total items rec'd per box"
I understand what the formula is doing - divide the number of validated items found by the number of submissions - yet somehow it is reporting back 3.36 items for both Tier 2 and Tier 3 boxes, which should only be able to drop a maximum of 2 or 3 items respectively.

I think that "Number of Submissions" is just looking at the raw data - not the validated data. But since we know the validated total MUST be lower than the submitted total (because there are bogus entries) then that would make the "Calculated total items rec'd per box" stat increase which puts it further from the in game text stating 2 for Tier 2, 3 for Tier 3, 4 for Tier 4.

This would suggest that the validated data is letting more data through than it should. Just looking at Tier 2 Green - it gives what should be an impossible figure - 768 Green drops from 298 submissions. If Tier 2 can only create 2 items there should be a max of 596 across all colours, while the validation formula should be excluding entries that don't total 2 (if I've read the array formula correctly).

Avenger
04-13-2013, 06:12 PM
So just to clarify... for data to be counted as validated can I still do multiple boxes per entry? (Ie 6 items for 2 T3, or 8 items for 4 T2) or do I need to be doing each box seperate?

Do each box seperately. The validation as it stands will only count entries that have a valid number of items for 1 box of the selected Tier. So if you input data for 4 Tier 2 boxes as one entry it will show in the raw data, but will not be validated as it is impossible for 1 Tier 2 box to generate 8 items.

Mogren
04-13-2013, 07:04 PM
Do each box seperately. The validation as it stands will only count entries that have a valid number of items for 1 box of the selected Tier. So if you input data for 4 Tier 2 boxes as one entry it will show in the raw data, but will not be validated as it is impossible for 1 Tier 2 box to generate 8 items.

Noupoi's post on the previous page seems to say that the validated results allow entering multiple lock boxes at once, as long as the number is a multiple of the type of box being entered. Unless I'm reading that wrong. That would also explain your previous post about number of submissions vs number of items.

Rat2man
04-13-2013, 09:34 PM
Do each box seperately. The validation as it stands will only count entries that have a valid number of items for 1 box of the selected Tier. So if you input data for 4 Tier 2 boxes as one entry it will show in the raw data, but will not be validated as it is impossible for 1 Tier 2 box to generate 8 items.

Hehe, between you and Mogr, thats why Im asking, guess Ill follow the 1 box rule for the time being...

*side note* Fing hate *******es that add false data b/c its somehow lolz funny

Edit: and are we tracking snergy mod caches in another post by chance?

Noupoi
04-14-2013, 05:13 AM
Noupoi's post on the previous page seems to say that the validated results allow entering multiple lock boxes at once, as long as the number is a multiple of the type of box being entered. Unless I'm reading that wrong. That would also explain your previous post about number of submissions vs number of items.

All it means is that validation includes all entries where a complete set of lockboxes, e.g. 4 T2 lockboxes, have been entered, but won't include entries where 4 T2 lockboxes and an extra item have been entered. If you have more than 10 items of a certain type, you will need to submit the data seperately, but otherwise, you can submit more than one box at a time.

ten4
04-14-2013, 05:19 AM
I have found greens with synergies and blues with same synergies/secondaries/stats as purps/oranges. Often times better, actually. And there is no different graphic for legendries either...it's same as any trash white item. :(

Itemization needs work imo...

Sorry to go a bit off topic, interesting to see data like this. Thanks!

Avenger
04-14-2013, 08:19 AM
All it means is that validation includes all entries where a complete set of lockboxes, e.g. 4 T2 lockboxes, have been entered, but won't include entries where 4 T2 lockboxes and an extra item have been entered. If you have more than 10 items of a certain type, you will need to submit the data seperately, but otherwise, you can submit more than one box at a time.
You created the sheet, entry form and rules for this, but I don't think allowing multiple entries on one form is a good idea.

Firstly if all the validation is to check if the number of items on an entry is correctly divisible by the number of items expected from that tier, it does nothing to prevent the 10/10/10/10 abuse since that represents 20 Tier 2 or 10 Tier 4 lockboxes - passes validation despite being highly suspect entries - particularly when you see time stamps only a few seconds / minutes apart for multiple entries doing the same thing.

Secondly if people do submit multiple boxes together but go over 10 items of one type and split it into a new entry, it can leave invalid results for both entries - say 5 Tier 3 boxes produces 11 green and 4 blue - that would be split 10+4 on the first entry and 1 on the second - both invalid as neither is divisible by 3.

I assumed from the start that 1 lockbox result = 1 entry, so that's how I've been submitting data - and from looking at the form responses it appears that is also the way the vast majority of people have been submitting data. It takes longer, but makes it possible to see individual box results, and also makes it easier to spot the data that stands out.

If you do intend for people to be able to enter multiple boxes at once, then is it possible to edit the submission form to ask people how many boxes they are submitting for? That could perhaps act as some kind of check - if the number x the tier don't match the number of items reported the data would be invalid?

Personally when looking at the data I am discounting any Tier 2 line showing more than 2 items, any Tier 3 with more than 3, and any Tier 4 with more than 4. If there are some people that have grouped their genuine data together into a few large entries before submitting it will unfortunately exclude them, but it will also exclude the majority of griefers who submit bogus entries just to piss on the work of others.

Bdtry
04-14-2013, 12:06 PM
Just bout a tier 4 lockbox and got a green in it.... something I read was not supposed to happen. Got a really cool orange pistol though :D

182 FRC Northstar Flare
Synergy Machinist

+4 mag
15% grenade recharge on kill
x1.10 Fire Rate

Dmg 1010
FR 1.1
Mag 5
Reload 2.5
Crit 2.5x
Bloom 4.90

First time I fired this thing I giggled like a little girl. A pistol that basically shoots grenades.. ;)


The green I got was a VOT Rebound Lobber.

Edit: Got another tier 4... 3 blues and a green.

Tayce
04-15-2013, 10:29 AM
Thanks for this info.
Seems like my hunch that t3 boxes gives best value was true.

Hope499
04-15-2013, 01:52 PM
Numbers...confusing....understanding...difficult.. ..

I have been doing T2's since I got my one and only lgndary from it...maybe I will try some T3 instead

Kazu
04-16-2013, 01:05 PM
Anyone have already drop a sub machine gun in a lock box ?

Shongjen
04-18-2013, 10:02 AM
Anyone have already drop a sub machine gun in a lock box ?
yes, its possible. i bought a VBI SMG (orange) from a guy, and he had it from a logbox.

Vorman
04-18-2013, 10:20 AM
I got a orange burstfire SMG from a lockbox, good damage but it just fires to slow and innacurate so I haven't really used it. Other orange I got was a VOT Mazu blast rifle which is pretty badass I don't know the stats off hand and can't look them up atm since the servers are down, but they both came from T4 boxes. and the last t4 box I opened had 3 blue 1 purple in it.

DeMoted
04-18-2013, 10:29 AM
I got a orange burstfire SMG from a lockbox, good damage but it just fires to slow and innacurate so I haven't really used it. Other orange I got was a VOT Mazu blast rifle which is pretty badass I don't know the stats off hand and can't look them up atm since the servers are down, but they both came from T4 boxes. and the last t4 box I opened had 3 blue 1 purple in it.

You need the synergy mods or those orange weapons are useless.

Tgreen
04-19-2013, 12:19 PM
Your item amount doesn't add up.
T4 lockboxes item count divised by four yields 470.5 lockboxes bought.
When viewing the list I noticed the last digits not adding up to be divisable by 4 and checked.
Same is true for T2 and T3 after checking the numbers, someone messed up their data input.

Other than that really good effort, although my personal odds up until now are far below what your data shows.
Guess my luck isn't what it used to be. ;>

Yokai
04-19-2013, 01:58 PM
Numbers...confusing....understanding...difficult.. ..

I have been doing T2's since I got my one and only lgndary from it...maybe I will try some T3 instead

Check the link in my sig block below. Those numbers will probably make more sense to you.

Noupoi: don't google spreadsheets have a BINOM formula? It would be easy to do the same thing I do in my chart on another worksheet in your google doc. I strongly feel that most people will intuitively understand: "if you spend n keycodes, your odds of getting at least 1 (or at least 2, or at least 3, etc.) oranges weapons is exactly this".

quietscribe
04-21-2013, 07:22 AM
What a great resource and idea. I will be adding my own finds from now on :)

BraveDude8
04-27-2013, 03:37 PM
I decided to give this a bump. It deserves it.

BraveDude8
04-28-2013, 07:30 AM
Really guys? I can't bump it any more after this in good conscience. PUT YOUR RESULTS IN, WE NEED MORE ACCURATE FIGURES.

Lahlaos
04-30-2013, 12:00 PM
Well Ive had 4 oranges out of about 16 t2 lockboxes, i've been advising people to just buy T2 lockboxes, because there isnt a major guarantee that you will get an orange in the T3/4's either, and 64 Keystones??? you can SUCK my balls.

T2 FTW:) dont be a silly billy and make the mistake I made of saving up 64 to get a T4 its bloody pointless.

Nic Nova
04-30-2013, 12:07 PM
I just started keeping track and I'll add my results to the graph. I did not post because I got something good, but rather the opposite:
4/27
Tier 4 G/G/G/B
4/28
Tier 2 G/G
Tier 3 G/G/B
4/29/13
Tier3 G/B/P
Tier 3 G/G/B
Tier 3 G/B/P
Tier 3 G/G/G
I've got 70 key codes now and I'm considering what to open next.

PowThePredat0r
04-30-2013, 12:33 PM
I have been collecting my own data and just added them..

No oranges from Tier 4 box after 704 keycodes.

2 oranges from Tier 3 box after 1056 keycodes.

Bad luck? Very. Lol.

Lahlaos
04-30-2013, 12:33 PM
can you buy T2's and just list what you get:) best way to prove something:)

Lahlaos
04-30-2013, 12:34 PM
youll also get more out of it technically

PowThePredat0r
04-30-2013, 12:39 PM
I never really bought T2's but I might start now.

To date I have only spent 64 keycodes on T2's and received 14 green and 2 blue.

What kind of oranges did you get out of T2's Lahlaos?

Barley McHops
05-02-2013, 08:34 AM
Sitting on a full 75 keys and 50k ark resource. Tired of opening T3 boxes and getting nothing but spanners and rocket launchers. At this point I'm more concerned with getting weapons I'll use than with getting orange weapons (that I may not even want), so I'm about to pop open up at least 15 T2 boxes, probably more when I break down the stuff I don't want.

I'll report back with what happens.

Barley McHops
05-02-2013, 01:22 PM
After breaking down stuff I didn't want (most of it), I ended up opening 17 T2 boxes.


2x Sniper
3x Pistol
2x Shield
1x BMG
2x Detonator
3x Rocket Launcher
1x LMG
6x Shotgun
1x Grenade
1x Infector
1x Assault Rifle


1x SMG
3x Detonator
1x BMG
1x Grenade
1x Shotgun
1x Sniper

1x SMG
2x Grenade

So, in 17 T2 boxes, 21 Green, 8 Blue, 3 Purple and 0 Orange.

Avedis
05-02-2013, 01:24 PM
Great idea. Will submit data frequently.

Wise Ginger
05-02-2013, 02:19 PM
For experiment purposes I did something similar I purchased Bits and focused on Tier 3 and Tier 4. With cost per item being 8 for Tier 3 and a whopping 16 for Tier 4 it certainly appears to me that the Tier 3 is a much better value. Oranges may be a little harder to come by but I receive quite a good amount of purple from T3 and it just seems better.

Wise Ginger
05-02-2013, 02:21 PM
Has anyone else noticed the high chances of receiving like items? As in 3 shotguns or 3 explosive types or 3 pistols when doing lock boxes. This is highly frustrating.

Lukezordz
05-07-2013, 02:23 AM
Added to my Sig :D

Great idea, I'll test out a few T4 lockboxes and post the results. I'll be happy with just a few more blues/purples :D

PS. This deserves a sticky!

FEAR Alpha KEQ
05-10-2013, 08:31 AM
GREAT IDEA!!! A very commendable effort, and I am sure a great many players are using this.


For experiment purposes I did something similar I purchased Bits and focused on Tier 3 and Tier 4. With cost per item being 8 for Tier 3 and a whopping 16 for Tier 4 it certainly appears to me that the Tier 3 is a much better value. Oranges may be a little harder to come by but I receive quite a good amount of purple from T3 and it just seems better.

On this subject, I would think that there are different algorithms, and to check maybe everyone should also state if it is a bit-purchase, or a resource purchase. Imho, the bit-costs are a little exhorbitant, so i think that they would be skewed differently from the resource lockboxes. I will be adding in my data now too, as well as the payment type utilized.


Has anyone else noticed the high chances of receiving like items? As in 3 shotguns or 3 explosive types or 3 pistols when doing lock boxes. This is highly frustrating.

I actually got 3 cluster-frags from one tier 3 box. And I wish that you could pay a premium to get weapons that you actually use lol!

cusman
05-10-2013, 08:50 AM
Looks to me like people are better off wasting their time grinding for Tier 4 if they want Orange than Tier 3

I think one improvement you could make is ask what version they are on at the time they received the Orange. There is also the case of whether they have any Boosts active (individual, clan, or group).

The only Orange I got was from a Tier 3 box and I really feel like certian Lox Box locations or time of day or something I get better items than other times. It could all be random. Their code base is a mess so I just don't trust that they would have a simple random chance roll that works as intended.

cusman
05-10-2013, 08:53 AM
After breaking down stuff I didn't want (most of it), I ended up opening 17 T2 boxes.


2x Sniper
3x Pistol
2x Shield
1x BMG
2x Detonator
3x Rocket Launcher
1x LMG
6x Shotgun
1x Grenade
1x Infector
1x Assault Rifle


1x SMG
3x Detonator
1x BMG
1x Grenade
1x Shotgun
1x Sniper

1x SMG
2x Grenade

So, in 17 T2 boxes, 21 Green, 8 Blue, 3 Purple and 0 Orange.

Those T2 boxes are worthless to grind/open

H3adstomp3r
05-18-2013, 02:50 AM
would it not be worth adding a section for the % of luck? dont know if this has already been said have only just stumbled across this!

DieSlow321
05-22-2013, 06:56 AM
Ah very well spotted. Indeed this would have some effect.

Ctullu
05-25-2013, 07:12 PM
I just entered all of the current data you have gathered into the probability calculator over at stattrek.com, and here is what I came up with.

Assuming you have only 64 Keycodes, and can buy either one Tier 4 Lock Box, three Tier 3 lock Box's, or eight Tier 2 Lock Box's, here are the probabilities of each if you spend all 64 of the Keycodes:

4 items from a single Tier 4 Lock Box yields a 23.7% chance of obtaining at least a single legendary(orange) item.
4 items from a single Tier 4 Lock Box yields a 2.34% chance of obtaining at least two legendary(orange) items.
4 items from a single Tier 4 Lock Box yields a 0.106% chance of obtaining at least three legendary(orange) items.


9 items from three Tier 3 Lock Box's yields a 22.9% chance of obtaining at least a single legendary(orange) item.
9 items from three Tier 3 Lock Box's yields a 2.5% chance of obtaining at least a least two legendary(orange) items.
9 items from three Tier 3 Lock Box's yields a 0.172% chance of obtaining at least three legendary(orange) items.

16 items from eight Tier 2 Lock Box's yields a 18.49% chance of obtaining at least a single legendary(orange) item.
16 items from eight Tier 2 Lock Box's yields a 1.72% chance of obtaining at least a least two legendary(orange) items.
16 items from eight Tier 2 Lock Box's yields a 0.101% chance of obtaining at least three legendary(orange) items.

This is using the current data collected, as follows:

6.54% drop rate from Tier 4 Lock Box's
2.86% drop rate from Tier 3 Lock Box's
1.27% drop rate from Tier 2 Lock Box's

Hopefully this will clear up any questions people have about the actual drop rates and which is better for the money. If you only have 64 key codes to spend and just want a chance at getting a single legendary item, go with a Tier 4 Lock Box. I may come back and make a more comprehensive listing of probabilities for getting epic(purple) items later, but this is what I was looking for at the time and just figured I would share my results with you.

Yokai
05-26-2013, 07:31 AM
I just entered all of the current data you have gathered into the probability calculator over at stattrek.com, and here is what I came up with.

Assuming you have only 64 Keycodes, and can buy either one Tier 4 Lock Box, three Tier 3 lock Box's, or eight Tier 2 Lock Box's, here are the probabilities of each if you spend all 64 of the Keycodes:

4 items from a single Tier 4 Lock Box yields a 23.7% chance of obtaining at least a single legendary(orange) item.
4 items from a single Tier 4 Lock Box yields a 2.34% chance of obtaining at least two legendary(orange) items.
4 items from a single Tier 4 Lock Box yields a 0.106% chance of obtaining at least three legendary(orange) items.


9 items from three Tier 3 Lock Box's yields a 22.9% chance of obtaining at least a single legendary(orange) item.
9 items from three Tier 3 Lock Box's yields a 2.5% chance of obtaining at least a least two legendary(orange) items.
9 items from three Tier 3 Lock Box's yields a 0.172% chance of obtaining at least three legendary(orange) items.

16 items from eight Tier 2 Lock Box's yields a 18.49% chance of obtaining at least a single legendary(orange) item.
16 items from eight Tier 2 Lock Box's yields a 1.72% chance of obtaining at least a least two legendary(orange) items.
16 items from eight Tier 2 Lock Box's yields a 0.101% chance of obtaining at least three legendary(orange) items.

This is using the current data collected, as follows:

6.54% drop rate from Tier 4 Lock Box's
2.86% drop rate from Tier 3 Lock Box's
1.27% drop rate from Tier 2 Lock Box's

Hopefully this will clear up any questions people have about the actual drop rates and which is better for the money. If you only have 64 key codes to spend and just want a chance at getting a single legendary item, go with a Tier 4 Lock Box. I may come back and make a more comprehensive listing of probabilities for getting epic(purple) items later, but this is what I was looking for at the time and just figured I would share my results with you.


Great minds think alike. See my sig block. I haven't updated my charts lately, but will soon. It's been busy at work the past three weeks.

I hope you were using the _validated_ drop rates, not the raw ones. That 6.54% drop rate for oranges at T4 seems surprisingly low.

Burgess
05-26-2013, 07:50 AM
If I remember correctly you only need a (N) of 20 to gather an accurate probability rating, either that or (N) 100.

Cavadus
06-05-2013, 10:25 AM
Bumping for the awesomesauce of this project.

Macbeth Asdwon
06-06-2013, 01:58 PM
Have you guys considered taking into account the scrap value of the lockbox as well and coming up with a net cost, since you'll scrap the vast majority of what you get? For example, using the drop percentages for tier 1, you would be able to break those drops down into 14338 salvage per 100 key codes spent, or 14.338 key codes, making the net cost of the chests 85.662. You then come up with a new drop rate per net 100 spent after salvaging the crap. It goes some interesting results and might be more useful as it is your true final cost in codes.

Macbeth Asdwon
06-07-2013, 07:26 AM
Have you guys considered taking into account the scrap value of the lockbox as well and coming up with a net cost, since you'll scrap the vast majority of what you get? For example, using the drop percentages for tier 1, you would be able to break those drops down into 14338 salvage per 100 key codes spent, or 14.338 key codes, making the net cost of the chests 85.662. You then come up with a new drop rate per net 100 spent after salvaging the crap. It goes some interesting results and might be more useful as it is your true final cost in codes.

Tier 2 in my example, my bad. I have it all done on an excel doc I have in google drive, just can't figure out how to make it public and get a link to it from my phone. It ends up showing, based on your drop rates, to go for T4 for oranges but T3 for blues and purples.

Xeno Hunter
06-25-2013, 09:54 AM
Is there a way to determine which lockboxes will get you an orange weapons with good synergy in it or it doesn't matter?

Reason why is because I've been getting orange weapons with good synergy in it from lockbox 4.

Myll_Erik
06-25-2013, 09:58 AM
Yikes - lots of math!

You guys are a busy bunch I'll give you that!

Sanguinesun
06-25-2013, 10:05 AM
This should be a nice noodle cooker for all of you. They can adjust on the fly the drop chances of legendaries on those boxes without any transparency and with any number of variables to the chance's algorithm. Thus the data collection, while helpful if and only if the drop chance is a constant, can be extremely skewed or erroneous if it is dynamic(which I believe it actually is).

Cavadus
06-25-2013, 10:20 AM
Thus the data collection, while helpful if and only if the drop chance is a constant, can be extremely skewed or erroneous if it is dynamic(which I believe it actually is).

It's the only data we have and I can personally attest to the fact that due to nature of the submissions the data can quickly reflect changes so long as it's getting enough entries.

I started using this data project very early on and at launch lower tier boxes were the best bet for oranges. TW made their changes and this data project quickly corrected to match reality.

So, IMO, it's actually very accurate since the data submissions themselves correct the results' presentations over time.

As long as this thing gets lots of entries it will reflect reality. I have submitted the results of every single lockbox I have opened to it and I constantly remind my joes in the 47th to enter their data as well.

I'm sure we're not the only ones using it.

Cale307
06-26-2013, 03:09 PM
I just got a headache ....

UnchartedDestiny
06-26-2013, 05:11 PM
i wouldnt spend real money on lockboxes.. ive put a ton of money in the game and the orange drop rate is crazy. at one time i got 643 items from lockboxes and got 21 oranges. nothing good. so there as you see, the drop rate is more crappier than Borderlnads. at least in Borderlands you have the chance to farm the legendaries. Defiance, you hv to pay for them considering at times it takes you at least 2 days to accumulate 75 keys which is completely redunkculous. but like any other franchise, they **** you cuz they want your money. but your money isnt used properly, like fixing the game server and bugs. they pocked it for themselves. EA is really good at pocketing your money and nit using it wisely.

Skoll
08-10-2013, 08:45 AM
it takes you at least 2 days to accumulate 75 keys
Do not exaggerate, it's couple hours imo, not 2 days

Doyle
10-07-2013, 07:20 AM
Would it be possible to revive/reset the date for the T$ lock boxes after the latest patch. I would love to see how the percentages changed to determine what to buy for what I need.

Thank you for your hard work!

Qhival
02-26-2014, 05:32 AM
is this data still relevant? has the data ever been purged/ rest after updates?

Skeletoriels
09-13-2014, 08:44 PM
Have you considered expanding the data sheet, Instead of purging the data and losing everyone's hard work, just start a fresh set of data after each major update (Or specifically updates that alter the lockbox drop rate) Which would keep the drop rates available and show us exactly how much each update effects the lockboxes.

Skeletoriels
09-18-2014, 04:37 AM
Update: Tier 2 lockbox no longer drops legendary, so now it would be completely reasonable to purge the results and start fresh. Also, Arkforge is a possible drop and should be included in the data collection.

Etaew
09-18-2014, 06:58 AM
DD data collection works on patches and handles arkforge.