PDA

View Full Version : Farewell Repo-Man - The history of a longstanding bug



PhantasieTrion
12-11-2014, 05:44 PM
Hello Arkhunters

Veteran's, and perhaps some of our newer F2P players, may be familiar with a bug that has been around since launch. Seemingly randomly, characters would be ejected from their vehicle and it would then despawn.

Players referred to this as the "Repo-Man" bug; he often visited at the most inopportune time.

http://i59.tinypic.com/nfi8o2.jpg

Since launch, engineers periodically spent time trying to track down this bug but had no luck. The "randomness" of when it would happen made it so we could never reliably reproduce it, nor did our QA team experience it internally.

Recently, one of our newer engineers, DeadGhost, took a crack it. He came up with a theory that made sense, but theories based on slogging through 1000's of lines of code don't fix bugs; we needed proof.

In September, we made a call to the PTS community (http://forums.defiance.com/showthread.php?173900-Disappearing-Vehicle-Bug&p=1615968&viewfull=1#post1615968) to do a "Repo-Run" with some of the Dev's to test DeadGhost's theory. It was successful, and with Aftermath we now believe the elusive Repo-Man bug has been fixed (http://forums.defiance.com/showthread.php?178033-Vehicles-Despawning-Bug-Feedback&p=1645817&viewfull=1#post1645817).

The Defiance Team wants to thank the PTS players who helped us track down this bug:


Market
Fuzzy
Etaew
Xaat
Infinity Eagle
Stoodakiss


We granted you 50 arkforge for your help back in September and now, as promised, we have also granted your live characters an exclusive title: "Repo Crew".

If you see an Arkhunter with this title, please take a moment to /wave and /cheer.

http://i58.tinypic.com/wvoppx.jpg

Here is a summary the event, if you would like more details on it:

The run started at Tranquility Road (outside the Tutorial) and headed south east to Iron Demon Ranch. We re-grouped, and then drove south to The Crater.

http://i57.tinypic.com/125jfc7.jpg

http://i58.tinypic.com/ngakig.jpg

We then teleported back to Iron Demon Ranch. Someone, ironically, aggroed Car Trouble and, at TL 10, all our vehicles got destroyed quickly.

http://i59.tinypic.com/2w3vs0m.jpg

After we rescued the ranchers, we headed east to Shondu's Consulate to finish the run. I got ejected from my vehicle, as did several others throughout the run.

http://i62.tinypic.com/2cfxpur.jpg

http://i59.tinypic.com/4sk26g.jpg

Here is a summary of what the bug turned out to be, if you're interested:

An assumption (a poor one, in hindsight) was made with with the vehicle code. One player could notify the server that another players vehicle was in an invalid location (the server was no longer the authority).
If one player had a bad internet connection they would not get vehicle updates for players around them for a longer periods of time.
This exposed a second bug with the physics system - that player would then think that vehicles for other players (near them) were in an invalid location (since they didn't get an update on their real position from the servers, and a collision check failed).
That player would notify the servers that the other player was in an invalid location, and the server would then despawn that players vehicle in an attempt to get them back in a valid location.
We verified this on PTS with help from the aforementioned PTS'ers. We setup a local client and simulated a terrible network connection and then followed the PTS players around and watched them despawn from their vehicles (with logging in place to confirm the actual issue). One might consider this a mean thing to do, but we can assure you we did it in the name of bug fixing; not one of laughed as we randomly ejected players from their PTS vehicle.


Part of the reason DeadGhost was able to track this down was from anecdotal evidence on the forums (yes, we all read them) - a recurring theme we saw was players saying they got despawned more frequently with other players around them (near events, and such).

We look forward to tracking down more bugs on PTS with the help of our great community!

--Phantasie

Nefarious
12-11-2014, 06:52 PM
That's real good. And cool title to the few who now have them. The Repo Man was a true menace to Defiance, will not be missed.

Fuzzy
12-11-2014, 07:11 PM
Very nice it got fixed! The bug will not be missed. :)

Looking forward to track down more bugs on PTS with you all. It was a nice event and I appreciate the title! I'll wear it with pride :D


http://i.imgur.com/mP4CIXE.png

konstantinov
12-11-2014, 07:24 PM
Nice work to all who helped solve this obnoxious bug.

Anyone have a recommendation on windows mirror programs that don't cost a grand? I really want to get on PTS.
Will also take suggestions on a computer tower that I can hook up to my tv via HDMI.

Market
12-11-2014, 08:01 PM
Can I be one of the first to say to this pesky, annoying bug "GOOD. RIDDANCE!" lol

As always, I'm proud to help out wherever I can! Having that bug gone is truly reward enough, but I will wear my new title with great reverence!

Thanks goes out to PT and the rest of the Defiance crew, especially Deadghost for figuring out what was causing it!! Somebody give that Dev a beer!! Cheers!!!

Amack
12-11-2014, 10:57 PM
Like I always say, better late than never.

stoodakiss
12-12-2014, 12:42 AM
awesome, so you found the invisible repo man. thanks for the cool title.

Dixie Cougar
12-12-2014, 03:08 AM
Nice work all around, that bug was infuriating.

hiban
12-12-2014, 04:20 AM
I definitely wont miss the repo man. Good work, guys!

Vega
12-12-2014, 11:05 AM
          

Bentu
12-12-2014, 01:08 PM
Yep I'm glad to say I haven't been repoed recently,

Chump Norris
12-13-2014, 08:57 PM
So glad this bug is finally gone. Thanks for for helping to fix this and congrats on the cool title.

Authorized
12-18-2014, 04:45 AM
I rather get repo on occasion then trion to call my roller in an wide open area then having my character spin in another direction and toss it up a tree or into a railing or bolder, to have it not go into reverse a good 45% of the time, talking only about the not reversing part being the 45% of the time.