Was the Beta Test wasted?

[VolDeVie]

Short answer : Beta was wasted, they're working over-time now. I bet they feel the same way.

Long Answer : I'd have had them working 60 hour works and sleeping in the office. I'd have done whatever it took to have a clean launch on something that had my name on it. Still, quite an enjoyable game and happy at the rate that things ARE being fixed.

[LordRaymond]

Seems to me the beta test was more of a demo of sorts for people to try the game and a way to test their servers. Rather than focusing solely on bugs that will regardless be present in the release product.

[Panthan]

I'm disappointed that none of the replys so far answered my questions: What decision would YOU have made if you were in charge? What would have caused you to make that decision?

I cant really add much to what you would have done, only thing I can think of Id add is a known issues post with regular updates and status.

[MacDeath]

Seems to me the beta test was more of a demo of sorts for people to try the game and a way to test their servers. Rather than focusing solely on bugs that will regardless be present in the release product.

It could well be true that Beta 3 was primarily a test of the servers, especially the consoles beta. None of us outside of Trion know what the primary focus of the tests were. However, Alpha test had been running for some time and there had already been two Beta tests for the PC platform going back to January and February.

[Kreegan]

I'm disappointed that none of the replys so far answered my questions: What decision would YOU have made if you were in charge? What would have caused you to make that decision?

The VAST majority of people who make statements like "all games have bugs" have obviously never been developers, never been leads, managers, or directors of large development products, or - at best - have been a programmer in an academic setting or in some low level role on some trivial effort.

Not trying to insult anyone; we all start there.

Reality is that major efforts have a target budget and a target schedule, and today's business world is focused on short term profits. The companies putting out MMORPGs are NOT targeting long term customers; they aren't so much looking at retention of players as they are looking at initial sales of boxes - especially in a B2P business model. As such, companies make decisions about how much risk to accept in knowingly releasing a flawed product relative to impact on sales.

Beta tests in MMORPG environments are marketing tools, not real tests. Real tests - especially for large programs - are highly structured and highly formal, not just a bunch of kids and wannabe kids stroking their egos so they can claim "firsts". Real tests take a lot of time, and there is a significant amount of time spent correcting problems in code after formal testing is completed.

I'm retired now, but I wrote, designed, lead, managed, directed, and produced millions of lines of code over my career and NEVER released software with known bugs; I can say quite honestly that the vast majority of code relased in products over the years that I worked on was bug-free.

What's the difference? Simple: products not focused on short term profits; products actually intended to be used for the long term are going to be "finished" before they are approved for release.

It isn't a question of schedule or budget; I can't think of any product I was ever involved in that did not have tight schedules and budgets (and, quite frankly, the products were much more complex algorithmically than MMORPGs). It was a question of focus: do things right the first time, not put out a piece of crap and expect the customers to accept that.

There is no excuse for Trion or any other MMORPG developer; they make a conscious choice to put out a flawed product because it really doesn't hurt their bottom line.

[Rasczak]

Not enough information on the legalities and day-to-day constraints of Trion around that timeframe.

"What I'd do" is an impossible question to answer due to this. All that matters is what was done produced a substandard product, even if that substandard product was the best Trion could manage under its constraints.

I have to agree with him, Mac. Sorry, I know what you are asking, but trying to answer it would be akin to me asking internet fans and players of my RPGs, who have never operated a full publishing company, never mind never actually produced a book for retail publication (from the concept, design, writing, editing, art, layout, printer files, proofs, and shipping stages, never mind the freelance hiring and coordinating across multiple countries) what they would have done different on our back end.

I can assure you, they would not have enough information on the logistics, schedules, deadlines, and technical issues to answer that; just as I don't for Defiance.

[baelrusk]

In my opinion I think alphas and betas are not typically wasted.
But from the sheer basics of the bugs that were reported then and are still in this game I do think they were.

First off......if a delay on release is not possible.........I would have it publicly noted right on the first page the known bugs and that they were being worked on.
Secondly......I would arrange there be some for of discounted purchase until a WORKING patch that addresses those fundamental issues is available.
Thirdly........the members of the team that creates things to add into the game.......I would move whoever I possibly could (if they actually had the skill to) over to the team fixing the existing bugs.
Fourth......every patch would require EXTENSIVE testing before release.
Fifth.......I think my goal in the beginning would have been to make it TRUE cross-platform (interact with players who have different platforms.....even if you have to choose a server)
Sixth.......I would try to very much to never add any vague comments
Seventh......no advertizing with an online stream that shows off the game (man it pisses me off......showing all these great things and all the stuff you plan to add and never talk about the problems that exist)
Eight........set strict deadlines for improvement
Nine......offer some full refund policy available (regardless of where it was bought) during the period those fundamental major issues still exist.........it's just good policy

That's all I can think of for now. Only been awake for ten minutes........but if I would probably have ended up thinking up more.

Just a side note since I saw people referring to this as an MMORPG again. It isn't. It is an MMO.......an MMOTPS.
Not all MMOs are MMORPG games. After all.....this is a Third Person Shooter, NOT an RPG. Side note over.

[MacDeath]

I have to agree with him, Mac. Sorry, I know what you are asking, but trying to answer it would be akin to me asking internet fans and players of my RPGs, who have never operated a full publishing company, never mind never actually produced a book for retail publication (from the concept, design, writing, editing, art, layout, printer files, proofs, and shipping stages, never mind the freelance hiring and coordinating across multiple countries) what they would have done different on our back end.

I can assure you, they would not have enough information on the logistics, schedules, deadlines, and technical issues to answer that; just as I don't for Defiance.

Fair enough Rasczak. We, the posters on these forums, don't have enough info to decide if we would have done it differently. Then how is it that so many posters think they have enough info to decide that Trion did it wrong?

[Schwa]

Fair enough Rasczak. We, the posters on these forums, don't have enough info to decide if we would have done it differently. Then how is it that so many posters think they have enough info to decide that Trion did it wrong?

Because I can tell when my soup's cold, just as I can tell when a game doesn't pose any decent challenge.

Co-op expert modes are one way to add challenge. An enemy tier is one way to add challenge. Running through the storyline with BMG to add challenge adds challenge... but is lame.

It's up to the developer (of a TPS or MMO) to add challenge that isn't lame, just as it's up to the chef to warm up my soup without burning it.

[Codebreaker]

What line of work did you do coding for?

The VAST majority of people who make statements like "all games have bugs" have obviously never been developers, never been leads, managers, or directors of large development products, or - at best - have been a programmer in an academic setting or in some low level role on some trivial effort.

Not trying to insult anyone; we all start there.

Reality is that major efforts have a target budget and a target schedule, and today's business world is focused on short term profits. The companies putting out MMORPGs are NOT targeting long term customers; they aren't so much looking at retention of players as they are looking at initial sales of boxes - especially in a B2P business model. As such, companies make decisions about how much risk to accept in knowingly releasing a flawed product relative to impact on sales.

Beta tests in MMORPG environments are marketing tools, not real tests. Real tests - especially for large programs - are highly structured and highly formal, not just a bunch of kids and wannabe kids stroking their egos so they can claim "firsts". Real tests take a lot of time, and there is a significant amount of time spent correcting problems in code after formal testing is completed.

I'm retired now, but I wrote, designed, lead, managed, directed, and produced millions of lines of code over my career and NEVER released software with known bugs; I can say quite honestly that the vast majority of code relased in products over the years that I worked on was bug-free.

What's the difference? Simple: products not focused on short term profits; products actually intended to be used for the long term are going to be "finished" before they are approved for release.

It isn't a question of schedule or budget; I can't think of any product I was ever involved in that did not have tight schedules and budgets (and, quite frankly, the products were much more complex algorithmically than MMORPGs). It was a question of focus: do things right the first time, not put out a piece of crap and expect the customers to accept that.

There is no excuse for Trion or any other MMORPG developer; they make a conscious choice to put out a flawed product because it really doesn't hurt their bottom line.