Friday, June 1

CCP, Singularity and You: How we can all make better use of the test servers


A Brief Introduction

This past Monday, I was fortunate enough to have the opportunity to speak with CCP Explorer while he was in Providence, RI. We kind of skipped around from topic to topic before he got swamped with people from 38 Studios to interview, but one of the subjects that came up was one that'd been in my mind for a while: how the playerbase makes use of the test server, and how it affects CCP's development of the game.

I figure it's probably safe to assume that the majority of people reading this know about the test server Singularity and its sister (cousin?) Duality. In case you're not in that majority, Singularity and Duality are CCP's public testing servers for the playerbase to test new features for upcoming patches using slightly outdated copies of your characters from the production server Tranquility. You can find installation instructions elsewhere on the forum or on EVElopedia.

So What's the Problem?

The problem is that there is a fairly wide gap between how CCP sees the test servers and how the playerbase uses them. While generally harmless, this gap can have some pretty nasty side effects when the things being put on the test server are going to change things on Tranquility at a very fundamental level.

Try Before You Buy: How CCP views the test servers

CCP wants to provide its customers with the best experience it can. One way that they attempt to do this is by including the playerbase in the development process for features with devblogs, forum posts and, you guessed it, builds on the test server.

Singularity and Duality provide the players with the opportunity to play with and provide feedback on upcoming features and changes to the EVE client while they are still in development. These features can range in scope from a graphical change to a whole new UI element or even new ships or modules. To facilitate testing of the latter group, most charges, modules and ships have been 'seeded' at 100 ISK/unit at several stations in each region.

CCP's hope is that players who have the time and spare disk space will install a copy of the Singularity (or on rare occasions, Duality) clients and test these in-development features, as well as fixes to bugs that have been reported by player and dev alike. The opportunity to get involved and get some practice and experience with upcoming features before they are pushed to production should be motivation enough to log into the test server on its own. Being able to play around with ships and modules for cheap is just a side dish.

EFT Warriors Unite: How the players use the test servers

Unfortunately, some of the things that CCP does to help facilitate testing of new features, changes and bug fixes are also a big part of why the players frequently use the test server for things that do very little to actually help CCP improve the game. (I'm sorry, but testing out your latest theorycrafted Moa fit in 6-C, while fun, does not help CCP in the slightest. Get over it.)

Seeding ships, modules and ammo at cheap prices has the side effect of turning the test server into not a place where testing features, changes and bug fixes is the priority, but into a magical playground where every kid can summon (nearly) any toy they want out of the aether. It brings people's focus and interest in the test server to what is effectively multiplayer EFT/pyfa/etc, rather than making sure things work. Who wants to check out some random new UI feature when you can fit and fly that ship you've always wanted to try out on Tranquility but never had the ISK to buy the hull? Who wants to take a look and see if that bug that makes the game unplayable for 3% of players was properly fixed when you can take that ship into combat and see how well it works?

The answer? Not enough people, which is a real shame.

The majority of the playerbase seems to view the test servers as the aforementioned playground - a simulator, if you will - but not as the opportunity to help participate in the development process that it is. Most people on the test server are interested in testing ship fits. This in and of itself isn't a problem, and in fact it's just fine. It's a perfectly valid use of the test server and I don't hold anything against anyone for it. The problem is that it typically comes at the expense of everything else that CCP needs the players to be testing.

Getting to the Point: How we can make more effective use of the test servers for everyone's benefit

The solution here is two-fold: both CCP and the playerbase need to bend a little to make this work.

What the players need from CCP:

  • More devblogs on features that are still in the prototyping stage, so that CCP can get player feedback right away, early in the process.
  • Timely iterations on these devblogs, either via posts on the forums or additional devblogs, addressing the feedback. Whether feedback is being incorporated into the design or not, players should get replies from the developers responsible for the feature saying so. If someone's idea is really good and you're going to add it, say so! If something we suggested just doesn't fit with the plan, tell us why. If you're not sure about what someone suggested, just say so and ask for a more detailed explanation. 
  • Public and easily visible notification that these new feature and changes are going on Singularity or Duality, and when they will be available for us to test. You have people who are on Twitter all the time, make use of them!
  • Remember that your use-cases for testing purposes may not cover all of the ones players have to experience on a daily basis. Ask us to come up with scenarios you may not have thought of - I can more or less guarantee we'll have a few that will make you wonder how you could have missed them, like the issue with POS towers with large numbers of modules and the Unified Inventory.

What CCP needs from us:

  • More testing of the client itself and less faffing about with multiplayer EFT. Before you go play with the seeded market, take a look at all of the new features and try to see how they fit into your routine. Try to break them. Hundreds of people show up for mass tests. Imagine what could happen if everyone using the test server spent half an hour testing new features before going and playing with the seeded markets.
  • Constructive feedback and criticism. Spewing bile at the devs won't get us anywhere; it just puts them on the defensive. Be polite and be detailed in your feedback. Give examples and screenshots if possible.
  • Properly-written bug reports! There are threads on the forum and articles on the wiki to help you out. The better-written and more detailed our bug reports are, the more likely it is that the developers can track down the issue and fix it quickly.
  • Help the devs increase awareness of features and changes that are in progress on the test server. Use Twitter, corp/alliance forums, blogs, even facebook or Jita local. Just get the word out!


Please note that I'm not advocating that people not be able to screw around as they please on the test server with ship fits and such. I'm advocating that people spend more time trying to help CCP by testing new things that affect the entire playerbase, not just your personal k/d or ISK/hour ratios.

In short: we need to help them help us, and CCP needs to help us help them. Do your part!

If you have comments, please post them to the  thread on the EVE forums here.