Jump to content
Double Fine Action Forums
Sign in to follow this  

[Advices] Your friendly neighboor's guide to QA Bug Report

Recommended Posts

Your frindly neighboor's guide to QA bug reports

Given I had my share of experiences on Game Developing QA testing , I thougth I might give some helpful advices to those who would like to contribute with bug reports in Spacebase DF-9 development.

So these are some really simple rules you can follow to achieve a successful contribution to development !

Buf first ... some terminology ... because ... everyone loves terminology , right ?

- White Box Testing : This is one is the QA guy's heaven. It just means you as a Game Tester has access to the game's source code ( and the correspondent debugging enviroment ), so your bug report's

not on include information about what you do to produce the actual bug but also information about what part of code ( variables , memory location , error handling a.k.a. exceptions , etc ) was affected by

this bug.

- Black Box Testing : Also know as "Functional Testing" . This is the most common type of testing. Basically means you find bugs by actually playing the game without ever caring about the game's internal

workings. There are various types of Black Box Testing depending on what you are trying to test :

+ Ad hoc : This means you play the game until you find (i.e. you "stomp" with) a bug.

+ Regression : This means repeating the sequence of actions that lead to a previously reported bug. It's done when you have found a bug on a previous build that has been notified as fixed and

you like to check this bug is actually fixed.

+ Scripted : This is done when you need to check is a new feature is working properly or "as intended". It just means you follow a sequence of actions designes by the developer or QA lead in

order to test the feature.

- Useful Acronyms ( because , you know , devs loves acronyms ) :

+KS/WF : Means "Known Shippable / Won't Fix" , it just means the developers run out of time or money to actually fix the bug given the shipping date is very very close trying to fix a not very

important bug may led to breaking up other things in their game.

+NMI : Means "Need More Info" , your bug report was accepted , but your description about how you got to that bug doesn't shed enough light on were this bug is actually occuring in the code ,

so you are asked to describe this sequence more thoughly.

+NR : Means "Non Reproduceable" , your bug report does not provide enough information on the sequence of actions that lead you to this particular bug , so the developer can't actually

reproduce it in order to debug the actual code and find out what is going wrong.

+AS/NAB : Means "As Designed / Not A Bug" , it just means that what you appreciated as a bug it's actually an intended game's feature , so no fix is necesary.

Okey ... let's start then !!!

1st - Be sure you are using the lastest possible build

I am sure most of you think this one is a no brainer , but , it's easy to forget that you may be reporting a bug that was already fixed in a previous build ( specially given developers actually use

different build versions that the public released ones ). So , first be sure that you are using only the latest build.

- Q - Grumpy Tester : What do i do if i found a bug in the build i am playing and a new build was released 15 mins ago ?????

- Answer : In that case all you need to do is install the new build , DELETE ALL PREVIOUS BUILD SAVED GAMES , and then repeat all the steps that lead you this bug. If the bug reappears like

in the previous build , you are free to report take notes about how you ever got to that situation.

- Q - Grumpy Tester : I worked very hard to build my awesome space base before this bug showed up , why do I have to delete all my previous saved games ?

- Answer : While in development a saved game file format may change from build to build for very different reasons. While a previous saved game may work with a currente build release , you

may be experiencing a bug actually produced by obsolete information in your saved game and not because of an actual game's defect.

Share this post

Link to post
Share on other sites

2nd - Triple check before you report

You might not know this , but bug reports repetition ( also know as "Bug Duplication" ) actually may hurt a game development . You see , game developers mantain big databases ( a.k.a. "Bug

Trackers" , you might have heard of "Bugzilla" , that's one of them )of every actual defect in the game. Now , if a developer receives 200 reports of the same bug , you are only wasting a developer's time

, because it has to sort out all those duplicated bug reports. Besides , some bugs affect more than one area , like animation , models , gameplay ( this is what's called a "Cross-Cutting Concern") and

fixing it might require for the bug report to bounce back and forth before it's actually fixed. So , before reporting a bug , check in the forums to make sure the bug you are reporting hasn't been already

being posted.

3rd - Repeat , repeat , reapeat ... and then ... repeat some more.

I can't enforce enough the case you must actually repeat several times the sequence of actions that lead you to a bug ( not to mention taking down notes of them ). A game developer will

actually require to repeat them at least 10-15 times before actually locating what's going wrong with the code. So its extremelly important that before you report a bug , you can actually assert it's a

reproduceable one. If you can't assure that then you are only wasting a developer's time.

4th - Use propper english grammar and sintax

It's not use for a developer to spend more time and effort trying to decipher a bug report that looks like being written as a SMS than to actually trying to fix the bug. So communicating effectively

what is that you did for this particular bug to show up is of crucial importance.

5th - Know your priorities (a.k.a not all bugs are created equal )

You may not know this but most bugs have some priority level to be fixed . So it's useful for a developer if you can communicate how important for the game is this bug.

-Q - Grumpy Tester : How can I even know what priority I must assign to a bug ?

- Answer : Some common prority examples are :

+ Priority A - "Must Fix" : Anything that severy interrupts a game continous playing

- The games crashes after a sequence of events.

- The characters fall out of the game's world

- An event need for the game to advance to a next level never happens

+ Prority B - "Urgent" : A disrupting event that doesn't stop you from progressing but it severy affects how you interact with the game

- A malfuntioning User Interface element that doesn't render properly

- A character animation that deprives you from using it effectively

+ Priority C - "Important" : A very frequent non disrupting or progress stopping event that affects severy the game

- A non propperly applied model texture

- A User Interface Element that is not propperly readeable for the player

+ Priority D - "Major" : A very infrequent non disrupting or progress stopping event that affects severy the game

- A character's model that mulfunctions in very specific areas

- A user interface element that disappears after a very specific action

+ Priority E - "Minor" : Almost unnoticeable and non disrupting defects

- A texture that is not properlly scaled or "jointed"( a.k.a texture seams)

- A game prop that is not located in a very expected o logical place (i.e. a mid-air crate)

- A game prop not accuratebly scaled

Share this post

Link to post
Share on other sites

6th - It's all about categories , son !

You will make a developer a great favor if you can give him/her some keywords to make his search easier. As an example , if your bug is about big frying pans on you space base , you can add

something like "Keywords : Art - 3D Model - Scale - Game Prop " to your bug report. Or , if your bug is about Bartenders eternally waiting for themselves to prepare some food , it can go like something

like this : "Keywords : Gameplay - Bartender - Food - Infinite Loop ". You are gonna be making your favorite dev's life a hella lot easier !.

I hope this little writing becomes a useful guide for your QA Bug reporting adventures !

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Create New...