Jump to content
Double Fine Action Forums
Sign in to follow this  
Greg Rice

Production Update #2: What's a Producer?

Recommended Posts

Woah. As a SCAD student and wannabe producer, this is really amazing and eye opening. Getting way more than I ever imagined by backing this project. Who knew I'd get the opportunity to learn as well? Please keep up these production updates. While I have heard of these methods before in class, it's pretty cool to see them in action. These are definitely things I will be researching more into and applying to the projects I am working on now. Thank you!

surprised-rainbow-face.jpg

surprised-rainbow-face.jpg.9be4829fed4d1

Share this post


Link to post
Share on other sites

This is a fantastic question, and touches on a subject close to my game developer heart. Forgive me if I write too much in response :)

This is a fantastic answer!!! It combines knowledge with real experience! Thanks Nathan!

Share this post


Link to post
Share on other sites
Do you use a actual scrum board or do you note it all digital on scrumtious?

During BL we actually used a board w/ physical note cards in addition to scrumtious. I, personally, grew to hate the physical notecards. They take a long time to write (many people have dozens of tasks in a sprint) and pin up and move around and once you've written them they aren't very easy to update. I suspect the note card approach would be awesome for a small team of 5 people with no budget for project management software, but on BL we were 60-70 people, 4-5 scrum teams, and it was just a crazy amount of overhead. Scrumtious (which Gabe wrote) is vastly superior on all fronts* and we've never done physical note cards since.

*Scrumtious is amaze, but if you don't have your own personal Gabe, you could probably get similar advantages using any of the existing commercial agile/scrum softwares or even Jira/Bugzilla/etc.

... Seeing note cards on the wall in various states of completion gives an instant visual graph of where everything is. Almost like a color coded defrag display. If you see a card that should be completed staying in the same place day after day, it raises a red flag for the team and invites conversation. In the digital space, you have to actively look for it, depend on each person to keep things updated properly, and then go find someone to talk things over if the data is even up to date. It is less organic. It also adds a bit of accountability for each member of the group when they have to show their work in front of the team.

...The cards in the standups put things more in your face. I haven't found a way to completely recreate all the benefits of cards in a digital tool. I thought it was worth mentioning in case someone is interested in taking a stab at the cards. There is some value there.

As N8 mentioned earlier, one of the core tenets of scrum is "people over process". Since the cards added too much overhead for the team on smaller projects with tighter schedules and generally made everyone grumpy, we dropped it for the good of the team.

This is all very new to me but I'll give it my newbie approach. I don't/can't know the program you are using but I suspect that there is a visual effectiveness in all this cards issue. The best way to handle it in a digital form is seeing always the big picture AND every aspect/area of focus together. Perhaps a zoom in/zoom out (like digital maps) approach could always get you to see the overview of the project without getting lost in the details.. See the trees AND the forest!

Just a thought...

Share this post


Link to post
Share on other sites

So much organization. Good thing some people out there can manage that. Frankly looking at all that just gives me a headache.

Share this post


Link to post
Share on other sites

One thing glossed over in the description of scrum here is that traditional scrum uses Story Points (SP) to gauge how much a team can handle in a sprint. Every team "commits" to a SP capacity, usually based off of the completed SPs from previous sprints. You reach your capacity for SPs when planning the next sprint, and you ae done! You determine SP by researching upcoming user stories, and assigning (usually) a Fibonacci-like value to them (1,2,3,5,8,13,20,50 is the standard set). The points are determined by team consensus, though usually the engineer who researched the story has a little extra sway. SPs are supposed to take into account effort (how long?), complexity (how hard?), and risk (how scary?). After thinking on a user story, you compare it to the values of stories completed in the past, and assigned the same value as a similar previous story. This is all because, people SUCK AT TIME ESTIMATION. It's a psychological deficiency inherent in the human animal. However, we are actually really good at comparisons. Like, really good.

The way to think about it is that if I have a fancy, purple, weirdly-shaped vase to fill with water, I will be way off if asked to estimate how long it will be to fill the vase, but given several other vases, I can very easily tell you which other vase probably would take the same amount of time. Then, by making sprints short, you keep things from spiraling out of control for too long , and you get lots of stories completed that can be used to refine your SP estimation.

Agile gurus do recommend estimating hours, but it's really the SP that guide when a sprint is "full". The hours are largely to let people managers figure out their people's utilization. At the company I work for, we actually stopped estimating hours all together, because the teams were, honestly, awful at it. Or worse, they would work themselves to the bone, but only log the hours to make everything "look right". Plus it stressed people out. The SPs were a much more accurate tool at determining if the work allotted could actually be completed.

NOTE: I was a (almost) certified scrum master, I just need to take the test. Everyone does scrum a bit differently (it's actually "allowed" by the rules of scrum), so everything I said above applies to a geneic ideal scrum team.

EDIT: BTW, given that a team (after a couple sprints of settling to build team bonds) levels out their SP capacity for a given sprint of a given length, the PO can then get a pretty good idea of when they can release, with a certain feature set.

Share this post


Link to post
Share on other sites
What would you say are the top skills or pieces of experience one would need to become a producer? What are people looking for?

There are a lot of different people make-ups that can make a good producer, but some of the keys to me are:

- Be a people person. Love working with people to achieve a goal and have empathy for them when things get tough. Your game is nothing without talented people, and you want to keep them happy. The producer rarely creates game content, rather his/her job is to ensure others do, so you will be running around doing little things to make the team's life easier and making sure the machine doesn't get gummed up!

- Be organized. Most people depend on the producer to be so they don't have to think about dates or deliverables so much. This starts with how you manage your own time. There is only so much time in the day and you need to be able to constantly prioritize the most important things while keeping an eye on the holistic view of the project. "Getting Things Done" by David Allen is a great book for picking up some time managing techniques.

- Be analytical! You need to constantly analyze situations and data to be able to judge how your team is tracking. When you see the team making trees, you need to be able to extrapolate a forest. Especially if you are making a canyon game! Stupid analogy.

- Thick skin! Being a producer is often a thankless job. When things are running smoothly, people assume that's how it should be and don't wonder why it's running smoothly (although some people notice!). You mostly only hear feedback when things aren't going well. You need to be able to mostly deal with with that sort of feedback and be able to find your pleasures in the team's success. You may also need to deal with personnel issues. Those aren't always fun.

Other things that help:

- Experience in other areas of the game. You should have a solid understanding of whatever you are responsible for. If you're managing a whole team, this can be difficult if you don't know a 1 from a 0 or have never opened Maya. You will have to lean more heavily on your team leads and can often get in trouble if you don't recognize problems early enough. You also need to be able to determine whether work estimates are reasonable or not. If you're managing something, take some time to learn the basics. You don't have to be an expert, but you should be able to speak the language.

- Play games! I'm sure you have this in the bag already, but it's good to know the competition and all the bars that are getting set in the industry. Part of your job is quality control and you have to know what that even means. Actually, if you want to kill two birds, one game that requires a TON of producer skills is Viva Pinata. If you can unlock all those stinking animals in that game, then you are ready to do some production consulting!

I have to manage my own time now, so I'm going to stop typing.

Share this post


Link to post
Share on other sites
One thing glossed over in the description of scrum here is that traditional scrum uses Story Points (SP) to gauge how much a team can handle in a sprint. Every team "commits" to a SP capacity, usually based off of the completed SPs from previous sprints. You reach your capacity for SPs when planning the next sprint, and you ae done! You determine SP by researching upcoming user stories, and assigning (usually) a Fibonacci-like value to them (1,2,3,5,8,13,20,50 is the standard set). The points are determined by team consensus, though usually the engineer who researched the story has a little extra sway. SPs are supposed to take into account effort (how long?), complexity (how hard?), and risk (how scary?). After thinking on a user story, you compare it to the values of stories completed in the past, and assigned the same value as a similar previous story. This is all because, people SUCK AT TIME ESTIMATION. It's a psychological deficiency inherent in the human animal. However, we are actually really good at comparisons. Like, really good.

The way to think about it is that if I have a fancy, purple, weirdly-shaped vase to fill with water, I will be way off if asked to estimate how long it will be to fill the vase, but given several other vases, I can very easily tell you which other vase probably would take the same amount of time. Then, by making sprints short, you keep things from spiraling out of control for too long , and you get lots of stories completed that can be used to refine your SP estimation.

Agile gurus do recommend estimating hours, but it's really the SP that guide when a sprint is "full". The hours are largely to let people managers figure out their people's utilization. At the company I work for, we actually stopped estimating hours all together, because the teams were, honestly, awful at it. Or worse, they would work themselves to the bone, but only log the hours to make everything "look right". Plus it stressed people out. The SPs were a much more accurate tool at determining if the work allotted could actually be completed.

NOTE: I was a (almost) certified scrum master, I just need to take the test. Everyone does scrum a bit differently (it's actually "allowed" by the rules of scrum), so everything I said above applies to a geneic ideal scrum team.

EDIT: BTW, given that a team (after a couple sprints of settling to build team bonds) levels out their SP capacity for a given sprint of a given length, the PO can then get a pretty good idea of when they can release, with a certain feature set.

Excellent points, Olhado! We tried the SPs and for whatever reason it didn't resonate with the entire team. We took a stab and it fell flat. We are definitely using a bastardized version of scrum, but it's working for us. Different strokes for different folks!

Share this post


Link to post
Share on other sites

Wow, I'm really glad I found this topic (unfortunately, my time management as of late has down-prioritized DFA and its updates. Finally carved out some time today).

I'd really like to return to the waterfall vs. agile/Scrum for a moment here. First of all, I'll flat out say that I'm a strong believer in agile. I tend to prefer Scrum, but I'm not religious about it. The most important factor for me is the necessity to deeply understand and apply the 4 values of agile and its principles (for those who don't know about it, look here Agile Manifesto)

You sound incredibly experienced and insightful Nathan, but I have to critique your description of Scrum a bit and also some of the ways it has been implemented.

First of all, Scrum does not ignore or even criticize the importance of long-term planning. Most texts on Scrum I've read, the Planning Onion has always been played a fundamental part. While it is true that the Scrum method itself primarily revolves around daily and iteration processes, it highly acknowledges the need for longer term planning. Essentially, it's all about decreasing the level of planning detail the further out in the onion you go.

The agile principles and practices were originally described by Nonaka & Takeuchi in a field analysis of Toyota's Just-in-Time and Lean practices, and an important part of their study revolved around a constant feedback cycle, where the field work of employees created new learnings that were reported back to upper management, which then adjusted the high-level plans accordingly and continously.

So I have to disagree with you on the statement that there is a subtle lie at the heart of Scrum. While waterfall is dangerous because it trusts estimates for longer periods of time and split phases, which increases the cost of change and completely obstructs feedback and learning loops, Scrum is only empirical about things and positions reality/working software at the front and center. In other words, waterfall often hides reality, while agile/Scrum (when utilized correctly) makes even the ugliest truth take the spotlight.

Utilizing Scrum requires high-level planning and a deep understanding of the complex dynamics of the product being made, in order to avoid what you describe: standing with functional, but individual parts that produce... nothing. That's why it's important to prioritize the high-level components and introduce them at the right time in the development cycle. Nonetheless, a strong focus on vertical slicing - however difficult it is - makes it much easier to live up to the promise of working, releasable software.

But bear in mind two things regarding the last statement:

1. Most agile methods put an emphasis on working software - not necessarily releasable. The reason why releasable can be a good CoS to use is because it puts a greater focus on the front-end part of the experience. If only backend, engine stuff works, it's not really releasable, is it?

2. Scrum and other agile methods acknowledges that most projects start with a heavy focus on architecture building, which necessarily makes it difficult to actually showcase something. What these methods do is merely encourage to always put something in there that a user can experience, and never forget that focus.

From what I hear, I wouldn't call your high-level plan for waterfall for the following reasons:

- You are aware of estimate errors and correct the planning as reality takes shape.

- You don't do detailed planning; it sounds like you're only doing exactly the planning needed for this level and don't drill down to levels of detail where the margin of error would become too big.

Also, remember that most agile experts highly value the old Eisenhower quote: "Plans are nothing; planning is everything." Your high-level plan sounds more to me like a process than a product.

That being said about Scrum and Agile, I do see the value in waterfall when these conditions are met:

- We know exactly what we're going to do

- We know exactly how to do it

- We know exactly how long it will take

All within margins of error, of course. But that often only applies to content production with the scene is already very strictly set.

Second, I'm curious about your implementation of the Scrum Board or whatever one would call it. You said that you went away from the idea because there were too many tasks and thus it took too long for people to write index cards and maintain the board. I can see why this became an issue, and I completely agree with you on the notion of people over processes as the reason for why you abandoned the board concept.

Personally, I see the problem as lying with the level of detail used for the index cards. A Scrum Board shouldn't contain every single task for every person. It's about finding just the right level of abstraction based on the team and project size. Make it detailed enough so that everyone has a good understanding of where the project is and how it's going, but no more detailed than that. It's a team tool, not your personal task organizer.

Also, for very large teams, many experts emphasizes the need for several boards (as well as several stand-up meetings), set up as such:

- Each team has their own board and stand-up, which is pretty detailed

- Each team has one representative that participates in a high-level stand-up (also called the Scrum of Scrums), which revolves about a high-level project board where the level of detail is low (fx the board has no specific tasks, only story cards)

Want to get a high-level overview? Look at the project board. Want to drill down into the performance of a team or the progres of a certain story? Look at the relevant team board.

BTW, I do acknowledge the fact that utilizing agile takes a whole lot of discipline, especially in the early, transitional phases, but the methods are specifically built to help your maintain discipline and not be able to cover up the truth. That's why so many experts always advice to initially introduce whatever practice you want to use (Scrum, XP, Lean etc.) by the book and be very rigid about it, so you don't skip certain parts because it doesn't fit or feel right. Then when you've properly learned the value and importance of each method, you can begin to evaluate how to tweak it to fit your environment, because now you know how it will impact your team and it won't just be for the sake of covering up some parts of the truth.

Of course, Nathan, you are much more experienced than I and I know that the true learning of producing comes from experience, so I'm not trying to disregard you in any way (I'm still young and relatively inexperienced). And it seems like you have exactly the right mindset and approach to production as I hope to employ. But it just feels like you've been a little burned from the early Scrum hype, where the practice is often blamed instead of oversimplifications and fanatic craze. "Hey, heard about this Scrum thing? It's super great and awesome and will make everything better very easily! It's impossible to fail with it." If anything, agile experts are the first to acknowledge that there is no silver bullet, no all-in-one-for-everyone methodology. But I still believe these two guidelines should always stand on the forefront:

- The agile values and principles should be the core guideline for everything you do, since values and principles are context-independent

- Always start by applying the chosen practice by the book; understand its advantages and disadvantages in your context, and once you why certain parts actually work and don't, then start altering it to suit your specific needs in the current context

Well, that was longer than I thought :D Anyway, thanks for some great insights, arguments and discussion so far. I'll be keeping my eye on this!

Share this post


Link to post
Share on other sites
Wow, I'm really glad I found this topic (unfortunately, my time management as of late has down-prioritized DFA and its updates. Finally carved out some time today).

I'd really like to return to the waterfall vs. agile/Scrum for a moment here. First of all, I'll flat out say that I'm a strong believer in agile. I tend to prefer Scrum, but I'm not religious about it. The most important factor for me is the necessity to deeply understand and apply the 4 values of agile and its principles (for those who don't know about it, look here Agile Manifesto)

You sound incredibly experienced and insightful Nathan, but I have to critique your description of Scrum a bit and also some of the ways it has been implemented.

I think we both agree on what good process looks like. Whether that process or similar but flawed processes are "true Scrum" or not, is difficult to answer. When a faith's practitioners all make similar mistakes is the flaw in the faith or in the practice? That is not for me to say. In fact, it's not really for anyone to say as there is no central authority to "canonize" the tenants of scrum, just a lot of prophets, err, consultants, espousing generally similar philosophies. I can say that for sure when we started adopting scrum 7 years ago many people only acknowledged long term planning in a token way and that I saw many teams (including our own) lured into a false sense of iterative security. All I want to do is highlight a very common mistake (or misunderstanding) let others do as we did.

From what I hear, I wouldn't call your high-level plan for waterfall for the following reasons:

- You are aware of estimate errors and correct the planning as reality takes shape.

- You don't do detailed planning; it sounds like you're only doing exactly the planning needed for this level and don't drill down to levels of detail where the margin of error would become too big.

Also, remember that most agile experts highly value the old Eisenhower quote: "Plans are nothing; planning is everything." Your high-level plan sounds more to me like a process than a product.

I think that statement is fair enough. I definitely love that quote. Does that make me an agile expert? If only...

Second, I'm curious about your implementation of the Scrum Board or whatever one would call it. You said that you went away from the idea because there were too many tasks and thus it took too long for people to write index cards and maintain the board. I can see why this became an issue, and I completely agree with you on the notion of people over processes as the reason for why you abandoned the board concept.

On BL we had an entire room dedicated to the notecards with different scrum teams on different walls or sections there of. We never had a formal scrum of scrums and hence no board dedicated to high level tasks. I'm sure that could work, and as Gabe says there are plenty of potential benefits to a physical manifestation of your schedule, but I never personally found the bother was worth the benefit, at least on a huge team full of individuals with a range of talents and interests.

BTW, I do acknowledge the fact that utilizing agile takes a whole lot of discipline, especially in the early, transitional phases, but the methods are specifically built to help your maintain discipline and not be able to cover up the truth. That's why so many experts always advice to initially introduce whatever practice you want to use (Scrum, XP, Lean etc.) by the book and be very rigid about it, so you don't skip certain parts because it doesn't fit or feel right. Then when you've properly learned the value and importance of each method, you can begin to evaluate how to tweak it to fit your environment, because now you know how it will impact your team and it won't just be for the sake of covering up some parts of the truth.

Totally reasonable advice that is very difficult to practice in the real world, at least the worlds I've lived in. How many game artists (or programmers or designers) do you think are really genuinely interested in adhering rigidly to a book's practices, especially at 10am the day before a milestone and they all resent every minute they are spending moving cards on a board and not actually getting work done on the game? The fundamental problem is that very few people really care about process, any process, and just want a basic structure within which to do their work, which makes strict/religious adherence to anything very difficult.

Of course, Nathan, you are much more experienced than I and I know that the true learning of producing comes from experience, so I'm not trying to disregard you in any way (I'm still young and relatively inexperienced). And it seems like you have exactly the right mindset and approach to production as I hope to employ. But it just feels like you've been a little burned from the early Scrum hype, where the practice is often blamed instead of oversimplifications and fanatic craze. "Hey, heard about this Scrum thing? It's super great and awesome and will make everything better very easily! It's impossible to fail with it." If anything, agile experts are the first to acknowledge that there is no silver bullet, no all-in-one-for-everyone methodology.

I definitely don't think you're dismissing my perspective (or anyone else's). I don't feel burned by scrum/agile or the hype. In fact I like many elements of it, especially the refreshingly honest take on problems inherent in detailed long term planning and the value of iteration in complex software. In fact, I think what you call "true scrum" and what I call "the scrum/hybrid waterfall I like" describe a very similar process. My goal is not to grind an axe, but to help point out mistakes I have seen many teams and companies make, rightly or wrongly, in the name of scrum/agile.

Share this post


Link to post
Share on other sites
I think we both agree on what good process looks like. Whether that process or similar but flawed processes are "true Scrum" or not, is difficult to answer. When a faith's practitioners all make similar mistakes is the flaw in the faith or in the practice? That is not for me to say. In fact, it's not really for anyone to say as there is no central authority to "canonize" the tenants of scrum, just a lot of prophets, err, consultants, espousing generally similar philosophies. I can say that for sure when we started adopting scrum 7 years ago many people only acknowledged long term planning in a token way and that I saw many teams (including our own) lured into a false sense of iterative security. All I want to do is highlight a very common mistake (or misunderstanding) let others do as we did.

First of all, thanks for answering so fast :) I love these kinds of production discussions!

I think my annoyance often lies with the fact that I feel that people criticize how Scrum has been implemented rather than how it has been described by its "founders", such as Schwaber. Of course, you can always argue that how it's implemented show its true nature - yet I feel that it's pointing the blame at a wrong direction. I'm not saying Scrum is perfect, but many of the negative things being said about it is often due to how it was put into practice, where it clearly did not follow many of the guidelines from the books.

It's almost like communism. I mean, I'm a very liberal person, but I can see the idea of communism described as a theory. The problem is we often criticize communism because the countries that state they have practiced this type of government (soviet, cuba, north korea etc.) have all failed to a large degree. Yet, can you honestly say that they actually practised communism? Communism is about sharing goods between everyone, and yet these countries were/are far more related to dictatorships, where few people sit on and control all the goods.

Oh and actually there is a central authority for Scrum. It's called the Scrum Alliance :) Of course, I know where you're getting it and I'm just posting the Scrum Alliance for kicks ;)

On BL we had an entire room dedicated to the notecards with different scrum teams on different walls or sections there of. We never had a formal scrum of scrums and hence no board dedicated to high level tasks. I'm sure that could work, and as Gabe says there are plenty of potential benefits to a physical manifestation of your schedule, but I never personally found the bother was worth the benefit, at least on a huge team full of individuals with a range of talents and interests.

How big was the Scrum teams on BL? Can you give like a short overview of the size of the teams, the amount of teams and how they organized?

Totally reasonable advice that is very difficult to practice in the real world, at least the worlds I've lived in. How many game artists (or programmers or designers) do you think are really genuinely interested in adhering rigidly to a book's practices, especially at 10am the day before a milestone and they all resent every minute they are spending moving cards on a board and not actually getting work done on the game? The fundamental problem is that very few people really care about process, any process, and just want a basic structure within which to do their work, which makes strict/religious adherence to anything very difficult.

Yes, I fear this is where my lack of experience hits me the most, and I totally get what you're saying. I have also found the most difficult part in convincing people to use these methods - especially many people have been burned by previous attempts at introducing glorified methodologies - but I do believe a few tricks can help this process:

1. Set it up as an experiment. Put a time-limit on it, fx 1-3 months, which should be enough to really get it under your skin, and convince people that it's very important to fully deploy the entire methodology for this period of time to fully learn and understand its benefits and disadvantages.

2. Do retrospectives - ideally once a week. While people may not be able to change course or tweak the practices while the experiment is still running, it's a great opportunity to vent frustrations and discuss them; which might often reveal that the underlying issue is either not with the methodology itself or because the metholodogy has been wrongfully implemented.

3. Once the experiment period is over, you are free to tweak the methodology as you see fit. The team will have embraced certain practices and learned to hate or just don't see the necessity of others. It's ok, because now they pick and choose based on real experiences with the real methodology.

Still, I know this is not an easy thing to do. But many experts advice to do experiment phases, and also experiment teams (beachhead teams) where people willing to try this out try it out, and then the learnings and experiences are shared with the rest of the organization, who might start to feel like trying it.

Of course, this is all about change management; a highly fascinating field which I know a little less about. It's so much about psychology and being a charismatic and convincing person. You're no longer just a Producer or Project Manager, you're a Director of sorts.

I definitely don't think you're dismissing my perspective (or anyone else's). I don't feel burned by scrum/agile or the hype. In fact I like many elements of it, especially the refreshingly honest take on problems inherent in detailed long term planning and the value of iteration in complex software. In fact, I think what you call "true scrum" and what I call "the scrum/hybrid waterfall I like" describe a very similar process. My goal is not to grind an axe, but to help point out mistakes I have seen many teams and companies make, rightly or wrongly, in the name of scrum/agile.

And so you should! :)

Anyway, I would love to know a bit more about how you practice it today (or practiced it before), if you wouldn't mind:

- How do you organize the overall product team into smaller sub-teams? If you do that at all?

- What is your workspace like? Who sits next to who and how is it all set up?

- Do you do Sprint Reviews/Iteration Demos, where you showcase the progress (from a user perspective) made in the past Sprint?

- What is your policy and mindset regarding working overtime and crunch?

- How do you ensure that Tim doesn't change too much of the goals during an iteration? And when is it too much?

Thanks Nathan :)

Share this post


Link to post
Share on other sites

How big was the Scrum teams on BL? Can you give like a short overview of the size of the teams, the amount of teams and how they organized?

...

Anyway, I would love to know a bit more about how you practice it today (or practiced it before), if you wouldn't mind:

- How do you organize the overall product team into smaller sub-teams? If you do that at all?

- What is your workspace like? Who sits next to who and how is it all set up?

- Do you do Sprint Reviews/Iteration Demos, where you showcase the progress (from a user perspective) made in the past Sprint?

- What is your policy and mindset regarding working overtime and crunch?

- How do you ensure that Tim doesn't change too much of the goals during an iteration? And when is it too much?

I, too, thiink dev process is fascinating. And also complicated and hard! I'll try to answer your questions as best I can.

Gabe may remember the numbers better than me, but at peak BL was about 5 scrum teams with roughly 10-15 people per scrum team. Standups would start at 10:30 and run every 15 mins or so until noon. Scrum teams would have a central theme like "the open world" or "skirmish (i.e. RTS) battles" or "cars and combat" to try to give some sense of unity, but it was a big game so the goals of each group were usually similarly expansive. Each team might have one or two dozen user stories, with many tasks for a given story. Thankfully, since BL, all of our games that have wanted to do standups have been small enough for a single scrum teams.

The biggest problems that we had with scrum teams were specialization and utilization. Attending multiple standups could be very time consuming/disruptive, especially if yours was first and next-to-last or something so we tried to only have people in one team. But what do you do about the physics expert who is needed in one group for combat and another group for open world driving? Similarly, we want all of our people to work a full time job. What do you do about the animator who's dedicated to combat, but only has 50 hours of tasks in a 4 week sprint? I've never seen a scrum text that deals with either of these topics well. Most seem to assume that everyone is sufficiently general that the former is not an issue and I've never seen the latter addressed at all.

Our studio is essentially open plan. The office is a series of rows with light dividers between rows. Desks are grouped in "pods" of 4 desks with their backs to each other. Rows are 1-3 pods long. We seat people based on who they work with most, which generally now means by team and then by discipline. On BL it was generally by discipline and then related disciplines (i.e. gameplay programming and animation) sat near one another.

We've never been great about formal sprint reviews, though we starting to get better. We did a few during BL, but we also tried to plan the entire sprint in a day (which was a different sort of crazy) so there was never time to cram everything in. Now each team does it differently, with some doing micro reviews weekly, others monthly, others only in an ad-hoc fashion. We've also dabbled with CPI style surveys as a way to collect feedback, with modest success.

Our OT policy is that it should be what people chose to do if they are passionate about an idea and want to take the game from amazing to phenomenal, not a tool to ship the bare minimum. We (almost) never schedule for overtime. On the occassions when there's more work that just has to be done than fits, we make sure that individuals chose how and when that extra work gets done (i.e. stay late vs work lunch vs come in on a saturday vs be extra focused vs etc...) and do everything we can not to be in that situation in the first place. In my experience, most game devs don't mind working hard or working extra, what they do mind is being told exactly when and for how long they need to work.

Managing change during a sprint isn't just about Tim but something we have to be careful about w/ all of our project leads. It's a delicate balance as change/iteration/exploration is part of the daily work of game design and we can't cut that out. In practice, it's something that our producers spend a lot of managing in a very daily way. Usually we're open to change within the scope of a user story or even scrum team, but cautious about any significant increases in scope or changes of fundamental direction during a sprint.

Share this post


Link to post
Share on other sites
Gabe may remember the numbers better than me, but at peak BL was about 5 scrum teams with roughly 10-15 people per scrum team. Standups would start at 10:30 and run every 15 mins or so until noon. Scrum teams would have a central theme like "the open world" or "skirmish (i.e. RTS) battles" or "cars and combat" to try to give some sense of unity, but it was a big game so the goals of each group were usually similarly expansive. Each team might have one or two dozen user stories, with many tasks for a given story. Thankfully, since BL, all of our games that have wanted to do standups have been small enough for a single scrum teams.

Sounds very reasonable, although it seems like the teams were leaning towards the "too big" category. How did some of the bigger teams fare compare to the smaller ones? Did you sense a pattern there? And why did you decide on that particular team size?

The biggest problems that we had with scrum teams were specialization and utilization. Attending multiple standups could be very time consuming/disruptive, especially if yours was first and next-to-last or something so we tried to only have people in one team. But what do you do about the physics expert who is needed in one group for combat and another group for open world driving? Similarly, we want all of our people to work a full time job. What do you do about the animator who's dedicated to combat, but only has 50 hours of tasks in a 4 week sprint? I've never seen a scrum text that deals with either of these topics well. Most seem to assume that everyone is sufficiently general that the former is not an issue and I've never seen the latter addressed at all.

I think you're getting into a very soft spot for Scrum right there: scalability. At least, that's one of the primary complications I've heard when introducing the methodology in larger organizations. There are some texts on the issue here and there, but to be honest, I'm not yet completely convinced by them.

I was at a talk at Nordic Game (an annual game conference in Sweden), where a guy from Remedy, Jay Ranki, did a great talk on agile, which he introduced at the company. It had a lot of useful information on scaling and making the information flow between different units and key people work.

A few tidbits:

- During Alan Wake, they had 6 week internal milestones, where the leads got together to plan what was the goal for this next period. It also included 6 weeks further ahead in lower detail, and then 6 weeks more ahead in even lower detail. This equally a 18 week lookahead plan. This was somewhat useful, but was not good enough for long-term planning.

- After some time, they did the "full jump", where they got Clinton Keith onboard to make sure everyone understand the terminology and show that this was a strong company effort - not just a fun excercise without real commitment.

- He talked about 3 pillars of scaling:

-- One person (at least) in every team understands the vision on an intimate level, reports directly to the Creative Director

-- Each team has its own Scrum Master, reporting to the main Scrum Master

-- Each team has its own Product Owner, ready to answer the questions and prioritize, reporting to the main Product Owner

-- The Creative Director, Main Scrum Master and Main Product Owner must then talk very closely together and keep everything tied up

- They did the following with the Backlog:

-- It had 3 layers: Project (big chunks), Milestone (medium chunks, a bit more meaningful) and Sprint (small chunks, possible to properly estimate)

-- They used a combination of tools: Hansoft and Excel for the Project and Milestone level, and Post-it's and Scrum boards for the Sprint level (they did not prefer software for this level, because it lacks the tactile feel and doesn't serve as much as a reminder)

- They used the following 4 planning layers:

-- Project Plan: these are defined in the publisher contract and is essentially the plan for every milestone ahead. This could be changed based on the learnings from the pre-production.

-- Phase Plan: Here they use the pre-production to investigate the contract milestones. Then every team lead prepares a plan with the team, and when all leads present it, the differences will reveal the biggest risks.

-- Milestone Plan: Here all individual Sprints are planned one sprint ahead, in negotiation with the publisher. They also have experience with the publisher being more tolerant towards change when they get early warnings that milestone goals will not be reached

-- Sprint Plan: Well, yeah, this is the standard internal Sprint planning.

- They current use 3-week Sprints. Have tried 2-week before and will probably try 4-week at some point.

-- In the first Sprint of a milestone, they do a LOT of planning (the entire first day is set aside for this). Next day is followed by a team kick-off, and it's also the final time to do a retrospective (based on the past Sprint). By the end of the first Sprint, the leads will do the planning a bit earlier than usual, and the last day is a show-and-tell (essentially a Sprint review).

-- In the last Sprint of the milestone, planning, retrospective and kick-off is all on the first day, since there's less to plan. Already midway during the sprint, the leads will start with the next milestone planning. It still ends with a show-and-tell

So yeah, they are essentially using milestones as the link between the traditional development phases and the Scrum practices, bridging the gap that Scrum doesn't talk about that much. Hope you found it useful :)

Our studio is essentially open plan. The office is a series of rows with light dividers between rows. Desks are grouped in "pods" of 4 desks with their backs to each other. Rows are 1-3 pods long. We seat people based on who they work with most, which generally now means by team and then by discipline. On BL it was generally by discipline and then related disciplines (i.e. gameplay programming and animation) sat near one another.

Sounds very neat!

We've never been great about formal sprint reviews, though we starting to get better. We did a few during BL, but we also tried to plan the entire sprint in a day (which was a different sort of crazy) so there was never time to cram everything in. Now each team does it differently, with some doing micro reviews weekly, others monthly, others only in an ad-hoc fashion. We've also dabbled with CPI style surveys as a way to collect feedback, with modest success.

I think you should really try to look into this more! I know it's hard to do, but it's great for two reasons: it does wonders for motivation/team-spirit, and it forces you (in a good way) to get your shit done by Sprint deadline in a REAL workable state. I mean, you're presenting this for everyone and for everyone to try, so there's not room to hide behind a "well, it's sorta finished, but I can't really show it". No, what you see and are able to try is the actual, real state of the game.

I think you should also look into doing this on a studio-scale, so all teams share the same review/show-and-tell date (or maybe do that one less often, for example once a month or every milestone), which can be a great knowledge sharing tool.

Our OT policy is that it should be what people chose to do if they are passionate about an idea and want to take the game from amazing to phenomenal, not a tool to ship the bare minimum. We (almost) never schedule for overtime. On the occassions when there's more work that just has to be done than fits, we make sure that individuals chose how and when that extra work gets done (i.e. stay late vs work lunch vs come in on a saturday vs be extra focused vs etc...) and do everything we can not to be in that situation in the first place. In my experience, most game devs don't mind working hard or working extra, what they do mind is being told exactly when and for how long they need to work.

Completely agree with you on that one! :) Have you ever had the situation where you've had to tell someone to go home, because you could see it impacted his/her performance, quality of work and/or personal life? And if so, how did you tackle that?

Managing change during a sprint isn't just about Tim but something we have to be careful about w/ all of our project leads. It's a delicate balance as change/iteration/exploration is part of the daily work of game design and we can't cut that out. In practice, it's something that our producers spend a lot of managing in a very daily way. Usually we're open to change within the scope of a user story or even scrum team, but cautious about any significant increases in scope or changes of fundamental direction during a sprint.

Makes sense. So basically, tweaks that do not prolong the story/tasks or change the core of it? Sounds very solid.

Share this post


Link to post
Share on other sites

A few quick things:

BL teams absolutely were at risk of being too large. But smaller teams would mean more teams and then more standups and more folks in multiple standups. It's one of the reasons I very much agree that scrum's core issue is scalability. It feels to me like a process designed originally for a small team of programmers working on a website or simple business app, people with pretty homogeneous perspectives/skillsets working on a software that's primarily functional, not narrative or entertaining. Dealing with large groups (> 10 or so), heterogeneous groups (i.e. programmers vs artists), problems complex enough to require deep specialization (i.e. PS3 programming), or products that are not easily disassociated (i.e. single player game narrative), are all sort of left as an "exercise for the reader". Even the best advice I've heard for solving those problems don't strike me as particularly elegant solutions.

Great recap of the Remedy process. I can totally see something like that working. I think the most critical point is something you touched on earlier - the key to success is having someone really committed to making the process work, in detail, every day of the week. Interestingly, Alan Wake was way over budget and late, MSFT even stopped paying for it for a while and Remedy was forced to self-fund until they had a more promising project to show.

Regarding the review, I totally agree on the value of being forced to show your work. On BL, we had a HOF (hour of fun) the Wednesday before the sprint end where the company was forced to play the game for an hour and then we met and discussed afterwards. It had the very powerful motivational effect you describe and also helped us identify critical items to fix before sending the build to the publisher. What we've been less good about is *process* review where we regularly review not just the product but the process that created it.

Since BL we do company wide HOFs, sometimes the very desirable HOF-brau (HOF w/ beer :). With multiple projects you can't really force everyone into a rigid, synchronized schedule but we do them as often as we can. This idea of studio uniform process is another challenge in general and w/ scrum specifically. Uniformity has many advantages, but it's hard to find a process that maps equally well to teams of varying sizes, scopes, budgets, financing models, etc.

Regarding sprint length, it's another fascinating topic. We experimented with all sorts of stuff but I personally really like 5 week sprints. 5 weeks feels like enough time to really get in the flow and tackle meaty features without having so much time you lose the sense of urgency. 6 weeks definitely makes slacking (a bit!) easy and 2-3 week sprints feel like trying to run a marathon composed of 100 meter dashes, just too intense and too much replanning/disruption. I also really love having the milestone/sprint end on a Wednesday rather than a Friday as it's the best insulation against weekend work (or working late on a Friday night) I've seen.

I've never sent anyone home because the late hours are hurting their productivity, no, though I probably should have sent myself home on many occasions. I feel like when someone is working late sending them home just treats the symptom, but doesn't cure the disease. I think it's much more important to understand why they are working late so often. Do they just really love it? Or do they have focus/time management issues? Is the team's work equitably distributed? Is the team excessively specialized? Etc.

And now I need to do some work of my own!

Share this post


Link to post
Share on other sites

Thanks for all the replies Nathan and for taking your time to do this. When your goal is to become a great Producer, you never really can't get enough of listening to other people's experiences, can you? Thanks for the discussion so far! :)

BTW, what is your role at DF at the moment? I assume you're not working on DFA?

Share this post


Link to post
Share on other sites
Thanks for all the replies Nathan and for taking your time to do this. When your goal is to become a great Producer, you never really can't get enough of listening to other people's experiences, can you? Thanks for the discussion so far! :)

BTW, what is your role at DF at the moment? I assume you're not working on DFA?

I'm the Studio Technical Director, so my main focus is tech (programmers, architecture, strategy, etc). Before I took this job I was the PL on OUAM and the Lead Programmer on BL, during which I was pretty deeply involved in a lot of our initial adoption of scrum.

Share this post


Link to post
Share on other sites

Whoa, .. You guys coud make school textbooks and learning materials.

Lots of usefull information here.

Share this post


Link to post
Share on other sites
10-15 minutes stand up meetings are taking over the world. I am currently implementing them in a healthcare setting.

I also believe it's one of the best methods to come out of the agile movement. It's quick and relatively simple to do, can be used in most settings, and adds a lot of value.

Although I know of many who don't do it right, using them as a report tool or not maintaining enough discipline in the standup (people starting to discuss details, making the length go way beyond 10-15 minutes). So... it's a simple method, but it can misused easily.

Share this post


Link to post
Share on other sites

I work at a Japanese company where stand-up meetings have been the norm since time immemorial. It's great to see the practice catching on, as I've always felt it's an effective way to get people interacting and on the same page right at the start of the day.

Share this post


Link to post
Share on other sites

Thanks for taking the time Greg! I didn't understand a word of it, but I appreciate the complexity. Basically, your job is to liaise between the different departments, and make sure everyone is working toward a singular goal... ? Am I even close?

Share this post


Link to post
Share on other sites

I'm sorry if this was answered before, but I have to ask something.

How do you control, at the middle or end of a project, planned resources vs. used resources, to figure out if you've spent more money than you should have? And maybe see who is being more efficient or if estimates were well done.

Maybe that was something for Update 4. If so, sorry for jumping ahead.

Share this post


Link to post
Share on other sites

Hi,

Thanks for all the information !

I've read a thing or two about Scrum, and I wished to know how it is being applied in game companies, so the original article and the Q&A so far have been very helpful to me.

There are some things that I would like to ask, though:

- In your scrum tool, I see tasks are being allocated in hours. Do the people responsible for the tasks allocate these numbers themselves, and

how many hours do you consider as one day ?

- When you enter the scrum phase after doing high-level planning, do you split each scrum into several phases, like design, implementation, test phases, etc ? Or do you have a scrum dedicated to each ? Or is it pretty random ?

- Is it possible to share a videotape showing how you are doing stand-ups ? It sounds like having a meeting consisting of around 15 people, talking about what they did and what they are doing, in less than 15 mins is too short, and it makes me wonder how you are doing it. Do people talk very fast? Are people listening to what the other guy is saying ?

- This last question is about the game engine. Maybe it's more appropriate to ask this in the programming area, but anyway, how are you guys iterating on the game engine development ?

Cheers

Share this post


Link to post
Share on other sites

invaluable information and insight from the comments on this thread, getting way more than i expected !!!

Share this post


Link to post
Share on other sites
Sign in to follow this  

×
×
  • Create New...