• Announcements

    • Spaff

      These Forums are closing!   10/04/2019

      After more than a decade of serving this community well, these forums have finally run their course and it's time to close them down. That doesn't mean we want to close the doors on our community, quite the opposite!
      Our discord server grows ever busier by the day, and we encourage all Double Fine fans to meet us over there www.discord.gg/doublefine In a short time these forums will become a read only archive and will remain that way until they become needed again.
      You never know, it might happen.  There is... a prophecy. Thank you all for being part of these forums, and remember that the fun is definitely not over - so please join us on Discord! Love ya, Spaff, Tim, Info Cow, and all of Double Fine.
Sign in to follow this  
Malena

Dialog Pipeline (or, how we manage every bit of dialog Tim writes)

Recommended Posts

Since Double Fine is known for witty dialog, we need to have a dialog pipeline that allows for easy integration and fast iteration. We want to give Tim as much time as possible to write, (and re-write, as necessary), until we have to lock the script. For us, that means finding ways to organize the data and be able to manipulate it without causing unnecessary work for everyone on the team. The goal is to implement the bulk of the dialog once, with only minor hand editing that could come up later, for such things as adjusting the timing or adding a new line for gameplay clarification.

Our main way of organizing the dialog is via a dialog database (DDB). We're on our 3rd iteration and it is an indispensable tool used by all members of the team. Gameplay programmers use it to integrate dialog trees and other interactive bits of dialog, animators use it for cutscenes, materials artists get translations to paint into textures, system programmers use it for any system error messages, TCRS, etc. Scripts exported from the DDB are sent to translators and later re-imported into the DDB; they are also used by actors in the recording studio.

Some of its features allow you to:

- generate a unique line ID (“linecode”) for each piece of text/dialog that represents the line regardless of the language it is eventually translated into (used for implementing the line into the game)

- store every piece of information about a line that is necessary for its implementation: the text, the translation, the voice direction, the animation gesture associated with it, etc

- drag & reorder lines of dialog (either within the same scene, or into a different one)

- apply audio templates (set parameters such as 2D or 3D, ducking, adjust reverb, etc)

- generate string tables, which external tools and the game engine use to gather the necessary information about the line

- generate text-to-speech audio for stubbing into the game before scratch or professional voice becomes available

- track recording status (not recorded, text-to-speech, scratch, professional, final)

- track localization status

- generate trophy & achievements files for consoles and Steam

- export scripted cutscene lua code

- export files used by FMOD to build our voice banks

- track which lines need auto-generated lip-sync

- track dependencies between a base-language line and its translation, or between a line and its recorded version

- track changes to lines, along with the the user who made them

- can be used by multiple people simultaneously

The DDB organizes lines into a 4-level hierarchy. For historical reasons, these levels are called Story, Chapter, Mission, and Scene. Lines can move from one Scene to another, Scenes can move from one Mission to another, Missions can move from one Chapter to another, and so on. This gives us the flexibility to group the dialog into chunks that make sense for us to work with. For Broken Age, each geographical location is a Chapter (e.g. Sugar Bunting), with sub-locations grouped underneath (e.g. Vella's House, Maidens Feast, etc).

This screenshot shows how Sugar Bunting is organized in the DDB:

DDB-main-window.jpg

For Broken Age, when Tim is ready to share what he's written, he does it in the form of a shared Google doc. This way, everyone on the team has a chance to comment on it, ask questions, and make notes about what tech, art, or animation is needed in order to pull off what is written in the script. The dialog script is a great blueprint for identifying the scope and complexity of the game.

If possible, I like to give the team a day or so to read through the Google doc and comment on it. Sometimes they point out continuity errors or ask for clarification about something which results in Tim editing the script. After that, I start the process of importing it into our DDB, where each line of dialog or on-screen text is given a unique ID (we call it a linecode), and breaking it up into separate scenes. In Broken Age, cutscenes, chatter, dialog trees, etc. all get their own scenes, grouped together by level or location. “Stage directions” are also included for context, as well as any notes to programmers about implementation (usually in the form of "pick one" or "play all, then repeat the last one"). “Voice direction” may also be added to individual lines, as notes to the voice director, actor, or animator.

Once the text is all in the DDB, we get to work on implementing it. Dialog trees are a big part of Broken Age and the DDB is able to export lua code for each dialog tree, making its initial implementation quite simple. That, along with the ability to create text-to-speech (TTS) wav files, means our gameplay programmers can quickly evaluate the logic and timing of each dialog tree, scripted cutscene, etc within the game. Later we'll record scratch VO using people on the team to give it more character before doing the final professional VO recording sessions. Once final VO is in, our animators will finalize the camera, adjust timing, and add animations via a library of animation files.

Since each line of dialog is implemented as a linecode (as opposed to hard-coded text), we have more flexibility with edits and changes. While there will always be some hand editing to adjust timing, or add or remove lines from a scene, oftentimes a change can be made without it impacting anyone on the team. Likewise, the linecodes allow us to drop in updated assets easily. Scratch VO overwrites TTS, professionally recorded wavs overwrite the scratch ones, and re-written/re-recorded lines overwrite previous revisions.

The recorded, edited wav files (whether TTS, scratch, or professionally recorded) get scanned by the DDB to ensure that the files are named appropriately and match a linecode in the DDB. Any errors are listed in a log file. Once any errors are fixed, we quality control (“QC”) the lines to make sure the audio matches the text of the linecode (if professionally recorded, we'll update the text if there were natural line changes that occurred in the studio). When finished processing all the wav files, the DDB is updated to reflect the current recording status of each affected linecode. The initial scanning process also generates the data files we use to build our voice banks; once we're done with a batch of new audio files we can export the data files needed to generate them. This process is repeated as linecodes move through the audio recording pipeline: TTS, scratch, professional VO; each time overwriting the previous revision.

We continue this notion of overwriting when it comes to localized assets, though in this case we're overriding, instead of overwriting, English with the localized version within the game engine itself. In order to avoid issues where missing localized assets (textures, subtitles, or audio) might crash the game or make it unplayable, missing localized assets are filled in with the English version instead. The upside of this is that once localized assets are added to the proper place (the DDB for text and audio, or a directory in Perforce for visual assets), it all just magically works!

Lastly, we never delete anything from our DDB. Sometimes we'll determine that a line is unnecessary so it will be "deprecated" and removed from the game. It's never deleted outright since we may want to bring it back at a later date, preserving its same linecode.

I feel like I could write a ton about the history of our DDB and go into the minutiae of how it all works, but that would probably be tl;dr. I am happy to answer any specific questions you may have! Thanks for reading!

Share this post


Link to post
Share on other sites

Thanks for this update - indeed a very insightful look at how many layers there are to writing and implementing dialogue.

Share this post


Link to post
Share on other sites

Looks like a very nice tool! I'd love to have that kind of stuff at home...

Does it also have some kind of graphic representation for dialog trees, like a node graph?

Share this post


Link to post
Share on other sites

Notably absent from the list of features is a "shuffle" mode. Just mix up all the lines and see what delights emerge during gameplay.

Share this post


Link to post
Share on other sites

This is the most amazing timing. I've spent most of my day digging around the internets researching methods of handling text localization and tools for making it easier. Just before I go to bed I check my feeds to find a DFA update and it's on the very topic! :)

While I may not need the flexibility of moving text around much it's still a nice window into one way to handle such things. I'd love to hear more; I'm just not sure what questions to ask at this point. :) Either way thanks for the great write up!

Share this post


Link to post
Share on other sites

Amazing stuff, very interesting. Can you let us know for how long you have been developing and implementing this system? Is it an in-house program and can it be licensed?

Share this post


Link to post
Share on other sites

The part about never deleting anything from the DDB makes SO MUCH sense but we've never done this. Great advice! Your localization time must be a huge time saver too! Thanks for the in-depth writeup Malena.

Share this post


Link to post
Share on other sites

Are you copying text over manually or do you have an automatic import/export path for google docs?

Btw, I found the translate function in google spreadsheets quite useful to generate placeholder translations. It is not good enough for release, but it gives a good idea about missing characters in a font or text spacing.

Share this post


Link to post
Share on other sites

Thank you for this interesting peek into your pipeline.

More would definitely not be tl;dr at least for me. As a fellow game dev I'd be really interested in knowing how everything is done and what sort of files the system generates and everything.

Share this post


Link to post
Share on other sites
Looks like a very nice tool! I'd love to have that kind of stuff at home...

Does it also have some kind of graphic representation for dialog trees, like a node graph?

No. But that is an awesome idea! :)

Share this post


Link to post
Share on other sites
This is the most amazing timing. I've spent most of my day digging around the internets researching methods of handling text localization and tools for making it easier. Just before I go to bed I check my feeds to find a DFA update and it's on the very topic! :)

Awesome! Glad the timing was fortuitous!

Share this post


Link to post
Share on other sites
Amazing stuff, very interesting. Can you let us know for how long you have been developing and implementing this system?

Our first DDB dates back to the days of Psychonauts. I joined Double Fine about a year before Psychonauts shipped (holy smokes, I've been here for almost 10 years!) There was no organization of the script or wav files and my first task was to get ready to do a big recording session for Psychonauts. It was crazy how disorganized things were. So the first DDB was very quickly put together by one of the programmers on the team. It wasn't feature-rich since we didn't have the luxury of time to develop it, but it was enough to get the project done. When we were figuring out what tools we'd need for Brütal Legend, we knew we wanted to overhaul our dialog pipeline and that meant expanding the DDB and its features. Much of what we rely on today dates back to 2006-2008. We worked with a contractor to build that DDB and we used it for all projects until sometime around 2012 when one of our internal programmers, Paul Du Bois, rewrote the whole thing. That's what we're using now. So I guess it's been in development/refinement for about 10 years!

Is it an in-house program and can it be licensed?

It is proprietary software and not really set up for licensing. Sorry!

Share this post


Link to post
Share on other sites
Are you copying text over manually or do you have an automatic import/export path for google docs?

We have an importer, but I usually have to futz with things a little bit before importing. I don't mind this at all -- for me, the most important thing is that Tim writes. He shouldn't have to think about formatting his writing, his mind should stay in the creative realm. Plus, it's a good time for me to try and catch typos.

Btw, I found the translate function in google spreadsheets quite useful to generate placeholder translations. It is not good enough for release, but it gives a good idea about missing characters in a font or text spacing.

I'm totally going to try this out! We've usually used leet as a language option to test for hardcoded strings but it's not so good for testing out localized strings that may be longer than English. We do have a way to verify via a debug command that all the characters needed for each language are in the font.

Share this post


Link to post
Share on other sites

This one was interesting to read, thanks.

For some strange reason i'm curious to know what kind of editor Schafer is using for writing text. Some ordinary word processing software, a so called distraction free creative juices empowering specialised text editor in fullscreen mode, some lowres amber coloured hex editor, painting everything with a flithy mouse in DeluxePaint and driving everyone crazy by saving it to tons of different image formats, ...?

Share this post


Link to post
Share on other sites

Very interesting read, its cool to see how the pipeline works on a game like this with so much dialogue. For anyone that is interested there is a great program called Chat Mapper that can be used to write dialogue trees for games http://www.chatmapper.com/. I've been using it for a puzzle adventure game I'm making and I find it awesome to use. Its really easy to customize and the information can be exported as an XML. It has become a big part of the pipeline of our game. We use it to trigger character animations and audio as well as handling the dialogue trees. I think it was even used for the new Leisure Suit Larry game.

Share this post


Link to post
Share on other sites

What's TCRS? What's leet?

Looks like a very nice tool! I'd love to have that kind of stuff at home...
What would you do with it at home?

Share this post


Link to post
Share on other sites

Thanks for the in-depth look at your dialogue pipeline! I'm an assistant editor at an animation studio, and had actually been curious as to how Double Fine records and processes dialogue. It's a big part of what we do as assistants, and we're always looking for ways to make it better.

Once any errors are fixed, we quality control (“QC”) the lines to make sure the audio matches the text of the linecode (if professionally recorded, we’ll update the text if there were natural line changes that occurred in the studio).

When you do this, do you give each variation of the line that was made in the studio a new linecode, Or are they put under the original linecode? Also, when you receive audio from the recording studio, do you receive one large file that your audio team has to chop up into clips, or is it already broken out by line for you? Is there any process of "marking" line reads (multiple reads of a single linecode), and if not, how are the various reads for each linecode auditioned? Are several reads sometimes edited together to make one "new" read for use in the game?

I apologize for the plethora of questions; I'm just very curious. :)

Thanks a lot!

- David

Share this post


Link to post
Share on other sites
Thanks for reading!

No, thanks to you Malena. Great reading! :)

Share this post


Link to post
Share on other sites
I should have read this before Amnesia Fortnight! :)

I should have read this before March!

Share this post


Link to post
Share on other sites

Hey there!

OK, phew, AF has been keeping me very, very busy, but I thought I'd take a moment to respond to some more questions...

Once any errors are fixed, we quality control (“QC”) the lines to make sure the audio matches the text of the linecode (if professionally recorded, we’ll update the text if there were natural line changes that occurred in the studio).

When you do this, do you give each variation of the line that was made in the studio a new linecode, Or are they put under the original linecode? Also, when you receive audio from the recording studio, do you receive one large file that your audio team has to chop up into clips, or is it already broken out by line for you? Is there any process of "marking" line reads (multiple reads of a single linecode), and if not, how are the various reads for each linecode auditioned? Are several reads sometimes edited together to make one "new" read for use in the game?

Lots of good questions here! Khris Brown has done a great job of posting about what happens in the studio with the talent. She is amazing at her job and I love working with her! She gets the script from me and keeps track of which takes she wants us to use as the actors do their line reads. When we get the raw session files back from the studio, she sends us her marked up script, which is her EDL (Editor Decision List). We edit the files and use only the ones she specifies, so we don't assign new linecodes for unused takes. If Tim wants to use multiple variations (like, Gus's underwear line), we will still assign new linecodes instead of grouping them under the original one. We do this because we want to be able to display accurate subtitles accounting for variation in the reads. (On Psychonauts we could have one linecode trigger any number of wav files; however, we weren't able to display accurate subtitles if there were variations in the line read, which is important not only for subtitling, but for localization as well.)

We do keep all the raw session files, so if Tim doesn't like a particular read, we are able to go back and edit all alts and let him choose. Once he's chosen which one he wants, we give that wav file the linecode filename and it overwrites the previous version in Perforce. This allows us to easily drop in updated assets without requiring a programmer to change it in code.

What's TCRS? What's leet?

TCRs are Microsoft's Technical Certification Requirements, (Sony's are called TRC - Technical Requirement Checklist). This is all that system error stuff you see. It's things like making sure you have a TM or copyright symbol in the right place, as well as using the correct terminology. We can't release a game if we don't comply with TCRs/TRCs.

Leet is this! http://en.wikipedia.org/wiki/Leet[url=http://en.wikipedia.org/wiki/Leet][/url]

A lot of people thought that we had leet in Psychonauts as an Easter Egg. I suppose it can be viewed that way, but it was vital to us in our quest to root out all instances of hardcoded text. I joined Double Fine about a year before Psychonauts shipped and it was full of hardcoded text strings... I am pretty sure it was Paul Du Bois who thought of implementing leet as a language. That guy is always full of genius solutions!

For some strange reason i'm curious to know what kind of editor Schafer is using for writing text. Some ordinary word processing software, a so called distraction free creative juices empowering specialised text editor in fullscreen mode, some lowres amber coloured hex editor, painting everything with a flithy mouse in DeluxePaint and driving everyone crazy by saving it to tons of different image formats, ...?

I think Tim starts writing in his notebook, then switches to something like Word as things come together. Personally, I don't feel the need to intrude on that part of the pipeline -- he should write in a manner that is most comfortable for him. No piece of software should restrict the creative process. It's for this reason that we don't employ any screenwriting software (which could make the DDB import more fluid). But shhh! Don't give him the idea to do everything in DeluxePaint! :) Gah!

Share this post


Link to post
Share on other sites
Sign in to follow this