I dedicate this Podcast Summary to [url=http://i868.photobucket.com/albums/ab249/Hylebosaiden/Shitake_zps98d7eb32.png]Shitake[/url]. Rest eternally, my Manly Lion. Another month, another Podcast. While my initial predictions for the subject matter of this Podcast turned out to be terribly inaccurate, this is still a very solid Podcast that can be enjoyed nicely alongside turkey. The subject this month is User Research and Localization, two important vectors from which Bungie can reach out to the community near and far. Before we begin, please remember that as always this summary is no substitute for the real deal. If you can carve out time to listen to this 51 minute Podcast this break, you should definitely do so, but if you can't find the time or you want to review what was discussed, that is what this summary exists for. Keep in mind that I am Human and I can make errors or portray things incorrectly, so if you spot any feel free to point them out. [b][u]Introduction[/u][/b] 1) DeeJ, Urk, and Halcylon are bringing us the Bungie Podcast from deep within the basement of Bungie in the Foley lab. 2) Time-wise, this podcast was recorded some time shortly after the launch celebration for the PS4. 3) Today’s guests will help us understand how Bungie interacts with their community both near and far. 4) They are: [url=http://halo.bungie.net/News/content.aspx?type=news&cid=32234]Jennifer Ash[/url] from Bungie User Research and [url=http://halo.bungie.net/news/content.aspx?type=news&cid=32214]Tom Slattery[/url] from Localization. [b][u]5:33 - Jennifer Ash[/u][/b] 5) Jennifer Ash is a User Researcher. In her own words, User Researchers look at the game, they look at the data going on behind the gameplay, a lot of it is analyzing what goes on in terms of actual interactions that people have but also in terms of the data going on behind the scenes and analyzing and visualizing that data to better inform game designers. 6) Her undergraduate degree was in Game Design and Psychology which led her to Human Computer Interaction which set her on the path towards User Research. In graduate school she did educational games which gave her a cognitive science background which helped her to understand statistics and view games from different perspectives. 7) A lot of people misunderstand what User Research is and in their minds they blend it together with what they think is focus group testing, what’s the difference? A lot of participants apparently mistake the User Research team for test, they’ll report bugs and the UR team will be like “That’s great… but that’s not what we’re going for today…”, they care more about your experience and how you play the game. Focus groups are more of a marketing angle, which is much more about how people perceive things. 8) There are two major perspectives from which they approach user research: usability and play testing. With Usability testing they pair one user with one researcher and they analyze “What did you do there? Talk out loud as you go through your subject area…”. With play testing, they have a group of people in the lab and they are just playing normally, and they want to know “How would you normally play? What is some gameplay data that we can get from this that would be useful to designers?”, things like balancing items and balancing terrain. 9) Is it more taking in the data and looking at it retrospectively, or more actively listening and actively interacting with the subjects? Usability is more in the moment, a lot of what the researcher is doing is taking notes as the subject plays and writing up a summary afterwards about their experience. With playtesting it’s more of mulling over the data after the fact because they cannot process 18 people at once. There are things they do to help, from recording in game data to recording their faces as they play which can be reviewed after the fact. 10) The Laboratorium has about 18 stations, each one has a computer, dev kits, and one of the stations is set up with an eye tracker which they can use to see what their eyes are looking at while they play the game, which is useful for things like cinematics, user interface, and the heads up display. Every station is wired internally so all data is kept within the walls. The user studies can also be streamed so designers can watch it live, often in the theatre which is directly adjacent to the User Research Labs. 11) There are some user research tools built directly into Grognok (Bungie’s world editing tool) which allows them to view the data in-engine; things like player deaths or kills. Designers can figure out where all the kills are happening, especially in PvP, it can inform design decisions like geometry changes and such. It can also be used to see which abilities are popular in which spaces. It’s like a heat map, but in a real space that you can walk through instead of a drawn image. 12) Is it the User Research team’s responsibility to draw conclusions for the Designers using the data? That responsibility is shared from the beginning to make sure the user research team is answering questions that the designers are interested in, and in the end they try to keep things very data-based instead of having subjective opinions. They try to give the data to the designers and let them draw their own inferences and offering recommendations, but mostly through past experiences. Most of the time the designers have to be on board to understand, they can’t force those sorts of things. 13) Halcylon points out that the User Research team isn’t exclusively working with the game, but also with Bungie.net on the mobile app and web companion. 14) User research can be difficult sometimes because you have to go up to people and tell them with data in hand that their baby is objectively ugly, maybe you should try and change it. 15) It can also be difficult to interact with players sometimes. You need to be careful not to ask leading questions and make things as objective as possible, like if you ask “What did you like?” they’ll focus on the positives when you were hoping to hear more about what they disliked and such. A lot of it involves asking questions in a way that doesn’t suggest an answer, which is especially difficult in usability testing where you have to think on your feet a lot, you have to take a step back and interact very broadly. 16) This entire “coloring of conversations” is one of the reasons they try to discourage research participants from talking to one another, especially when they are filling out surveys. 17) How do they choose people? Basically they do random selection, they have a database with a profile for everyone who has registered for beta testing, and depending on the study that they are running they’ll want different profile types and they’ll invite them to the studio. Being local gives you a better chance of being called into the studio, but ideally anyone worldwide can be given an email survey at any point. They bring in all sorts of players, it all depends on what the designers are interested in. 18) After undergraduate college Jennifer worked at IBM in various divisions. The game industry is very different from the software industry, both in terms of schedule, the variety of things you work with on a daily basis, dress code, working towards a single goal… 19) Previous experience for designers is super useful, but User Research can be super useful for informing them about emergent activities, which are very prevalent when you are doing things that have never been done before. 20) The user research team is a team of four, and they’ve split the game up into various areas that they’ve divided amongst the team to focus on, but they’re all working together in making the experience of the game as good as possible and help each other out when they run into problems. 21) There’s a few amusing anecdotes about various User Research tests. One where a guy didn’t realize that you couldn’t bring a gun into the tests and ran back out to his car to drop it off before proceeding through the metal detector. Another generalized one where users would input their antiquated and embarrassing usernames that they would try to explain away as the researcher and all the designers in the other room are all silently judging them. 22) On the subject of the NDA that all user researchers sign, you’re not allowed to talk about anything. You’re allowed to wear the free t-shirt you’ve been given, you’re allowed to say that you’ve done testing at Bungie, and that’s about it, and that lasts forever. 23) User researchers do get fed, but no beer. DeeJ and friends share an amusing anecdote about how the user research would be more applicable if the subjects were in their natural gaming environment: probably wasted and wearing no pants on a couch. 24) One of the more surprising data-points were the reactions shared social experiences that people had in the playtests. People responded very well to them, from recognizing that the person they were fighting with was someone in the room, to two guardians sharing a silent sunset with one another. 25) Jennifer was one of the Bungie Employees who went to Gamescom, there’s a few amusing anecdotes about that. 26) They’re always looking for more people to join the User Research Program. [url=http://www.bungie.net/en/Questionnaire/UserResearch]GO HERE TO SIGN UP.[/url]