CMS.339 / CMS.839
Virtual Reality and Immersive Media Production
MORE ABOUT YOU
Pioneers, pirates and makers
Hacking XR is all about pushing the boundaries of immersive media production. We believe in fostering a creative and collaborative experience. Students from various disciplines are encouraged to enroll. We’re looking for a diversity of skills and backgrounds – coding, game design, programming experience but also storytelling and creative wildcards – to build complementary teams that will produce innovative and original VR experiences. Tell us more about yourself - fill in the questionnaire!
MORE ABOUT THE COURSE
Mens et manus ... mind and hand ... VR and Immersive Media Production will engage both. What better way to explore a new technology than to push it to its limits, re-imagine it, and to rebuild it in unexpected ways?
Offered as part of a relationship between MIT's Comparative Media Studies / Writing and in partnership with the Oculus NextGen Program and Unity, Hacking XR will enable students to experiment with VR/AR technologies and current developments - including Oculus Rift, touch controllers, 360 cameras and new this year, Mixed Reality headsets. The course will survey historical developments and present day innovation in virtual reality and immersive media - gear, software, and emerging storytelling techniques, languages and approaches. It will cover essential steps in immersive media production - from conception, storyboard, UX/UI and human-centered design, sound and story.
Finally, the class invites leading makers of the industry, to explore with them the fundamentals of immersive creation, and with luck, develop innovative ways to build immersive experiences. Join us!
Register for 12 units. HASS-A
Meets Mondays 1:30-4:30 pm
MEET THE INSTRUCTIONAL TEAM
Sandra Rodriguez, PhD., is a creative director (interactive/VR/MR) and a sociologist of new media technology. A Lecturer and a Visiting Scholar at MIT CMS, she is a Fellow of the MIT OpenDocLab, and currently Heads the Creative Reality Lab at EyeSteelFilm, and Emmy Awarded company based in Montreal, where she explores the future of immersive documentary in VR/AR/AI.
hours - Mon. 4:30-6:30
Annie is a first-year Masters student in the Comparative Media Studies/Writing department. She works in the Education Arcade and previously worked on a microbiology-based educational VR game at MIT. She is interested in the affordances of VR and the usage of new media technology in K-12 and public education.
hours - Thu. 1:30-4:30
IS PREVIOUS EXPERIENCE WITH VIRTUAL REALITY REQUIRED FOR THIS COURSE?
No, the course will move quickly from the basics of understanding VR production and workflow, and will encourage participants to go as far as possible during the semester. Projects will be completed by teams of 3 students, with each team having members with programming, design, and creative skills, so experience in any of these areas is far more important.
HOW DO I APPLY FOR THE COURSE?
This is the first course of its kind at MIT. It was the 1st course offered at MIT on Immersive media and VR, and launched the beginning of new curriculum focused on VR/AR offered in today's MIT curriculum. You can apply for the class like you would any CMS class - see here
IS THIS A CRITICAL COURSE OR A HANDS-ON PRODUCTION COURSE?
Both! The course will use hands on experimentation to push up against the hype surrounding VR, but also AR and MR. It will use critical and historical approaches to interrogate some of the key default assumptions about VR, while probing to find unexplored alternatives and under-considered consequences. What can we learn from current immersive media projects, and can VR/AR/XR enable new kinds of stories to be told? What kind of cognitive processes underlie immersion? Are there particular ethical considerations that this medium requires? What can touch controllers add to stories and games? Theory and practice will work together in the course, and the two will hopefully inform the semester-long development of VR/AR and MR projects.
DOES THE COURSE FULFILL ANY HASS ELECTIVES?
The course will count as a 12-unit general HASS-A.
WHAT IS THE COURSE’S RELATIONSHIP TO OCULUS OR OTHER COMPANIES?
Thanks to a special relationship with MIT's CMS/W and the Open Documentary Lab and the Oculus NextGen Program, companies like Oculus, Samsung, Unity and HP provide equipment and services to support the course. MIT is one of 20 campuses working with Oculus NextGen. But the class is technology agnostic - we also have access to Microsoft MR headsets, HTC and new technology soon arriving. Our job is to push these technologies and their storytelling capacities and to come up with fresh ideas. All intellectual property produced in the course will belong to the students who produce it, as per standard MIT policy.
WHAT IS YOUR POLICY ON AUDITING SPECIFIC CLASSES WHILE NOT ENROLLED IN THE COURSE?
The course focuses on team production of VR or AR/MR experiences. Class attendance by all members of a team is required to push the projects gradually, and will be evaluated as part of participation. Auditing will not be permitted in regular classes. However, the course will have multiple guest-lectures and special workshops which will be opened to wider audiences, held the next day (Tuesday) at lunch. Stay tuned by following our blog and the HackingXR Speaker Series here above!
HOW IS THIS CLASS PART OF AN OCULUS NEXT GEN NETWORK?
Beyond the course, there will be several ways to communicate and network with other Oculus NextGen participants in universities across the US :
A Facebook group to which you will be invited to register, contribute, post, share, comment, etc
Regular skype/in-person lectures and check-ins from Oculus chosen picks, artists and staff
If you have unanswered questions, please contact the instructor. Also, if you'd like to join us but aren't registered in the class, do note that we have a public and open HackingXR Speaker Series, featuring some of the class special guests (organized in collaboration with the MIT Open Documentary Lab and the MIT Center for Arts, Science and Technology).
BUILDING IMMERSIVE WORLDS
MARCH 4, 2019
Building worlds - from pros to beginners, you’ll need to create worlds. How does one create worlds that are simultaneously compelling, immersive and comfortable for a user? What about interaction design? Pacing? Music and tone setting? What about aesthetics and narrative approaches? A full walk-through of the Award-Winning project Spheres, a 45min 3 part interactive VR series that lets you discover the sounds of space. Winner for best Immersive Experience (Venice Mostra del Cinema 2018).
PRODUCTION 1: MAKE IT HAPPEN!
Image capture, prototyping
MARCH 11, 2019
There’s never a time like now to start your projects. Will you choose 360 live capture techniques? Real time capture systems such as laser scanning (LIDAR), photogrammetry, Kinekt can also offer significant possibilities. Hands-on session on examples of production and post-production methods. Feedback and team learning exchangeds in VR and AR - hackathon style!
PRODUCTION 2- The Evolution of a Headset
MARCH 18, 2019
Creating worlds also means making sure the experience is enjoyable and comfortable for your audience. How does it translate to new platforms and hardware? What's coming down the pike in 360 and MR developments? Varjo is a new kid on the block, promising changes in hardware and software that will bring big step is Bionic Display™, letting us see VR in human-eye resolution.
MEMORIES FROM THE
FUTURE OF IMMERSION
APRIL 1, 2019
What does it mean to be immersed? Pioneering a new medium sometimes means being able to envision a future, before it is even built. This Master Class will be offered by no other than true futurist and visual effects pioneer, Doug Trumbull. We owe him the special effects of 2001: A Space Odyssey, Close Encounters of the Third Kind, Star Trek: The Motion Picture, and Blade Runner, Brain Storm, all of which earned him multiple Academy Award nominations for Best Visual Effects. From the past, we discover the future in visual effects using virtual digital sets and electronic cinematography.
APRIL 8, 2019
Rethinking the rules of immersive storytelling. What else could we create? 3D printing, painting and sculpting in VR. What about scanning the real? What about touch? Smell? Taste? What can we learn from filmmakers, game designers, magicians and architects? What if you could see through the eyes of an insect? What if you could be a tree? What if you feel, touch, smell in immersive experiences?
IMMERSIVE APPLICATIONS: EXTENDING REALITY
FUTURE APPLICATIONS: LOCATION BASED AND MULTI-USER XR
APRIL 22, 2019
Game culture and design have long been thinking about user experience, interaction and emergent narratives. How can interactive UX/UI learn from theatre, touring shows nd gameplay? Exploring future VR/AR/XR applications currently trending, with LOCATION-BASED VR (LBVR), multi-user and multi sensory strategies, with a special visit from the DV group team (Alice, Jack, and The Horrifically Real Virtual Reality).
APRIL 29, 2019
Is VR just a tool for entertainment? How does interaction, innovation and magic converge to create original works? Recent trends have been experimenting with AI to create seemingness interactions with virtual characters. The recent work Wolves in the Walls, has been heralded as a hallmark moment for immersive and interactive storytelling, exploring how to emotionally bond with the audience through interaction, AI innovation, and a little magic.
MAY 6, 2019
VR is still in its infancy, but it’s taking flight quickly. How do we extend other human senses? This year has been a game changer in appearance and resurgence of haptic devices, and exploration of new senses in immersive experience. How do we bring touch into VR? What about cold? Heat? and sensory details of skin perception?
IMMERSIVE APPLICATIONS: VR MEETS AI
FUTURE APPLICATIONS - touch, haptics and exploring other senses
INTRODUCTION TO IMMERSION
FEBRUARY 11, 2019
Introduction to course mechanics, definitions and the distinctions among various VR/AR/MR systems. This session will include a historical overview of immersive media; lessons learned from earlier media hypes; and an assessment of the current ecosystem. We will get acquainted with the lab, Oculus headsets and controllers, computers and schedules; explain assignments and the Kino method.
CREATING in XR - some guidelines
FEBRUARY 19, 2019 (TUESDAY!!)
How does one construct a compelling VR/AR or immersive media experience? What makes it "good"? What makes it stand out? We’ll explore some of the fundamental mechanics (visual, sensory, depth, sound and story!) that allow or impede presence and immersion. From the “Swayze effect” to simulator sickness, interactive or passive VR, this session offers important learning steps in creating immersive content - with a visit lecture from a true VR pioneer! Get your bases covered!
SO YOU HAVE AN IDEA...From ideation, conception and production
FEBRUARY 25, 2019
Should we create immersive experience that look or that feel real? After taking up issues such as user attention, interaction, and transitions, it is paramount to also think about what makes an immersive experience real, relevant or felt. Thinking about space, time, interaction and sound are important. But what about empathy? Accessibility? Representation? As immersive media expands, diversity is key to create compelling, meaningful and thriving ecosystem of experiences.
FINAL PROJECT PRESENTATIONS AND CRITIQUES
MAY 13, 2019
Projects will be presented and discussed in class. We can't wait to see them!