Introduction

For content creators seeking to transition from producing 360° (monoscopic) video to creating more immersive mixed reality experiences with;

  • 3D assets,
  • positional tracking,
  • 360° (steroscopic/3D) video, and
  • interaction,

there are a number of major differences in workflow that can be challenging to understand and plan for. Being unaware of missing workflow processes can drastically impact the overall success of the transition.

Department leaders coming from a purely cinematic background may be new to the programming-side of the industry, and thus, are potentially unaware of all of the pieces that make up a virtual reality (VR) or augmented reality (AR) pipeline. Let’s look at several ways that processes differ and may be improved by preparing for scaling and efficiency early on in the transition or startup stage.

Many startups in this sector do not believe they need to include data capture and mining in their workflow process from the onset. This is a critical mistake. The mixed reality sector is experiencing a boom, so companies need to be prepared to meet with angel investors, venture capital firms and banks. A few reports with valuable metrics can make a substantial difference in outcomes of meetings. Visuals that can be placed in the hands of decision makers and key stakeholders help facilitate keeping up at the pace the industry demands.

Despite publicly available and low-cost tools for collecting data, many small scale companies believe that they may not have the labor, budgets or capability to handle data collection tasks and further data analysis.  Some of these companies may rely solely on verbal exchanges with the client(s) to gauge product success.  This is a wasted opportunity. These are areas that vital to bring into the process and document.

Current Workflow Process

As companies wish to cultivate virtual reality (VR) capability, they must strive  to produce content, at the highest level of immersion possible based on the technology being deployed. In order to handle the ever growing variety of challenging tasks, businesses need to deploy an agile and scalable workflow model  that captures data and delivers valuable insights in the future. Moreover, in today’s economic climate, it is pragmatic to include some type of data mining, exploration and analysis embedded into content creation workflows from the onset of such a venture.

Many startups in this sector do not believe they need to include data capture and mining in their workflow process from the onset. This is a critical mistake. The mixed reality sector is experiencing a boom, so companies need to be prepared to meet with angel investors, venture capital firms and banks. A few reports with valuable metrics can make a substantial difference in outcomes of meetings. Visuals that can be placed in the hands of decision makers and key stakeholders help facilitate keeping up at the pace the industry demands.

Despite publicly available and low-cost tools for collecting data, many small scale companies believe that they may not have the labor, budgets or capability to handle data collection tasks and further data analysis.  Some of these companies may rely solely on verbal exchanges with the client(s) to gauge product success.  This is a wasted opportunity. These are areas that vital to bring into the process and document.

Current Production Flowchart

The diagram above shows the current content creation production workflow of a startup mixed reality studio and production house with multiple issues present. This type of workflow is often present in startup studios who do not have foundations and/or principals with backgrounds in the gaming sector.

The process begins with obtaining any specified client files, setting up the project’s file structure and storing the original client files on the server. Then, if needed, editing and creating pre-visualizations from the client files, capturing video content in the field and/or generating any needed CGI, VFX, titles, sound and other art files. Then the files are composited, rendered and sent to the client(s) for approval. If the client(s) approve files, aside from following up, the project considered concluded. If the client(s) requests additional edits, files are re-delegated to team members and then reenter the pipeline for editing, compositing and rendering. They are then resent to the client(s) for final approval, and so on.

Experienced mixed reality artists, programmers and technical directors may realize there is quite a bit left off of the above flowchart. The missing processes represent the beginning of numerous missed opportunities. As the workflow stands currently, there is;

  • no mechanism for data collection, mining or analysis,
  • no production scheduling/queues,
  • no code or interaction implementation,
  • no game play, bug and/or quality testing.

For media companies seeking to transition from their current film, 360° video and photography workflows into creating mixed reality experiences, being unaware of those missing processes can drastically impact the success of a production. In today’s economic climate, it would also be pragmatic to include some type of data mining, exploration and analysis embedded into the content creation workflow. Small scale companies may not have any mechanism for data collection at all and may rely solely on verbal exchanges with the client(s) to gauge product success. These are areas that vital to bring into the process.

Note: the flowchart also does not show concept development or design, as they are considered to be a part of the separate pre-production process. For similar reasons, the chart does not include processes for sales and marketing and deployment, as these would fall into post-production processes.

Proposed Production Workflow Revision

The diagram on below shows a revised and improved mixed reality content creation production workflow. Notably, the diagram has been divided into segments and introduces a few new processes into the workflow. Also, data capture and analytics have been added.

Revised Production Workflow

The diagram above shows a revised and improved mixed reality content creation studio’s production workflow. Notably, the diagram has been divided into segments and introduces a few new processes into the workflow. Also, data capture and analytics have been added.

Key Features

This lifecycle of mixed reality media content creation still has room for improvement, but now it is much closer to being on the right track. Segments were put in place to denote the distinct phases of the production workflow. The 8 new segments are; Design/Documentation, Principle Photography + Audio Capture, Art & Audio Creation, Conversion, VFX & Credits, Compositing, Code & Interaction, Testing and Final Edits & Client Submission. These phases can be further used for aid in the creation of production queues and to identify particular processes for affecting scheduling.

Workflow Planning

“The transition occurring in the… media production industry is a profound game changer. It is one of the most dramatic and significant changes that impacts every aspect of creation, production, management, distribution and monetization” [1].

Under design documentation, workflow planning is the first piece of our first new data capture and analytics layer. This layer feeds into collaboration enablement and creates a mechanism for the files to be shared seamlessly across the entire production workflow. While the files are being created across the process, this new workflow gathers the data that allows for rich indexing. This will lead to savings later on, in terms of speed and efficiency. Additionally, this workflow planning process addresses media and format obsolescence and how to handle legacy files.

Rich Indexing

One of the biggest wastes of time for staff is looking for lost or misplaced files. By adding rich indexing to our workflow, the process now allows for rapid and accurate retrievals and more in-depth queries. Over the years, rich media has been gaining in usage and is now commonplace online. In light of this, the issue of “facilitating efficient access” of rich media has become a formative research problem [2]. Using our solution, we can limit the separation of versions as a barrier and easily complete tasks, such as normalizing formats.

Cognitive & Game Play Analytics

Under the code and interaction and testing section another layer has been added to capture cognitive and game play analytics. Here the goal is to secure real-time capture and analysis of game play performance data, while working out any bugs and/or quality issues. This data analysis will inform our ability to make more engaging experiences and to add adaptive learning mechanisms into future processes. This particular process has been set-up to be seamlessly woven into the experience to capture “play-based competency development” [3]. In this way, the system is setting the foundations for “targeted and dynamic learner support” [3]. This is key to generating more revenues and creating more relevant and enjoyable mixed reality experiences.

Cost Breakdown

The current workflow costs $509,500, including the loss from lack of an effective data storage and analytics package. Failing to use an effective data storage system costs $75,000 in wasted time and mistakes. It is also significant that failing to use analytics, costs roughly $125,000.

Consider the impact deep learning systems and reinforced learning could have on cost savings. These valuable tools can be deployed seamlessly within the system, quickly and affordably. By implementing the revised workflow a studio migrating from the current workflow would have a  production cost of $449,000, giving us a $60,500 savings each time a full production cycle runs.

Although, there are some nuanced areas that would need to be clarified and researched before launching any real initiatives, this is a solid start for exploration.  Below is an example breakdown of a cash analysis, along with a cash flow statement, income statement and statement of financial position.

"The internet explodes when somebody has the creativity
to look at a piece of data that's put there for one reason
and realize they can connect it with something else."

Tim Berners-Lee, Inventor of the World Wide Web