Li LiuJun

Exclusive Interview with Li LiuJun, Director of "Farewell to Walnuts":What is the concept of XR film based on Metaverse technology?


随着Quest系列、HTC Vive 、Pico、大朋VR等VR设备在市场上的普及,VR电影、VR游戏两大娱乐体验也逐渐被熟知,但你听说过XR电影吗?
With the popularity of VR devices such as Quest series, HTC Vive,Pico and Dapeng VR in the market, VR movies and VR games are gradually known, but have you ever heard of XR movies?

告别核桃》就是第一个吃螃蟹的XR影片。该影片由温州商学院的XR_LAB团队打造,总时长约10分钟,已经入围了北美电影节、英国Lift-off Global Network、新西兰亚太国际电影节金蕨奖、香港国际青年电影节等多个海内外知名电影节提名。
Farewell to the Walnut is the first XR movie to eat crab.Created by the XR_LAB team of Wenzhou Business School, the film has a total duration of about 10 minutes. It has been nominated by many well-known film festivals at home and abroad, such as the North American Film Festival, the UK Lift-Off Global Network, the Golden Fern Award of the New Zealand Asia Pacific International Film Festival, and the Hong Kong International Youth Film Festival.


"Farewell to the Walnut" combines the 3D digital live-action dope-in technology in the era of YouTube, and its most unique experience is that the audience's digital people can replace the roles in the film.According to chief director Li Liujun, Farewell to the Walnut has already released trailers on iQiyi VR channel and Kuaishou Panorama VR platform, but the specific release date has not yet been determined because it is still participating in major film festivals.

What's the difference between XR films and VR films?What's the story behind Farewell to the Walnut?How is the metaverse developing in the field of film and television?We have an interview with Li Liujun, chief director of Farewell to the Walnut.

Li Liujun used to be engaged in new media and exhibition related work. In 2015, he began to get into the VR field and taught in the Digital Media Teaching and Research Department of Xiamen University Jiaeng College. In 2017, he was introduced to Wenzhou Business School and taught in the Network and New Media Department of the School of Media and Design Art. He opened courses such as "Digital Media Technology", "MR Mixed Reality" and "E-Sports and Game Communication", and led the XR_LAB team.


《告别核桃》总导演 李柳君

Li Liujun, chief director of Farewell to the Walnut

The following is the interview transcript:

Game Daily: How did you come up with the idea to make a short film like Farewell to Walnuts?
For many years I study in the field of VR, originally wanted to be in the yuan digital universe come and go and XR display technology creation, similar to a TV drama "in the western world" (science fiction series, probably worldview is the bionic people live in high tech park, visitors can enter into the "world" in the new drama/life), let the audience can be involved in the film.

Game Daily: Farewell to Walnut's label "the first Metaverse XR movie" has attracted a lot of attention. How do you understand what an XR movie is?How is it different from a regular VR movie?
Actually, when we first mentioned this concept, many people didn't understand it. In short, VR is the technical basis of XR movies, but its presentation form is not limited to VR.Let me give you two points to explain.
On the one hand, XR movies stretch the boundaries of "watching".In addition to VR devices, mobile phones can also be used to view. We are also trying to combine the real environment to create video content, such as shooting a certain area in Wenzhou center, so that the audience can quickly integrate into the scene, or directly walk to the scene to watch XR film.
The other aspect, which is what Farewell to the Walnut is already doing, is integrating into the digital human form.We worked with UberLink to replace the protagonist with each audience's own digital avatar, so as to give viewers an immersive experience.


Game Daily: Is it only possible to replace the main character?

No, the passer-by in the story can also be replaced. Of course, the story must have a main line. It won't be said that if the passer-by is replaced, the passer-by can be seen all the time.In addition, what's special about this film is that many characters use digital doppelgangers.These digital doppelgangers aren't specifically looking for actors, they're just ordinary people around us.

Game Daily: It seems a bit like a simplified version of the scene in Ready Player One that goes into the movies?

Yeah, it's kind of like that.

Game Daily: How long did it take us to make the short film Farewell to the Walnut?What difficulties did we encounter along the way?
The project was launched in June last year. After the project was approved, I kept revising the script, the plan and the characters. From scratch, I created a script of more than 20,000 words and produced a 3D material file close to 2T.The main difficulty was that there was no product to refer to, because the concept was completely innovative and we were the first explorer.


The latest animation design of character image


Latest character image animation design draft

Game Daily: Farewell to Walnut has been shortlisted for so many awards. Do you think it is a good story telling or a gain of XR new experience?
The most important thing is the new changes brought by the new technology, which will bring innovative changes to the traditional media and even promote the reform of the film industry.As far as I know, current Hollywood films such as The Lion King, The Jungle Book and Blade Runner 2049 all use Unity for script previews.Secondly, Farewell to the Walnut itself is based on a true story, full of realism.


Game Daily: A true story?Can you tell me a little bit about it?
Yes, it comes from a real interview we had more than 10 years ago, when we met a girl like the main character in Farewell to the Walnut.


Game Daily: I heard that Farewell Walnut is preparing IP mobile game and VR game?Will you develop it yourself or cooperate with any other company?

We have related team game reserves, the development of VR application technology in my courses (" digital media technology, the spread of e-sports game with "MR mixed reality", etc.) it will introduce how to design and develop a mobile game and VR games, involving hundreds of mobile phone games, VR applications, XR movie plot, and demonstrates the demo.Some students from Wenzhou Business College want to participate in project development after they have mastered certain skills, and we also encourage them to participate in it.
The graduates of the class of 20 I supervised are currently engaged in mobile game development and have received nearly 10 million yuan in financing.Of course, most students do not have the ability to do complete system development, and they mainly do creative work and plan cases.At present, the game project is in the early stage of preparation, which has not been quickly landed, and now it is still looking for funds.

Game Daily: What do you plan to do after you finish your game?
I will continue to work on the IP of Farewell Walnut until it has enough influence. This is a clear future goal.This includes IP based series, mobile games and VR games.


The trailer for Farewell to the Walnut has been viewed more than 140,000 times in Kuaishou

Game Daily: Talking about the metaverse, about the application of VR, AR and other virtual technologies in animation/film, how do you think it has changed from five years ago?

2016 is the first year of VR explosion. VR technology is gradually becoming more mature, gaining market acceptance and ushering in large-scale commercial application.So far, on the one hand, it is the improvement of hardware technology, on the other hand, it is the creation of animation/film content based on VR and AR.This is very important, any new technology must have enough rich content as a carrier to gradually cultivate a new consumer group, so as to form a huge industry.

Game Daily: Do current audiences in the film and television field have enough awareness of the concept of the metaverse?

To be honest, it is relatively unfamiliar. There are some large-scale professional international film festivals that may set up this section, and they will be willing to embrace new technologies.As for the audience, we will not emphasize the meta-universe, but mainly focus on XR films. Even so, the audience may not understand very well.Therefore, this needs to be constantly promoted, and more excellent works are needed.

Game Daily: Have you researched VR devices on the market today?How do you see the current coverage and features?

李柳君:这个领域我可以说是比较了解,目前我们大概有50多套设备,涉及各个类型和品牌,是买来用于课程教学实验和做项目开发测试的。现在内容大多是基于Unity去做,这也限制了国内一些设备的功能。本人开发体验感觉最好的是Meta的Quest系列,基本无缝兼容各种需求,其他硬件还有华为VR、PICO、HTC Vive、影创VR等,但是都会对软硬件有限制。
I know a lot about this field. At present, we have about 50 sets of equipment, covering various types and brands, which are purchased for course teaching experiments and project development and testing.At present, most of the content is based on Unity, which also limits the functions of some domestic equipment.My best development experience is Meta's Quest series, which is basically seamless and compatible with all kinds of needs. Other hardware includes Huawei VR, PICO, HTC Vive, Film Creation VR, etc., but all of them have restrictions on hardware and software.
In fact, it also reflects a trend that there is a lack of self-developed engine of sufficient volume in China. Once the engine fails to keep up with the upgrade, the previous content will be limited. Without the support of software content, the hardware market will not be activated.Therefore, one of the most important reasons why I want to work in the field of XR films is to adapt to various kinds of hardware.And of course there was a lot of trial and error early on.

Game Daily: XR emphasizes touch and interaction on the basis of VR and AR. Some people think that brain computer interface is the first step to truly achieve XR experience. What do you think of this view?
Extended Reality (XR) includes virtual reality (VR), augmented reality (AR) and mixed reality (MR), which is called the final form of virtual reality interaction in the future.IDC previously predicted that the global AR/VR/XR device market would exceed $800 billion in 2025. The new generation of XR glasses, which are close to ordinary glasses in shape and weight, will be produced and become the key entry of the next generation of Internet.Bci is also developing well, mainly in the medical and health field, and V Society is developing one that can be used in games. I think the two will push each other in the future.