News

On Virtual Idol@HKDI 

Virtual characters have been around for quite some time. Many Asian millennials can still recall when songs by Hatsune Miku were on their MP3 playlist. Today, almost 15 years later, we still don't see this trend going away anytime soon. As a matter of fact, it is growing more rapidly than ever: dozens of virtual influencers are invited to attend A-list fashion shows each season; mega K-pop agency is introducing the first ever girl group with virtual characters; guests attending global panel discussions are showing up at the venue with their own virtual characters. Now might be the worst of times for flesh-and-bone figures to actively involve in social activities, but it surely is the spring of hope for virtual characters to blossom. 

A young female idol is dancing alongside her fellow backup dancers during a live stream session at HKDI. Yedda, the name of the idol, is not only known for her smooth dance moves and edgy appearance, there's something more about her identity that fascinates us, for she is not a real person. 

Yedda is a virtual idol created by students and instructors from HKDI's Department of Digital Media and Department of Fashion and Image Design. Two graduates from HKDI's Animation and Visual Effects programme, Fay and Dennis participated in the project. They said: "We both wanted the character to be more relatable to the younger generations. Thus, we studied many popular celebrities including IU from South Korea. We wanted to fuse these pop culture elements and characteristics when creating the virtual idol." 

During their extensive research, Fay and Dennis also took inspirations from Japanese virtual model imma and virtual girl group K/DA from computer game League of Legends . 

On the technical side, Yedda is able to perform her sleek dance moves thanks to an actual dancer controlling her. The said dancer wears a specially engineered black bodysuit with 17 sensors and batteries with a life span of 9.5 hours. Combining with motion capture system Xsens, the whole setup reaches a wireless range of 50 metres indoor and 150 metres outdoor. The dancer also needs to wear a headband and gloves to capture their head and hands movements. 

During the performance, students need to control 4 other systems to monitor Yedda's body movements and facial motions. These systems include real-time camera tracking system Ncam, body motion tracking system XSen, facial motion capture technology Faceware as well as a hand controller called Manus. In total, Yedda has 52 basic facial shapes, and that gives endless variations of facial expression when combined. 

The Yedda project was still a budding concept in September 2020, and in June 2021 we already see Yedda's debut. Since then, she has performed duets with real dancers and attended events including HKDI and IVE (Lee Wai Lee)'s annual design show Emerging Design Talents 2021 and Motion in Music 2021. 

When dancers vary between performances, the entire Yedda system has to go through calibration for it to fit with the current dancer's body type. Rounds of rehearsals and calibrations are required before each performance, it is definitely not an easy task for a team of 6. "We have hands and body movements, facial motion, camera, etc. to be in charge of. If one thing goes wrong, everything goes wrong. This is also why each live stream is equally nerve-wrecking and exciting." adds Fay. 

On Virtual Production by HKDI 

IT Sarah is a virtual character designed and developed by students and teachers from the Information Technology Department of Hong Kong Institute of Vocational Education (IVE) (Lee Wai Lee) with MR technology. Student at IVE's Higher Diploma of Games and Animation course designed IT Sarah's appearance. Then, it employs 3D animation building software to add body movements. Combined with real-time image output, Sarah is able to interact with audience without lag time. 

IT Sarah and Yedda are proud productions by students and teachers from IVE and HKDI respectively, but it is only a beginning. Ken Lee, Acting Senior Lecturer at HKDI Department of Digital Media, says, "The pandemic has proven outdoor shooting difficult, and that in a way has promoted the application and development of virtual production." 

Karma is an XR short film produced by students at HKDI. The 15-minute film is based on cyberpunk and detective story. Terrance, one of the student creators of Karma tells us the biggest challenge during producing the short film was the how it differs from conventional filmmaking. He says: "Normally we do post-production after filming, but now special effects and filming happen at the same time. We have to follow orders given by the direction team to make changes on spot." 

It was not only a new experience for students, but also for instructors. "When we taught filmmaking before, we separate the education by pre-, mid-, and post-production. Now the difference is that we have the CG (computer graphic) effects ready on the day of shooting." says Ken Lee. 

He mentions: "Extended reality revolutionised the filmmaking process, subsequently changing qualification requirements for people in the industry." He believes that digital media creators today have to maintain a broader vision. Not only do they need to obtain fundamental knowledge of filmmaking, but also computer imaging, scenography and extended reality filming techniques. 

Filming with Extended Reality is developing in Hong Kong. HKDI believes in its potentials, and actively collaborate with the industry to promote Extended Reality, in the hope that local digital media field can foster more skilled specialists adapted to the future. 

Virtual production belongs to the broader term of Arts Technology of which the development is supported by the local government. Local film production company Free D Workshop and HKDI both believe in collaborating between academia and professional industry, and agree that education is the key. 

Younger generations are hoping to revitalise Hong Kong film culture. Many of them prefer to be in the industry as opposed to staying in academia long term. Both the technical artist and technical director in the Yedda project are already hired by local art tech companies. HKDI also has a designated recruitment platform for students, new graduates and employers to get in touch and interact with one another.