Imperial College create immersive language learning experience using Watson and Virtual Reality

Share this post:

The Call for Code Challenge was created by philanthropist David Clark, founder of the David Clark Cause, in partnership with the Office of the United Nations High Commissioner for Human Rights, the Linux Foundation and IBM. This Challenge is open to developers, start-ups, academics, NGOs etc. from around the world. The objective is to imagine innovative open-source solutions based on AI, IoT, blockchain, cloud computing etc. to address major global environmental challenges: water sanitation, world hunger, responsible production and consumption. This competition is aligned with IBM’s initiative to invest $30 million over five years to use the full potential of technologies in order to protect the environment, save human lives and improve the future of the next generations.

Venice of Secrets – Imperial College, UK

Maybe it’s the fact that they all come from different cultures and speak a multitude of languages that Alexander Scott, Sebastiano Zane, Gabrielle Rubin, George Chen, and Christy Wong gravitated toward a Call For Code Global Challenge 2021 project that centers around language learning. The Imperial College students argue that access to complementary education should not be restricted to those who can afford to pay for it. They also conclude that when learning a language, immersion in the environment is best, but that discretionary travel has a negative impact on the environment. Their solution: a playful online- and AI-based learning platform that puts the student into a virtual scenario as it could happen in real life.

How did your participation in Call For Code 2021 start?

Christy: It started with a Learning a Language Project earlier this year. We collaborated with IBM on building an adventure-based educational game that teaches Italian in a fun and easy way through simulating a visit to Venice in our virtual reconstruction of the town. Not only does our game address the social problems within the education system but also the increasing carbon footprint from traveling to the world’s most iconic travel destinations.

Gabrielle: As we were already working with IBM as a “client”, we spoke to John McNamara who encouraged us to participate in the challenge. Then we said, “OK, let’s do this!”, entered, and quickly realized what an impact this project actually had.

Tell us about your group.

Sebastiano: We are a group of friends with a shared interest in engineering and/or computer sciences. Coming from different countries – the UK, Italy, Hong Kong, Trinidad and Tobago – all of us speak at least two or more languages. So we know about the importance of being able to communicate properly in different cultures. Collaborating on this project definitely brought us even closer together as friends.

What were your expectations when entering the competition?

Christy: We wanted to offer an affordable and sustainable solution that combines AI with science-based teaching methodology to engage learners and bring the benefits of international study to devices everywhere — while shrinking the carbon footprint and localized damage caused by international travel.

How was the process of preparing the 2021 CFC submission for you?

George: We were all Covid-stuck in our relevant locations. So as a first step we had to split up the tasks into different teams. This unique situation is also why we could only present a skeleton structure. But we are proud of what we have accomplished so far, and every completed step was rewarding.

Gabrielle: The process was joyful. Due to the pandemic-related logistics it was different from any other experiences before but we were glad we faced and overcame the challenges; it was in fact a lot of fun. As time went on we became more and more invested in the project. And all helped with the design.

How does your solution help to save the planet?

Gabrielle: Climate change is a big issue today. We wanted to make our solution relevant to it and reach a large audience.

Sebastiano: Venice, which happens to be my home town, is very much at risk from climate change. At the same time it is such an iconic travel destination. Every year Venice receives more than 20 million visitors whose travels leave a large carbon footprint in their wake. A part of those are language students. So if we connect learning, tourism, and environmental issues, we can help mitigate some of that damage.

Which technology did you choose for your solution, and why?

George: The solution is built on a Unity 3D game engine, and we integrated IBM Watson Assistant for translations, IBM Watson Text To Speech and Speech To Text for the challenge scenes. We also use the IBM Watson Language Translator.

Did you ask IBM for support? Did you get any help from outside experts?

Christy: We asked John McNamara of IBM for guidance and feedback, and he was very helpful.

Gabrielle: Outside IBM, we turned to educators for advice about Best Practices, student retention etc.

What did this adventure bring you on a personal level, what did you discover by way of experience?

Christy: I never thought that developing video games would be this interesting. I was responsible for the characters. Also, the communication went better than expected, given the different time zones.

Gabrielle: I learned a new language, as well as new tools when integrating IBM API.

Sebastiano: For me it was a completely different and exciting experience.

George: I found it rewarding because we could see what we were doing.

Are you thinking about deploying your solution?

George: Yes, totally, I would love to do that one we have a finished solution!

What advice do you have for potential FC 2022 participants?

Christy:  Early planning and good communication is crucial.

Sebastiano: Pick something you’re passionate about – excitement is key!

If you are interested in exploring the IBM technologies that made these projects possible. IBM provides free access to both technology and enablement via the IBM Academic Initiative. Simply register with your university email address and you will have access to an array of enterprise grade technologies and learning experiences, which you can use to create your own projects – https://www.ibm.com/academic/home