Alquist made it to the Alexa finals

The CVUT Alquist team managed to get with other two teams to the finals of a $2.5 million Alexa Prize, university competition. Our team has developed the Alquist social bot.

The whole team has met in the eClub during summer 2016. That time we have been working on a question answering system YodaQA. YodaQA is a somewhat complex system, and students learned the classic NLP. Of course, everybody wanted to use Neural Networks and design End to End systems. That time we have also been playing with simple conversational systems for home automation. Surprisingly Amazon announced the Alexa Prize and all clicked together.  We have quickly put together the team and submitted a proposal. One Ph.D., three MSc, and one BSc student completed a team with strong experience in NLP. In the beginning, we have been competing with more than a hundred academic teams trying to get to the top twelve and receive the 100k USD scholarship funding. We were lucky, and once we were selected in November 2016, we began working hard.  We started with many different incarnations of NNs (LSTM, GRU, attention NN, ....) but soon we have realized the bigger problem, a lack of high-quality training data. We tried to use many, movies scripts, Reddit dialogues, and many others with mixed results. The systems performed poorly. Sometimes they picked an interesting answer, but mostly the replies were very generic and boring. We have humbly returned to the classical information retrieval approach with a bunch of rules. The final design is a combination of the traditional approach and some NNs. We have finally managed to put together at least a little reasonable system keeping up with a human for at least tenths of seconds. Here started the forced labor. We have invented and implemented several paradigms for authoring the dialogues and acquiring knowledge from the Internet. As a first topic, we have chosen movies since it is also our favorite topic. Then, we have step by step added more and more other dialogues. While perfecting dialogues, we have been improving the IR algorithms. We had improved the user experience when Amazon introduced the SSML. Since then Alexa voice started to sound more natural.

While developing Alquist, we have gained a lot of experience. A significant change is a fact that we have to look at Alquist more as a product than an interesting university experiment. The consequences are dramatic. We need to keep Alquist running, which means we must very well test a new version. Conversational applications testing is by itself a research problem. We have designed software to evaluate users behavior statistically. First, a task is to find dialogues problems, misunderstanding, etc. Second, we try to estimate how happy are users with particular parts of the conversation to make further improvements. Thanks to the Amazon we have reasonably significant traffic, and while we are storing all conversations, we can accumulate a large amount of data for new experiments. Extensive data is a necessary condition for training more advanced systems. We have many new ideas in mind for enhancing the dialogues. We will report about them in future posts.

Many thanks for the scholarship go to Amazon since it was a real blessing for our team. It helped us to keep the team together with a single focus for a real task. Students worked hard for more than ten months, and it helped us to be successful.

Today we are thrilled we made it to the finals with the University of Washington in Seattle and their Sounding Board and the wild card team from Heriot-Watt University in Edinburgh, Scotland, with their What’s up Bot. Celebrate with us and keep the fingers crossed. There is a half a million at stake.

New projects for this summer

This year we are opening the eClub Summer Camp new CIIRC building. We have prepared exciting projects from the field of AI, IoT, and Internet. We will focus on conversational IA, how to program assistants to control your household, Natural Language Processing, and other topics, see the projects page.

Two years ago we started to work on the question answering engine YodaQA. Last year during the eClub Summer Camp we have designed the first bot. Our primary goal for this summer is to create a great Echo application. Echo is a voice controlled smart speaker made by Amazon. You can only ask to play music, ask factoid question, carry a simple dialog or control your household. There is an amazing technology behind the set of new Amazon devices. First of all the speech recognition, directional microphone, conversational AI, knowledge database, etc. The eClub team is among the first in the world working directly with the Amazon research group on making the Alexa even smarter. We want to make her sexy, catchy and entertaining and it requires a lot of different skills. Starting with the linguistics up to Neural Networks design. We have many well-separated problems for any level of expertise. Come to see us, we are preparing an introductory course to teach you how they do it. We will help you to create your first app with initial skills. You can meet a lot of students who work in the Conversational AI who will help you to get over the underlying problems.

We want to make the Conversational apps not only entertaining but also knowledgeable. Alexa must also be very informative. It must know for example the latest news in politics, the Stanley Cup results, what are the best movies and I am sure we can continue with many other topics. The knowledge is endless, and it is steadily growing. To handle to alway increasing data requires processing many news feeds, different sources, accessing different databases, accessing the web, etc. The news streams must be understood, and the essential information must be extracted. There are many steps before we retrieve the information. Especially today we need to be careful, and every piece of information must be verified. We try to create a canonical information using many sources of the same news. As soon as the information is clear, we need to store it in a knowledge database. The facts need to be linked to information already in the database. And how about the fake news, how to recognize them?

Building the Conversational AI does not include only the voice controlled devices. We may want to create a system automatically replying to the user email or social media requests. Imagine for example a helpdesk where users are asking many different questions from IT to HR topics. For example very frequently how to reset a password, or how to operate a printer or a projector, why not to answer them automatically? And we can be much more ambitious. Many devices are quite complex, and it is not easy to read a manual. It is much faster to ask a question such as “How do I reset my iPad,” or “How do I share my calendar.” These apps are put together from two major parts. The understanding of the question and a preparation of the answers. Both use the NLP pipeline. If you expand on this idea, you may find a million of applications with a similar scenario. An automated assistant can at least partly handle every company-customer interaction. To make a qualified decision, the executives need fast access to business intelligence. Why not ask questions such as “What was the company performance last week,” “What is the revenue of my competitors” etc.

Let me mention another aspect of our effort. The latest manufacturing lines are extensively using robots, manipulators, etc. (INDUSTRY 4.0) The whole process is controlled by a large number of computers. What if something stops working, it is a very complicated task to fix a line like this? Every robot or manipulator might be from a different manufacturer, programmable in a slightly different dialect. Is there anybody in the company who can absorb the complete knowledge to be useful in localizing the problem? Yes, it is a robot, which has all the knowledge in a structured form. The robot can apply optimization to find the best set of measurements or tests to help the maintenance technician. To make this happen, we need in addition to a productive dialog and knowledge database also an optimization to suggest the shortest path for fixing a problem. The robot can guide humans to repair the problem most efficiently.

Yes, I have almost forgotten. It is recently very popular to use the robots to control the household. Alexa, turn off all the lights. Alexa, what is the temperature in the wine seller? We want to invent and build some of these goodies to our new eClub space during the summer. Our colleagues have developed a Robot Barista application shaking drinks on demand. A voice user interface will make it even more entertaining. We have other exciting devices and small gizmos deserving voice control. You also may come with your ideas. Join us we will assist you to be successful.

These are just few use cases we will try to tackle during this season. If you want to learn the know-how behind join us, we will help you, and we also will award a scholarship.

Conversational AI for Dungeons and Dragons

we start the 2017 eClub Summer Camp. eClub has moved to a new CIIRC building. We are competing in the Alexa Prize competition. Join eClub and learn the latest machine learning, NLP algorithms.

A few years ago we have started with a question answering system YodaQA. It has been inspired by the IBM Watson beating the best player in Jeopardy. Today we continue our journey in even more challenging projects. We are creating dialogs for the latest voice-controlled appliances. We have entered the Alexa Prize competition and it helped us to develop Alquist the social bots. We have a free access to an immense power of AWS, we are in constant touch with the Amazon research staff. Every member of the Alquist team became an NLP expert. The Alquist system is day by day getting better. Currently, Alquist can conduct a sensible short dialog. The user can choose from several topics: sports, politics, celebrities, jokes, etc.

Thousands of users are using the chat and we receive valuable logs making us very busy. It takes a lot of time to get through details, to discover why the user stopped the conversation but we are learning a lot. What is the social dialog? What are the catchy questions? How to respond quickly and interestingly? There are still a lot of questions, but we are ambitious, we want to extend the Alquist knowledge to handle a long and interesting dialog. If you like tested say Alexa let's chat.

The Natural Language Understanding (NLP) underlines Alquist. NLP is also the essential part of Siri, Cortana, Alexa, Google Assitant and other latest bots. The Conversational AI is building on machine learning, optimization etc., it takes advantage of all the latest development in machine learning, starting with the classical algorithms up to the latest deep neural network, sequence to sequence and memory networks etc. This summer we want to considerably improve the Alquist capabilities. To achieve our goals we need to enlarge the Alquist team and focus on the Conversational AI. If you are a BSc, MSc or Ph.D. student join us. We have various programs including Ph.D. candidates.

In addition to AI, we also need creative individuals knowing how to handle a dialog, being innovative. We all know that carrying an interesting dialog is an art. To teach Alquist interesting dialogs is even more complicated. Creative young people with many different skills in the human to the human conversation are welcome to join us.

A dialog is also about information and experience exchange. Imagine for example a bot helping you playing an adventure game. The RPG have many rules and it is very boring to search an information in a handbook. One of the very popular RPG games is the Dungeons and Dragons. We want to design an interactive Alexa D&D handbook and improve level and XP progression. If you are interested in D&D join us helping us to design a voice-controlled interactive manual.

The NLP space is huge and we have a large number of interesting topics to work on. If you are interested in AI, machine learning, neural nets etc. join us. We have great resources, a lot of experience and funds to award you scholarships.

Alquist mission continues

The Alquist team has just returned from the Alexa Prize Summit. We have met with Alexa researchers, developers business developers, and all competing university teams. We enjoyed the three days of great fun discussing the conversational AI.

The summit opened with a visit to the Amazon Fulfillment Center. We were stunned observing the Amazon robots smoothly transporting racks with goods to pickers. The center is an excellent example of automation, optimization, and efficiency.

We have started the summit with a review of our Alquist social bot. Amazon experts commented on our accomplishment. No problems, we have passed.

The key part of the summit were the Alexa team presentations. The development of a social bot is a complex task requiring knowledge from many different disciplines. Therefore the presentations were covering many topics ranging from speech technology, NLP, Deep Learning to how to carry an engaging dialog. The Alquist team members are CVUT students with in-depth knowledge in AI. Therefore the most revealing for us were the non-technical presentations. I liked the Celeste Headlee presentation. She is a reporter on the public radio and a professional opera singer. She taught us how to Make Great Conversations. The key message:  Researchers discovered that talking about yourself activates the same pleasure centers in your brain as sex and cocaine. We learn how the Alquist needs to behave. We also got a lesson about Best Practices for Promoting Our team. The advice is simple: promote better the Alquist team and the Conversational AI.

The summit was not only about technology, on Wednesday evening we joined the screening of just released Amazon Studios Movie “The Lost City of Z” produced by Brat Pitt.

We also looked around the Seattle City on Friday afternoon. We were lucky it was one of the few days with blue sky. I gave up queuing for the lift to the Space Needle Observation Deck and instead I have climbed one of the Seattle hills. The sky was clear and the more than 4000 meters high Mount Rainier with its snow cap seemed like just behind the city.

The Alquist team was excited by the Alexa Price Summit. We had a unique opportunity to meet with the Amazon Alexa team and the competing university teams. We had the chance to experience the Amazon enthusiasm and the entrepreneurial spirit. We got back charged with fresh energy to push the conversational qualities of Alquist even further.

If you are interested in building interesting conversational applications, join eClub and the Alquist team and help us to make Alquist even better. There are many AI tasks we need to improve, some of them we have not touched yet. It does not matter if you do not have any experience, we will teach you. Thanks to Amazon we can give you a scholarship. Join us and enjoy the feeling of success!

Alexa socialbot testers wanted!

We are finishing the last details, of the Alquist social bot.  Today we are opening the first preview for testers.  If you are interested, leave your email. We will send you instructions. The total number of testers is limited. Hurry up!

If you do not own Echo or other Alexa appliance, install the Echosim.io or Reverb on your cell, and you are ready to start. Upon signing up, we will email you all required details. We are interested in your feedback.

The chatbot is not perfect but, it is the time to start learning how users interact and collect the feedback. We need real users who are not familiar with the system. Amazon is helping us with their internal testing system, and it is a unique opportunity. Help us!

What can you expect?  We have trained Alquist to carry a simple dialog, it knows how to answer factoid questions using the Wikidata DB, and how to give help. Recently, we have made advances in creating a simple dialog within limited domains with frequently changing data.

All is still under development with a lot of quirks and twists, but your feedback will help us to improve. Thanks!

New projects - join us

eClub will again organize the Summer Camp (ESC). The ESC 2016 was incredibly successful. Five eClubbers worked on a question answering system YodaQA. At the end of summer, they have entered the Amazon Alexa Prize competition and got in between top twelve selected for development of a social bot. They received $100k scholarship for a social bot development. Currently, they are busy working on the first version of Alquist.

We would like to continue in the direction of developing dialog applications in ESC 2017. The social bot task is very challenging, and many required technologies are still in development. While developing YodaQA, we have looked at the well known classical NLP algorithms as well as to new mainly neural networks based ones, such as LSTM, GRU, etc. to process text in many different ways.

It is beginning of March, but we are already prepared to incubate new, eager students interested joining us on this journey toward smarter systems. We have strong support not only from Amazon but also from the local company Seznam. Seznam is one of the few competing successfully with Google on the domestic market. They are 100% Machine Learning company with a lot of problems of mutual interest.

Here is a sneak preview of the projects for this year. Join us, and work with us tomorrow, if you are interested. We are offering scholarship equivalent to what you would earn working for a company. We are moving to the new CVUT building, it is gorgeous, as soon as it opens, it is just a matter of weeks. Join us! You can start any time.

Automatic email reply generation
In this project, we want to research methods for automatic generation of short responses to emails or social networks messages. Specifically, on a cell phone, it can be a great advantage to select from a selection of semantically diverse replies. We want first cover few words long messages. The initial steps will include a review of Recurrent Neural Networks architectures and a meaningful training set construction.

Amazon Echo conversational application
We have a set of tasks we would like to cover as a spoken dialog. We want to design interactive conversational bots for Amazon Echo. We want the application to be engaging, entertaining and informative, bringing the user latest news from specific areas, such as sports, celebrities, movies, etc. This project is OK for students who are only entering the field with no or small experience.
   
Knowledge extraction
There is vast of information on the Internet. A lot of the information is in the form of a text; the information is unstructured. In this project, we want to review the methods for retrieving and extracting the information, learning the dependencies of statements in the texts. We want to create ontologies from a selected, limited content and store the knowledge for further use. These are very challenging problems but do not hesitate to join. We have students currently working on these topics. We know what the first steps are.

Text summarization
Journalists write the Internet news in a particular language, frequently using idioms, slang or infrequent expressions. In this project, we want to extract what is important and create a summary in a clear language. Initially, we want to summarize long sentences. Next, select a suitable method, implement and test it on a chosen domain.

Events Extraction from Text
This project is only an extension of the previous one. We want to design and implement a system for extracting events from the Internet. The primary goal is selecting news messages based on identified topics (or events). Extraction of economic events like mergers & acquisitions, stock splits, dividend announcements, etc., play an essential role in decision making, in risk analysis applications and monitoring systems.