PRE2017 3 Groep4

Museum Tour Robot

The museum robot is an artificially intelligent robot guide to enhance a new visitors’ experience in a museum by providing an interactive experience. By conducting a dialogue, the museum visitor is shown around the museum. The robot adjusts his dialogue and guided tour to the visitor. This will be done on the basis of age, gender, interests and cultural background. Every person gets a unique tour through the museum. The robot is also able to stimulate the senses of the visitors by giving audio and in some cases a certain smell. During the tour, the robot is able to respond to questions asked by the visitors and he must also be able to act on certain actions. In addition to answering questions, the robot will also take in account the background, preferences and interest of the visitors. He can respond on it by giving recommendations about what the visitors must have seen on the basis of their interests. This way of touring will serve to attract more younger and foreign visitors to museums. This because the tour is more personalized and the robot will take in account the preferences of the visitors. The robot is also able to give a tour in another language, this makes a tour more attractive to foreign visitors. At the same time, the museum needs to remain accessible to all possible visitors, including those who are not familiar with these robots or do not wish to use them like elderly people. These are generally the people who are visiting museums the most. The robot must therefore also be accessible to this group.

Orientation robot

Objectives

When developing and researching the museum tour robot, the objectives give a clear direction of the goals. The objectives are the following;

• The robot must be able to enhance the museum experience by giving personal tour based on interest and background.
• The robot must be able to attract more and younger visitors to the museum by taking into account the preferences of the visitors.
• The robot must be able to provide the experience for a wide audience by giving tours in multiple language.
• The robot must be easy to use also by elderly people

Users

A robot in a social environment must take all other agents (users) into account. The users in this scenario are;

• Museum visitors
• Tourists
  • “Regular” museum visitors
  • “Irregular” museum visitors (who we want to attract)
• Elderly, Young people, children
• Families
• Museum owners
• Museum employees
• Maintenance technicians
• Government

User Requirements

Visitors:

• Personal, positive experience
• Experience on their level / their background (age, home-country, language)
• Good balance between time with friends/family and information.

Museum owner(s):

• More visitors going to museum, people visiting museum more than once
• System is cheap to maintain

Museum employees:

• The system does not interfere with the employees’ duties (does not get in the way)
• The system can easily be controlled by an employee

Maintenance technicians:

• System does not require much maintenance
• Maintenance is easy to perform (parts are accessible)

Task Environment

Performance measure: Safe, fast (enough), personal for visitors, legal (rules museum), //(something with reacting well to visitors)

Environment: Entrance, visitors, museum staff, museum art, paths, other robots

Actuators: Moving, talking

Sensors: Location mechanism, cameras (see visitors/artworks), speech recognition system

fully observable vs partially observable ; agent can only observe visitors in the current room.

single agent vs multiagent ; There are more entities in the environment; visitors, other robots.

Deterministic vs stochastic ; movement of visitors is not to be exactly predicted.

Episodic vs sequential ; discussable. Episodic, because every room has new artworks and new information, which does not depend on previous room. But sequential, because visitors keep knowledge of previous room and might recall on that, or make links.

Static vs dynamic ; Environment changes while robot is thinking. People will keep moving and discuss/talk with each other (potentially about the artwork)

Discrete vs continuous

Known vs unknown ; this is about if the robot knows its environment, which is true in this case, since the robot has pre-knowledge about the artworks and the museum layout.

Persona’s

William and Alice, parents of 14 year-old Emma and 8 year-old Dylan, like to visit museums in the holidays to familiarize their children with art and culture. William and Alice hope their kids will appreciate culture and art in their lives and that it will educate them by looking at the world from different perspectives. William and Alice really like museums themselves, however, their daughter Emma is entering puberty and she thinks museums are boring, and their son Dylan is really active and doesn’t like to read. Therefore, William and Alice struggle to find a museum that fits all the different interests of the different family members.

John and Lauren are enjoying their retirement. They love art and they like to visit museums regularly. They enjoy many different kinds of art and they like to go to a different museum every week. They can really enjoy spending a day in a museum looking at art and taking their time to take in the piece of art.

A group of fifteen French tourists are visiting the Netherlands and as a part of their trip they visit a museum. However, because they only speak and read French, it is hard for them to understand the information that is presented alongside the art pieces as they only provide information in Dutch and English. Their experience of the museum depends only on the art pieces themselves, without the extra information that possibly could enhance this experience.

Scenario’s

William and Alice choose to go to the Noordbrabants Museum because of its versatile exhibitions and they hope the whole family will like it. There are multiple exhibitions of art from the past until now. Which William and Alice think they will enjoy and for their youngest Dylan, there is said to be a floor in the museum to show art more interactively. For their daughter Emma there is a bit of modern art which they hope she will like. Because it is a family trip, William and Alice like to stay together a bit. But when William and Alice are enjoying the art, Emma and Dylan are whining they like to walk faster and that they have seen everything in this room already. When they are on the more interactive floor for Dylan, Emma is whining that she wants to continue, she finds it childish there, and when Dylan is in the modern part of the museum he is whining that he wants to go back to the more interactive part. At the end of the day, no one really enjoyed the museum as much as they could have, and the family is a bit disappointed.

John and Lauren also decide to go to the Noordbrabants museum. They really enjoy the art and they take their time to take in each piece of art and the accompanying information. They make use of the benches to sit on every now and then, to be able to fully enjoy all the art. They also like the peace and quiet in the museum.

The group of French tourists decide to go to the Noordbrabants museum as well. They enjoy the pieces of art, but they cannot read the accompanying information. Unfortunately, guided tours are only available in German and English, which they also don’t understand. So, at the end of the day, they liked the museum but they wish their was some information in their language so that they would understand the art better, and could have enjoyed it better.

Orientation project

This section will be about the personal orientation of the project.

Approach

First, a literature review will be done to gather information about the current state-of-the-art in multiple disciplines. Of course it is important to have knowledge on similar ideas and how they were approached. The necessary technology to realise the technical implementation in the end has to be looked at. For that, it is important to look at the state-of-the-art of Artificial Intelligence, smart home systems, person localization systems, chatbots, virtual personal assistants (like Siri from Apple), speech language processing systems, and probably more. Further, literature about user experiences in museums is necessary to see what the users do and do not like and be able to respond on these findings.

Secondly, a survey study will be performed on the user experience in general museums. The survey will be based on the literature review and created online through Google Forms. The results of the survey study will be mainly analysed qualitatively. The small survey will be spread out online.

Finally, the technical implementation phase will start by creating an implementation plan. The technical implementation will consist of three parts: The set-up of a virtual 3D model of a museum An AI dialogue system implemented in the museum A smart person localization system implemented in the museum

If everything goes well according to the planning, an optional test phase will be added after realizing the prototype. In this test phase, a small experiment study will be performed. Participants will experience the virtual museum with the developed AI tour guide. After experiencing, the participants will fill in a small questionnaire on their experience and opinion.

Planning

The planning can be found on this page: Planning_Group_4.

Milestones

• Deciding which exact deliverables we want
• Pepper can answer 1 question
• Pepper can detect/distinguish whether a question fits its knowledge of the museum, and can respond accordingly

Deliverables

• Code of pepper with documentation
• A video clip with Pepper at work
• A small report with information about the design choices of Pepper

State of the art

In this section, the current situation and most recent ideas and methods will be discussed. All used articles and sources can be found at the end of the wiki. Researching this "state of the art" will help to discover which steps can be made, and can help focus on more specific aspects of the museum tour robot.

First, the focus will be on more broad aspects around the museum tour robots. Topics include Museum/AI experience, person localization systems, Speech recognition, natural language processing and tour guides in other locations than musea. All research can be found on the page Broad SotA.

After researching the broad SotA, its decided that the focus will be on having a useful dialogue between robot and human. This is why extra research is done about the state of the art of dialogue/interaction between robot and human. This research can be found on this page Specific SotA

Specification robot

In this section, there will be more focus on dialogue between robot and human, and some important topics will be discussed to specify the direction of the work that will be done.

Useful conversation

Before making decisions, and to make objectives and methods more clear, the definition "useful conversation" between robot and human, in the context of a museum, will be made clear.

A conversation is defined as oral communication between at least two person in which information is exchanged. In this project, we define a conversation as oral communication between a robot and a human in which information is exchanged.

In this project, a useful conversation between a robot and a user shall take place in the context of a museum. This means that the conversation will take place in the museum about information regarding the museum and its artwork. The conversation shall take into account the personal preferences of the user. If the robot does not have knowledge of something the user asks, it shall respond that it doesn’t have knowledge on that, but that it can share information on a different subject. Also, the robot shall react to a question in a way that the question is actually answered (with the exception of a question that is irrelevant in the context, in that case it will respond as described above). On top of that, the questions, answers and remarks, provided by the robot, have to be sensible (contribute the context, experience or information-basis) and the exchanged information has to be truthful.

Conclusion

A useful conversation is oral communication between a robot and a human in a museum. In this useful conversation, all questions, answers and remarks provided by the robot are:

• Relevant/useful in the context (museum)
• Truthful
• A question that suits an answer
• Fitting the background of the user (language, origins, age)

Coaching Questions

Questions of the coach and answers of the group can be found on this page (Coaching Questions Group 4) every week.

Sources

APA citation is used here. Some sources lack official citation, this will follow later.