PRE2022 3 Group12

Problem Definition

Problem Statement:

Society is currently faced with an ageing population. By around 2040, it is expected that one-quarter of the population will be aged 65 years or older. Compared to today, the size of this group of people will have increased by about 1.2 million people by 2040, all while the number of people working (in the age group 20 to 64 years old) will stay roughly the same. This means a large shortage of healthcare workers will arise, causing some elderly to not receive all care they might be expecting. One important aspect of this care that might easily be overlooked are ways to combat their loneliness. This is often prevalent among the elderly, especially those aged 75 years or older. One possible way to battle loneliness is to provide activities. However, with the reduced availability of care, it will become harder for healthcare workers to provide these activities. In these circumstances, robots can be used to support the workers.

Users:

Our design provides people with an opportunity to play physical card games without the need for other players. This is beneficial for anyone who is for some reason unable or uninclined to play with others. While it is great to have many potential people that are able to use the product, it also results in a large and ill-defined target group. In order to combat this general target group as well as form a starting point for the design and make it feasible considering the size of this project, a subset of the target group is taken. This new target group focuses on elderly people.

The target group of elderly people is chosen as they are generally assumed to have more difficulties with technology. It’s therefore expected that if the elderly people are able to properly use and understand the product, the younger generations will be able to do so as well.

We hope to increase the Quality of Life (QoL) of the elderly by creating this product. For example when they are unable to visit others, or unable to have visitors, they can still play with the robot and enjoy a game of cards.

Related Work:

One research focuses on tabletop technology. They aimed to make a game for children and elderly while combining digital and physical aspects of games. Because of the digital aspect there came a new element of uncertainty in the game, which was very appreciated. The users of the game, also appreciated that there where physical objects to hold, such as gaming tiles and cards.

Another research focuses on whether a robot can detect important information of its human partner’s inner state, based on eye movement. The robot also had a second goal, namely doing an entertaining activity with a human. The human is only lying about the secret card, and the robot has to then guess which card is the secret card out of six cards. Although the robot, did not have a 100% accuracy of guessing the secret card, their human partners still considered the game entertaining.

There’s also one research which focuses on the trust levels between robots and humans. Such that they would be able to create a social robot that would be able to entertain elderly, whom suffer from social isolation. For their experiment they used only one robot with a card game. Basically there were groups where the robot could either be your partner with whom you needed to work together, or the robot could be your opponent. The authors of this paper only looked at what the trust levels where of the human partner, and concluded that humans are able to trust robots. But to get higher trust levels, they need more interactions with the same robot.

Research Questions:

For our research questions, we tried to figure out what other researches did not have yet. So did the first project focus on board games and there was no robot player.  Whereas, the second project had a robot “player”, but this project used a different type of game. Namely, a game with tricks and guessing, instead of a game usually played between two people. And while the third project has a robot player, who plays a card game. The main focus of this project was on the trust-levels between partners, and not opponents.

Therefore, we decided that as main research question we want to focus on how we can create a card game-playing robot that the elderly want to use. While the robot could have easily played any other types of games, we decided to go for a card game since it might be more familiar than a digital game. To be able to answer the main question better, we have decided to specify it through dividing it up in the two following sub questions.

As has already been stated, the target group experiences difficulties with current technology. Therefore, it is important to figure out how to make the system of the robot easy and understandable to use. If our target group is unable to figure out how to use the robot, or it takes too much effort they are less likely to engage with the robot.

During another study, it has been stated that difficulty systems are important to engage more people. But this study only focused on the motivational reasons for elderly to play games. If the game is too easy it could become boring, and if the game is too hard it could cause anxiety. So this sub question focuses on how to implement the difficulty system, such that the different levels are appropriate for elderly.

User Requirements:

Due to their age, most elderly have increased problems with their sight, hearing, or motor skills. Therefore, it is important that the design has options built in to deal with this. For example, an easy-to-read font and text size, clear and loud audio implementations, and a lightweight and easy-to-move design.

Through our literary research, it was also noted that elderly people often experience more difficulties when learning something new. Because of this, it is assumed that using concepts that the elderly are already familiar with, or at least similar to those, is better as they will understand and learn them faster. Therefore, we should choose a game that is easy to understand and known by elderly people. As well as implement a simple interface and design.

Other aspects that could be added in order to improve the user experience, but are not necessary. Are the implementation of motivational messages during the game and multiple difficulty settings as a balance between ability and difficulty is important.

To engage the users’ more while playing with the robot, it is important that the robot has a competitive nature. Instead of having a robot that is relationship driven.

Deliverables:

The aim of this project is to design and build a card-game playing robot to provide social support in the form of entertainment for the increasingly ageing society. Therefore, it is the goal to deliver a prototype version of a card-game playing robot that satisfies some basic requirements that are necessary for the robot to be functional. The most important requirements are listed below.

1. The robot must have a strategy to play one specific card-game with equal performance as the average non-professional human player.
2. The robot must be able to successfully recognize and distinguish all the cards that make up a standard 52-card deck with an accuracy of at least 95%.
3. The robot must be embodied and it must be able to move the cards physically or have an integrated virtual environment by means of a multi-touch table.

In view of these requirements, the following components that make up the prototype will be delivered.

1. An algorithm that uses available information to select the next action or move.
2. An object recognition algorithm trained for a standard 52-card deck.
3. A physical implementation of the robot that is able to move cards or display them virtually.

The physical prototype will then be created by combining these components into one system. All progress of the project will be documented on this wiki, which will serve as the group report at the end of this project. Furthermore, a final presentation will be given at the end of the project combined with a demonstration of the prototype.

Approach:

Graphical representation of the steps of the design process.

During the project, the prototype will be developed by means of a design process. This process consists of multiple phases. Firstly, the problem will be defined as well as the design goal to solve the problem. Secondly, the functional and technical requirements of the prototype will be specified taking into account time, money and resources. From these requirements, it is possible to create concepts for the design. Next, details will be added to the design and the design will be realized. Lastly, tests will be performed to see if the prototype satisfies the requirements. This process can be summarized in the following steps:

1. Define problem
2. Specify requirements
3. Preliminary design
4. Detailed design
5. Prototyping
6. Testing

Given that the final presentations will be planned in week 8 of the course, this process will have a time period of seven weeks. Therefore, every step is expected to be completed within one week, with the exception of the detailed design since this is the most important and difficult step which will therefore require more time. So the time will be managed as follows, step 1 will be completed in week 1, step 2 in week 2, step 3 in week 3, step 4 in weeks 4 and 5, step 5 in week 6 and step 6 in week 7. In this way, all deliverables are finished in week 8.

Another important aspect of the design process is communication. Therefore, a weekly meeting is scheduled to discuss the progress and task division as well as a weekly work session. The meetings and work sessions will be scheduled on Wednesday from 10:30-11:30 and Monday 11:30-12:30 respectively.

Milestones:

The project milestones are both related to the steps of the design process and deliverables in that they break down the project into smaller sections for the tasks that need to be completed. They can be defined as follows:

1. Have a problem definition that clearly states the broader issue and the targeted user as well as the design goal
2. Have created a list of requirements for both functional and technical requirements according to the MoSCoW method
3. Have created an object recognition algorithm to identify a standard 52-card deck
4. Have created an algorithm that implements a strategy to select actions for one specific card game
5. The algorithm has a notion of 'fun' which dictates which moves should be picked (such as to maintain some win/loss ratio)
6. The game can be launched and played without any technical knowledge or elderly-unfriendly UI
9. The system satisfies the requirements as shown by means of a user test
10. (Optional) The project knows valid strategies for 3 popular card games
11. (Optional) The project can classify non-standard cards (like those in Uno)
12. (Optional) Have created a 3D design of the robot's design ready to be built and integrated with the program

Each milestone is planned to be achieved in a logical order to each step of the design process as shown by the following table.

Week	Step	Planned Milestones
1	Define problem	1
2	Specify requirements	2
3	Preliminary design	3
4	Detailed design	4
5	Detailed design	5
6	Detailed design	6
7	Testing	7, 8, 9, 10

Specified Requirements:

Functional MoSCoW table for the project:

Must have (project won’t function without these)	Ability to classify regular playing cards (95%) in a live camera feed	Have a valid strategy at playing the game	Be understandable and interactable by elderly people	Give general commands to a phantom robot to play the game
Should have (improves the quality of the project)	Ability to track moving/obscured/out-of-frame cards	Strategy should optimize user fun	Communicate intent with the user; display/audio
Could have (non-priority nice to have features)	Have a self-contained user interface window to play in	Know strategies for multiple popular card games	Ability to classify non-standard cards from other games	A robot design ready to be built and integrated with the program	Robust ability to keep playing after user (accidentally) cheats
Won’t have (indicates maximum scope of the project)	No physical device except for the camera

Technical MoSCoW table for the project:

Concrete requirements for the functional entries above.

Must have (project won’t function without these)	At least 95% accuracy when classifying any playing card (suit and number)	Must be able to decide on a valid move within <=3 seconds	The UI is designed with proven methods to ease interactivity for the elderly	The code must be compatible with some robot movement interface
Should have (improves the quality of the project)	The project should remember all cards it was confident it saw (95%) during a match as if they are in play	The project should have a notion of a ‘fun’ level and adjust its moves based on this (combined with the selected difficulty)	The project should communicate clearly at any time (English & Dutch) what it is doing at the moment; thinking, moving, waiting for the user’s turn
Could have (non-priority nice to have features)	A game can be launched & played without the use of a terminal, inside a browser-tab or separate window	Know valid strategies for at least 3 popular card games	Ability to classify (95% accuracy) non-standard cards from other games (like Uno)	A 3D robot CAD design ready to be built and integrated with the program without much extra work	After noticing an (unexpected) state-change of the game the project should continue playing, not try to correct the user
Won’t have (indicates maximum scope of the project)	No physical device except for the camera

User fun

In designing a card-game robot, it is important to consider how the card-game playing robot will deal with the difficulty level of the game. It is given that the robot has perfect memory and with an effective game playing strategy, the robot can exceed the average human player. However, the design is centered around the user whose most important aspect of playing card games is pleasure. The user’s pleasure may vary depending on the difficulty level of the game. This necessitates the development of a difficulty model to dynamically adjust the difficulty level of the game playing strategy according to the user in order to maximise user fun. To develop the difficulty model, it will first be defined what user fun is and how fun is often quantified. Next, it will be explored how user fun can be measured. Lastly, it will be argued what the most important measurement technique is for this application based on several requirements.

According to the article “Assessment of fun in interactive systems”, there are three different aspects of fun, namely interaction, immersion and emotion. Firstly, interaction is defined as the phenomenon of mutual influence. Humans interact with the environment through the six senses. However, they tend to limit their attention more to the precepts that are most in line with the internal goals. Whenever limiting attention is effortless for humans, it is called flow. At this moment, humans experience enjoyment due to the feeling of being in control. Secondly, immersion is closely related to flow since it describes to which degree a person may be involved with or focuses attention on the task at hand. The difference between immersion and flow is that the initial stages of immersion do not guarantee enjoyment, but are still required to achieve flow. Lastly, the subjectivity of fun is related to human emotions, such as those related to past experiences, preferences and current mood. Human emotions are made up of three hierarchical levels, namely the visceral, behavioural and reflective level. The visceral level is triggered by sensory perception resulting in physiological responses such as a change in heart rate, sweating or facial expressions. Next, the behavioural level involves the unconscious execution of routines. The emotions experienced in this level are related to the satisfaction of overcoming challenges. Lastly, the reflective level allows for conscious considerations in an attempt to control physical and mental bodily changes. Consequently, from the three different aspects of fun, it can be described as the feeling of being in control and overcoming challenges.

What is most important is in designing a difficulty model that maximises user fun, is how to measure fun. Following upon the previous discussion of the aspects of fun, it can be classified in two dimensions. The first dimension is valence which describes the extent to which an emotion is positive or negative. For example, joy has a high positive valence while sadness has a very negative valence. In contrast, the second dimension is often referred to as arousal, which refers to the intensity of the emotion or the energy felt. Anger is, for example, a state of high arousal while boredom is a state of low arousal.

In “The FaceReader: Measuring instant fun of use” there are mentioned two methods to measure emotions: non-verbal and verbal. Some of those methods can be automated as well, whereas others cannot or are harder to.

Non-verbal methods focus on aspects for which “words” are not needed. For example, using heart rate, skin conductivity or facial expressions to learn how someone is feeling. These processes can also be automated. Pros of non-verbal methods are that there is generally no bias, as well as that there is no language barrier, and they do not bother the users during a task.

Verbal methods, require some form of input of the user. This can be done through rating scales, questionnaires, or interviews. Some of it can be automated, through using standard questions. But it cannot be fully automated, as the users still have to give input themselves. One way, is that instead of doing it everytime a target user is playing a game. We can also do an expert analysis beforehand, in other words the developers test the product themselves and decide whether the way it currently works is fun. This method can be used to check whether the current assessment actually matches the expected outcomes. An advantage of the verbal methods is that it can be used to evaluate all emotions. Although, a disadvantage is, is that people fill in how they felt after the task and not during the task. So a difference between non-verbal and verbal methods is, is that one is good for noting how people experience the process, whereas the other is good for noting how people experienced the end result.

Code Repository

Github repository

Task Division & Logbook:

The logbook with member task and time spend per week can be found on the Logbook page.

The tasks will be divided weekly among the group members during the team meetings. The task division for each week is also shown on the Logbook page.

Literature Study:

The articles read for the literature study accompanied with their summary can be found in the Literature page.

Members

• Abel Brasse (1509128) - a.m.brasse@student.tue.nl
• Linda Geraets (1565834) - l.j.m.geraets@student.tue.nl
• Sander van der Leek (1564226) - s.j.m.v.d.leek@student.tue.nl
• Tom van Liempd (1544098) - t.g.c.v.liempd@student.tue.nl
• Luke van Dongen (1535242) - l.h.m.v.dongen@student.tue.nl
• Tom van Eemeren (1755595) - t.v.eemeren@student.tue.nl

Links

• User interview
• AI Object Detection
• Literature study articles
• Logbook of group12
• Task division

References

note: The references only from the literature study can be found on the literature study page.