PRE2017 3 Groep18

This page describes the group's effort at planning, logging and noting the meetings. Our product specific page can be found at Clairvoyance.

week 1

Meet the group members

Louis: Studied a year of Software Science in 2015. Switched to Psychology and Technology, now in second year of PT. I have some experience with Java, Python, CSS, Javascript, Arduino, Stata. French, but mostly studied in English so I can also help on documentation, writing. Decent at presenting.

Clara: Mechanical engineering Master. Can code C++, Python, Matlab, a tiny bit C. Worked on image recognition before. Know a bit about neural networks, Caffe, Datasets, Json.

Joëlle: Currently in the second year of Biomedical Engineering. I don’t have a lot of experience with programming, only in Python, but I’m very interested and would like to learn more.

Rens: Last year of Software science. Experience with low-level, high-level and webscale code. Also did some data science and machine learning for my job.

Bas: Second year Mechanical Engineering student. Lots of experience with practical group work. Since I'm a mechanical engineer, I know a lot about mechanics and dynamic, control of systems and some flow mechanics. I worked with arduino once, but I'm not too good at it. I can work with matlab pretty well.

Nosa Dielingen: Second year Electrical Engineer. I can program C and Arduino and a couple other languages. Furthermore building a circuit is almost second nature when having Google.

What has been done in week 1?

In week 1 all the teammates had to come with 5 ideas.

week 2

Two subjects have been chosen. A Smart Mirror which makes to user prettier and a house system that places all the furniture to their spot and is able to find objects that have been lost. At the end of week 2 one of the two subjects will be chosen.

Paper

In week 2 we have collected papers which can be used to gain knowledge about our subject. Below links to the papers can be found. This section of papers are for the House System Feng Shui Geomancy and the Environment in premodern Taiwan QuadCopter Dynamics [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]

This section of papers are for the smart mirror:

Dynamic Hair Manipulation in Images and Videos

This paper describes that high-resolution image manipulation is doable but that it is much harder to use in a video because of issues of depth and extravagant hairstyles. However simple video manipulation is doable.

Computer Vision

This book on 3D modeling explains how using multiple images from different viewpoints we can recreate images as well as know their depth. Useful for both the mirror idea or the home robot(furniture moving). Contains a lot of algorithms and extensive information about the different types of image/video reconstruction or alterations (600+ pages). This is more to be used in the future if we want to make it. Contains an extensive list of sources that can be relevant too.

Automating Image Morphing using Structural Similarity on a Halfway Domain

This article explains that using specific points in images and two distinct images we can create a morph from one image to another. This can be interesting when trying to make someone look different/better, by using a standard image and morphing slightly based on another image. Currently only works on images and thus not on video, but together with other research could be interesting.

I like those glasses on you, but not in the mirror: Fluency, preference, and virtual mirrors

A very relevant paper about different effects of a mirror in a shop. This article details the idea that people prefer their image in the mirror to their actual image and that acquaintances prefer how you look face to face over your reflection in the mirror. This is because someone who is familiar with you is more likely to have seen you from face to face rather than as a reflection in the mirror and thus the reflection seems unfamiliar and thus unusual.This is based on a previously done study [14].

Furthermore, it hits on how virtual mirrors already exist in different forms, but not in the way we want to use them. You can, for example, upload a picture of yourself to some makeup or glasses companies where they will return an image including how their product would look on you. Moreover, it claims that “neither virtual mirror technology itself nor its potential as a basic research tool has received much attention in consumer research.”. It focusses on fluency processing (how easily a visual cue can be processed by someone) and explains the different variables for fluency. It concludes that it increases aesthetical pleasure for the perceiver. It then goes on to explain that processing facilitation creates positive affect and activates smiling muscles [15]. Then conducts an experiment to test people prefer people they do or don’t know in the mirror versus face to face. The results favour seeing someone face to face with familiar people and are indistinguishable for either case with strangers.

Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression

State of the art AI technique for reconstructing facial meshes from 2D pictures (and videos). Does so by using volumetric CNNs. Can be used in our product to prevent having to use expensive depth cameras. When we have these meshes, we can transform them according to some beauty standard (for example make people's faces thinner, their eyes bigger, etc.)

Photorealistic Facial Texture Inference Using Deep Neural Networks

Generates facial textures from 2D images. These facial textures can be used for "photoshopping" purposes (we can egalize skin, remove blemishes, change skin and eye color (ethics!!)). After having performed our automatic photoshopping we can project the textures back onto our mesh (see above).

Snapchat patents

In 2015 Snapchat bought the company Looksery, which uses technology very related to our purposes.

METHOD FOR REAL TIME VIDEO PROCESSING INVOLVING CHANGING A COLOR OF AN OBJECT ON A HUMAN FACE IN A VIDEO

METHOD FOR REAL-TIME VIDEO PROCESSING INVOLVING CHANGING FEATURES OF AN OBJECT IN THE VIDEO

METHOD FOR REAL TIME VIDEO PROCESSING FOR CHANGING PROPORTIONS OF AN OBJECT IN THE VIDEO

First detects faces by using the Viola Jones algorithm. Then fits a face shape model to the faces (Active Shape Model). Then it transforms this model to the Candide-3 model, a popular low-computation face mask/mesh. Which then can be used for actually transforming the faces.

[16] [17] [18] [19] [20] [21] [22] [23] [24] [25]

planning

A planning will be made at the end of the week when a subject is chosen.

week 4

planning

Rens

Waiting for Clara to finish the RPC’s in order to figure out the technical (research) side of the app. Found a website for generating a 3D model from a single picture. Will update the product wiki with research and will write a demo/prototype/wizard-of-oz description. Will work on prototype after tutor meeting.

Nosa

Will update wiki with existing apps/products and writes the introduction.

Joëlle

Did research on ethics & laws

Coming week: updating the wiki on ethics & laws and making it up to date

Bas

Is going to work on 3D hair.

Louis

Created user-survey

Clara

FILL IN

Questions for tutor

How ‘’wizard-of-oz” does it have to be? Do we need to create a real fake

Wiki page for the app structure

• Introduction
• Existing work
  • Research
  • Existing apps/products
• User Survey
• Requirements Preferences Constraints
• Prototype Description (Wizard of Oz)
• Laws and Ethics

week 5

test

week 6

test

week 7

test

week 8

test

Coaching

Coaching Questions Group 18