Talk:PRE2015 4 Groep2

Minutes

Meeting 20--04

Goal: Create a demo in which a robotic system is able to detect ripe strawberries and harvest them effectively.

Subsystems: Robotic arm Machine Vision/Learning

Requirements for presentation: -USE needs (Yannick + Raomi Team Awesome) -Scientific literature -> EU projects -> How to go beyond? (Cameron) -Requirements --Moving to A to B along a fixed axis --Cutting fruit --Sensing for ripeness --Ambient sensing --Locating fruit --Collecting/Handling fruit effectively --Feedback from farmer to system

-Idea/Solution to the problem

--Moving to A to B along a fixed axis Fixed railing Cable Treads for conveying 4-wheel/2-wheel drive Yo-yo

--Cutting fruit Scissors Custom cutting mechanism Laser cutting Sharp knife (fruit ninja)

--Sensing for ripeness Kinect Color sensor pH sensor Force sensor Machine learning ripeness (with Kinect using training set of ripe fruit)

--Ambient sensing Temperature sensor Humidity Air pressure Sunlight exposure

--Locating fruit Kinect for depth map Probe for fruit (use color sensor to identify where fruit is located) Touch sense for fruit (and then use color sensor) Fixed location for fruit

--Collecting/Handling fruit effectively Basket collection Soft packaging Grip from stem

--Feedback from farmer to system

-Plan of approach Divide task into sub-groups: -Building the robotic arm (mechanical) (TB) -Machine Vision/Learning (software) (TC) -Control of the robotic arm (software and electronics) (TBC) -Feedback from farmer (software) (TA) -USE aspects (TA)

Key: TA: Yannick & Raomi TB: Mark & Maarten TC: Cameron and Birgit

Deadline Week1: Defining project plan and timeline Specify USE aspects and identify multiple solutions Elucidate requirements Compile Literature Create a presentation

Deadline Week2: First presentation Parts list (BOM) Order parts Begin drawings/concepts

Deadline Week3: Beginning of meetings Mark abandons us Consolidate drawings

Deadline Week4: Separately working components Working base for movement from A to B

Deadline Week5: Build the arm

Deadline Week6: Control of arm

Deadline Week7: Integration of subsystems

Deadline Week8: Testing of system Deadline Week9: Phase-out period

Meeting 25-04

Idea in steps:

Problem description:

Less people needed to work on farms, they can work in other fields (like help elderly)

Facts and figures!

Move away from 3 group structure, no The whole group working on the use aspect first? The groups should not be too large,

We should first redesign the whole system with the focus of the way the USE aspects interact and change the design (as seen from a purely technical perspective)

We should specify more requirements with respect to the USE aspects, not as much from the technical side.

Users: -Primary: -Farmers: -Higher efficiency -More harvest -Lower costs due to less employees -They will have more time -Less physical work / Health benefits -Don’t have to train seasonal workers

-Secondary: -Supermarkets, Distributors: -Higher quality food -Lower cost for food -There will be more fresh food available(maybe?) -More efficient supply chain -The supermarkets might also be able to give feedback. -Tertiary: -Company that creates the robots and maintains them: -

Society -There will not be enough food in the near future for all the people. -Not enough workers, due to aging. -Decrease in wage gap due to overabundance of food -Post scarcity

Enterprise: -Large farms will start to dominate the market which will result in an increase in the gap between rich and poor. -Lower worker costs -Lower food prices might result in more sales

Results from USE -The supermarket might want to have a say in which fruits are ripe. -Direct and faster communication -We can better specify the amount of food that we need. -There is a lot of food waste, even due to food not looking good. -Robots can determine what the best use for fruits is based on many factors (e.g. looks) -Farmer feedback: -Via his/her smartphone. -Online database for machine learning -Many farmers can have access -> lots of pictures means high accuracy -Farmers and secondary users should have a separate application -Farmer should have more control (e.g. stop button) -Greenhouses -Robots save space -Verticality -More plants -Can work 24/7

Society idea:

-every buyer: can indicate which fruit it wants and how many at which time. -every farmer: can indicate which fruits it can deliver at what time.

Raomi: Society part (needs with h2020) Birgit: identify supply chain Cameron: technical plan Yannick: planning Maarten: User part Mark: Enterprise part

Meeting 28-04

How it's done right now (researched by Birgit)
Supply chain state of the art: what they do now is sell not-nice-looking food to farmers for animals (not thrown away). There exists a machine that sorts potatoes on size, ugliness, etc., but probably not findable online (no research done on it).

User part: not done, put on Drive AP.

Society: found on H2020 (expand..), no figures and facts there.

Enterprise: not done, put on Drive AP.

Idea:
- 1 system with Kinect, Raspberry Pi, … to classify fruit. Connects to database for classifying fruit.
- Encasing with custom lighting for equal light situations.
- Cloud database for the pictures with machine learning.

Problem: images taken by different types of camera's with different color. Possible solution: one type of tablet. Other possible solution: only use Kinect for sensing (something else for interface). Thus we have only 1 system. This solution is chosen.

Teams:
App team: UI/UX, app, interface, Raomi, Birgit, Yannick.
Database/backend/Kinect: machine learning, neural networks. Cameron, Maarten, Mark.

Ethical part
- How broad, enterprise scale/impact analyzation? AP ask - During the project make sure that the user is taken into consideration (in design).

[1]

Meeting 02-05

Focus Isolated fruit detection in controlled lighting

Tasks
Learning about CNN and basics, implement a non-fruit NN (look into TensorFlow) - MV, CW, MdJ
Read literature about previously implemented CNNs - MV, CW, MdJ
Basic CNN, demonstrate fruit detection- CW
Create a basic API to POST images and catch the response - MV
Screenflow for App - Raomi
USE aspects of App, why does app help the solution? - YA, RvR
Basic visuals of app with walkthrough - BvdS, YA
Look into Mendix - BvdS

Find a U, S, and E to interview about automation in their process/harvesting as well as our solution and how it impacts their sector. -MV ,U(E), BvdS (U,S)

Meeting 09-05

Updated Planning:
-Before thursday 12-05

Maarten: Find local farmer

How we will define color?

Which lighting conditions do we want?

Mark: Why CNN? (compared with for example support vector machine)

Yannick: Powerpoint of app

Birgit: First version of the app

LED light ring

-Before Thursday 19-05

Cameron, Maarten, Mark: Gathering database with rasberry pi and kinect

Farmer: [2]

Farmer: Needed to assure the use of our product

-Week 3

Preliminary design for app

First implementation of app

Database/Server setup

CNN, and basics of neural networks

-Week 4

App v1.0 with design fully implemented

Kinect interfacing to Raspberry Pi completed

USEing intensifies

Finish back-end design and choose frameworks

-Week 5

App v2.0 with design fully implemented and tested

Further training of CNN

Working database classification (basic)

Casing (with studio lighting LED shining on fruit)