PRE2023 3 Group3: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
Line 208: Line 208:
|}
|}


=== Sensors Required For Movement ===
=== Sensors Required For Navigation, Movement and Positioning ===
Sensors are a fundamental component of any robot that is required to interact with its environment, as they aim to replicate our sensory organs which allow us to perceive and better understand the environment around us<ref>https://www.wevolver.com/article/sensors-in-robotics-the-common-types</ref>. However, unlike with living organisms, engineers are given the choice to decide what exact sensors their robot needs and must be careful with this decision in order to pick the sufficient options to be able to allow the robot to have its full functionality without picking any redundant options that will make the robot unnecessarily expensive. This decision is often based on researching and considering all possible sensors that are available on the market which are related to the problem the engineer is trying to solve and selecting the one which fulfills the requirements of the the robot most accurately<ref>[https://www.sciencedirect.com/science/article/abs/pii/S0079642500000116#:~:text=The%20selection%20of%20an%20appropriate,as%20cost%2C%20and%20impedance%20matching. https://www.sciencedirect.com/science/article/abs/pii/S0079642500000116#:~:text=The%20selection%20of%20an%20appropriate,as%20cost%2C%20and%20impedance%20matching.]</ref>. In this section we will specifically be looking into sensors which will aid our robot in traversing our environment, a garden. This means that we must consider the fact that the sensors we select must be able to work in environments where the lighting level is constantly changing as well as possible mis inputs due to high winds and or uneven terrain. Additionally, it is important to note that unlike the discussion in the previous section, one type of sensor/system is rarely sufficient to fulfill the requirements and most robots must implement some form of sensor fusion in order to operate appropriately, this is no different in our robot<ref>[https://www.sciencedirect.com/topics/engineering/sensor-fusion#:~:text=Sensor%20fusion%20is%20the%20process,navigate%20and%20behave%20more%20successfully. https://www.sciencedirect.com/topics/engineering/sensor-fusion#:~:text=Sensor%20fusion%20is%20the%20process,navigate%20and%20behave%20more%20successfully.]</ref>.
 
==== LIDAR sensors ====
[[File:LIDAR.jpg|thumb|337x337px|Lidar Sensor in automotive industry]]
LIDAR stands for Light Detection and Ranging. These types of sensors allow robots which utilize them to effectively navigate the environment they are placed in as they provide the robot with object perception, object identification and collision avoidance<ref>[https://www.mapix.com/lidar-applications/lidar-robotics/#:~:text=LiDAR%20(Light%20Detection%20and%20Ranging,doors%2C%20people%20and%20other%20objects. https://www.mapix.com/lidar-applications/lidar-robotics/#:~:text=LiDAR%20(Light%20Detection%20and%20Ranging,doors%2C%20people%20and%20other%20objects.]</ref>. These sensors function through sending lasers into the environment and then calculating how long it takes them to return back to the receiver to determine their distance and shape. As can be seen, LIDAR’s provide robots with a vast amount of crucial information and even allow them to see the world in a 3D perspective. This means that not only are robots able to see their closest object, they can instantaneously derive possible methods of avoidance and to traverse around it<ref>(Jie Shan and Charles K. Toth (2018). Topographic Laser Ranging and Scanning: Principles and Processing (2nd ed.). CRC Press.)</ref>.
 
LIDAR’s are often the preferred option by engineers in robots that operate outdoors as they are minimally influenced by weather conditions<ref>[https://intapi.sciendo.com/pdf/10.2478/agriceng-2023-0009#:~:text=The%20LiDAR%20sensor%20allows%20the,which%20allows%20clustering%20and%20positioning. https://intapi.sciendo.com/pdf/10.2478/agriceng-2023-0009#:~:text=The%20LiDAR%20sensor%20allows%20the,which%20allows%20clustering%20and%20positioning.]</ref>. Often sensors rely on visual imaging or sound sensors which both get heavily disturbed in more difficult weather conditions, whether that is rain on a camera lens or the sound of rain disturbing sound sensors. However, an issue that our robot is likely to face when utilizing a LIDAR sensor is that of sunlight contamination<ref>https://opg.optica.org/oe/fulltext.cfm?uri=oe-24-12-12949&id=344314</ref>. Sunlight contamination is the effect the sun has on generating noise in the sensor’s data during the daytime and therefore possibly introducing errors within it. Since our robot needs to work optimally during the daytime it is crucial that this is considered. However, the LIDAR possesses many additionally positive aspects that would be truly beneficial to our robot such as the ability to function in complete darkness and immediate data retrieval. This would allow the users of our robot to turn on the robot before they go to sleep and wake up to a complete report of their garden status. Furthermore, these features are necessary for the robot as they would allow it to work in a dynamic and constantly changing environment, which is of high importance to where our robot wants to operate in, the garden. The outdoors can never be a fully controlled environment and that has to be considered into the design of the robot.
 
As can be seen the LIDAR sensor has many excellent features that our robot will likely require, therefore it is a very important candidate when making our next design decisions.
 
==== Boundary Wire ====
[[File:Boundary wire.jpg|thumb|Boundary Wire being placed by user]]
A boundary wire is likely the most cost efficient and commonly implemented technique in state-of-the-art garden robots that are on the private consumer market today. It is not a complicated technology but still a very effective one when it comes to robot navigation. A boundary wire in the garden acts as a virtual barrier that the robot cannot cross, similar to a geo-cage in drone operation<ref>https://www.thalesgroup.com/en/markets/aerospace/drone-solutions/scaleflyt-geocaging-safe-and-secure-long-range-drone-operations</ref>. In order to begin utilizing the robot the user must first lay out the wire on the boundaries of their garden and then dig them approximately 10 cm below the ground's surface. This is a tedious task for the user but has to only be completed once and the robot is now fully operational and will never leave the boundaries set by the user.
 
The robot communicates with the robot by emitting a low voltage, around 24V, signal which is picked up by a sensor on the robot<ref>https://www.robomow.com/blog/detail/boundary-wire-vs-grass-sensors-for-robotic-mowers</ref>. This means that when the robot detects the wire it knows not to continue moving in its direction. As is displayed above, the boundary wire is a very simple technology which with a slight amount of effort of the user can perform the basic navigability tasks. However, its functionality is fairly limited, it cannot detect any objects within the area of its operation and therefore avoid them meaning that its environment has to be maintained and clear throughout its operation.
 
==== GPS/GNSS ====
[[File:GNSS.jpg|thumb|GNSS operation depiction]]
GPS/GNSS are groups of satellites deployed in space that allow robots and devices to receive signals from them which aid them in positioning. Over the past few years these systems have gotten extremely accurate and can position devices to the meter<ref>[https://www.ardusimple.com/rtk-explained/#:~:text=Introduction%20to%20centimeter%20level%20GPS%2FGNSS&text=Under%20perfect%20conditions%2C%20the%20best,accuracy%20of%20around%202%20meters. https://www.ardusimple.com/rtk-explained/#:~:text=Introduction%20to%20centimeter%20level%20GPS%2FGNSS&text=Under%20perfect%20conditions%2C%20the%20best,accuracy%20of%20around%202%20meters.]</ref>. This happens through a process called triangulation, where multiple satellites calculate their distance to the device and establish its location<ref>[https://first-tf.com/general-public-schools/how-it-works/gps/#:~:text=GNSS%20positioning%20is%20based%20on,each%20of%20the%20visible%20satellites. https://first-tf.com/general-public-schools/how-it-works/gps/#:~:text=GNSS%20positioning%20is%20based%20on,each%20of%20the%20visible%20satellites.]</ref>. The usage of this sensor in our robot is very encouraging as it has been proven to be effective in the large scale gardening industry for many years. As can be seen in many robots in the precision farming domain<ref>https://therobotmower.co.uk/2021/12/02/robot-mowers-without-a-perimeter-wire/</ref>. If equipped with a sensor that can communicate with satellites and fetch its location at all times, our robot will be able to precisely ping the location it has found sick plants or any plants that need care and send that information to the users device. Once again promising to be a very key component in our robot design.


== Work Distribution ==
== Work Distribution ==
Line 492: Line 511:
|Briana Isaila
|Briana Isaila
|Meeting with tutors (0.5h)
|Meeting with tutors (0.5h)
|
|-
|Raul Hernandez Lopez
|
|
|-
|Ilie Rareş Alexandru
|
|
|-
|Ania Barbulescu
|
|
|-
| rowspan="6" |Week 4
|Patryk Stefanski
|
|
|-
|Raul Sanchez Flores
|
|
|-
|Briana Isaila
|
|
|
|-
|-

Revision as of 20:49, 8 March 2024

Group members

Name Student ID Current Study Programme
Patryk Stefanski 1830872 CSE
Raul Sanchez Flores 1844512 CSE
Briana Isaila 1785923 CSE
Raul Hernandez Lopez 1833197 CSE
Ilie Rareş Alexandru 1805975 CSE
Ania Barbulescu 1823612 CSE

Problem statement

[1] Manchester University survey on why people don't garden.

In western society, having a family, a house, and a good job are what many people aspire to have. As people strive to achieve such aspirations, their spending power increases, which allows them to be able to afford to buy a nice home for their future family, with a nice garden for the kids and pets. However, as with many things, in our capitalist world, this usually comes at a sacrifice: free time. According to a study conducted by a student team at Manchester University [2], the three main reasons why people don't garden which made up 60% of the survey responses included time constraints, lack of knowledge/information, and space restraints. Gardening should be encouraged, due to its environmental benefits and many other advantages[3].

In the past decade, robotics has been advancing across multiple fields rapidly as tedious and difficult tasks become increasingly automated [4], this is not any different in the field of agriculture and gardening [5]. As the world's population continues to rapidly increase, there are increasingly many people to feed, putting pressure on farmers to optimise their crop growing and production. In recent years, many robots have become available that aid farmers in important aspects such as irrigation, plantation, and weeding. These robots are large mechanical structures sold at a very high price meaning their only true usage is in large-scale farming operations. Unfortunately, one common user group has been left behind and not considered when developing this new technology in gardening and agriculture, the amateur gardener. Amateur gardeners, often lacking in-depth knowledge about plants and gardening practices, face challenges in maintaining their gardens. Identifying issues with specific plants, understanding their individual needs, and implementing corrective measures can be overwhelming for their limited expertise. It is no surprise that traditional gardening tools and resources often fall short in providing the necessary guidance for optimal plant care, so another solution must be found. This is the problem that our team's robot will be the solution to.

Objectives

The objectives for the project that we hope to accomplish throughout the 8 weeks that are given to us are the following:

  • The robot should be able to cut grass.
  • The app which controls the robot should be user-friendly and fulfil all the robot's requirements
  • The robot should be able to map out the garden's terrain and location of plants growing in it.
  • The robot should be able to identify the type of a plant and if it is healthy
  • The robot should be able to recommend specific actions after spotting a disease/infestation
  • The app which control the robot should display the location of a unhealthy plant and recommended actions to take care of it.

Users

Who are the users?

The users of the product are garden-owners who need assistance in monitoring and maintaining their garden. This could be due to the fact that the users do not have required knowledge to properly maintain all different types of plants in their garden, or would prefer a quick and easy set of instruction of what to do with each unhealthy plant and where that plant is located. This would optimise the users routine of gardening without taking away the joy and passion that inspired the user to invest into plants in their garden in the first place.

What do the users require?

The users require a robot which is easy to operate and does not need unnecessary maintenance and setup. The robot should be easily controllable through a user interface that is tailored to the users needs and that displays all required information to the user in a clear and concise way. The user also requires that the robot may effectively map their garden and identify where certain plants are located. Lastly, the user requires that the robot is able to accurately describe what actions must be taken, if any are necessary, for a specific plant at a specific location in the garden.

Deliverables

  • Research into AI plant detection mapping a garden and best ways of manoeuvring through it.
  • Research into AI identifying plant diseases and infestations.
  • Survey confirming that the problem we have selected to solve is a solution users desire.
  • Interactive UI of an app that will allow the user to control the robot remotely that implements the user requirements that we will obtain from the survey. The UI will be able to be run on a phone and all its features will be able to be accessed through a mobile application.
  • Interview with a specialist in biology or AI
  • This wiki page which will document the progress of the group's work, decisions that have been made, and results we obtained.

State of Art

Garden Planning applications[6]

The current state of the art presents numerous mobile applications to deal with planning what to grow in a garden. This can help the user to keep track of what plants are planted and where.

  • Robots such as roombas, and automated grass-cutting robots.
  • Existing plant detection AI systems

Regular Robot Operation

As with any piece of technology it is important that the users are aware of its proper operation method and how the robot functions is general. It is important that this is clear for our robot as well. Upon the robot's first use in a new garden or when the garden owner has made some changes to the garden layout, the mapping process must be instantiated in the app. This mapping will be a 2D map of the garden which will then later allow the robot to efficiently traverse the entire garden during its regular operation without leaving any part of the garden unvisited. In order to better understand this feature one, can compare it to the iRobot Roomba. After the initial setup phase has been completed the robot will be able to begin its normal operation. Normal operation includes the robot being let out into the garden from its storage place, and traversing through the garden cutting grass while its camera scans the plants in its surroundings. Whenever the robot detects an irregularity in one of the plants, it will notify the user through the user of the app, where the robot will send over a picture of the plant with an issue as well as its location on the map of the garden. The user will then able to navigate in the app to view all plants that need to be taken care of in his garden. This means that not only will the user have a lawn which is well kept but also be aware of all unhealthy plants keeping the users garden in optimal condition at all times.

Robots-1 (1).jpg

Maneuvering: Patryk

Movement

One of the most important design decisions when creating a robot or machine with some form of mobility is deciding what mechanism the robot will use to traverse its operational environment. This decision is not always easy as many options exist which have their unique pros and cons. Therefore is is important to consider the pros and cons of all methods and then decide which method is most appropriate for a given scenario. In the following section I will explore these different methods and see which are expected to be most beneficial and work the best in the task environment our robot will be required to function in.

Wheeled Robots

It may be no surprise that the most popular method for movement within the robot industry is still a robot with circular wheels. This is due to the fact that robots with wheels are simply much easier to design and model[7]. They do not require complex mechanism of flexing or rotating a actuator but can be fully functional by simply altering rotating a motor in one of two directions. Essentially they allow the engineer to focus on the main functionality of the robot without having to worry about the many complexities that could arise with other movement mechanisms when that is not necessary. Wheeled robots are also convenient in design as they rarely take up a lot of space in the robot. Furthermore, as stated by Zedde and Yao from the University of Wagenigen, these types of robots are most often used in industry due to their simple operation and simple design.[2] Although wheeled robots seem as a single simple category there are a few subcategories of this movement mechanism that are important to distinguish as they each have their benefits and issues they face.

Differential Drive
Differential Drive Robot Functionality

Differential drive focuses on independent rotation of all wheels on the robot. Essentially one could say that each wheel has its own functionality and operates independently of the other wheels present on the robot. Although rotation is independent it is important to note that all wheels on the robot work as one unit to optimize turning and movement. The robot does this by varying the relative speed of rotation of its wheels which allow the robot to move in any direction without an additional steering mechanism. [3] order to better illustrate this idea consider the following scenario - suppose a robot wants to turn sharp left, the left wheels would become idle and the right wheel would rotate at maximum speed. As can be seen both wheels are rotating independently but are doing so to reach the same movement goal.

Differential Drive
Pros Cons
Easy to design Difficulty in straight line motion on uneven terrains
Cost-effective Wheel skidding can completely mess up algorithm and confuse the robot of its location
Easy maneiveribility Sensitive to weight distribution - big issue with moving water in container
Robust - less prone to mechanical failures
Easy control
Omni Directional Wheels
Omni Wheel produced by Rotacaster

Omni-directional wheels are a specialized type of wheel designed with rollers or casters set at angles around their circumference. This specific configuration allows a robot which has these wheels to easily move in any direction, whether this is lateral, diagonal, or rotational motion.[4] By allowing each wheel to rotate independently and move at any angle, these wheels provide great agility and precision, which makes this method ideal for applications which require navigation and precise positioning. The main difference between this method and differential drive is the fact that omni directional wheels are able to move in any direction easily and do not require turning of the whole robot when that is not necessary due to their specially designed roller on each wheel.

Omni Directional Wheels
Pros Cons
Allows complex movement patterns Complex design and implementation
Superior maneuverability in any direction Limited load-bearing capacity
Efficient rotation and lateral movement Higher manufacturing costs
Ideal for tight spaces and precision tasks Reduced traction on uneven terrains
Enhanced agility and flexibility Susceptible to damage in rugged environments

Legged Robots

Legged robot traversing a terrain

Over millions of year organisms have evolved in thousands of different ways, giving rise to many different methods of brain functioning, how an organisms perceives the world and what is important in our current discussion, movement. It is no coincidence that many land animals have evolved to have some form of legs to traverse their habitats, it is simply a very effective method which allows a lot of versatility and adaptability to any obstacle or problem an animal might face.[5] This is no different when discussing the use of legged robots, legs provide superior functionality to many other movement mechanisms due to the fact that they are able to rotate and operate freely in all axis's. However, with great mobility comes the great cost of their very difficult design, a design with which top institutions and companies struggle with to this day.[6]

Legged Robots
Pros Cons
Versatility in Terrain Complexity in Design
Obstacle Negotiation Power Consumption
Stability on Uneven Ground Sensitivity to Environmental Changes
Human-Like Interaction Limited Speed
Efficiency in Locomotion Maintenance Challenges

Tracked Robots

Tracked robots used for navigating rough terrain

Tracked robots, which can be characterized by their continuous track systems, offer a dependable method of traversing a terrain that can be found in applications across various industries. The continuous tracks, consisting of connected links, are looped around wheels or sprockets, providing a continuous band that allows for effective and reliable movement on many different surfaces, terrains and obstacles.[7] It is therefore no surprise that their most well known usages include vehicles which operate in uneven and unpredictable, such as tanks. Since tracks are flexible it is even common that such robots can simply avoid small obstacles by driving over them without experiencing any issues. This is particularly favorable for the robot we are designing as naturally gardens are never perfectly flat surfaces often littered by many naturally cause obstacles such as stone, dents in the surface or even possibly branches that have fallen on the ground due to rough wind.

Tracked Robots
Pros of Tracked Robots Cons of Tracked Robots
Superior Stability Complex Mechanical Design
Effective Traction Limited Maneuverability
Versatility in Terrain Terrain Alteration
High Payload Capacity Increased Power Consumption
Efficient Over Obstacles
Consistent Speed

Hovering/Flying Robots

Flying robot in action

Hovering/Flying robots provide without a doubt the most unique way of movement from the previously listed. This method unlocks a whole new wide range of possibilities as the robot no longer has to consider on-ground obstacles; whether that is rocks or uneven terrain. This method also unlocks the possibility of the robot to optimize its movement distance as it is able to move from point A to point B directly in a straight line. However, as is the case with any solution, flying/hovering has its major problems. It is by far the most expensive method, as flying apparatus is far more costly and high maintenance than any other solution. This makes this unreliable and likely a method far out of the technological needs and requirements of our gardening robot.

Hovering/Flying Robots
Pros Cons
Versatile Aerial Mobility Limited Payload Capacity
Rapid Deployment Limited Endurance
Remote Sensing Susceptibility to Weather
Reduced Ground Impact Regulatory Restrictions
Dynamic Surveillance Security and Privacy Concerns
Efficient Data Collection Initial Cost and Maintenance

Sensors Required For Navigation, Movement and Positioning

Sensors are a fundamental component of any robot that is required to interact with its environment, as they aim to replicate our sensory organs which allow us to perceive and better understand the environment around us[8]. However, unlike with living organisms, engineers are given the choice to decide what exact sensors their robot needs and must be careful with this decision in order to pick the sufficient options to be able to allow the robot to have its full functionality without picking any redundant options that will make the robot unnecessarily expensive. This decision is often based on researching and considering all possible sensors that are available on the market which are related to the problem the engineer is trying to solve and selecting the one which fulfills the requirements of the the robot most accurately[9]. In this section we will specifically be looking into sensors which will aid our robot in traversing our environment, a garden. This means that we must consider the fact that the sensors we select must be able to work in environments where the lighting level is constantly changing as well as possible mis inputs due to high winds and or uneven terrain. Additionally, it is important to note that unlike the discussion in the previous section, one type of sensor/system is rarely sufficient to fulfill the requirements and most robots must implement some form of sensor fusion in order to operate appropriately, this is no different in our robot[10].

LIDAR sensors

Lidar Sensor in automotive industry

LIDAR stands for Light Detection and Ranging. These types of sensors allow robots which utilize them to effectively navigate the environment they are placed in as they provide the robot with object perception, object identification and collision avoidance[11]. These sensors function through sending lasers into the environment and then calculating how long it takes them to return back to the receiver to determine their distance and shape. As can be seen, LIDAR’s provide robots with a vast amount of crucial information and even allow them to see the world in a 3D perspective. This means that not only are robots able to see their closest object, they can instantaneously derive possible methods of avoidance and to traverse around it[12].

LIDAR’s are often the preferred option by engineers in robots that operate outdoors as they are minimally influenced by weather conditions[13]. Often sensors rely on visual imaging or sound sensors which both get heavily disturbed in more difficult weather conditions, whether that is rain on a camera lens or the sound of rain disturbing sound sensors. However, an issue that our robot is likely to face when utilizing a LIDAR sensor is that of sunlight contamination[14]. Sunlight contamination is the effect the sun has on generating noise in the sensor’s data during the daytime and therefore possibly introducing errors within it. Since our robot needs to work optimally during the daytime it is crucial that this is considered. However, the LIDAR possesses many additionally positive aspects that would be truly beneficial to our robot such as the ability to function in complete darkness and immediate data retrieval. This would allow the users of our robot to turn on the robot before they go to sleep and wake up to a complete report of their garden status. Furthermore, these features are necessary for the robot as they would allow it to work in a dynamic and constantly changing environment, which is of high importance to where our robot wants to operate in, the garden. The outdoors can never be a fully controlled environment and that has to be considered into the design of the robot.

As can be seen the LIDAR sensor has many excellent features that our robot will likely require, therefore it is a very important candidate when making our next design decisions.

Boundary Wire

Boundary Wire being placed by user

A boundary wire is likely the most cost efficient and commonly implemented technique in state-of-the-art garden robots that are on the private consumer market today. It is not a complicated technology but still a very effective one when it comes to robot navigation. A boundary wire in the garden acts as a virtual barrier that the robot cannot cross, similar to a geo-cage in drone operation[15]. In order to begin utilizing the robot the user must first lay out the wire on the boundaries of their garden and then dig them approximately 10 cm below the ground's surface. This is a tedious task for the user but has to only be completed once and the robot is now fully operational and will never leave the boundaries set by the user.

The robot communicates with the robot by emitting a low voltage, around 24V, signal which is picked up by a sensor on the robot[16]. This means that when the robot detects the wire it knows not to continue moving in its direction. As is displayed above, the boundary wire is a very simple technology which with a slight amount of effort of the user can perform the basic navigability tasks. However, its functionality is fairly limited, it cannot detect any objects within the area of its operation and therefore avoid them meaning that its environment has to be maintained and clear throughout its operation.

GPS/GNSS

GNSS operation depiction

GPS/GNSS are groups of satellites deployed in space that allow robots and devices to receive signals from them which aid them in positioning. Over the past few years these systems have gotten extremely accurate and can position devices to the meter[17]. This happens through a process called triangulation, where multiple satellites calculate their distance to the device and establish its location[18]. The usage of this sensor in our robot is very encouraging as it has been proven to be effective in the large scale gardening industry for many years. As can be seen in many robots in the precision farming domain[19]. If equipped with a sensor that can communicate with satellites and fetch its location at all times, our robot will be able to precisely ping the location it has found sick plants or any plants that need care and send that information to the users device. Once again promising to be a very key component in our robot design.

Work Distribution

Navigation Algorithm and Netlogo Implementation: Patryk

Week Task Deliverable to meeting
2 Define Deliverables and brainstorm new idea Completed wiki with new idea
3 Write survey questions about garden maintenance robot, research methods to maneuver in an area, specifically in a garden. Wiki
4 Send survey results, complete research maneuvering in gardens and begin looking into state-of-the art AI classification methods. Wiki
5 Analyze survey results, research mapping techniques, sensors that might be required for this to be effective and complete research of AI classification models. Wiki
6 Decide on the final requirements for the sensors and mapping of robot. Wiki
7 Update wiki to document our progress and results Wiki
8 Work on and finalize presentation Final Presentation

Navigation Algorithm and Netlogo Implementation: Raul S.

Week Task Deliverable to meeting
2 Literature Review and State of the Art Wiki
3 Write survey questions about garden maintenance robot, research about what hardware, equipment and materials

that the robot would need.

Wiki
4 Send survey questions, complete research about hardware, equipment and materials. Wiki
5 Analyse survey results. Wiki
6 Decide on a final set of requirements for the hardware of the robot. Wiki
7 update wiki to document our progress and results Wiki
8 Work on and finalize presentation Final Presentation.

Research into AI identifying plant diseases and infestations: Briana.

Week Task Deliverable to meeting
2 Research on state of the art AI. Wiki
3 Research on plant diseases and infestations Google Doc
4 Research on best ways to detect diseases and infestations (where to point the camera, what other sensors to use) Google Doc
5 Research on AI state recognition (healthy/unhealthy) Google Doc
6 Research on limitations of AI when it comes to recognising different states of a plant (healthy/unhealthy) Google Doc
7 Conducting interviews with AI specialist. Google Doc
8 Work on and finalize presentation. Presentation

Research into AI identifying plant diseases and infestations: Rareş.

Week Task Deliverable to meeting
2 Research on state of the art AI. Wiki
3 Research on plant diseases and infestations Google Doc
4 Research on best ways to detect diseases and infestations (where to point the camera, what other sensors to use) Google Doc
5 Research on AI state recognition (healthy/unhealthy) Google Doc
6 Research on limitations of AI when it comes to recognising different states of a plant (healthy/unhealthy) Google Doc
7 Conducting interviews with AI specialist. Google Doc
8 Work on and finalize presentation Presentation

Interactive UI design and implementation: Raul H.

Week Task Deliverable to meeting
2 Literature Review and State of the Art Wiki
3 Write interview questions in order to find out what requirements users expect from the application, and research the Android application development process in Android Studio. Wiki
4 Based on the interviews, compile a list of the requirements and create UI designs based on these requirements. Wiki
5 Start implementing the UI designs into a functional application in Android Studio.
6 Finish implementing the UI designs into a functional application in Android Studio. Completed demo application
7
8 Work on and finalize presentation

Interactive UI design and implementation: Ania

Week Task Deliverable to meeting
2 Literature Review and State of the Art of Garden Robots and Plant Recognition Software Wiki
3 Write interview questions in order to find out what requirements users expect from the application, start creating UI design based on current concept ideas. Wiki
4 Based on the interviews, compile a list of the requirements and create UI designs based on these requirements. Wiki
5 Start implementing the UI designs into a functional application in Android Studio.
6 Finish implementing the UI designs into a functional application in Android Studio. Completed demo application
7 Testing and final changes to UI design.
8 Work on and finalize presentation

Individual effort

Break-down of hours Total Hours Spent
Week 1 Patryk Stefanski Attended kick-off (2h), Research into subject idea (2h), Meet with group to discuss ideas (2h), Reading Literature (2h), Updating wiki (1h) 9
Raul Sanchez Flores Attended kick-off (2h) 2
Briana Isaila Attended kick-off (2h), Meet with group to discuss ideas (2h), Research state of the art (2h), Look into different ideas (2h) 8
Raul Hernandez Lopez
Ilie Rareş Alexandru
Ania Barbulescu Attended kick-off (2h), Reading Literature (2h) 4
Week 2 Patryk Stefanski Meeting with tutors (0.5h), Researched and found contact person who maintains Dommel (1h), Brainstorming new project ideas (3h), Group meeting Thursday (1.5h), Created list of possible deliverables (1h), Group meeting to establish tasks (4.5h), Literature review and updated various parts of wiki (2h) 13.5
Raul Sanchez Flores Meeting with tutors (0.5h), Group meeting Thursday (1.5h) 2
Briana Isaila Meeting with tutors (0.5h), Group meeting Thursday (1.5h), Brainstorming new project ideas (2h), Updating wiki (2h), Group meeting to establish tasks (4.5h) 10.5
Raul Hernandez Lopez Meeting with tutors (0.5h), Group meeting Thursday (1.5h) 2
Ilie Rareş Alexandru Meeting with tutors (0.5h), Group meeting Thursday (1.5h) 2
Ania Barbulescu Group meeting Friday (4h), Research Literature (2h), Updated Wiki (2h) 8
Week 3 Patryk Stefanski Meeting with tutors (0.5h), Research to specify problem more concretely (3.5h), Found literature that backs up problem is necessary (1h), Group meeting Tuesday (1h), Finished Problem statement, objectives, users (2h), Research into maneuvering and reporting on findings (3h) 11
Raul Sanchez Flores
Briana Isaila Meeting with tutors (0.5h)
Raul Hernandez Lopez
Ilie Rareş Alexandru
Ania Barbulescu
Week 4 Patryk Stefanski
Raul Sanchez Flores
Briana Isaila
Raul Hernandez Lopez
Ilie Rareş Alexandru
Ania Barbulescu

Literature Review

1.     TrimBot2020: an outdoor robot for automatic gardening (https://www.researchgate.net/publication/324245899_TrimBot2020_an_outdoor_robot_for_automatic_gardening)

·      The TrimBot2020 program aims to build a prototype of the world’s first outdoor robot for automatic bush trimming and rose pruning.

·      State of the art: ‘green thumb’ robots used for automatic planting and harvesting.

·      Gardens pose a variety of hurdles for autonomous systems by virtue of being dynamic environments: natural growth of plants and flowers, variable lighting conditions, as well as varying weather conditions all influence the appearance of objects in the environment.

·      Additionally, the terrain is often uneven and contains areas that are difficult for a robot to navigate, such as those made of pebbles or woodchips.

·      The design of the TrimBot2020 is based on the Bosch Indigo lawn mower, on which a Kinova robotic arm is then mounted. (It might, therefore, be worthwhile to research both of these technologies.)

·      The robot’s vision system consists of five pairs of stereo cameras arranged such that they offer a 360◦ view of the environment. Additionally, each stereo pair is comprised of one RGB camera and one grayscale camera.

·      The robot uses a Simultaneous Localization and Mapping (SLAM) system in order to move through the garden. The system is responsible for simultaneously estimating a 3D map of the garden in the form of a sparse point cloud and the position of the robot in respect to the resulting 3D map.

·      For understanding the environment and operating the robotic arm, TrimBot2020 has developed algorithms for disparity computation from monocular images and from stereo images, based on convolutional neural networks, 3D plane labeling and trinocular matching with baseline recovery. An algorithm for optical flow estimation was also developed, based on a multi-stage CNN approach with iterative refinement of its own predictions.

2.     Robots in the Garden: Artificial Intelligence and Adaptive Landscapes (https://www.researchgate.net/publication/370949019_Robots_in_the_Garden_Artificial_Intelligence_and_Adaptive_Landscapes)

·      FarmBot is a California-based firm and designs and markets open-source commercial gardening robots, and develops web applications for users to interface with these robots.

·      These robots employ interchangeable tool heads to rake soil, plant seeds, water plants, and weed. They are highly customizable: users can design and replace most parts to suit their individual needs. In addition to that, FarmBot’s code is open source, allowing users to customize it through an online web app.

·      Initially, the user describes the garden’s contents to a FarmBot as a simple placement of plants from the provided plant dictionary on a garden map, a two-dimensional grid visualized by the web app. FarmBot stores the location of each plant as a datapoint (x, y) on that map. Other emerging plants, if detected by the camera, are treated uniformly as weeds that should be managed by the robot.

·      The robotic vision system employed by the Ecological Laboratory for Urban Agriculture consists of AI cameras that process images with OpenCV, an open-source computer vision and machine learning software library. This library provides machine learning algorithms, including pre-trained deep neural network modules that can be modified and used for specific tasks, such as measuring plant canopy coverage and plant height.


3.      Indoor Robot Gardening: Design and Implementation (https://www.researchgate.net/publication/225485587_Indoor_robot_gardening_Design_and_implementation)

·      

4. Building a Distributed Robot Garden (https://www.researchgate.net/publication/224090704_Building_a_Distributed_Robot_Garden)

5. A robotic irrigation system for urban gardening and agriculture (https://www.researchgate.net/publication/337580011_A_robotic_irrigation_system_for_urban_gardening_and_agriculture)

6. Design and Implementation of an Urban Farming Robot (https://www.researchgate.net/publication/358882608_Design_and_Implementation_of_an_Urban_Farming_Robot)

7. Small Gardening Robot with Decision-making Watering System (https://www.researchgate.net/publication/363730362_Small_gardening_robot_with_decision-making_watering_system)

8. A cognitive architecture for automatic gardening (https://www.researchgate.net/publication/316594452_A_cognitive_architecture_for_automatic_gardening)

9. Recent Advancements in Agriculture Robots: Benefits and Challenges (https://www.researchgate.net/publication/366795395_Recent_Advancements_in_Agriculture_Robots_Benefits_and_Challenges)

10. A Survey of Robot Lawn Mowers (https://www.researchgate.net/publication/235679799_A_Survey_of_Robot_Lawn_Mowers)

11. Distributed Gardening System Using Object Recognition and Visual Servoing (https://www.researchgate.net/publication/341788340_Distributed_Gardening_System_Using_Object_Recognition_and_Visual_Servoing)

12. A Plant Recognition Approach Using Shape and Color Features in Leaf Images (https://www.researchgate.net/publication/278716340_A_Plant_Recognition_Approach_Using_Shape_and_Color_Features_in_Leaf_Images)

13. A study on plant recognition using conventional image processing and deep learning approaches (https://www.researchgate.net/publication/330492923_A_study_on_plant_recognition_using_conventional_image_processing_and_deep_learning_approaches)

14. Plant Recognition from Leaf Image through Artificial Neural Network (https://www.researchgate.net/publication/258789208_Plant_Recognition_from_Leaf_Image_through_Artificial_Neural_Network)

15. Deep Learning for Plant Identification in Natural Environment (https://www.researchgate.net/publication/317127150_Deep_Learning_for_Plant_Identification_in_Natural_Environment)

16. Identification of Plant Species by Deep Learning and Providing as A Mobile Application (https://www.researchgate.net/publication/348008139_Identification_of_Plant_Species_by_Deep_Learning_and_Providing_as_A_Mobile_Application)

17. Path Finding Algorithms for Navigation (https://nl.mathworks.com/help/nav/ug/choose-path-planning-algorithms-for-navigation.html)

18. NetLogo Models (https://ccl.northwestern.edu/netlogo/models/)

19. Path Finding Algorithms (https://neo4j.com/developer/graph-data-science/path-finding-graph-algorithms/#:~:text=Path%20finding%20algorithms%20build%20on,number%20of%20hops%20or%20weight.)

20. How Navigation Agents Learn About Their Environment (https://openaccess.thecvf.com/content/CVPR2022/papers/Dwivedi_What_Do_Navigation_Agents_Learn_About_Their_Environment_CVPR_2022_paper.pdf)

21. NetLogo Library (https://ccl.northwestern.edu/netlogo/docs/dictionary.html

22. Adapting to a Robot: Adapting Gardening and the Garden to fit a Robot Lawn Mower (https://dl.acm.org/doi/abs/10.1145/3371382.3380738)

23. Automatic Distributed Gardening System Using Object Recognition and Visual Servoing (https://link.springer.com/chapter/10.1007/978-981-15-7345-3_30)

24. An Overview Of Smart Garden Automation (https://ieeexplore.ieee.org/abstract/document/9077615)

25. A cognitive architecture for automatic gardening (https://www.sciencedirect.com/science/article/pii/S0168169916304768)

Sources

  1. https://www.growertalks.com/Article/?articleid=20101
  2. https://www.growertalks.com/Article/?articleid=20101
  3. https://schultesgreenhouse.com/Benefits.html#:~:text=Plants%20act%20as%20highly%20effective,streams%2C%20storm%20drains%20and%20roads.
  4. https://www.bbvaopenmind.com/en/articles/a-decade-of-transformation-in-robotics/#:~:text=The%20advancements%20in%20robotics%20over,their%20environment%20in%20unique%20ways.
  5. https://www.mdpi.com/2075-1702/11/1/48
  6. https://foodgardening.mequoda.com/daily/garden-design/vegetable-garden-planner-apps/
  7. https://www.robotplatform.com/knowledge/Classification_of_Robots/legged_robots.html#:~:text=First%20and%20foremost%20reason%20is,%2C%20orientation%2C%20efficiency%20and%20speed.
  8. https://www.wevolver.com/article/sensors-in-robotics-the-common-types
  9. https://www.sciencedirect.com/science/article/abs/pii/S0079642500000116#:~:text=The%20selection%20of%20an%20appropriate,as%20cost%2C%20and%20impedance%20matching.
  10. https://www.sciencedirect.com/topics/engineering/sensor-fusion#:~:text=Sensor%20fusion%20is%20the%20process,navigate%20and%20behave%20more%20successfully.
  11. https://www.mapix.com/lidar-applications/lidar-robotics/#:~:text=LiDAR%20(Light%20Detection%20and%20Ranging,doors%2C%20people%20and%20other%20objects.
  12. (Jie Shan and Charles K. Toth (2018). Topographic Laser Ranging and Scanning: Principles and Processing (2nd ed.). CRC Press.)
  13. https://intapi.sciendo.com/pdf/10.2478/agriceng-2023-0009#:~:text=The%20LiDAR%20sensor%20allows%20the,which%20allows%20clustering%20and%20positioning.
  14. https://opg.optica.org/oe/fulltext.cfm?uri=oe-24-12-12949&id=344314
  15. https://www.thalesgroup.com/en/markets/aerospace/drone-solutions/scaleflyt-geocaging-safe-and-secure-long-range-drone-operations
  16. https://www.robomow.com/blog/detail/boundary-wire-vs-grass-sensors-for-robotic-mowers
  17. https://www.ardusimple.com/rtk-explained/#:~:text=Introduction%20to%20centimeter%20level%20GPS%2FGNSS&text=Under%20perfect%20conditions%2C%20the%20best,accuracy%20of%20around%202%20meters.
  18. https://first-tf.com/general-public-schools/how-it-works/gps/#:~:text=GNSS%20positioning%20is%20based%20on,each%20of%20the%20visible%20satellites.
  19. https://therobotmower.co.uk/2021/12/02/robot-mowers-without-a-perimeter-wire/

Appendix

Meeting Notes

Week 4

  • We need to have an explicit link from the user needs to our product:
    • ai recognition of plants. how many weeds, plants? how will the ai recognise disease? video, multiple images, 1 image?
    • we need to come up with requirements for our app, and back them up with literature or the survey
  • how will the robot behave in the garden? will it map out the entire garden by itself at first? how often will it use the ai to detect problems in the plants?