PRE2023 3 Group3
Group members
| Name | Student ID | Current Study Programme |
|---|---|---|
| Patryk Stefanski | 1830872 | CSE |
| Raul Sanchez Flores | 1844512 | CSE |
| Briana Isaila | 1785923 | CSE |
| Raul Hernandez Lopez | 1833197 | CSE |
| Ilie Rareş Alexandru | 1805975 | CSE |
| Ania Barbulescu | 1823612 | CSE |
Problem statement
In western society, having a family, a house, and a good job are what many people aspire to have. As people strive to achieve such aspirations, their spending power increases, which allows them to be able to afford to buy a nice home for their future family, with a nice garden for the kids and pets. However, as with many things, in our capitalist world, this usually comes at a sacrifice: free time. According to a study conducted by a student team at Manchester University [2], the three main reasons why people don't garden which made up 60% of the survey responses included time constraints, lack of knowledge/information, and space restraints. Gardening should be encouraged, due to its environmental benefits and many other advantages[3].
In the past decade, robotics has been advancing across multiple fields rapidly as tedious and difficult tasks become increasingly automated [4], this is not any different in the field of agriculture and gardening [5]. In recent years, many robots have become available that aid farmers in important aspects such as irrigation, plantation, and weeding. These robots are large mechanical structures sold at a very high price meaning their only true usage is in large-scale farming operations. Unfortunately, one common user group has been left behind and not considered when developing many features of this new technology in gardening and agriculture, the amateur gardener. Amateur gardeners, often lacking in-depth knowledge about plants and gardening practices, face challenges in maintaining their gardens. Identifying issues with specific plants, understanding their individual needs, and implementing corrective measures can be overwhelming for their limited expertise. It is no surprise that traditional gardening tools and resources often fall short of providing the necessary guidance for optimal plant care, so another solution must be found. This is the problem that our team's robot will be the solution to. We cannot help the fact that some people do not have a space to garden, but we can address the other two common problems. So, the questions we asked ourselves were:
"How do we make gardening more accessible for the amateur gardeners?"
"How do we provide the necessary guidance and information?"
"How can we make our product aid time efficiency?"
Objectives
The objectives for the project that we hope to accomplish throughout the 8 weeks that are given to us are the following:
- Create a mobile application prototype that connects to a robot.
- Look into state-of-the-art technology that can be brought to an amateur user's fingertips through an application.
- The robot's application should be user-friendly and easy to use.
- The robot should map out the garden's explorable terrain.
- The robot should be able to scan and identify a plant and if it is healthy. (the type of plant as well?)
- The robot should be able to cut grass while scanning for diseases for time efficiency purposes.
- The robot should be able to recommend specific actions after spotting a disease/infestation.
- The application should display the location on the map at which it found an unhealthy plant and a picture of the plant, and recommend actions.
Users
Who are the users?
The users of the product are garden-owners who need assistance in monitoring and maintaining their garden. This could be due to the fact that the users do not have required knowledge to properly maintain all different types of plants in their garden, or would prefer a quick and easy set of instruction of what to do with each unhealthy plant and where that plant is located. This would optimise the users routine of gardening without taking away the joy and passion that inspired the user to invest into plants in their garden in the first place.
What do the users require?
The users require a robot which is easy to operate and does not need unnecessary maintenance and setup. The robot should be easily controllable through a user interface that is tailored to the users needs and that displays all required information to the user in a clear and concise way. The user also requires that the robot may effectively map their garden and identify where certain plants are located. Lastly, the user requires that the robot is able to accurately describe what actions must be taken, if any are necessary, for a specific plant at a specific location in the garden.
Deliverables
- Research into AI plant detection mapping a garden and best ways of manoeuvring through it.
- Research into AI identifying plant diseases and infestations.
- Survey confirming that the problem we have selected to solve is a solution users desire.
- Interactive UI of an app that will allow the user to control the robot remotely that implements the user requirements that we will obtain from the survey. The UI will be able to be run on a phone and all its features will be able to be accessed through a mobile application.
- Interview with a specialist in biology or AI
- This wiki page which will document the progress of the group's work, decisions that have been made, and results we obtained.
State of Art
Garden Planning applications[6]
The current state of the art presents numerous mobile applications to deal with planning what to grow in a garden. This can help the user to keep track of what plants are planted and where. Other applications such as Agrio [7] use artificial neural networks to identify the types of diseases that plants have.
AI disease Identification
- Robots such as Roombas, and automated grass-cutting robots.
- Existing plant detection AI systems
Regular Robot Operation
As with any piece of technology it is important that the users are aware of its proper operation method and how the robot functions is general. It is important that this is clear for our robot as well. Upon the robot's first use in a new garden or when the garden owner has made some changes to the garden layout, the mapping process must be instantiated in the app. This mapping will be a 2D map of the garden which will then later allow the robot to efficiently traverse the entire garden during its regular operation without leaving any part of the garden unvisited. In order to better understand this feature one, can compare it to the iRobot Roomba. After the initial setup phase has been completed the robot will be able to begin its normal operation. Normal operation includes the robot being let out into the garden from its storage place, and traversing through the garden cutting grass while its camera scans the plants in its surroundings. Whenever the robot detects an irregularity in one of the plants, it will notify the user through the user of the app, where the robot will send over a picture of the plant with an issue as well as its location on the map of the garden. The user will then able to navigate in the app to view all plants that need to be taken care of in his garden. This means that not only will the user have a lawn which is well kept but also be aware of all unhealthy plants keeping the users garden in optimal condition at all times.
.jpg)
Maneuvering: Patryk
Movement
One of the most important design decisions when creating a robot or machine with some form of mobility is deciding what mechanism the robot will use to traverse its operational environment. This decision is not always easy as many options exist which have their unique pros and cons. Therefore is is important to consider the pros and cons of all methods and then decide which method is most appropriate for a given scenario. In the following section I will explore these different methods and see which are expected to be most beneficial and work the best in the task environment our robot will be required to function in.
Wheeled Robots
It may be no surprise that the most popular method for movement within the robot industry is still a robot with circular wheels. This is due to the fact that robots with wheels are simply much easier to design and model[8]. They do not require complex mechanism of flexing or rotating a actuator but can be fully functional by simply altering rotating a motor in one of two directions. Essentially they allow the engineer to focus on the main functionality of the robot without having to worry about the many complexities that could arise with other movement mechanisms when that is not necessary. Wheeled robots are also convenient in design as they rarely take up a lot of space in the robot. Furthermore, as stated by Zedde and Yao from the University of Wagenigen, these types of robots are most often used in industry due to their simple operation and simple design[9]. Although wheeled robots seem as a single simple category there are a few subcategories of this movement mechanism that are important to distinguish as they each have their benefits and issues they face.
Differential Drive
Differential drive focuses on independent rotation of all wheels on the robot. Essentially one could say that each wheel has its own functionality and operates independently of the other wheels present on the robot. Although rotation is independent it is important to note that all wheels on the robot work as one unit to optimize turning and movement. The robot does this by varying the relative speed of rotation of its wheels which allow the robot to move in any direction without an additional steering mechanism[10]. In order to better illustrate this idea consider the following scenario - suppose a robot wants to turn sharp left, the left wheels would become idle and the right wheel would rotate at maximum speed. As can be seen both wheels are rotating independently but are doing so to reach the same movement goal.
| Pros | Cons |
|---|---|
| Easy to design | Difficulty in straight line motion on uneven terrains |
| Cost-effective | Wheel skidding can completely mess up algorithm and confuse the robot of its location |
| Easy maneiveribility | Sensitive to weight distribution - big issue with moving water in container |
| Robust - less prone to mechanical failures | |
| Easy control |
Omni Directional Wheels
Omni-directional wheels are a specialized type of wheel designed with rollers or casters set at angles around their circumference. This specific configuration allows a robot which has these wheels to easily move in any direction, whether this is lateral, diagonal, or rotational motion[11]. By allowing each wheel to rotate independently and move at any angle, these wheels provide great agility and precision, which makes this method ideal for applications which require navigation and precise positioning. The main difference between this method and differential drive is the fact that omni directional wheels are able to move in any direction easily and do not require turning of the whole robot when that is not necessary due to their specially designed roller on each wheel.
| Pros | Cons |
|---|---|
| Allows complex movement patterns | Complex design and implementation |
| Superior maneuverability in any direction | Limited load-bearing capacity |
| Efficient rotation and lateral movement | Higher manufacturing costs |
| Ideal for tight spaces and precision tasks | Reduced traction on uneven terrains |
| Enhanced agility and flexibility | Susceptible to damage in rugged environments |
Legged Robots
Over millions of year organisms have evolved in thousands of different ways, giving rise to many different methods of brain functioning, how an organisms perceives the world and what is important in our current discussion, movement. It is no coincidence that many land animals have evolved to have some form of legs to traverse their habitats, it is simply a very effective method which allows a lot of versatility and adaptability to any obstacle or problem an animal might face[12]. This is no different when discussing the use of legged robots, legs provide superior functionality to many other movement mechanisms due to the fact that they are able to rotate and operate freely in all axis's. However, with great mobility comes the great cost of their very difficult design, a design with which top institutions and companies struggle with to this day[13].
| Pros | Cons |
|---|---|
| Versatility in Terrain | Complexity in Design |
| Obstacle Negotiation | Power Consumption |
| Stability on Uneven Ground | Sensitivity to Environmental Changes |
| Human-Like Interaction | Limited Speed |
| Efficiency in Locomotion | Maintenance Challenges |
Tracked Robots
Tracked robots, which can be characterized by their continuous track systems, offer a dependable method of traversing a terrain that can be found in applications across various industries. The continuous tracks, consisting of connected links, are looped around wheels or sprockets, providing a continuous band that allows for effective and reliable movement on many different surfaces, terrains and obstacles[14]. It is therefore no surprise that their most well known usages include vehicles which operate in uneven and unpredictable, such as tanks. Since tracks are flexible it is even common that such robots can simply avoid small obstacles by driving over them without experiencing any issues. This is particularly favorable for the robot we are designing as naturally gardens are never perfectly flat surfaces often littered by many naturally cause obstacles such as stone, dents in the surface or even possibly branches that have fallen on the ground due to rough wind.
| Pros of Tracked Robots | Cons of Tracked Robots |
|---|---|
| Superior Stability | Complex Mechanical Design |
| Effective Traction | Limited Maneuverability |
| Versatility in Terrain | Terrain Alteration |
| High Payload Capacity | Increased Power Consumption |
| Efficient Over Obstacles | |
| Consistent Speed |
Hovering/Flying Robots
Hovering/Flying robots provide without a doubt the most unique way of movement from the previously listed. This method unlocks a whole new wide range of possibilities as the robot no longer has to consider on-ground obstacles; whether that is rocks or uneven terrain. The robot is able to view and monitor a very large terrain from one position due to its ability to position itself at a high altitude and quickly detect major problems in a very large area. This method also unlocks the possibility of the robot to optimize its movement distance as it is able to move from point A to point B directly in a straight line saving energy and time. However, as is the case with any solution, flying/hovering has its major problems. It is by far the most expensive method, as flying apparatus is far more costly and high maintenance than any other solution. This makes this unreliable and likely a method far out of the technological needs and requirements of our gardening robot. Furthermore, its operation is best in large open fields which perfectly suits the large farms of the agriculture industry, however, this is not the aim of the robot we are designing. Most private gardens are of a small size meaning its main strength could not be used. Additionally, it is likely that a robot which has aerial abilities would find difficulty in maneuvering through the tight spaces of a private garden and would have to avoid many low hanging branches or pushes ultimately making its operation unsafe.
| Pros | Cons |
|---|---|
| Versatile Aerial Mobility | Limited Payload Capacity |
| Rapid Deployment | Limited Endurance |
| Remote Sensing | Susceptibility to Weather |
| Reduced Ground Impact | Regulatory Restrictions |
| Dynamic Surveillance | Security and Privacy Concerns |
| Efficient Data Collection | Initial Cost and Maintenance |
Sensors are a fundamental component of any robot that is required to interact with its environment, as they aim to replicate our sensory organs which allow us to perceive and better understand the environment around us[15]. However, unlike with living organisms, engineers are given the choice to decide what exact sensors their robot needs and must be careful with this decision in order to pick the sufficient options to be able to allow the robot to have its full functionality without picking any redundant options that will make the robot unnecessarily expensive. This decision is often based on researching and considering all possible sensors that are available on the market which are related to the problem the engineer is trying to solve and selecting the one which fulfills the requirements of the the robot most accurately[16]. In this section we will specifically be looking into sensors which will aid our robot in traversing our environment, a garden. This means that we must consider the fact that the sensors we select must be able to work in environments where the lighting level is constantly changing as well as possible mis inputs due to high winds and or uneven terrain. Additionally, it is important to note that unlike the discussion in the previous section, one type of sensor/system is rarely sufficient to fulfill the requirements and most robots must implement some form of sensor fusion in order to operate appropriately, this is no different in our robot[17].
LIDAR sensors
LIDAR stands for Light Detection and Ranging. These types of sensors allow robots which utilize them to effectively navigate the environment they are placed in as they provide the robot with object perception, object identification and collision avoidance[18]. These sensors function through sending lasers into the environment and then calculating how long it takes the signals they send to return back to the receiver to determine the distance to the nearest objects and their shapes. As can be seen, LIDAR’s provide robots with a vast amount of crucial information and even allow them to see the world in a 3D perspective. This means that not only are robots able to see their closest object, whenever faced with an obstacle they can instantaneously derive possible methods of avoidance and to traverse around it[19].
LIDAR’s are often the preferred option by engineers in robots that operate outdoors as they are minimally influenced by weather conditions[20]. Often sensors rely on visual imaging or sound sensors which both get heavily disturbed in more difficult weather conditions, whether that is rain on a camera lens or the sound of rain disturbing sound sensors, this is not the case with LIDAR's as their laser technology does not malfunction in these scenarios. However, an issue that our robot is likely to face when utilizing a LIDAR sensor is that of sunlight contamination[21]. Sunlight contamination is the effect the sun has on generating noise in the sensor’s data during the daytime and therefore possibly introducing errors within it. Since our robot needs to work optimally during the daytime it is crucial that this is considered. However, the LIDAR possesses many additionally positive aspects that would be truly beneficial to our robot such as the ability to function in complete darkness and immediate data retrieval. This would allow the users of our robot to turn on the robot before they go to sleep at night and wake up to a complete report of their garden status. Furthermore, these features are necessary for the robot as they would allow it to work in a dynamic and constantly changing environment, which is of high importance to where our robot wants to operate in the garden. The outdoors can never be a fully controlled environment and that has to be considered into the design of the robot.
As can be seen the LIDAR sensor has many excellent features that our robot will likely require, therefore it is a very important candidate when making our next design decisions.
Boundary Wire
A boundary wire is likely the most cost efficient and commonly implemented technique in state-of-the-art garden robots that are on the private consumer market today. It is not a complicated technology but still a very effective one when it comes to robot navigation. A boundary wire in the garden acts as a virtual barrier that the robot cannot cross, similar to a geo-cage in drone operation[22]. In order to begin utilizing it, the robot user must first lay out the wire on the boundaries of their garden and then dig them approximately 10 cm below the ground's surface, so that the wire is safe from any external factors. This is a tedious task for the user but has to only be completed once and the robot is now fully operational and will never leave the boundaries set by the user. It is important for the user to take their time in the first setup as any change they will want to make will require digging up many meters of wire and once again putting it in the ground after relocation.
The boundary wire communicates with the robot by emitting a low voltage, around 24V, signal which is picked up by a sensor on the robot[23]. This means that when the robot detects the signal it knows that the wire is underneath it and it should not to continue moving in its direction. As is displayed above, the boundary wire is a very simple technology which with a slight amount of effort of the user can perform the basic navigability tasks. However, its functionality is fairly limited, it cannot detect any objects within the area of its operation and therefore avoid them meaning that its environment has to be maintained and clear throughout its operation.
GPS/GNSS
GPS/GNSS are groups of satellites deployed in space that allow robots and devices to receive signals from them which aid them in positioning. Over the past few years these systems have gotten extremely accurate and can position devices to the nearest meter[24]. This happens through a process called triangulation, where multiple satellites calculate their distance to a device and establish its location[25]. The usage of this sensor in our robot is very encouraging as it has been proven to be effective in the large scale gardening industry for many years, more specifically including the precision farming domain[26]. An important distinction to note is that of GPS and GNSS. Although GPS is likely the term many are more familiar with from navigation applications they have used in the past, it is really just a subpart of GNSS which represents all Constellation Satellite Systems and GPS is simply one of them. If equipped with a sensor that can communicate with satellites and fetch its location at all times, our robot will be able to precisely ping the location it has found sick plants or any plants that need care and send that information to the users device. Once again promising to be a very key component in our robot design.
Bump Sensors
Bump sensors, commonly referred to as collision or impact sensors, are sensors designed to detect physical contact or force a robot could encounter while traversing its environment[27]. These sensors can be seen being utilized across various industries in robotics in order to increase the safety and allow for greater automation of the robots they are integrated into. Additionally, these devices are crucial in robotics and many different types of vehicles as they allow the robot to replicate the human ability of touch and have a tactile interface with the environment.
In order to have this feature, bump sensors are typically composed of accelerometers, devices that are able to detect and measure a change in acceleration forces, or simply a switch which gets pressed as soon as the robot applies pressure on it from factors in its environment[28]. In its many applications the contact the robot experiences is with larger objects so the sensor must not be extremely sensitive. This could not be the case in our gardening robot as the robot would have to consider smaller and more fragile objects requiring the sensor to have a much higher sensitivity. Bump sensors are most commonly used to make sure a robot does not drive into and collide with large objects, it allows the robot to detect that a change of direction in its motion must occur before continuing its operation. Although in many industries bump sensors are a last resort form of defense against the robot breaking or destroying important elements in our environment, in the robot we are designing it is a lot less of an issue if the robot were to detect an object through this sensor[29]. The robot would have to signal contact that it has collided with an object and change its direction of motion without further applications. However, this should still be unlikely to happen as in our robot the LIDAR sensor should have detected the problem beforehand and dealt with it, nevertheless technology is not always reliable and having a backup system ensures the robot experiences fewer errors in its operation especially with the possibility of faults of the LIDAR due to sun contamination.
Ultrasonic sensors
Ultrasonic sensors, or put in more simple terms, sound sensors, are another type of sensor which allows a robot to measure distances between objects in its environment and its current position. These sensors also find widespread applications in robotics whether that is liquid level detection, wire break detection or even counting the number of people in an area. Their strength is that it allows robot that have them to replicate human depth perception in a method similar to that of a dolphin[30].
Ultrasonic sensors function by emitting high-frequency sound waves through its transmitters and measuring the time it takes for the waves to bounce back and be received by its receivers. This data allows the robot to calculate the distance to the object, enabling precise navigation and obstacle detection. Although this once again is similar to the function of a LIDAR sensor, it allows the robot to work in a frequently changing environment without the use of state-of-the-art and expensive technology. One thing that must be considered in the usage of this sensor is that it tends to perform worse when attempting to detect softer materials which our team will have to take into account and make sure the sensor is able to detect plants it is approaching[31].
In our gardening robot, ultrasonic sensors could once again play an important role in supplementing the functionality of more advanced sensors like the LIDAR. Through its simple and reliable solution, ultrasonic sensors provide essential functionality, improving the robot's operational reliability in a very wide range of gardens it could encounter in its deployment.
Gyroscopes
Gyroscopes are essential components in the field of robotics that help in providing stability and precise orientation control in a wide range of industrial areas. These devices use the principles of angular momentum to constantly maintain the same reference direction, in order to not change its current orientation[32]. This allows robots to improve their stability and therefore enhance their operational abilities.
In order to perform their functionality, gyroscopes consist of a spinning mass, mounted on a set of gimbals. When the orientation of the gyroscope changes, its mechanism of conserving angular momentum means that it will apply counteracting force essentially keeping it in the same orientation. This feature is very important in the field of robotics as it allows the robot to know its current angle with regards to the ground and when that gets too large the gyroscope helps the robot to not flip over or fall during its operation.
Since thousands of relevant sensors exist, we can only discuss the most important ones. Sensors such as lift sensors, incline sensors and camera systems can also be included in the robot for navigation purposes, however in the design of our robot they are either too complex or unnecessary.
Mapping
Mapping will be one of the most important features of our robot as it will be the very first thing the robot performs after being taken out of its box by the user and will rely on the quality of this process for the rest of its operation. Mapping is the idea of letting the robot familiarize itself with its operational environment by traversing it freely without performing any of its regular operations and simply analyzing where the boundaries of the environment are and how it is roughly shaped. This allows the robot to gain the required knowledge so that during its normal operation throughout its lifecycle it is aware of its positioning in the garden, areas it has visited in the current job and areas it still must visit. Essentially it turns the robot from a simple-reflex-agent to a robot which has knowledge stored in its database and can access it to make better informed decisions for more efficient operation. Mapping can be done both in 3D and in 2D depending on the needs of the robot and user. Initially, we considered 3D mapping in this project, enabling the robot to also memorize plant locations in the garden for easier access in the future, however since plants grow very rapidly this environment would change quickly and the mapping process would have had to be repeated on a daily basis, a very inefficient process. Now that the decision was made to implement 2D mapping, similar to that of the Roomba vacuum cleaning robots, the purpose of the map would be to learn the dimensions and shape of the garden. It may come at no surprise but as is the case in the design problems, there is rarely one solution and that is no different in the case of mapping. Nonetheless, the most optimal method for our robot is simply Random Exploration Mapping.
Random Exploration Mapping
Random Exploration Mapping bases its operation on the robot freely moving across the garden and detecting obstacles on its way and boundaries it cannot cross. Eventually after sufficient time has passed for the robot to explore its surroundings it will be able to localize all areas that are available for its traversal and connect them into one coherent map. Although this method is likely going to take quite a bit of time, after the robot begins discovering areas and closing them off with all boundaries that are present within it, it will continue to new areas and gain knowledge effectively.
Occupancy Grid Mapping
VSLAM (Visual Simultaneous Localization and Mapping)
NetLogo Simulation
Our NetLogo deliverable aims to simulate and display how the functioning of the robot will look visually as well as how the robot will communicate and interact with the app. The simulation will display the robot on a regular operation, traversing the mapped garden trimming grass and scanning for plants which are sick. In the simulation when a robot detects a sick plant it will drive up as close as possible to its location and send the coordinates of the sick plant to the application. The application will then be able to receive this information and display it graphically for the user to access.
Interview
In order to confirm our decisions and ask for clarifications and recommendations on features that our user group truly desires we will perform an interview. The interview will be completed with the owner of two different private gardens. The interviewee also owns two grass trimming robots, one produced by Bosch and the other by Gardena which he utilizes in both gardens. We believe his expertise will allow us to solidify our requirements and improve on our product as a whole.
Identifying plant Diseases
Diseases to identify.
Artificial Intelligence recognising diseases.
Methods of plant identification (sensors etc, is the camera the best option??).
Types of AI (algorithms used)
Work Distribution
| Week | Task | Deliverable to meeting |
|---|---|---|
| 2 | Define Deliverables and brainstorm new idea | Completed wiki with new idea |
| 3 | Write survey questions about garden maintenance robot, research methods to maneuver in an area, specifically in a garden. | Wiki |
| 4 | Complete interview questions, complete research maneuvering in gardens and begin looking into mapping techniques | Wiki |
| 5 | Analyze survey results, research mapping techniques, sensors that might be required for this to be effective and complete research of AI classification models. | Wiki |
| 6 | Decide on the final requirements for the sensors and mapping of robot. | Wiki |
| 7 | Update wiki to document our progress and results | Wiki |
| 8 | Work on and finalize presentation | Final Presentation |
| Week | Task | Deliverable to meeting |
|---|---|---|
| 2 | Literature Review and State of the Art | Wiki |
| 3 | Write survey questions about garden maintenance robot, research about what hardware, equipment and materials
that the robot would need. |
Wiki |
| 4 | Send survey questions, complete research about hardware, equipment and materials. | Wiki |
| 5 | Analyse survey results. | Wiki |
| 6 | Decide on a final set of requirements for the hardware of the robot. | Wiki |
| 7 | update wiki to document our progress and results | Wiki |
| 8 | Work on and finalize presentation | Final Presentation. |
Research into AI identifying plant diseases and infestations: Briana.
| Week | Task | Deliverable to meeting |
|---|---|---|
| 2 | Research on state of the art AI. | Wiki |
| 3 | Research on plant diseases and infestations | Google Doc |
| 4 | Research on best ways to detect diseases and infestations (where to point the camera, what other sensors to use) | Google Doc |
| 5 | Research on AI state recognition (healthy/unhealthy) | Google Doc |
| 6 | Research on limitations of AI when it comes to recognising different states of a plant (healthy/unhealthy) | Google Doc |
| 7 | Conducting interviews with AI specialist. | Google Doc |
| 8 | Work on and finalize presentation. | Presentation |
Research into AI identifying plant diseases and infestations: Rareş.
| Week | Task | Deliverable to meeting |
|---|---|---|
| 2 | Research on state of the art AI. | Wiki |
| 3 | Research on plant diseases and infestations | Google Doc |
| 4 | Research on best ways to detect diseases and infestations (where to point the camera, what other sensors to use) | Google Doc |
| 5 | Research on AI state recognition (healthy/unhealthy) | Google Doc |
| 6 | Research on limitations of AI when it comes to recognising different states of a plant (healthy/unhealthy) | Google Doc |
| 7 | Conducting interviews with AI specialist. | Google Doc |
| 8 | Work on and finalize presentation | Presentation |
Interactive UI design and implementation: Raul H.
| Week | Task | Deliverable to meeting |
|---|---|---|
| 2 | Literature Review and State of the Art | Wiki |
| 3 | Write interview questions in order to find out what requirements users expect from the application, and research the Android application development process in Android Studio. | Wiki |
| 4 | Based on the interviews, compile a list of the requirements and create UI designs based on these requirements. | Wiki |
| 5 | Start implementing the UI designs into a functional application in Android Studio. | |
| 6 | Finish implementing the UI designs into a functional application in Android Studio. | Completed demo application |
| 7 | ||
| 8 | Work on and finalize presentation |
Interactive UI design and implementation: Ania
| Week | Task | Deliverable to meeting |
|---|---|---|
| 2 | Literature Review and State of the Art of Garden Robots and Plant Recognition Software | Wiki |
| 3 | Write interview questions in order to find out what requirements users expect from the application, start creating UI design based on current concept ideas. | Wiki |
| 4 | Based on the interviews, compile a list of the requirements and create UI designs based on these requirements. | Wiki |
| 5 | Start implementing the UI designs into a functional application in Android Studio. | |
| 6 | Finish implementing the UI designs into a functional application in Android Studio. | Completed demo application |
| 7 | Testing and final changes to UI design. | |
| 8 | Work on and finalize presentation |
Individual effort
| Break-down of hours | Total Hours Spent | ||
|---|---|---|---|
| Week 1 | Patryk Stefanski | Attended kick-off (2h), Research into subject idea (2h), Meet with group to discuss ideas (2h), Reading Literature (2h), Updating wiki (1h) | 9 |
| Raul Sanchez Flores | Attended kick-off (2h) | 2 | |
| Briana Isaila | Attended kick-off (2h), Meet with group to discuss ideas (2h), Research state of the art (2h), Look into different ideas (2h) | 8 | |
| Raul Hernandez Lopez | |||
| Ilie Rareş Alexandru | |||
| Ania Barbulescu | Attended kick-off (2h), Reading Literature (2h) | 4 | |
| Week 2 | Patryk Stefanski | Meeting with tutors (0.5h), Researched and found contact person who maintains Dommel (1h), Brainstorming new project ideas (3h), Group meeting Thursday (1.5h), Created list of possible deliverables (1h), Group meeting to establish tasks (4.5h), Literature review and updated various parts of wiki (2h) | 13.5 |
| Raul Sanchez Flores | Meeting with tutors (0.5h), Group meeting Thursday (1.5h) | 2 | |
| Briana Isaila | Meeting with tutors (0.5h), Group meeting Thursday (1.5h), Brainstorming new project ideas (2h), Updating wiki (2h), Group meeting to establish tasks (4.5h) | 10.5 | |
| Raul Hernandez Lopez | Meeting with tutors (0.5h), Group meeting Thursday (1.5h) | 2 | |
| Ilie Rareş Alexandru | Meeting with tutors (0.5h), Group meeting Thursday (1.5h) | 2 | |
| Ania Barbulescu | Group meeting Friday (4h), Research Literature (2h), Updated Wiki (2h) | 8 | |
| Week 3 | Patryk Stefanski | Meeting with tutors (0.5h), Research to specify problem more concretely (3.5h), Found literature that backs up problem is necessary (1h), Group meeting Tuesday (1h), Finished Problem statement, objectives, users (2h), Research into maneuvering and reporting on findings (3h) | 11 |
| Raul Sanchez Flores | |||
| Briana Isaila | Meeting with tutors (0.5h) | ||
| Raul Hernandez Lopez | |||
| Ilie Rareş Alexandru | |||
| Ania Barbulescu | |||
| Week 4 | Patryk Stefanski | Meeting with group (1.5h), Research into maneuvering and fixing parts of wiki (2h), Describing robot operation on wiki (1h), Research into sensors and updating wiki (6h), Research into NetLogo writing files and environment setup (2h), Prepare and organize interview (1h), Research mapping and watch videos about it (3h), Created References in APA (0.5h) | 17 |
| Raul Sanchez Flores | |||
| Briana Isaila | |||
| Raul Hernandez Lopez | |||
| Ilie Rareş Alexandru | |||
| Ania Barbulescu |
Literature Review
1. TrimBot2020: an outdoor robot for automatic gardening (https://www.researchgate.net/publication/324245899_TrimBot2020_an_outdoor_robot_for_automatic_gardening)
· The TrimBot2020 program aims to build a prototype of the world’s first outdoor robot for automatic bush trimming and rose pruning.
· State of the art: ‘green thumb’ robots used for automatic planting and harvesting.
· Gardens pose a variety of hurdles for autonomous systems by virtue of being dynamic environments: natural growth of plants and flowers, variable lighting conditions, as well as varying weather conditions all influence the appearance of objects in the environment.
· Additionally, the terrain is often uneven and contains areas that are difficult for a robot to navigate, such as those made of pebbles or woodchips.
· The design of the TrimBot2020 is based on the Bosch Indigo lawn mower, on which a Kinova robotic arm is then mounted. (It might, therefore, be worthwhile to research both of these technologies.)
· The robot’s vision system consists of five pairs of stereo cameras arranged such that they offer a 360◦ view of the environment. Additionally, each stereo pair is comprised of one RGB camera and one grayscale camera.
· The robot uses a Simultaneous Localization and Mapping (SLAM) system in order to move through the garden. The system is responsible for simultaneously estimating a 3D map of the garden in the form of a sparse point cloud and the position of the robot in respect to the resulting 3D map.
· For understanding the environment and operating the robotic arm, TrimBot2020 has developed algorithms for disparity computation from monocular images and from stereo images, based on convolutional neural networks, 3D plane labeling and trinocular matching with baseline recovery. An algorithm for optical flow estimation was also developed, based on a multi-stage CNN approach with iterative refinement of its own predictions.
2. Robots in the Garden: Artificial Intelligence and Adaptive Landscapes (https://www.researchgate.net/publication/370949019_Robots_in_the_Garden_Artificial_Intelligence_and_Adaptive_Landscapes)
· FarmBot is a California-based firm and designs and markets open-source commercial gardening robots, and develops web applications for users to interface with these robots.
· These robots employ interchangeable tool heads to rake soil, plant seeds, water plants, and weed. They are highly customizable: users can design and replace most parts to suit their individual needs. In addition to that, FarmBot’s code is open source, allowing users to customize it through an online web app.
· Initially, the user describes the garden’s contents to a FarmBot as a simple placement of plants from the provided plant dictionary on a garden map, a two-dimensional grid visualized by the web app. FarmBot stores the location of each plant as a datapoint (x, y) on that map. Other emerging plants, if detected by the camera, are treated uniformly as weeds that should be managed by the robot.
· The robotic vision system employed by the Ecological Laboratory for Urban Agriculture consists of AI cameras that process images with OpenCV, an open-source computer vision and machine learning software library. This library provides machine learning algorithms, including pre-trained deep neural network modules that can be modified and used for specific tasks, such as measuring plant canopy coverage and plant height.
3. Indoor Robot Gardening: Design and Implementation (https://www.researchgate.net/publication/225485587_Indoor_robot_gardening_Design_and_implementation)
·
4. Building a Distributed Robot Garden (https://www.researchgate.net/publication/224090704_Building_a_Distributed_Robot_Garden)
5. A robotic irrigation system for urban gardening and agriculture (https://www.researchgate.net/publication/337580011_A_robotic_irrigation_system_for_urban_gardening_and_agriculture)
6. Design and Implementation of an Urban Farming Robot (https://www.researchgate.net/publication/358882608_Design_and_Implementation_of_an_Urban_Farming_Robot)
7. Small Gardening Robot with Decision-making Watering System (https://www.researchgate.net/publication/363730362_Small_gardening_robot_with_decision-making_watering_system)
8. A cognitive architecture for automatic gardening (https://www.researchgate.net/publication/316594452_A_cognitive_architecture_for_automatic_gardening)
9. Recent Advancements in Agriculture Robots: Benefits and Challenges (https://www.researchgate.net/publication/366795395_Recent_Advancements_in_Agriculture_Robots_Benefits_and_Challenges)
10. A Survey of Robot Lawn Mowers (https://www.researchgate.net/publication/235679799_A_Survey_of_Robot_Lawn_Mowers)
11. Distributed Gardening System Using Object Recognition and Visual Servoing (https://www.researchgate.net/publication/341788340_Distributed_Gardening_System_Using_Object_Recognition_and_Visual_Servoing)
12. A Plant Recognition Approach Using Shape and Color Features in Leaf Images (https://www.researchgate.net/publication/278716340_A_Plant_Recognition_Approach_Using_Shape_and_Color_Features_in_Leaf_Images)
13. A study on plant recognition using conventional image processing and deep learning approaches (https://www.researchgate.net/publication/330492923_A_study_on_plant_recognition_using_conventional_image_processing_and_deep_learning_approaches)
14. Plant Recognition from Leaf Image through Artificial Neural Network (https://www.researchgate.net/publication/258789208_Plant_Recognition_from_Leaf_Image_through_Artificial_Neural_Network)
15. Deep Learning for Plant Identification in Natural Environment (https://www.researchgate.net/publication/317127150_Deep_Learning_for_Plant_Identification_in_Natural_Environment)
16. Identification of Plant Species by Deep Learning and Providing as A Mobile Application (https://www.researchgate.net/publication/348008139_Identification_of_Plant_Species_by_Deep_Learning_and_Providing_as_A_Mobile_Application)
17. Path Finding Algorithms for Navigation (https://nl.mathworks.com/help/nav/ug/choose-path-planning-algorithms-for-navigation.html)
18. NetLogo Models (https://ccl.northwestern.edu/netlogo/models/)
19. Path Finding Algorithms (https://neo4j.com/developer/graph-data-science/path-finding-graph-algorithms/#:~:text=Path%20finding%20algorithms%20build%20on,number%20of%20hops%20or%20weight.)
20. How Navigation Agents Learn About Their Environment (https://openaccess.thecvf.com/content/CVPR2022/papers/Dwivedi_What_Do_Navigation_Agents_Learn_About_Their_Environment_CVPR_2022_paper.pdf)
21. NetLogo Library (https://ccl.northwestern.edu/netlogo/docs/dictionary.html
22. Adapting to a Robot: Adapting Gardening and the Garden to fit a Robot Lawn Mower (https://dl.acm.org/doi/abs/10.1145/3371382.3380738)
23. Automatic Distributed Gardening System Using Object Recognition and Visual Servoing (https://link.springer.com/chapter/10.1007/978-981-15-7345-3_30)
24. An Overview Of Smart Garden Automation (https://ieeexplore.ieee.org/abstract/document/9077615)
25. A cognitive architecture for automatic gardening (https://www.sciencedirect.com/science/article/pii/S0168169916304768)
Sources (What citation style should we use)
- ↑ A College Class Asks: Why Don’t People Garden? (n.d.). Www.growertalks.com. https://www.growertalks.com/Article/?articleid=20101
- ↑ A College Class Asks: Why Don’t People Garden? (n.d.). Www.growertalks.com. https://www.growertalks.com/Article/?articleid=20101
- ↑ Benefits of Gardening. (n.d.). Schultesgreenhouse.com. https://schultesgreenhouse.com/Benefits.html#:~:text=Plants%20act%20as%20highly%20effective
- ↑ Rus, D. (n.d.). A Decade of Transformation in Robotics. OpenMind. Retrieved March 9, 2024, from https://www.bbvaopenmind.com/en/articles/a-decade-of-transformation-in-robotics/#:~:text=The%20advancements%20in%20robotics%20over
- ↑ Cheng, C., Fu, J., Su, H., & Ren, L. (2023). Recent Advancements in Agriculture Robots: Benefits and Challenges. Machines, 11(1), 48. https://doi.org/10.3390/machines11010048
- ↑ 6.0 6.1 6.2 MacArthur, A. (2023, December 5). 6 Vegetable Garden Planner Apps Compared. Food Gardening Network. https://foodgardening.mequoda.com/daily/garden-design/vegetable-garden-planner-apps/
- ↑ An app that identifies plant diseases and diagnoses plant problems. (2022, June 15). Agrio. https://agrio.app/An-app-that-identifies-plant-diseases-and-pests/
- ↑ Robot Platform | Knowledge | Wheeled Robots. (n.d.). Www.robotplatform.com. Retrieved March 9, 2024, from https://www.robotplatform.com/knowledge/Classification_of_Robots/legged_robots.html#:~:text=First%20and%20foremost%20reason%20is
- ↑ van de Zedde, R. (2022). Field robots for plant phenotyping. Advances in Plant Phenotyping for More Sustainable Crop Production, 153–178. https://doi.org/10.19103/as.2022.0102.08
- ↑ Wheeled mobile robotics : from fundamentals towards autonomous systems | WorldCat.org. (n.d.). Search.worldcat.org. Retrieved March 9, 2024, from https://search.worldcat.org/title/971588275
- ↑ Admin, Gtfr. (2020, December 12). What is Omni Wheel and How Does it Work? GTFRobots | Online Robot Wheels Shop. https://gtfrobots.com/what-is-omni-wheel/
- ↑ How fins became limbs. (n.d.). Scientific American. https://www.scientificamerican.com/article/how-fins-became-limbs/#:~:text=Four%2Dlegged%20creatures%20may%20have
- ↑ Zhu, Q., Song, R., Wu, J., Masaki, Y., & Yu, Z. (2022). Advances in legged robots control, perception and learning. IET Cyber-Systems and Robotics, 4(4), 265–267. https://doi.org/10.1049/csy2.12075
- ↑ Robot Platform | Knowledge | Tracked Robots. (n.d.). Www.robotplatform.com. Retrieved March 9, 2024, from https://www.robotplatform.com/knowledge/Classification_of_Robots/tracked_robots.html
- ↑ Ayodele, A. (2023, January 16). Types of Sensors in Robotics. Wevolver. https://www.wevolver.com/article/sensors-in-robotics-the-common-types
- ↑ Shieh, J., Huber, J. E., Fleck, N. A., & Ashby, M. F. (2001). The selection of sensors. Progress in Materials Science, 46(3-4), 461–504. https://doi.org/10.1016/s0079-6425(00)00011-6
- ↑ Sensor Fusion - an overview | ScienceDirect Topics. (n.d.). Www.sciencedirect.com. https://www.sciencedirect.com/topics/engineering/sensor-fusion#:~:text=Sensor%20fusion%20is%20the%20process
- ↑ Robotics. (n.d.). Mapix Technologies. https://www.mapix.com/lidar-applications/lidar-robotics/#:~:text=LiDAR%20(Light%20Detection%20and%20Ranging
- ↑ Shan, J., & Toth, C. K. (2018). Topographic Laser Ranging and Scanning. CRC Press.
- ↑ Taras Hutsol, Alexey Kutyrev, Nikolay Kiktev, & Mykola Biliuk. (2023). Robotic Technologies in Horticulture: Analysis and Implementation Prospects. Inżynieria Rolnicza, 27(1), 113–133. https://doi.org/10.2478/agriceng-2023-0009
- ↑ Optica Publishing Group. (n.d.). Opg.optica.org. https://opg.optica.org/oe/fulltext.cfm?uri=oe-24-12-12949&id=344314
- ↑ ScaleFlyt Geocaging: safe and secure long-range drone operations. (n.d.). Thales Group. https://www.thalesgroup.com/en/markets/aerospace/drone-solutions/scaleflyt-geocaging-safe-and-secure-long-range-drone-operations
- ↑ Robomow. (n.d.). Robomow. Retrieved March 9, 2024, from https://www.robomow.com/blog/detail/boundary-wire-vs-grass-sensors-for-robotic-mowers
- ↑ RTK in detail. (n.d.). ArduSimple. Retrieved March 9, 2024, from https://www.ardusimple.com/rtk-explained/#:~:text=Introduction%20to%20centimeter%20level%20GPS%2FGNSS&text=Under%20perfect%20conditions%2C%20the%20best
- ↑ GNSS - FIRST-TF. (2015, June 4). https://first-tf.com/general-public-schools/how-it-works/gps/#:~:text=GNSS%20positioning%20is%20based%20on
- ↑ http://111914033483285. (2021, December 2). 12 Robot Mowers without a Perimeter Wire | The Robot Mower. https://therobotmower.co.uk/2021/12/02/robot-mowers-without-a-perimeter-wire/
- ↑ Products | Joy-IT. (n.d.). Joy-It.net. https://joy-it.net/en/products/SEN-BUMP01
- ↑ Sensors - Accelerometers | Farnell Nederland. (2023). Farnell.com. https://nl.farnell.com/sensor-accelerometer-motion-technology#:~:text=An%20accelerometer%20is%20an%20electromechanical
- ↑ Mukherjee, D., Saha, A., Pankajkumar Mendapara, Wu, D., & Q.M. Jonathan Wu. (2009). A cost effective probabilistic approach to localization and mapping. https://doi.org/10.1109/eit.2009.5189643
- ↑ https://www.facebook.com/Piwsfl. (n.d.). How Do Dolphins Communicate? Https://Ponceinletwatersports.com/. https://ponceinletwatersports.com/how-do-dolphins-communicate/#:~:text=Dolphins%20emit%20high%2Dfrequency%20ultrasound
- ↑ MaxBotix. (2019, September 11). Ultrasonic Sensors: Advantages and Limitations. MaxBotix. https://maxbotix.com/blogs/blog/advantages-limitations-ultrasonic-sensors
- ↑ How the Gyroscope Works. (1970, January 1). HowStuffWorks. https://science.howstuffworks.com/gyroscope.htm#:~:text=A%20gyroscope%20is%20a%20mechanical
- ↑ 33.0 33.1 Frontiers in Plant Science | Articles. (n.d.). Www.frontiersin.org. https://www.frontiersin.org/journals/plant-science/articles/
Appendix
Meeting Notes
Week 4
- We need to have an explicit link from the user needs to our product:
- ai recognition of plants. how many weeds, plants? how will the ai recognise disease? video, multiple images, 1 image?
- we need to come up with requirements for our app, and back them up with literature or the survey
- how will the robot behave in the garden? will it map out the entire garden by itself at first? how often will it use the ai to detect problems in the plants?