PRE2022 3 Group1: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
m (Citations)
 
(147 intermediate revisions by 6 users not shown)
Line 33: Line 33:
<br />
<br />


==Project plan==
==Abstract==


<br />
In this wiki our group examined the issue of man overboard (MOB) incidents and proposes a solution to increase the survival chances of victims. A state-of-the-art literature review was conducted, which explored various aspects of MOB incidents such as search area, search patterns, oceanic currents and temperatures, and current person in water detection technology such as drones. A proof of concept was then developed using an XY-rig, which incorporated a thermal camera connected to a Raspberry Pi. The results showed that the thermal camera was able to successfully locate a heat element in a bland environment, representing the ocean. In future work, the solution will be extended to a drone swarm, where multiple drones can be deployed to search for a victim. This solution has the potential to improve the effectiveness of search and rescue operations, and ultimately increase the survival rate of MOB victims.
===Problem statement and objectives===
In a 2020 study<ref name=":0">Gonel, O., & Çiçek, I. (2020, 25 december). ''STATISTICAL ANALYSIS OF MAN OVER BOARD (MOB) INCIDENTS''. ResearchGate. <nowiki>https://www.researchgate.net/publication/348266442_STATISTICAL_ANALYSIS_OF_MAN_OVER_BOARD_MOB_INCIDENTS</nowiki></ref> researchers examined 100 reports of a man overboard (MOB) incident using 114 parameters to create a MOB event profile. In 88 cases the casualty was deceased as a result of the MOB incident, from those 88 cases 34 were assumed dead and 54 were witnessed dead. From the witness deaths 18 died before the rescue and 31 after the rescue, in 5 cases it is unknown. The cause of death was indicated for 42 cases, the most common cause is identified as drowning (26), followed by trauma (9), cardiac arrest (4) and hypothermia (3). Based on this study the survival chance of an MOB event is slim (12%). We aim to increase the survival chance by developing a device that can locate the victim within 10 minutes and provided the victim with life saving equipment such as a ring life buoy.  
 
===Scenario===
The 2020 study<ref name=":0" /> showed that 53% of the MOB events happen on cargo ships, our focus therefore lies on developing a drone for a cargo ship. We assume that the weather conditions are mild at the time of accident and during the search and rescue operation, i.e. mild waves, a mild breeze, little rain, and an average sea air temperature. We also assume that is nighttime at the time of accident and that there is a sole victim who is capable of maintaining afloat, i.e. no major/life-threating injuries. 
 
Assume that we know a man has fallen overboard within '''X''' minutes so that we know the general search area.  
===Users===
The end users of our product are personnel of cargo ships, shipping companies that own a fleet of cargo ships, ports, and possibly the military.
===Requirements: MoSCoW method===
'''Must'''
 
*Have sensors to detect a person within 10 minutes
 
*Have a communication system to communicate with the involved ship
*Be able to withstand an air temperature range of -3'''°'''C to 36'''°'''C'''.'''
*Be able to operate in the dark
*Be able to operate in the rain
*Be able to withstand wind forces of 4 beaufort.
*
 
'''Should'''
 
*Have life assist systems
*Resist (bad) weather to a degree
*A person recognition algorithm
 
'''Could'''
 
*Have communication between victim and ship
 
'''Won’t'''
 
*Have the ability to take the person to safety
 
===Approach milestones and deliverables===
TODO
 
===Task division===
{| class="wikitable"
|Person
|Week 1
|Week 2
|Week 3
|Week 4
|Week 5
|Week 6
|Week 7
|Week 8
|-
|Luc
|Literature study (4-5 articles)+Subject picking
|Further Brainstorming and subject refining
|Literature study (4-5 articles):  '''Search patterns, Remote drone control''' ''> necessary equipment''
|Search pattern and area, TIC, (remote control)
|get familiar with raspberry pi
|
|
|
|-
|Thijs
|Literature study (4-5 articles)+Subject picking
|Further Brainstorming and subject refining
|Literature study(4-5 articles):  '''Communication systems''' ''> necessary equipment''
|Communication systems
|get familiar with raspberry pi
|
|
|
|-
|Geert
|Literature study (4-5 articles)+Subject picking
|Further Brainstorming and subject refining
|Literature study(4-5 articles):  '''Influence of cold water, Oceanic Weather'''
|Ethics of rescue robots, write on scenario (specifics)
|get familiar with raspberry pi, look into code for search algorithm/pattern.
|
|
|
|-
|Victor
|Literature study (4-5 articles)+Subject picking
|Further Brainstorming and subject refining
|Literature study(4-5 articles):  '''Image recognition /sensors''' ''> necessary equipment, Take care of the wiki page''
|TIC, other sensors and image recognition
|get familiar with raspberry pi, look into code for search algorithm/pattern.
|
|
|
|-
|Adrian
|Literature study (4-5 articles)+Subject picking
|Further Brainstorming and subject refining
|Literature study(4-5 articles):  '''Night time deck procedures'''  ''> What does the victim have on him/her?''
|Finalize state-of-the-art, communication system (from thursday)
|get familiar with raspberry pi
|
|
|
|-
|Aron
|Literature study (4-5 articles)+Subject picking
|Further Brainstorming and subject refining
|Literature study(4-5 articles):  '''Current tech''' ''> write a section on  the current state of person detection tech in the wiki''
|Look into dropping equipment from the drone. (take into account wind direction and ocean currents)
|get familiar with raspberry pi
|
|
|
|}


==State of the art: Literature Study==
==Introduction==
Working on board a cargo ship is a dangerous occupation, with man overboard (MOB) incidents being a particularly serious concern. 91 fatalities were reported as a result of MOB events aboard cargo ships between 2014 and 2021<ref name=":9">ANNUAL OVERVIEW OF MARINE CASUALTIES AND INCIDENTS 2022. (2022). In ''EMSA'' (Ares(2022)8241169). European Maritime Safety Agency. from <nowiki>https://www.emsa.europa.eu/newsroom/latest-news/download/7362/4867/23.html</nowiki></ref>. Numerous factors, including; bad weather, unstable loads, and a lack of adequate safety equipment, are the main cause of such accidents. Unfortunately, locating a MOB victim in the ocean can be a challenging and complicated task. The search procedure can be affected by all sorts of elements such as the climate, ocean currents, and visibility.


===Search patterns===
To better understand the scope of the issue, a 2020 study analyzed 100 MOB incidents using 114 parameters to generate a MOB event profile<ref name=":0">Gonel, O., & Çiçek, I. (2020, 25 december). ''STATISTICAL ANALYSIS OF MAN OVER BOARD (MOB) INCIDENTS''. ResearchGate. <nowiki>https://www.researchgate.net/publication/348266442_STATISTICAL_ANALYSIS_OF_MAN_OVER_BOARD_MOB_INCIDENTS</nowiki></ref>. 53 of the 100 events that were looked at occurred on cargo ships. In 88 cases, the casualty was deceased as a result of the MOB incident, from those 88 cases 34 were assumed dead and 54 were witnessed dead. From the witness deaths 18 died before the rescue and 31 after the rescue, in 5 cases it is unknown. The cause of death was indicated for 42 cases, the most common cause is identified as drowning (26), followed by trauma (9), cardiac arrest (4) and hypothermia (3).
Search procedures for when a man goes overboard already exist. In the IAMSAR manual some of these procedures are already explained. I want to start discussing some of the search patterns described.


For different environments and conditions different patterns are recommended. When the location of the Man overboard is known well, Expanding Square or Sector search is recommended. If the location of the accident is not accurately known. different patterns are recommended such as sweep search.  
Despite the challenges involved in locating MOB victims in the ocean, we believe that technology can play a crucial role in increasing the survival chances of those involved in such incidents. With this in mind, we aim to develop a device that can improve the efficiency of locating MOB victims on cargo ships. Specifically, we will focus on MOB incidents that occur on cargo ships, as the 2020 study indicated that over half of such incidents occur on these vessels. The end users of our product are personnel of cargo ships, shipping companies that own a fleet of cargo ships, ports, and possibly the military.


<br />
It's worth noting that searching for a victim in the ocean can be a challenging and complex task, we therefore make several assumptions. First, we presume that both at the time of the accident and during the search and rescue effort, the weather conditions are mild. This means mild waves, a mild breeze, little rain, and average sea and air temperatures. We also assume that it is night-time at the time of the incident and that there is only one victim who is capable of maintaining afloat and does not have any major injuries. Additionally, we assume that the time of the incident is known, allowing for a general understanding of the search area.


*Expanding Square search - expanding square search can only be done by one ship at a time. the pattern is starts at the approximate location of the man over board and spirals outwards with course alterations of 90°
To develop our device, we will conduct a state-of-the-art literature study to investigate various factors that can impact the search process. This will include examining search areas, search patterns, oceanic currents and temperatures, and current person-in-water detection technology.


*Sector Search - this can be done using one vessel, or using a vessel and an aircraft. the method is used to search a spherical area. the pattern is depicted in .....
We hope to increase the efficiency of the search process and further improve the survival chances of those involved in MOB incidents on big (cargo) ships. By developing this device, we believe that we can make a meaningful contribution to the safety of those who work at sea.
==State-of-the-art literature study==
Before we can get started with the project it is important to gather knowledge about the state-of-the-art (SotA) concerning safety measures and pre-existing technology that aids victims of a MOB incident. We therefore performed a SotA literature study.


*Sweep search - this is a pattern used to search a bounded area. the ship zig zags down it.
===Current search methods===
Search procedures for when a man goes overboard already exist. In the IAMSAR manual some of these procedures are explained. In this paragraph we discuss and analyze these procedures and go over some different search patterns used.


====Current Procedures<ref>https://www.marineinsight.com/marine-safety/man-overboard-situation-on-ship-and-ways-to-tackle-it/#:~:text=1</ref>====
The existing procedures for people overboard on big cargo ships are quite elaborate. The procedure basically consists of a few key steps: it all starts when a victim goes overboard, from there the following will happen:


IAMSAR also mentions other factors need to be taken into account. If a person falls into the water they will for example be moved away by currents.
Step 1: Notify staff on board


===Remote Drone Control===
Step 2: Mark the position the victim went overboard / release a marker
A lot of research has already gone into autonomous drone flight. Take a look at autonomous drone racing for example<ref>https://link.springer.com/article/10.1007/s10514-021-10011-y</ref> . A lot of consumer drones also are capable of autonomous flight, for example DJI has a waypoint system based on GPS coordinates and their drones are also capable of tracking a moving person/object. DJI's waypoint system works by loading a set of GPS coordinates onto the drone, the drone will navigate itself to the first set coordinates. From there it goes on to the next. Until it has reached the final set of coordinates. The moving object tracking is a bit more complicated. This involves image recognition.<ref>http://acta.uni-obuda.hu/Stojcsics_56.pdf</ref> 


However one problem with the autonomous drone flight available in most commercial drones is that it is not possible to interface with the controller via a raspberry pi. So as of now we have three main options:
Step 3: Turn the ship around


#We built a drone ourselves
Step 4: Final search (finding the marker and victim)
#We manage to find a drone somewhere that can be controlled via a raspberry pi
#We try to work around a separation of the drone control and the camera input


Right now to me (Thijs) option two seems the best but it would entirely depend on us being able to find a drone we can use, and that is not very likely unless the university has such a drone available. If we can't find a drone that we can control it might be best to look a bit more into building our own drone. One problem with that is that none of the group members really have experience with this and that this add extra complexity in the manufacturing of the prototype/scale model that might not be desirable since the actual construction of the base drone is not really a focal point for this project.
Step 5: Rescue


If we do want to make our own drone there are some tutorials available, but it will probably still be difficult because not all of the exact materials they use are readily available and some parts are quite expensive. For example the pixhawk flight controller (the most popular option for autonomous flight as far as I can tell) costs between €60 for an older model and €185 for a newer model. There is also a tutorial that uses an alternative flight controller that seems to be cheaper (€30) but it is a little unclear how well that controller would work for autonomous flight.
At first this process seems quite efficient, but what should be taken into account is that Step 3, can take a long time for big ships<ref>https://content.iospress.com/download/international-shipbuilding-progress/isp23-260-02?id=international-shipbuilding-progress%2Fisp23-260-02<br /></ref> . For some ships this can take over 20 minutes while traveling a few kilometers. Then there are also some factors going into the step 4: For different environments and conditions different patterns are recommended. Some of these will be explained below.<ref>https://owaysonline.com/iamsar-search-patterns/</ref>


For the option where we work around not being able to control the drone and the camera's together we could make the flight autonomous via the provided app/drone interface and then give alerts to the ship when the camera detects a heat signature without then also stopping the drone to follow the heat signature and provide better localization.
=====Expanding Square Search=====
The expanding square search is a method that can be used to systematically inspect a body of water for signs of an MOB victim. The search should start at the last known location of the victim, from that point the search starts and expands outwards with course alterations of 90°, as can be seen in the image provided. The course of expansion heavily depends on the specification of the device that will be used. Thermal imaging can be used to scan the area, looking for any signs of the victim such as floating debris or the person themselves.


https://www.hackster.io/korigod/programming-drones-with-raspberry-pi-on-board-easily-b2190e
=====Sector Search=====
Sector search is another technique that can be used to search for a victim in the ocean. The sector search technique involves dividing the search area into sectors or pie-shaped sections and systematically searching each sector. It again starts at the last known location of the victim. From there the search area is divided and each sector is carefully searched in a clockwise fashion. Sector search can be effective in searching large areas, especially in cases where the search area is relatively circular or symmetric. However, sector search may not be the most efficient or effective method in all situations, especially if the search area is irregular or if weather and ocean conditions make it difficult to maintain a consistent search pattern.


https://www.instructables.com/The-Drone-Pi/
=====Sweep search=====
The sweep search pattern is another technique that can be used to search for a victim in the ocean. It involves moving the search vessel or search device in a back-and-forth sweeping motion across the search area.[[File:Expanding Square.png|thumb|Expanding Sqaure, IAMSAR search patterns<ref name=":4">O. (2021, 5 april). ''IAMSAR Search Patterns Explained with Sketches - Oways Online''. Oways Online. <nowiki>https://owaysonline.com/iamsar-search-patterns/</nowiki></ref>|left]]
[[File:Sweep Search.png|thumb|Sweep Search, IAMSAR search patterns<ref name=":4" />]]
[[File:Sector Search.png|center|thumb|Sector Search, IAMSAR search pattern<ref>O. (2021, 5 april). ''IAMSAR Search Patterns Explained with Sketches - Oways Online''. Oways Online. <nowiki>https://owaysonline.com/iamsar-search-patterns/</nowiki></ref>]]


https://all3dp.com/2/raspbery-pi-drone-simply-explained/#:~:text=A%20Raspberry%20Pi%20can%20enable,while%20the%20UAV%20is%20flying


https://hobbyking.com/en_us/multiwii-328p-flight-controller-w-ftdi-dsm2-comp-port.html
When the location of the MOB victim is known well, the Expanding Square or Sector Search is recommended. If the location of the accident is not accurately known different patterns are recommended such as Sweep Search. IAMSAR also mentions other factors that need to be taken into account during the search. If a person falls into the water they will for example be moved away by currents.  


http://ftec-shop.nl/shop/FMPro?-db=ftec%20producten.fp3&-format=hwkiosk.htm&-lay=cgi&-sortfield=code&subcat1=pixhawk&-max=25&-find
===Search area===
 
A person floating in the ocean can drift up to 10 nautical miles per day, this is equal to 0.2 meters per second. If the current is unknown this will mean the search area would expand in a circle if the starting location would be known. If we define the search area as a circle with a radius ''r'', the radius of the circle will depend on time, we describe the search area with the following formula: ''r = 0.2 t''. Here ''t'' is the time since the person fell overboard. The following table shows the search area size compared to time.
https://ramyadhadidi.github.io/files/Drone-Build-Guide.pdf
 
https://www.researchgate.net/figure/Raspberry-Pi-camera-attached-to-the-UAV_fig4_338096276
 
 
====Search Area====
Drift upto 10 (nautical) miles per day (<nowiki>http://www.oregonbeachcomber.com/2012/07/mechanics-of-drifting-on-open-sea.html#:~:text=Your%20average%20ocean%20wind%20is,blows%20552%20miles%20a%20day</nowiki>.)
 
This will be about 0.2 m/s
 
We define the search area as a circle, this circle has radius r, where r = 0.2t (where t is the time since the person fell overboard.
 
This following table shows the search area size compared to time.
{| class="wikitable"
{| class="wikitable"
|Time overboard (minutes)
|Time overboard (minutes)
|Search area size (m^2)
|Search area size ()
|-
|-
|1
|1
Line 216: Line 99:
|101788
|101788
|}
|}
<br />
The table clearly shows the search problem can become quite big. Especially when the search takes a long time. This shows the importance of finding the victim quickly.<br />[[File:Shipping routes.png|thumb|Image showing the world's shipping routes on the Atlantic ocean<ref name=":3">[https://www.vox.com/2016/4/25/11503152/shipping-routes-map "Shipping routes are changing with the climate" by Brad Plumer, published on Vox on April 25, 2016 (https://www.vox.com/2016/4/25/11503152/shipping-routes-map).]</ref>]]
 
===Influence of cold water and oceanic temperatures===
====Drone search speed====
Having a good understanding on how factors such as water temperature influence the rescue mission is crucial in ensuring that the rescue mission is successful. Depending on the water temperature, the time frame of the rescue mission changes and different measures need to be taken into account. Ideally the victim should be rescued as soon as possible, but having a known time frame of within which the mission needs to be executed gives the crew members a good idea on how to approach the situation.
The drone search speed will be limited by multiple things such as flight speed, sensor resolution and quality/speed of the detection algorithm. Since we still have some unknowns I would like to make some assumptions. Lets assume the algorithm is able to detect a human if they take up at least a space of 25 by 25 pixels. Lets also assume a human at sea takes up 25 by 25 cm.
 
A good sensor to use for the detection of humans at sea would be a IR camera. These cameras take images in the IR spectrum. They have a field of view(fov), a resolution and a refresh rate.
 
The height the drone can fly will be determined by the fov, and resolution. A typical fov for raspberry pi cameras is about 60 degrees of horizontal (<nowiki>https://www.raspberrypi.com/documentation/accessories/camera.html</nowiki>) and a typical resolution would be 1080*1920 pixels. Here 1920 is the horizontal direction.
 
If the drone would have a sensor of these specs it would be able to fly about 15 meters high, here it would be able to scan a line of 19.2 meters wide.
 
Lets assume the algorithm needs to have the human in shot for 5 seconds for it to be detected by the algorithm (<nowiki>https://www.researchgate.net/publication/350981551_Real-Time_Human_Detection_in_Thermal_Infrared_Images</nowiki>)
 
The vertical fov of raspberry pi sensors is about 40 degrees, this would result in a distance of about 10 meters vertical to be seen by the drone. This means it can fly at about 2 m/s.
 
The area the drone can covers is thus 20m*2m/s*t where t is the time since dispatched.
[[File:Drone search speed.png|thumb|X(axis) = time in seconds, Y(axis) = area, the blue line is area to be searched and the yellow line is area searched. ]]
<br />
 
====Can a drone keep up?====
in the graph on the right we see that after about 5 minutes the drone is not able to keep up with the search area anymore (even with a perfect search pattern). If we start taking dispatch time into account it would mean these 5 minutes would be about  result in about 2.5 of effective search time) this would move the yellow line to the left, this would also mean that one drone is not quick enough. and we would likely need to use multiple drones in order to search fast enough..
 
 
 
 
===Communication Systems===
For sea communication it might be a good idea to use RF (radio frequency) communication. The main limit to this seems to be the line-of-sight requirement, the transimission distance and the available bandwidth. The bandwidth might mostly be a problem if we want to send (live) video in a reasonable quality to the ship.
 
There is also reasearch into using a long range wifi network for sea connection, but this probably wouldn't work too well for a scale model since we most likely want to test the model outside where there is little to no wifi signal.<ref>https://ieeexplore.ieee.org/abstract/document/7151030</ref><ref>https://ieeexplore.ieee.org/abstract/document/6805843</ref>
 
For a model/prototype we would probably either have to use a seperate RF module for communication with the raspberry pi or use an inbuilt RF module in the drone controller if that is available.<ref>https://behind-the-scenes.net/433mhz-rf-communication-to-a-raspberry-pi/</ref>
 
 
 
In a scenario where a drone needs to send data, such as thermal images, to a container ship on the open sea, there are several reliable methods of communication that can be used, including:
 
#Satellite Communication: One of the most reliable methods of communication in remote areas, such as the open sea, is through satellite communication. The drone can use a satellite modem to transmit the data to a satellite in orbit, which then relays the data to a ground station or directly to the container ship. Satellite communication can support high data rates and can be used to transmit data over long distances.
#Long-Range Wi-Fi: Another option for communication between the drone and the container ship is through long-range Wi-Fi. The drone can be equipped with a Wi-Fi transmitter that is capable of transmitting data over a long distance, and the container ship can be equipped with a Wi-Fi receiver that is capable of receiving the data. Long-range Wi-Fi can support high data rates and can be used to transmit data over several kilometers, depending on the power of the transmitter and receiver.
#Cellular Communication: Depending on the location of the container ship and the availability of cellular coverage, the drone may be able to use cellular communication to transmit data to the ship. The drone can be equipped with a cellular modem that is capable of connecting to a cellular network, and the container ship can be equipped with a cellular receiver that is capable of receiving the data. Cellular communication can support moderate data rates and can be used to transmit data over several kilometers, depending on the strength of the cellular signal.
#Radio Communication: Another option for communication between the drone and the container ship is through radio communication. The drone can be equipped with a radio transmitter that is capable of transmitting data over a long distance, and the container ship can be equipped with a radio receiver that is capable of receiving the data. Radio communication can support low to moderate data rates and can be used to transmit data over several kilometers, depending on the power of the transmitter and receiver.
 
Overall, the choice of communication method will depend on the specific requirements of the scenario, such as the distance between the drone and the container ship, the data rate requirements, and the availability of communication infrastructure. Satellite communication is typically the most reliable and versatile option in remote areas, but it may also be the most expensive.<ref>https://dev.px4.io/v1.11_noredirect/en/qgc/video_streaming_wifi_broadcast.html</ref>
 
 
FPV (First Person View) drones use a variety of wireless communication technologies to transmit live video feeds to the ground station or the pilot's goggles. The combination of wireless technologies used by an FPV drone depends on the drone's design and the user's preference. Here are some of the most common wireless communication technologies used by FPV drones:
 
#Analog video transmission: Analog video transmission is the most common technology used by FPV drones for video transmission. It uses a 5.8GHz radio frequency (RF) band to transmit the video signal wirelessly to the ground station or the pilot's goggles. The video signal is usually low-latency and provides a real-time video feed for the pilot.
#Digital video transmission: Digital video transmission is becoming increasingly popular among FPV drone users due to its higher image quality and reliability compared to analog video transmission. Digital HD (DHD) systems, such as DJI's HD FPV system, use a 2.4GHz RF band to transmit the video signal wirelessly to the ground station or the pilot's goggles. DHD systems provide high-definition video with low latency and low interference, making them ideal for professional FPV pilots.
#Wi-Fi video transmission: Some FPV drones use Wi-Fi to transmit the video signal wirelessly to a smartphone or a tablet. Wi-Fi video transmission is not as reliable as analog or digital transmission and can suffer from latency and interference issues. However, it is a convenient option for casual FPV users who want to view the drone's video feed on their mobile device.
#Cellular data transmission: Some FPV drones can use cellular data networks to transmit live video feeds to a remote location. This option requires a cellular data plan and may suffer from latency and data transfer limitations. It is typically used for long-range FPV flights and commercial applications such as search and rescue.<ref>https://www.dronezon.com/learn-about-drones-quadcopters/learn-about-uav-antenna-fpv-live-video-transmitters-receivers/</ref>
 
<br />


===Influence of cold water and oceanic temperatues===
Since we're dealing with people who have fallen in water, we need to consider their survival time. When being submerged in water, your greatest risk is developing hypothermia as your body cools down 25 times faster in water than in air of the same temperature. The effects of hypothermia go from confusion and exhaustion, to unconsciousness and coma, and in the end lead to death. It is thus of utmost importance to detect and rescue a person in the water before the severe effects of hypothermia set in. The following table shows the relation between the temperature of the water and the time frame of the effects of hypothermia. The time windows are wide as the time when the effects of hypothermia begin depend on several factors, like age and percentage of body fat.<ref name=":1">[https://aceboater.com/en-us/cold-water-immersion-and-hypothermia "Cold Water Immersion and Hypothermia" by Aceboater, published on their website on an unknown date (https://aceboater.com/en-us/cold-water-immersion-and-hypothermia).]</ref>
Since we're dealing with people who have fallen in water, we also need to consider their survival time. When being submerged in water, your greatest risk is developing hypothermia as your body cools down 25 times faster in water than in air of the same temperature. The effects of hypothermia go from confusion and exhaustion, to unconsciousness and coma, and in the end lead to death. It is thus of utmost importance to detect and rescue a person in the water before the severe effects of hypothermia set in. The following table shows the relation between the temperature of the water and the time frame of the effects of hypothermia. The time windows are wide as the time when the effects of hypothermia begin depend on several factors, like age and percentage of body fat.<ref name=":1">https://aceboater.com/en-us/cold-water-immersion-and-hypothermia</ref>
{| class="wikitable"
{| class="wikitable"
|+Relation between water temperature and hypothermia<ref name=":1" />
|+Relation between water temperature and hypothermia<ref name=":1" />
Line 300: Line 133:
|3 - 12 hours
|3 - 12 hours
|3 - indefinite
|3 - indefinite
|}
|}[[File:Water temp.png|thumb|Image showing oceanic temperatures on 22 March 2023<ref name=":2">[https://www.seatemperature.org/ "SeaTemperature.org - Sea Surface Temperature (SST) map for the world's oceans and seas" by SeaTemperature.org, accessed on April 9, 2023 (https://www.seatemperature.org/).]</ref>]]Next, oceanic temperatures need to be considered. The two figures on the right show the world's shipping routes on the Atlantic ocean, and oceanic temperatures respectively. From this we can deduce that there are busy shipping routes in the colder regions of the Atlantic ocean, ranging from 3 to 7 degrees Celsius, and also the occasional ship in water at 1 degrees Celsius. The temperatures were taken in late March so it might even be colder in the middle of winter. Since these ships will benefit most from a technology that might result in faster rescue times, it can be concluded at the most desirable performance of the system is to find a person in the water in less than 10 minutes, leaving enough time for the rescue operation to start.<ref name=":2" /><ref name=":3" />
Next, oceanic temperatures need to be considered. The two figures on the right show the world's shipping routes on the Atlantic ocean, and oceanic temperatures respectively. From this we can deduce that there are busy shipping routes in the colder regions of the Atlantic ocean, ranging from 3 to 7 degrees celsius, and also the occasional ship in water at 1 degrees celsius. The temperatures were taken in late March so it might even be colder in the middle of winter. Since these ships will benefit most from a technology that might result in faster rescue times, it can be concluded at the most desirable performance of the system is to find a person in the water in less than 10 minutes, leaving enough time for the rescue operation to start.<ref name=":2">https://www.seatemperature.org/</ref><ref name=":3">https://www.vox.com/2016/4/25/11503152/shipping-routes-map</ref>[[File:Shipping routes.png|thumb|Image showing the world's shipping routes on the Atlantic ocean<ref name=":3" />]]
[[File:Water temp.png|thumb|Image showing oceanic temperatures on 22 March 2023<ref name=":2" />]]
<br />
<br />
===Possible detection methods===
There already exists technology that can detected a person in a body of water. Radars are frequently used to detect the distance and velocity of an object by sending electromagnetic waves out and detecting the echo that results from objects. In water environments, a radome is needed to protect the radar but let the electromagnetic waves through. However, even with a radome the signal strength already falls to a third in a wet environment compared to dry. Multiple radar technologies exist with each their different advantages and disadvantages. Pulse Coherent Radar (PCR) can pulse the transmitting signal so it only uses 1% of the energy. Another aspect of this radar is coherence. This means that the signal has a consistent time and phase so the measurements can be incredibly precise. It can also separate the amplitude, time and phase of the received signal to identify different materials. This is not necessarily needed in our use case. The 'Sparse' service is the best service for detecting objects since it samples waves every 6 cm. You don't want millimeter precise measurements in a rough environment like the ocean so these robust measurements are ideal.<ref>Frotan, D. F., & Moths, J. M. (2022). ''Human Body Presence Detection in Water Environments Using Pulse Coherent Radar'' [Bachelor Thesis]. Malmo University.</ref>


===Sensors===
Another detection method that does not yet exist is to send out a swarm of drones to search for victim. The research paper "Intelligent Drone Swarm for Search and Rescue Operations at Sea"<ref>Lomonaco, V., Trotta, A., Ziosi, M., De Dios Yáñez Ávila, J., & Díaz-Rodríguez, N. (2018b). Intelligent Drone Swarm for Search and Rescue Operations at Sea. ''HAL (Le Centre pour la Communication Scientifique Directe)''.</ref> presents a novel approach for using a swarm of autonomous drones to assist in search and rescue operations at sea. The authors propose a system that uses machine learning algorithms and computer vision techniques to enable the drones to detect and classify objects in real-time, such as a person in distress or a life raft. The paper describes the hardware and software architecture of the drone swarm system, which consists of a ground station, multiple drones equipped with cameras, and a cloud-based server for data processing and communication. The drones are programmed to fly in a coordinated manner and share information with each other to optimize the search process and ensure efficient coverage of the search area. The authors evaluate the performance of their system in various simulated scenarios, including detection of a drifting boat, a person in the water, and a life raft. The results show that the drone swarm system is capable of detecting and identifying objects accurately and efficiently, with a success rate of over 90%. The research paper presents a promising solution for improving search and rescue operations at sea, leveraging the capabilities of autonomous drones and artificial intelligence. For larger scale operations, drone swarm can be used to cover greater areas for search and rescue. This method is useful when other factors such as the wind, current etc. make finding the victim more difficult. The drone swarm would be implemented in such a way so that areas around the ship, or any search area for that matter, can be searched in a shorter time span, increasing the chances of finding the victim and increasing the chances of survival. When multiple drones are deployed, they all take up a specific search pattern in unison. Depending on the situation different search patterns will be used. If a man overboard is detected rather quickly, one search pattern may be implemented in order to optimize the time it takes in order to locate the victim. When other factors influence the case, such as wind or current, different search patterns will be used in order to adapt to the situation.  
The main sensor that will be used to locate a victim is a thermal imaging camera (TIC). The sheer size of the ocean makes it nearly impossible to use a normal camera to detect a person, it is as looking for a needle in a hay stack. Since the general search area is small relative to the complete ocean, temperatures of the water will all be relatively the same. The victim will have a much higher temperature than their surroundings, making it easy to detect them. There are a couple important requirements for the TIC. The camera needs a sufficiently high resolution so that we can clearly distinguish the victim from other warm object (think of the marine life). It also needs to have a high refresh rate. Our goal is to detect and locate a victim as soon as possible so that the survival chance is the highest, we therefore need to 'scan' the search area fast and that requires a sufficient refresh rate. For the same reason the thermal camera is required to have a large field of view (FOV). It also needs the right temperature sensitivity, and a temperature bar in the interface.  


Other sensors that we might want to include are a microphone and a loudspeaker. This would allow us to make contact with the victim. However, the issue with a microphone is that the sound of the ocean is most likely so loud that it is unlikely to understand the victim, trying to conversate with the victim might also fatigue them. A loudspeaker can be used to notify the victim rescue is on the way and help them calm down.  
There are multiple papers that discuss drone swarms as a method of detecting people on the ocean. The research paper "AutoSOS: Towards Multi-UAV Systems Supporting Maritime Search and Rescue with Lightweight AI and Edge Computing"<ref>Queralta, J. P., Raitoharju, J., Gia, T. N., Passalis, N., & Westerlund, T. (2020). AutoSOS: Towards Multi-UAV Systems Supporting Maritime Search and Rescue with Lightweight AI and Edge Computing. ''arXiv (Cornell University)''. <nowiki>https://arxiv.org/pdf/2005.03409.pdf</nowiki></ref> proposes a similar approach for using multiple unmanned aerial vehicles (UAVs) equipped with lightweight AI and edge computing to support maritime search and rescue (SAR) operations. The authors describe the AutoSOS system, which consists of a ground station, multiple UAVs, and a cloud-based platform for data processing and communication. The UAVs are equipped with cameras and other sensors to detect and classify objects in real-time, such as a person in the water or a life raft. The lightweight AI algorithms are designed to operate on the UAVs, minimizing the need for high-bandwidth communication with the cloud-based platform. The paper highlights the advantages of using a multi-UAV system for SAR operations, such as increased coverage area, improved situational awareness, and faster response times. The authors also discuss the technical challenges of designing and implementing the AutoSOS system, such as optimizing the UAV flight paths and ensuring the reliability of the communication and data processing systems. To demonstrate the effectiveness of the AutoSOS system, the authors present a case study of a simulated SAR mission in a coastal area. The results show that the system is capable of detecting and identifying objects accurately and efficiently, with a success rate of over 90%.  


We also aim to provide life saving equipment such as a ring life buoy, this buoy can be attached to the drone and dropped by a mechanism making use of servo motors. It is harder than it sounds to drop something near the victim from an altitude above the ocean, wind sensors should be added so that calculations can be made to drop the buoy in close proximity of the victim.  
Another paper that is of relevance is the "A Review on Marine Search and Rescue Operations Using Unmanned Aerial Vehicles"<ref>Yeong, S. P., King, L. M., & Dol, S. S. (2015). A Review on Marine Search and Rescue Operations Using Unmanned Aerial Vehicles. ''Zenodo (CERN European Organization for Nuclear Research)''. <nowiki>https://doi.org/10.5281/zenodo.1107672</nowiki></ref>. It provides a comprehensive review of the current state of research on using unmanned aerial vehicles (UAVs) for search and rescue (SAR) operations in marine environments. The paper discusses the main challenges of conducting SAR operations in marine environments, such as limited visibility, harsh weather conditions, and the vast and complex nature of the search areas. The authors then review the different types of UAVs and their applications in SAR operations, such as fixed-wing UAVs for long-range surveillance and multirotor UAVs for close-range inspection and search missions. The paper highlights the advantages of using UAVs for SAR operations, such as increased coverage area, improved situational awareness, and reduced risk to human rescuers. The authors also discuss the technical challenges of designing and operating UAVs in marine environments, such as optimizing the flight paths, ensuring reliable communication and data transmission, and complying with regulatory and ethical guidelines. To demonstrate the effectiveness of UAVs for SAR operations, the authors review several case studies of real-world applications, such as detecting and rescuing distressed vessels and locating missing persons in coastal areas. The results show that UAVs can provide valuable support to SAR operations, improving response times, and increasing the chances of successful rescue.


====TIC====
Another paper, "Requirements and Limitations of Thermal Drones for Effective Search and Rescue in Marine and Coastal Areas"<ref>Burke, C., McWhirter, P. R., Veitch-Michaelis, J., McAree, O., Pointon, H. A. G., Wich, S. A., & Longmore, S. N. (2019). Requirements and Limitations of Thermal Drones for Effective Search and Rescue in Marine and Coastal Areas. ''Drones'', ''3''(4), 78. <nowiki>https://doi.org/10.3390/drones3040078</nowiki></ref>, explores the potential of thermal drones for search and rescue (SAR) operations in marine and coastal areas. The authors review the main features and capabilities of thermal drones, including their ability to detect heat signatures and identify objects in low light or night conditions. The paper highlights the key requirements and limitations of using thermal drones for SAR operations, such as the need for high-resolution thermal cameras, long flight time, and reliable communication systems. The authors also discuss the challenges of operating drones in harsh weather conditions and the importance of complying with regulatory and ethical guidelines. To demonstrate the effectiveness of thermal drones for SAR operations, the authors present a case study of a simulated rescue mission in a coastal area. The results show that thermal drones can provide valuable assistance in locating and identifying missing persons or distressed vessels, particularly in remote or inaccessible areas. The research paper provides a comprehensive overview of the potential of thermal drones for SAR operations in marine and coastal areas, while also highlighting the technical and operational challenges that need to be addressed to ensure their effective use.
https://reprapworld.nl/elektronica/electronische-3d-merken/raspberry/raspberry-pi-camera-v2-infrarood-8mp-3280x2464-resolutie/&f=gs?utm_source=beslist&utm_medium=referral&bl3nlclid=e6745299-134d-4654-a46a-bc0da1bb118f&utm_term=BESLIe6745299-134d-4654-a46a-bc0da1bb118f


This TIC is able to take pictures with a resolution of 3280 x 2464 pixels, and videos of 1080p at 30 fps, 720p at 60 fps and 640 x 480p at 90 fps. It has an angle of view (AOV) of 67 degrees
From these studies we conclude that the most efficient detection method is a drone swarm with thermal cameras attached to the drones.
====Sensors====
The main sensor that will be used to locate a victim is a thermal imaging camera (TIC). Since the general search area is small relative to the complete ocean, temperatures of the water will all be relatively the same. The victim will have a much higher temperature than their surroundings, making it easy to detect them. There are a couple important requirements for the TIC. The camera needs a sufficiently high resolution so that we can clearly distinguish the victim from other warm object (think of the marine life). It also needs to have a high refresh rate. Our goal is to detect and locate a victim as soon as possible so that the survival chance is the highest, we therefore need to 'scan' the search area fast and that requires a sufficient refresh rate. For the same reason the thermal camera is required to have a large field of view (FOV). It also needs the right temperature sensitivity.


Other sensors and actuators that we might want to include are a microphone and a loudspeaker. This would allow us to make contact with the victim. However, the issue with a microphone is that the sound of the ocean is most likely so loud that it is unlikely to understand the victim, trying to conversate with the victim might also fatigue them. A loudspeaker can be used to notify the victim rescue is on the way and help them calm down.
====Thermal image recognition====
For the image recognition we can make use of TensorFlow, Google's Neural Network library. We can, as mentioned before, also choose to not do advanced image detection and instead as soon as a heat source that could reasonably be the victim is detected, send the camera images to the ship where a human operator can verify whether the heat source is actually the victim.


Lets assume a human head has a width of 20 cm and length of 25 cm, the surface area would be 500 cm². If we fly the drone 500m above the water we would cover an area of 438km², however, this means each pixel represents an area of 541.6 cm² in picture mode, meaning we would not be able to recognize a person since it would not even be represented by one pixel. The drone has to fly pretty low in order to detect a person. If we fly the drone 10 meters above sea level a person would be represented by approximately 2306 pixels which would be a square of approximately 48x48 pixels. In video mode the TIC has a resolution of 1920x1080 pixels, flying at 10 meters above sea level a person would roughly be represented by 591 pixels, i.e. a square of about 24x24 pixels. At 720p and 10 meters above sea level a person would be represented by a square of approximately 16x16 pixels.  
Another option that could be looked into more would be using a hybrid of these two strategies. Namely, using the fast simple heat source detection until a heat source is found and then activating the more advanced AI image detection to verify if it is a human. This would then solve a part of the efficiency problem and would eliminate some need for human operators and higher data transfer requirements.


The downside of flying 10 meters above sea level is that only a limited area of  ~175m² can be covered.
===Communication Systems===
In a scenario where a drone needs to send data, such as thermal images, to a container ship on the open sea, there are several reliable methods of communication that can be used, including:


https://www.flir.eu/ this company offers top-of-the-line thermal imaging systems.
#Satellite Communication: One of the most reliable methods of communication in remote areas, such as the open sea, is through satellite communication. The drone can use a satellite modem to transmit the data to a satellite in orbit, which then relays the data to a ground station or directly to the container ship. Satellite communication can support high data rates and can be used to transmit data over long distances.
#Long-Range Wi-Fi: Another option for communication between the drone and the container ship is through long-range Wi-Fi. The drone can be equipped with a Wi-Fi transmitter that is capable of transmitting data over a long distance, and the container ship can be equipped with a Wi-Fi receiver that is capable of receiving the data. Long-range Wi-Fi can support high data rates and can be used to transmit data over several kilometers, depending on the power of the transmitter and receiver.
#Cellular Communication: Depending on the location of the container ship and the availability of cellular coverage, the drone may be able to use cellular communication to transmit data to the ship. The drone can be equipped with a cellular modem that is capable of connecting to a cellular network, and the container ship can be equipped with a cellular receiver that is capable of receiving the data. Cellular communication can support moderate data rates and can be used to transmit data over several kilometers, depending on the strength of the cellular signal.
#Radio Communication: Another option for communication between the drone and the container ship is through radio communication. The drone can be equipped with a radio transmitter that is capable of transmitting data over a long distance, and the container ship can be equipped with a radio receiver that is capable of receiving the data. Radio communication can support low to moderate data rates and can be used to transmit data over several kilometers, depending on the power of the transmitter and receiver.


====Servomotors====
Overall, the choice of communication method will depend on the specific requirements of the scenario, such as the distance between the drone and the container ship, the data rate requirements, and the availability of communication infrastructure. Satellite communication is typically the most reliable and versatile option in remote areas, but it may also be the most expensive.<ref>Long-distance Video Streaming and Telemetry via Raw WiFi Radio<nowiki/>https://dev.px4.io/v1.11_noredirect/en/qgc/video_streaming_wifi_broadcast.html</ref>
https://www.amazon.nl/AZDelivery-Versnellingsbak-Helikoptervliegtuigen-compatibel-Inclusief/dp/B07H89JH74/ref=asc_df_B07H89JH74/?tag=nlshogostdde-21&linkCode=df0&hvadid=594570766932&hvpos=&hvnetw=g&hvrand=7221170963569094828&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=1010481&hvtargid=pla-656765183755&th=1.


This servomotor is supported by a raspberry pi, it has an active force of 11 kg/cm. These can be used to drop possible equipment and to power the rotors of the drone in case we need to construct one.
==Design==
===Thermal image recognition===
For the image recognition we can make use of TensorFlow, Google's Neural Network library.
===Sea rescue drones (SOTA)===
The research paper "Intelligent Drone Swarm for Search and Rescue Operations at Sea"<ref>https://arxiv.org/abs/1811.05291</ref> presents a novel approach for using a swarm of autonomous drones to assist in search and rescue operations at sea. The authors propose a system that uses machine learning algorithms and computer vision techniques to enable the drones to detect and classify objects in real-time, such as a person in distress or a life raft.


The paper describes the hardware and software architecture of the drone swarm system, which consists of a ground station, multiple drones equipped with cameras, and a cloud-based server for data processing and communication. The drones are programmed to fly in a coordinated manner and share information with each other to optimize the search process and ensure efficient coverage of the search area.
===Requirements===
{| class="wikitable"
|+MoSCoW method
!Must
!Should
!Could
!Won't
|-
|Have sensors to detect a person within 10 minutes
|Have life assist systems
|Have communication between victim and ship
|Have the ability to take the person to safety
|-
|Have a communication system to communicate with the involved ship
|Resist (bad) weather to a degree
|
|
|-
|Be able to withstand an air temperature range of -3'''°'''C to 36'''°'''C'''.'''
|A person recognition algorithm
|
|
|-
|Be able to operate in the dark
|
|
|
|-
|Be able to operate in the rain
|
|
|
|-
|Be able to withstand wind forces of 4 beaufort.
|
|
|
|}
====Drone requirements====
A couple of requirements need to be met in order for the drone to be functioning as intended. In this section we will discuss the possibilities of remote drone control and how it ties into our case. Different communication methods will be discussed and we have to choose the best one for our scenario. And the detection method using thermal imaging will also be discussed. This is also tied with the drone search speed, which also influences our approach.


The authors evaluate the performance of their system in various simulated scenarios, including detection of a drifting boat, a person in the water, and a life raft. The results show that the drone swarm system is capable of detecting and identifying objects accurately and efficiently, with a success rate of over 90%.
=====Remote Drone Control=====
A lot of research has already gone into autonomous drone flight. Take a look at autonomous drone racing for example<ref>AlphaPilot: autonomous drone racing, Philipp Foehn, Dario Brescianini, Elia Kaufmann, Titus Cieslewski, Mathias Gehrig, Manasi Muglikar & Davide Scaramuzza


Overall, the research paper presents a promising solution for improving search and rescue operations at sea, leveraging the capabilities of autonomous drones and artificial intelligence.
https://link.springer.com/article/10.1007/s10514-021-10011-y</ref> . A lot of consumer drones also are capable of autonomous flight, for example DJI has a waypoint system based on GPS coordinates and their drones are also capable of tracking a moving person/object. DJI's waypoint system works by loading a set of GPS coordinates onto the drone, the drone will navigate itself to the first set coordinates. From there it goes on to the next. Until it has reached the final set of coordinates. The moving object tracking is a bit more complicated. This involves image recognition.<ref>Autonomous Waypoint-based Guidance Methods for Small Size Unmanned Aerial Vehicles, Dániel Stojcsics


"We propose an Intelligent Drone Swarm for tackling the aforementioned problems. The central idea is to use an autonomous fleet of intelligent UAVs to accomplish the task. The self-organizing UAVs network would enable the coverage of a larger area in less time, reducing the impact of the limited battery capacity. Recent works on UAVs networks shown their ability of coverage and connectivity improvements in emergency scenarios [15]. Indeed, the UAVs swarm is capable of generating a multi-hop communication network that will guarantee a higher bandwidth due to aerial-to-aerial communication links provided by longer LoS connections. Artificial Intelligence tools, in this case, would greatly improve not only the autonomy and coordination of the UAVs network, but also the communication efficiency. On one hand, autonomy and coordination are improved due to the autonomous adaptation of mobility actions and communication parameters to non-stationary environmental conditions. On the other, communication efficiency is ensured with the usage of smart detection algorithms locally running on the UAV, which limit the amount of information sent to the base station. Recent advances in AI techniques, such as specialized AI chips [16] and quantization/pruning techniques [17], may also help reducing the energy and computational resources needed for detection."
http://acta.uni-obuda.hu/Stojcsics_56.pdf</ref>


However, the firmware on most of these commercial drones is not open to being interfaced with. This makes is much more difficult to make a more autonomous drone without building one from scratch. Another point is that drones, and especially the autonomous kind, are quite expensive. And due to the aforementioned lack of interfacing options with the flight controllers of commercial drones it is also not really possible to take a cheaper non-autonomous drone and hook it up to a computer that will act as a module to make the drone fly autonomously.
=====Drone search speed=====
[[File:Drone search speed.png|thumb|X(axis) = time in seconds, Y(axis) = area, the blue line is area to be searched and the yellow line is area searched. ]]
The drone search speed will be limited by multiple things such as flight speed, sensor resolution and quality/speed of the detection algorithm. Since we still have some unknowns we would like to make some assumptions. Lets assume the algorithm is able to detect a human if they take up at least a space of 25 by 25 pixels. Lets also assume a human at sea takes up at least 25 by 25 cm. A good sensor to use for the detection of humans at sea in the night would be a heat camera. These cameras take images in the IR spectrum and thus aren't hindered by low light conditions. They have a field of view (FOV), a resolution and a refresh rate. The height the drone can fly will be determined by the FOV, and resolution. A typical FOV for Raspberry Pi cameras is about 60 degrees of horizontal<ref>https://www.raspberrypi.com/documentation/accessories/camera.htm</ref> and a typical resolution would be 1080*1920 pixels. Here 1920 is the horizontal direction. If the drone would have a sensor of these specs it would be able to fly about 15 meters high, here it would be able to scan a line of 19.2 meters wide. Lets assume the algorithm needs to have the human in shot for 5 seconds for it to be detected by the algorithm.<ref>https://www.researchgate.net/publication/350981551_Real-Time_Human_Detection_in_Thermal_Infrared_Images</ref> The vertical FOV of Raspberry Pi sensors is about 40 degrees, this would result in a distance of about 10 meters vertical to be seen by the drone. This means it can fly at about 2 m/s. The area the drone can covers is thus 20m*2m/s*''t'' where ''t'' is the time since dispatched. Another solution to the detection could be to not look for humans specifically, but for relatively hot areas in the image. This might be an option since out at sea there is a significant heat difference between a human with a body temprature of approximately 37 degrees celcius and the environment that consists mostly of cold water. When using this strategy it is however necessary to include some kind of verification in the system so that the drone will not signal that the victim is found when it, for example, finds a maritime animal close to the surface of the water.


The research paper "Requirements and Limitations of Thermal Drones for Effective Search and Rescue in Marine and Coastal Areas"<ref>https://www.mdpi.com/2504-446X/3/4/78</ref> explores the potential of thermal drones for search and rescue (SAR) operations in marine and coastal areas. The authors review the main features and capabilities of thermal drones, including their ability to detect heat signatures and identify objects in low light or night conditions.
However, for our estimation of the search speed we will still use the option where an algorithm is used that actually detects a human since this might still be necessary and we prefer to work from the worst case scenario with regards to bounding the minimum required efficiency. In the graph on the right we see that after about 5 minutes the drone is not able to keep up with the search area anymore (even with a perfect search pattern). If we start taking dispatch time into account it would mean these 5 minutes would be about  result in about 2.5 of effective search time) this would move the yellow line to the left, this would also mean that one drone is not quick enough. and we would likely need to use multiple drones in order to search fast enough..


The paper highlights the key requirements and limitations of using thermal drones for SAR operations, such as the need for high-resolution thermal cameras, long flight time, and reliable communication systems. The authors also discuss the challenges of operating drones in harsh weather conditions and the importance of complying with regulatory and ethical guidelines.
===Components===
Sensor


To demonstrate the effectiveness of thermal drones for SAR operations, the authors present a case study of a simulated rescue mission in a coastal area. The results show that thermal drones can provide valuable assistance in locating and identifying missing persons or distressed vessels, particularly in remote or inaccessible areas.
*Thermal imaging camera


Overall, the research paper provides a comprehensive overview of the potential of thermal drones for SAR operations in marine and coastal areas, while also highlighting the technical and operational challenges that need to be addressed to ensure their effective use.
Communication Equipment


*Wireless communication module


"We used a custom hexacopter drone based around a Tarot 680 airframe and Pixhawk 2.1 flight controller. The drone was equipped with a custom designed two-axis brushless gimbal mechanism to provide stabilisation of the camera in both roll and pitch. The gimbal also allowed independent and remote control of the camera’s pitch angle. The thermal camera used was a FLIR Duo Pro R. This consists of a Tau2 TIR detector (
Drone with autonomous/waypoint flight capabilities


pixels, 60 Hz) with a 13 mm lens ( field of view), and a 4K RGB camera (4000 × 3000 pixels,
Small computer capable of:


field of view) affixed side-by-side. The camera was interfaced to the flight controller to enable remote triggering and geotagging information to be recorded. Live video feedback was available in flight via a 5.8 GHz analogue video transmitter and receiver system."
*Multithreading/multiprocessing
*Running TensorFlow or other image recognition AI software
*Interfacing with electronics (reading the camera image and possibly interfacing with servo motors)
*Running autonomous flight control software


====Drone====
For our real design we would like to use a drone that is able to fly accurately in mild wind and is able to carry some weight. Since the drone has to fly autonomously it will most likely be necessary to make a custom drone for this project since almost all commercially available drones don't allow interfacing with the flight controller in such a way that an external device/computer can control the drone. We have not really researched into how to make a drone from scratch since this is not really in the scope of the project, however for the autonomous flight a pixhawk flight controller can be used. The pixhawk is currently the most popular option for autonomous flight as far as we can tell. The pixhawk can then be interfaced with with a Raspberry Pi to controll the autonomous movement of the drone.


The research paper "AutoSOS: Towards Multi-UAV Systems Supporting Maritime Search and Rescue with Lightweight AI and Edge Computing"<ref>https://arxiv.org/abs/2005.03409</ref> proposes a novel approach for using multiple unmanned aerial vehicles (UAVs) equipped with lightweight AI and edge computing to support maritime search and rescue (SAR) operations.
We have also not researched into the details of the control of autonomous drones since we already found out pretty early that using an actual drone like this for the proof of concept would not be feasable. The details of the control software would not really contribute to the rest of the design so for our demo we assumed it was possible to transform carthesian waypoints to flight instructions for the drone.


The authors describe the AutoSOS system, which consists of a ground station, multiple UAVs, and a cloud-based platform for data processing and communication. The UAVs are equipped with cameras and other sensors to detect and classify objects in real-time, such as a person in the water or a life raft. The lightweight AI algorithms are designed to operate on the UAVs, minimizing the need for high-bandwidth communication with the cloud-based platform.
====Computer====
We have decided to use a Raspberry Pi as our computer due to it's computing power to size/weight ratio. The Raspberry Pi can do all the things we wish our computer to do so it is a great choice.


The paper highlights the advantages of using a multi-UAV system for SAR operations, such as increased coverage area, improved situational awareness, and faster response times. The authors also discuss the technical challenges of designing and implementing the AutoSOS system, such as optimizing the UAV flight paths and ensuring the reliability of the communication and data processing systems.
===Search pattern===
[[File:Expanding square search.jpg|thumb|Expanding Square Search]]
[[File:Adapted square search.jpg|thumb|Adapted Square Search]]
[[File:Cone.jpg|thumb|Cone Search]]
[[File:Two drone cone.jpg|thumb|Multi drone Cone Search]]
[[File:Wait and sweep no target.jpg|thumb|Wait And Sweep Search]]


To demonstrate the effectiveness of the AutoSOS system, the authors present a case study of a simulated SAR mission in a coastal area. The results show that the system is capable of detecting and identifying objects accurately and efficiently, with a success rate of over 90%.


Overall, the research paper presents a promising solution for improving SAR operations at sea, leveraging the capabilities of UAVs, lightweight AI, and edge computing.
A critical factor that can determine the effectiveness of a search operation and its ability to meet our requirements is the search pattern employed. We have identified three potential search patterns that could be utilized, and we plan to test each of these methods using a model to evaluate their effectiveness. The selection of an appropriate search pattern is crucial as it can have a significant impact on the chances of success in locating the person who has fallen overboard. By testing each of the proposed search patterns using a model, we can determine which method is best suited for the specific circumstances of the search and optimize our search efforts accordingly.


One already existing search pattern is the expanding square search. This involves the drone moving outward from the last known location of the person in a square pattern, with each pass covering an increasing amount of area. This method is efficient and can cover a lot of ground quickly, but it does not take into account the direction of the current, so could in theory only be efficiently used in situations where the victim doesn’t drift.


The research paper "A Review on Marine Search and Rescue Operations Using Unmanned Aerial Vehicles"<ref>https://publications.waset.org/10001953/a-review-on-marine-search-and-rescue-operations-using-unmanned-aerial-vehicles</ref> provides a comprehensive review of the current state of research on using unmanned aerial vehicles (UAVs) for search and rescue (SAR) operations in marine environments.
To address this issue, an improvement on the expanding square search is to incorporate the direction of the current. We call this Adapted Square Search. This involves adding a few repetitions in the direction of the current to increase the chances of finding the missing person. The amount of repetitions is determined by the strength of the current; No current would mean no repetitions (regular expanding square), a weak current one extra repetition in that direction, a bit stronger two repetitions, etc. This method can be especially useful when there is a strong current that could have carried the person further away from their original location.


The paper discusses the main challenges of conducting SAR operations in marine environments, such as limited visibility, harsh weather conditions, and the vast and complex nature of the search areas. The authors then review the different types of UAVs and their applications in SAR operations, such as fixed-wing UAVs for long-range surveillance and multirotor UAVs for close-range inspection and search missions.
Another possible search pattern is the cone pattern. This involves the drone moving outward from a small distance before the last known location of the person in a cone-shaped pattern, with the width of the cone expanding in the direction of the current. This method could be the most efficient as it doesn’t return back to the initial position and doesn’t search against the current.


The paper highlights the advantages of using UAVs for SAR operations, such as increased coverage area, improved situational awareness, and reduced risk to human rescuers. The authors also discuss the technical challenges of designing and operating UAVs in marine environments, such as optimizing the flight paths, ensuring reliable communication and data transmission, and complying with regulatory and ethical guidelines.
Then there is also a different approach, since the current moves the victim will move, we can also wait for it to come to us. This method is called wait and sweep, here the drone does a sweep search pattern, but it sweep the same area over and over.


To demonstrate the effectiveness of UAVs for SAR operations, the authors review several case studies of real-world applications, such as detecting and rescuing distressed vessels and locating missing persons in coastal areas. The results show that UAVs can provide valuable support to SAR operations, improving response times, and increasing the chances of successful rescue.
===Programming===


Overall, the research paper provides a comprehensive overview of the potential of UAVs for SAR operations in marine environments, while also highlighting the technical and operational challenges that need to be addressed to ensure their effective use.
All of the code used in this project is uploaded on the following GitHub repository: https://github.com/ThijsEgberts/ProjectRobotsEverywhere.
===Proof of concept===
In order to prove the concept not all Moscow requirements need to be fulfilled. Since the most important part of our system is the detection, this needs to be shown working. The algorithms for both search and detection need to be proven and the hardware also needs to be shown functioning. However weather resistibility, does not have to be taken into account yet. The test also does not have to be done at full scale.


====Designing and building====
Materials:


- commercially available/are they trial runs (yes)
*Plastic container 40cm x 60cm
*2 Metal rods 8 mm x 100cm
*Linear ball bearing
*ABS (to print pieces)


-are they autonomous (nope)
[[File:Slider.png|thumb|3D model of slider]]
The design process of the prototype was done with Fusion 360. This program allows for detailed 3D-modeling and easy conversion to Prusaslicer which is the slicer program. After that the pieces that were designed are ready to be printed. The 3D-printer that was used is the Voron V0. We made a total of 4 corner attachment pieces, 2 sliders and 1 slider platform. The whole process required a lot of little design changes to properly fit and assemble the entire setup. Eventually the final assembly process was easily done with some screws and bolts. The search pattern was drawn on the bottom of the container to help the people moving the camera with following the movement instructions and make it more clear for the demonstration video what was going on.[[File:Corner piece.png|thumb|3D model of corner attachment piece|none]][[File:Hele setup.png|thumb|3D-model of entire setup|center]]
[[File:Platform.png|thumb|3D model of slider platform|left]]


-cover area (in the papers they talk about covering areas of 10x10km, for our scenario, the areas we have to deal with are way smaller)


<br />
===Current Person in water detection tech===
Currently, radars are used to detect the distance and velocity of an object by sending electromagnetic waves out and detecting the echo that results from objects. In water environments, a radome is needed to protect the radar but let the electromagnetic waves through. However, even with a radome the signal strength already falls to a third in a wet environment compared to dry. Multiple radar technologies exist with each their different advantages and disadvantages. Pulse Coherent Radar (PCR) can pulse the transmitting signal so it only uses 1% of the energy. Another aspect of this radar is coherence. This means that the signal has a consistent time and phase so the measurements can be incredibly precise. It can also seperate the amplitude, time and phase of the received signal to identify different materials. This is not necessarily needed in our use case.


The 'Sparse' service is the best service for detecting objects since it samples waves every 6cm. You don't want millimeter precise measurements in a rough environment like the ocean so these robust measurements are ideal.
<br /><br /><br />


<br />


Frotan, D., & Moths, J. (n.d.). ''Human body presence detection in water environments using pulse ...'' www.diva-portal.org. Retrieved March 6, 2023, from <nowiki>https://mau.diva-portal.org/smash/get/diva2:1668701/FULLTEXT02.pdf</nowiki>


===Dropping equipment from drone===
After locating and confirming a person in need we must drop a life jacket or lifebuoy to help them keep afloat. We need a device that can carry and drop the life jacket or lifebuoy on command. We can also use a rope or wire to lower the life jacket or lifebuoy to make sure the victim receives it properly. This reduces the risk of the victim being unable to reach for it and it floating away with the current or a sudden wave. We might even be able to use the drone to pull the victim to the shore or the boat if the water is not too rough. A life jacket can weigh from 3.5 kg to 10 kg depending on the type. Lifebuoys generally don't weigh more than 2.5 kg. Our drone should be able to carry about 15 - 20 kg so this should be enough for a life jacket/ lifebuoy and dropping mechanism.


Rescue drone saving boy in Spain:  <nowiki>https://youtu.be/fWiw5qMF-Jo</nowiki>


Home made payload dropping device: <nowiki>https://www.youtube.com/watch?v=iZnQN5AmDdE</nowiki><br />
==Seven core ethically relevant themes==
From the paper "Ethical concerns in rescue robotics: a scoping review"<ref>https://link.springer.com/article/10.1007/s10676-021-09603-0</ref>
===Fairness and discrimination===
Hazards and benefits should be fairly distributed to avoid the possibility of some subjects incurring only costs while other subjects enjoy only benefits. This condition is particularly critical for search and rescue robot systems, e.g., when a robot makes decisions about prioritizing the order in which the detected victims are reported to the human rescuers or about which detected victim it should try to transport first.


====Fairness and discrimination in our scenario====
When there’s a situation where multiple people have fallen over board, discrimination is something to look out for. When searching for people, discrimination can’t take place because if the robot has found one person it will continue looking for the others and since it doesn’t take people out of the water it doesn’t need to make decisions about prioritizing people. However, we are also considering dropping lifebuoys and here prioritizing order does come into account. In the situation where the robot can only carry one life vest, we propose that it drops the lifebuoy one the first person it finds, then comes back to the ship to restock, and continues the search. This will be the most efficient and also prevents discrimination between multiple victims.


===False or excessive expectations===
Stakeholders generally have difficulties with accurately assessing the capabilities of a rescue robot. May cause them to overestimate and rely too much on the robot and give false hopes for the robot to safe a victim. On the other hand, if a robot is underestimated, it may be underutilized in cases where it could have saved a person.


====False or excessive expectations in our scenario====
It is important for us to accurately inform stakeholders about the capabilities and limitations of the rescue robot.
<br />
<br />
====Demo====
A Raspberry Pi and a thermal camera were mounted on a rig to demonstrate the proof of concept. Ropes were used to manually modify the camera's position. To make it easier to follow the search path, a map of the search pattern was drawn on paper. A tiny tea light was used as a heat source to imitate the relevant scenario and provide a distinct heat signature. Also, a rope that was tied to the lamp served as an ocean current replica. The heat source was identified by following the search pattern according to instructions being output by the Raspberry Pi.


===Labour replacement===
=====Thermal camera=====
Robots replacing human rescuers might reduce the performance of human contact, situational awareness and manipulation capabilities. Robots might also interfere with rescuers attempt to provide medical advice and support.
For our demo we used the MLX90640 thermal camera. This camera has a resolution of 32x24 pixels and a 55° field of view. The camera itself has a refresh rate of 60fps, however in our application this is limited due to the speed at which the I<sup>2</sup>C interface works. Therefore in our application we could process images for heat sources at about 7fps when not displaying the image and at about 1fps when displaying the image. This speed may in part be due the use of python but python did make the image manipulation and output much easier.


====Labour replacement in our scenario====
=====Heating element=====
Since it is important to not fully take the human aspect out of the equation, the robot should be equipped with speakers for people on the ship to talk to the person. They can check how he’s responding and reassure the victim that they’re coming to help. This will most likely help to calm the person. When the rescue attempt is being made by the crew or other rescuers, the drone should fly away since it’s no longer useful and as to not interfere with the rescue attempt.
For the experiment we needed to have a detectable heat signature. For this we chose the simplest possible solution, which was just having a candle. This gave us a detectable heat signature that we can work with.


===Privacy===
====Design of the Model====
The use of robots generally leads to an increase in information gathering, which can jeopardize the privacy of personal information. This may be personal information about rescue workers, such as images or data about their physical and mental stress levels, but also about victims or people living or working in the disaster area.
In order to investigate how efficient the drone is at finding the victim a model was developed. The workings of the model will be explained here:


====Privacy in our scenario====
The model simulates the victim moving in the water, for a few minutes before the drones arrive at the scene. After this dispatch time, it starts moving the drone(s) simultaneously to the victim. From here it checks if the victim is in view of the drone’s view every time the drone is moved to a new location. If this is the case, the victim is found and the search ends.
The information gathered by the robots is not shared with anyone outside professional rescue organizations and is exclusively used for rescue purposes. We should also try to limit the gathering of irrelevant data as much as possible.


===Responsibility===
Victim movement is simulated using some randomness. The victim is moved every second at a random angle, this angle is taken using a normal distribution (centered at the angle the current is at). The victim is also moved forward every second, this distance is uniformly varied between the a flow rate of 5cm per second below and above the current. The randomness is included into the victim movement in order to have every search be a unique one, and sufficiently test the search algorithms.
The use of (autonomous) rescue robots can lead to responsibility ascription issues in the case when for example accidents happen.


====Responsibility in our scenario====
Drone movement is a bit different from victim movement, this is totally not random. The drones are moved along a pre-determined search pattern.  
Clearly state who is (legally) responsible in the case accidents happen during rescue (like operator, manufacturer etc.) Proposal: To ensure trust in our product, we should claim full (legal) responsibility as a manufacturer when accidents happen caused by design flaws or decisions made by the autonomous system.


===Safety===
The full code of the model can be found on our programming Github
Rescue missions necessarily involve safety risks. Certain of these risks can be mitigated by replacing operators with robots, but robots themselves, in turn, may determine other safety risks, mainly because they can malfunction. Even when they perform correctly, robots can still be harmful: they may, for instance, fail to identify and collide into a human being. In addition, robots can hinder the well-being of victims in subtler ways. For example, the authors argue, being trapped under a collapsed building, wounded and lost, and suddenly being confronted with a robot, especially if there are no humans around, can in itself be a shocking experience


====Safety in our scenario====
Risks should be contained as much as possible, but we do acknowledge that rescue missions are never completely risk free


===Trust===
Trust depends on several factors; reputation and confidence being the most important once. Humans often lack confidence in autonomous robots as they find them unpredictable.


====Trust in our scenario====
Ensure that the robot has a good reputation by informing people about successful rescue attempts and/or tests, and limit unpredictable behaviour as much as possible. We can also ensure trust by claiming full (legal) responsibility as a manufacturer when accidents happen caused by design flaws or decisions made by the autonomous system.


==Design phase==


====Components:====
<br />
Sensors
==Results==
In this section we will talk about our results surrounding tests with our design, namely our physical demo test and our model tests.


*TIC
====Demo results====
*Wind sensor
[[File:Adapted square.jpg|thumb|Model of Adapted Square Search]]
*Servomotors
Our main demo results are that it is possible to detect heat sources and follow a search pattern with a Raspberry Pi drone model. For the demo we only executed the adapted expanding square search pattern due to practicalities with our test setup, namely that our test rig was relatively small and rectangular.
*


Communication Equipment
====Model results====
 
Different search patterns also influence the speed it takes to find the target. Multiple different patterns were investigated using a model made in Python. Using this model different search patterns were each tested for 10000 trials. A current of about 25 cm/s is used in the positive y direction. The drone has a field of view of 3 m by 2.4 m (this matches the FOV of the camera we used for the demo), and is moved at a max speed of 3m/s. 4 different patterns were investigated. These are Square Search, Adapted Square Search, Wait and Sweep, and a Cone search. The full results can be seen in the Appendix
*Microphone
*Loudspeaker
 
Drone


<br />
<br />


===Programming:===
======Single drone======
Square search is one of the most simple patterns. The patterns simply spiral out, in a square. This pattern is however not very applicable to our case. Since the target moves in (mostly) one direction, and is not standing still. This can also be seen in the simulation results. Of the 10000 trials done, only 1.4 % succeeded. With an average total time of 313 seconds including 4 minutes of dispatch time.


====Python: threshold image detection:====
Since the Square search did not seem to work very well, an adapted version of the algorithm was set up, this Adapted Square search puts more focus on the flow direction, by looping back on itself . This pattern saw serious improvement over the ‘normal’ square search. The success rate went up to 22.4% with a total time of 326 seconds including the dispatch time.  [[File:Wait and sweep.jpg|thumb|Model of single drone Wait And Sweep]]Cone search is different from the previous two patterns. It only searches in one direction and sweeps with increasing width. For this pattern multiple different cones were used, one with an angle of ¼ pi radians, one with an angle of ½ pi radians, and finally one with an angle of ¾ pi radians. The success rate varied between these, with the highest of 57.7% corresponding to the ½ pi radians cone. The other success rates were 16.9% and 51.3% belonging to the ¼ pi radian and ¾ pi radian cones respectively. The average total time of success for the ½ pi cone was 343 seconds including the dispatch time.  
checks image for pixels of a certain colour.


Wait and Sweep is different from the other patterns, this pattern does not focus on covering new areas it instead focuses on searching the same area over and over again, waiting for the victim to drift into the frame. The parameters going into this search are mainly the distance from the spot the victim dropped into the water and the sweep width. Multiple different distances (2.4 m, 7.2 m, 12 m, 16.8 m, and 24 m) were tried with both a 15 m and 30 m sweep width. A distance of 16.8 m with a sweep with of 15 meters resulted in the highest success rate of 70.0% with a success time of 428 seconds including dispatch time.
<br />[[File:Multi drone cone search.jpg|thumb|Model of multi drone Cone Search]]One note here is that both Square search and Adapted Square search are a lot more efficient at lower flow rates. reducing the flow rates from 25 cm/s to 10 cm/s increases the success rates to 49.5% for normal square search and 92.5% for the adapted version. this is a very big improvement. While cone performs similarly and decrease in efficiency is seen for Wait and Sweep.


code:
======Multi drone======
Both Square Search algorithms were not tested for multiple drones since the ‘normal’ square search was not very efficient and the adapted version is difficult to scale for multiple drones. The cone and sweep and wait patterns were however tested for multiple drones.


def check_image(image): ##image name as string example: 'image.png'
For both the cone and sweep and wait an increase in efficiency was found when implementing more drones. For both we saw more drones equal to a higher efficiency, however, for the sweep and wait the efficiency reaches a limit of about 95% (distance 15m, width 30m) while the cone also slowly gets more efficient, and reaches about the same limit of 95% (cone angle ½ pi radians). The cone has a 260-second average detection time while the sweep and wait takes a bit longer (288 seconds). These include the dispatch time. This means the cone will be a bit better on average.


    #import the neccecary packages
==Ethical analysis==
The paper "Ethical concerns in rescue robotics: a scoping review"<ref>Battistuzzi, L., Recchiuto, C.T. & Sgorbissa, A. Ethical concerns in rescue robotics: a scoping review. ''Ethics Inf Technol'' 23, 863–875 (2021). <nowiki>https://doi.org/10.1007/s10676-021-09603-0</nowiki></ref> describes seven ethical concerns regarding rescue robots. In this chapter, each ethical concern from the paper is summarized and then applied to our case.


    import numpy as np
===Fairness and discrimination===
Hazards and benefits should be fairly distributed to avoid the possibility of some subjects incurring only costs while other subjects enjoy only benefits. This condition is particularly critical for search and rescue robot systems, e.g., when a robot makes decisions about prioritizing the order in which the detected victims are reported to the human rescuers or about which detected victim it should try to transport first. For our scenario, when there is a situation where multiple people have fallen over board, discrimination is something to look out for. When searching for people, discrimination can not take place because if the robot has found one person it will continue looking for the others and since it does not take people out of the water it does not need to make decisions about prioritizing people. However, we are also considering dropping lifebuoys and here prioritizing order does come into account. In the situation where the robot can only carry one life vest, we propose that it drops the lifebuoy one the first person it finds, then comes back to the ship to restock, and continues the search. This will be the most efficient and also prevents discrimination between multiple victims.


    from PIL import Image
===False or excessive expectations===
Stakeholders generally have difficulties with accurately assessing the capabilities of a rescue robot. May cause them to overestimate and rely too much on the robot and give false hopes for the robot to safe a victim. On the other hand, if a robot is underestimated, it may be underutilized in cases where it could have saved a person. For our scenario, it is important to accurately inform stakeholders about the capabilities and limitations of the rescue robot.


   
===Labor replacement===
Robots replacing human rescuers might reduce the performance of human contact, situational awareness and manipulation capabilities. Robots might also interfere with rescuers attempt to provide medical advice and support. In our case it is important to not fully take the human aspect out of the equation, the robot should be equipped with speakers for people on the ship to talk to the person. They can check how he is responding and reassure the victim that they’re coming to help. This will most likely help to calm the person. When the rescue attempt is being made by the crew or other rescuers, the drone should fly away since it’s no longer useful and as to not interfere with the rescue attempt.


    #setting treshold for detection (red and blue values)
===Privacy===
The use of robots generally leads to an increase in information gathering, which can jeopardize the privacy of personal information. This may be personal information about rescue workers, such as images or data about their physical and mental stress levels, but also about victims or people living or working in the disaster area. What this means for our scenario is that The information gathered by the robots is not shared with anyone outside professional rescue organizations and is exclusively used for rescue purposes. We should also try to limit the gathering of irrelevant data as much as possible.


    treshold_red = 200
===Responsibility===
The use of (autonomous) rescue robots can lead to responsibility ascription issues in the case when for example accidents happen. What this means for us is that we clearly need to state who is (legally) responsible in the case accidents happen during rescue (like operator, manufacturer etc.) Proposal: To ensure trust in our product, we should claim full (legal) responsibility as a manufacturer when accidents happen caused by design flaws or decisions made by the autonomous system.


    treshold_blue = 100
===Safety===
Rescue missions necessarily involve safety risks. Certain of these risks can be mitigated by replacing operators with robots, but robots themselves, in turn, may determine other safety risks, mainly because they can malfunction. Even when they perform correctly, robots can still be harmful: they may, for instance, fail to identify and collide into a human being. In addition, robots can hinder the well-being of victims in subtler ways. For example, the authors argue, being trapped under a collapsed building, wounded and lost, and suddenly being confronted with a robot, especially if there are no humans around, can in itself be a shocking experience. Risks should be contained as much as possible, but we do acknowledge that rescue missions are never completely risk free.


       
===Trust===
 
Trust depends on several factors; reputation and confidence being the most important once. Humans often lack confidence in autonomous robots as they find them unpredictable. What this means for us is that we need to ensure that the robot has a good reputation by informing people about successful rescue attempts and/or tests, and limit unpredictable behavior as much as possible. We can also ensure trust by claiming full (legal) responsibility as a manufacturer when accidents happen caused by design flaws or decisions made by the autonomous system.
    image_array = np.array(Image.open(image).convert('RGB')) #import the image as an array
 
   
 
    #compare pixels to threshold values
 
    red_mask = image_array[:,:, 0] > treshold_red
 
    blue_mask = image_array[:,:,2] < treshold_blue
 
   


    mask = red_mask & blue_mask #combine both masks
   
    coordinates = np.where(mask) #convert mask to coordinates
   
    #cluster the data
    clust = [np.mean(coordinates[0][:]),np.mean(coordinates[1][:])]
   
    return clust
<br />
<br />


====Python: square search====
==Conclusion==
a program that deterimines the next location based on the current one. only works with integer coordinates. these will be converted to movement commands later on.
For the chosen scenario of search and rescue, the drones prove to be efficient tools in locating and helping victims. Specifically for man overboard situations, where the drone is able to cover large areas. These sorts of technologies prove to be efficient in increasing the rescue chances of such scenarios and can be expanded further than only maritime operations.  


As we assumed in the beginning of the wiki that the chosen scenario excludes extreme weather conditions. In our case, mild wind and waves are tolerable as long as these sorts of conditions don't affect the operations of the drone. Certain cargo ship pathways were also chosen such that we avoid extremely cold waters (which are not common paths for ships anyways as seen in the state-of-the-art literature study).


code:
When defining the search area, the current was also taken into account. Assuming that a person can drift up to 0.2 m/s, we calculated that the search area would be around 450 m². As for the search patterns, we can conclude that in the case where we know the exact direction a victim is drifting in, wait and sweep search is the most effective. However for a scenario where the exact drifting direction is less certain, the cone search is probably the best option. For single drone applications with a low drift rate the adapted square search pattern can also be useful. 


def nextloc(location):
For the construction of the drone we can conclude that it is a good idea to take a custom drone with a flight controller like the Pixhawk and then control the autonomous flight with Raspberry Pi. The Raspberry Pi will be connected to a thermal camera with a decently good resolution and framerate. The images captured by the camera will then be analyzed with TensorFlow to find victims in the image. When victims are found the ship will be notified of the location over radio communication.<br />


    new_loc = []
==Discussion==
During this project there were quite some things that could have been done better. In the beginning of the course the biggest issue was defining the goal of the project and narrowing down the case. The first couple of weeks were spent on deciding on what we would do for this project and defining the specific preferences, requirements and constraints. This left us with a lot less time to go through all of the necessary design phases. The scale of the project was also changing as the weeks were going by. Some things that we wanted to do just weren't feasible in the amount of time that was left. The main example here was that we found out that actually getting an autonomous drone to work turned out to be very hard and expensive given how specific that use case is. The main problem was a combination of us either having to find a very expensive (in the thousands of euros) drone that can be made autonomous or built our own drone from scratch, which would still be pretty expensive and take a lot of effort.


    x = location[0]
Looking back on the course, the biggest issue that should have been tackled was the planning and making the decision to go for the vision that we had in the beginning. The weekly goal had to be more ambitious as we were left with little time for the project. The vision of the project should have been narrowed down for a specific case like the one that we have right now somewhere around the first week. After this a concrete plan on the whole approach should have been made rather than making up on the spot what needed to be done.<br />It might also have been a good idea to look into the feasibility of making a physical prototype earlier on in the project so that we could make better decisions surrounding this given that in the way it turned out, our physical demo was more a way of showing that the control software and automatic detection is possible instead of also testing our search patterns in real life.


    y = location[1]
As for our simulations, they are definitely useful, but do not account for more complex movement of the victim or the influence of the weather on the drone. Therefore it might be useful for future research to try to more accurately model oceanic current and the influence of wind and rain on the movement and detection accuracy of the drone.


    if abs(x) > abs(y):
==Appendix==
 
        if x > 0:
 
            new_loc = [x, y - 1]
 
        else:
 
            new_loc = [x, y + 1]
 
    else:
 
        if abs(x) < abs(y):
 
            if y > 0:
 
                new_loc = [x + 1, y]
 
            else:
 
                new_loc = [x - 1, y]
 
        else:# abs(x) = abs(y)
 
            if x > 0:
 
                if y > 0:
 
                    new_loc = [x + 1, y]
 
                else:
 
                    new_loc = [x - 1, y]
 
            else:
 
                if x < 0:
 
                    if y > 0:
 
                        new_loc = [x + 1, y]
 
                    else:
 
                        new_loc = [x, y + 1]
 
                else:
 
                    new_loc = [x + 1, y]
 
    return new_loc
 
 
==Brainstorm Phase==
 
===Possible projects===
 
====Man over board (MOB) drone (Final project)====
During the MOB protocol, the most challenging part is locating the victim. This can prove to be especially difficult during stormy weather or night time. Creating a drone that is equipped with adequate sensors to locate the victim and life saving equipment would drastically increase the chances of survival for a man overboard. Another problem that our drone needs to tackle is providing appropriate care for the possibilities of drowning, hypothermia or any other injury.
 
'''User:''' Ship's crew, rescue teams (coast guard); '''Problem:''' MOB, '''Requirement:''' Locate and provide appropriate care for the victim.
 
'''References:''' https://www.ussailing.org/news/man-overboard-recovery-procedure/, https://doi.org/10.1016/j.proeng.2012.06.236
====Manure Silo suffacation====
Manure Silo's need to be cleaned. When this is done, people can sufficate in the toxic gasses released by the manure (even if the silo is almost empty). We want to develop a robot that alarms people when conditions become dangerous, and if this person is not able to leave the silo in time, supply clean air to them.
 
'''User:''' Farmers, '''Problem:''' Manure silo suffication, '''Requirement:''' Supply clean air before suffication.
 
'''Some references:''' https://www.ad.nl/binnenland/vader-beukt-wanhopig-in-op-silo-maar-zoon-bezwijkt~a4159109/, https://www.mestverwaarding.nl/kenniscentrum/1309/twee-gewonden-bij-ongeval-met-mestsilo-in-slootdorp
 
====Extreme Sports Accidents====
Thrillseekers are often a bit reckless when it comes to safety. We want to design a flying drone that can help people get out of sticky situations during parachute-jumping, base-jumping or even rock-climbing. The victim will be able to attach themselves to the drone using the parachute equipment or rock-climbing equipment and the drone will put them safely on the ground.
 
'''User:''' Extreme sporters, rescue teams, '''Problem:''' Dangerous accidents, '''Requirement:''' Can safely attach to people and put them on the ground.
 
'''Some references:''' https://www.nzherald.co.nz/travel/aussie-base-jumpers-two-hour-ordeal-after-parachute-gets-stuck-in-tree/HCN6DYMSSA4ZUBE3GVV2WCTNRQ/, https://www.tmz.com/2022/11/30/base-jumper-crash-cliff-dangling-parachute-death-defying-video-moab-tombstone-utah/
 
===Literature study===
To determine the state of the art surrounding our project we will do a literature study.
 
====Disaster robotics====
This article gives an overview of rescue robotics and some characteristics that may be used to classify them. The article also contains a case study of the Fukushima-Daiichi Nuclear power plant accident that gives an overview of how some robots where used. On top of that the article gives some challenges that are still present with rescue robotics.
 
https://link.springer.com/chapter/10.1007/978-3-319-32552-1_60
 
====A Survey on Unmanned Surface Vehicles for Disaster Robotics: Main Challenges and Directions====
This article gives an overview of the use of unmanned surface vehicles and gives some recommendations around USV's.
 
https://www.mdpi.com/1424-8220/19/3/702?ref=https://githubhelp.com
 
====Underwater Research and Rescue Robot====
This article is about an underwater rescue robot that gives necessary feedback in rescuing missions. This underwater robot has more computng power than the current underwater drones and reduces delay by the use of ethernet cable.
 
https://www.researchgate.net/publication/336628369_Underwater_Research_and_Rescue_Robot
 
====Mechanical Construction and Propulsion Analysis of a Rescue Underwater Robot in the case of Drowning Persons====
This article is about a unmanned life-saving system that recovers conscious or unconscious people. This prevents other people from getting themselves in a dangerous situation by trying to save others. This drone is not fully autonomous since it needs to be operated by humans.
 
https://www.mdpi.com/2076-3417/8/5/693
 
====Design and Dynamic Performance Research of Underwater Inspection Robots====
Power plants along the coastline use water as cooling water. The underwater drone presented in this paper is used to research water near power plants and clean filtering systems to optimize the efficiency of the powerplant.
 
https://www.hindawi.com/journals/wcmc/2022/3715514/
 
====Semi Wireless Underwater Recue Drone with Robotic Arm====
This article highlights the challenges concerning underwater rescue of people and valuable object. The biggest challenge is wireless communication due to the harsh environment. The drone is also equipped with a robotic arm to grab objects and a 4K camera with foglights to navigate properly underwater.[https://www.researchgate.net/publication/363737479_Semi_Wireless_Underwater_Rescue_Drone_with_Robotic_Arm https://www.researchgate.net/publication/363737479_Semi_Wireless_Underwater_Rescue_Drone_with_Robotic_Armhttps://www.researchgate.net/publication/363737479_Semi_Wireless_Underwater_Rescue_Drone_with_Robotic_Arm]
 
====Rescue Robots and Systems in Japan====
This paper discusses the development of intelligent rescue systems using high-information and robot technology to mitigate disaster damages, particularly in Japan following the 1995 Hanshin-Awaji earthquake. The focus is on developing robots that can work in real disaster sites for search and rescue tasks. The paper provides an overview of the problem domain of earthquake disasters and search and rescue processes.
 
https://ieeexplore.ieee.org/abstract/document/1521744
 
====Two multi-linked rescue robots: design, construction and field tests====
This paper proposes the design and testing of two rescue robots, a cutting robot and a jack robot, for use in search and rescue missions. They can penetrate narrow gaps and hazardous locations to cut obstacles and lift heavy debris. Field tests demonstrate their mobility, cutting, and lift-up capacity, showing their potential use in rescue operations.
 
https://www.jstage.jst.go.jp/article/jamdsm/10/6/10_2016jamdsm0089/_pdf/-char/ja
 
====The current state and future outlook of rescue robotics====
This paper surveys the current state of robotic technologies for post-disaster scenarios, and assesses their readiness with respect to the needs of first responders and disaster recovery efforts. The survey covers ground and aerial robots, marine and amphibious systems, and human-robot control interfaces. Expert opinions from emergency response stakeholders and researchers are gathered to guide future research towards developing technologies that will make an impact in real-world disaster response and recovery.
 
https://doi.org/10.1002/rob.21887
 
====Mobile Rescue Robot for Human Body Detection in Rescue Operation of Disaster====
The paper proposes a mobile robot based on a wireless sensor network to detect and rescue people in emergency situations caused by disasters. The robot uses sensors and cameras to detect human presence and condition, and communicates with a network of other robots to coordinate rescue efforts. The goal is to improve the speed and efficiency of rescues in order to save more lives.https://d1wqtxts1xzle7.cloudfront.net/58969822/12_Mobile20190420-67929-tn7req-libre.pdf?1555765880=&response-content-disposition=inline%3B+filename%3DMobile_Rescue_Robot_for_Human_Body_Detec.pdf&Expires=1676230737&Signature=YQXJqYheT6M0hsHXSWDx4FbuCauvv9o9uvDR1Hl8dJL~SmI~KObXAhXbq7dDYZAMLhsydh7ipP5RBOayNkzsM~K0xP7pcXLmOKcW3-WFdt1aTyHvQWeG5hUKzhb5KLaVAj4Frfb313Yi5oyhFaHVb~ODSxbtpN73SGd3YE3UouzuexfeGSVqFyWTWi-3qMqMIQ3qfUKGiBF24QfyArHlj9mKkq8gVItdJsAS9OGBUGeBQaf~8j37WsIauoABw8cO5V73RFxhfLR~ehXXMgJegTRxzwT1tBMhE14OVMK~PkfcpYSAVkHFi3gqf~sawW4SFIut7MetNdUcKfcAwHEBHA__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA
 
====Mine Rescue Robot System – A Review====
Underground mining has a lot of risks and it is a very difficult task for rescuers to reach trapped miners. It is therefore great to deploy a wireless robot in this situation with gas sensors and cameras, to inform rescuers about the state of the trapped miners.
 
<nowiki>https://www.sciencedirect.com/science/article/pii/S187852201500096X</nowiki>
 
====Ethical concerns in rescue robotics: a scoping review====
We also have to take the ethics of rescue robots into account. There are seven core ethical themes: fairness and discrimination; false or excessive expectations; labor replacement; privacy; responsibility; safety; trust
 
<nowiki>https://link.springer.com/article/10.1007/s10676-021-09603-0</nowiki>
 
====Rescue robots for mudslides: A descriptive study of the 2005 La Conchita mudslide response====
Robots assisted the rescuers who responded to the 2005 mudslide in La Conchita. The robots were waterproof and could thus be deployed in wet conditions, but they failed to navigate through the rubble, vegetation and soil. The paper thus suggests that rescue robots should be trained in a variety of environments, and advises manufacturers to be more conservative with their performance claims.
 
<nowiki>https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.20207</nowiki>
 
====Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots====
The 2011 earthquake and tsunami in Japan resulted in a meltdown of the Fukushima nuclear power plant. Due to the radiation levels, robots were deployed because it was too dangerous for humans. First various issues needed to be resolved, like the ability of the robot’s electrical components to withstand radiation. The ability to navigate and communicate was tested at a different nuclear powerplant similar to Fukushima.
 
<nowiki>https://onlinelibrary.wiley.com/doi/full/10.1002/rob.21439</nowiki>
 
====A Coalition Formation Algorithm for Multi-Robot Task Allocation in Large-Scale Natural Disasters====
Robots are more reliable then humans in a lot of cases. This paper discusses a bit of prior research concerning older algorithms and looks into a new algorithm considering multi-robot task allocation is rescue situations. These algorithms should take a lot into account, like sensors needed for problems. They compare their algorithm with older ones in multiple cases like different problem sizes.
 
https://www.researchgate.net/publication/316283106_A_Coalition_Formation_Algorithm_for_Multi-Robot_Task_Allocation_in_Large-Scale_Natural_Disasters


===Full model results===
[[File:Square.jpg|thumb|Model of Square Search]]
[[File:Multi drone wait and sweep, flaws.jpg|thumb|Model of multi drone Wait And Sweep]]
[[File:Single drone cone search.jpg|thumb|Model of single drone Cone Search]]
{| class="wikitable"
|Source.Name
|Count
|Mean
|STD
|Median
|Min
|Max
|Succes Ratio
|Succes Speed
|-
|Sweep5_d5_w10.txt
|9006
|306.4168332
|66.22582738
|280
|240
|590
|0.9006
|66.41683
|-
|Sweep4_d5_w10.txt
|8840
|313.3597285
|69.5844632
|290
|240
|590
|0.884
|73.35973
|-
|Sweep3_d5_w10.txt
|8448
|321.1316288
|71.79807856
|310
|240
|590
|0.8448
|81.13163
|-
|cone5_halfpi.txt
|8266
|261.1154125
|10.50518206
|260
|240
|310
|0.8266
|21.11541
|-
|Cone_2_halfpi.txt
|7900
|282.4898734
|40.08713795
|270
|240
|590
|0.79
|42.48987
|-
|cone_3_halfpi.txt
|7684
|267.8227486
|21.28586226
|260
|240
|560
|0.7684
|27.82275
|-
|cone4_halfpi.txt
|7494
|261.1956232
|11.1747414
|260
|240
|350
|0.7494
|21.19562
|-
|Sweep2_d5_w10.txt
|7417
|328.5371444
|71.95174355
|310
|240
|590
|0.7417
|88.53714
|-
|Cone_2_3quarterpi.txt
|7303
|316.4042174
|73.25601744
|280
|240
|570
|0.7303
|76.40422
|-
|Sweep1_d7_w5.txt
|7007
|428.0034251
|85.79344366
|420
|240
|590
|0.7007
|188.0034
|-
|Sweep1_d5_w5.txt
|6940
|335.2247839
|73.24696142
|320
|240
|590
|0.694
|95.22478
|-
|Cone3_3quarterpi.txt
|6801
|284.2067343
|42.58235158
|270
|240
|590
|0.6801
|44.20673
|-
|cone4_3quarterpi.txt
|6644
|268.385009
|22.70430898
|260
|240
|580
|0.6644
|28.38501
|-
|cone5_3quarterpi.txt
|6350
|262.5905512
|15.23111931
|260
|240
|430
|0.635
|22.59055
|-
|Cone_1_ halfpi.txt
|5777
|342.7367146
|83.92058695
|320
|240
|590
|0.5777
|102.7367
|-
|Cone_1_3quarterpi.txt
|5125
|357.8887805
|85.33027697
|330
|240
|590
|0.5125
|117.8888
|-
|Sweep1_d7_w10.txt
|4534
|430.8557565
|87.55154123
|440
|240
|590
|0.4534
|190.8558
|-
|Sweep1_d5_w10.txt
|3661
|364.1873805
|73.35725043
|340
|240
|590
|0.3661
|124.1874
|-
|Sweep1_d3_w5.txt
|2635
|288.2049336
|46.36198948
|280
|240
|540
|0.2635
|48.20493
|-
|cone5_quarterpi.txt
|2396
|258.6978297
|10.86522538
|260
|240
|300
|0.2396
|18.69783
|-
|cone4_quarterpi.txt
|2242
|258.8492417
|11.08224585
|260
|240
|300
|0.2242
|18.84924
|-
|Adapted_SQ.txt
|2236
|325.8184258
|71.30174178
|350
|240
|590
|0.2236
|85.81843
|-
|Cone3_quarterpi.txt
|2060
|257.7669903
|10.84412396
|260
|240
|290
|0.206
|17.76699
|-
|Cone_1_quarterpi.txt
|1689
|279.2243931
|37.61462698
|270
|240
|580
|0.1689
|39.22439
|-
|Cone_2_quarterpi.txt
|1514
|257.3051519
|11.50892696
|260
|240
|330
|0.1514
|17.30515
|-
|Sweep1_d3_w10.txt
|1436
|288.6699164
|70.27055499
|240
|240
|590
|0.1436
|48.66992
|-
|Sweep1_d10_w5.txt
|968
|528.7086777
|40.5658764
|530
|340
|590
|0.0968
|288.7087
|-
|Sweep1_d10_w10.txt
|530
|533.1320755
|50.86772116
|560
|430
|590
|0.053
|293.1321
|-
|Sweep1_d1_w5.txt
|194
|271.4948454
|45.12885953
|240
|240
|440
|0.0194
|31.49485
|-
|Square.txt
|141
|313.9007092
|52.69552773
|300
|240
|460
|0.0141
|73.90071
|}
<br />
<br />
==Appendix==
===Logbook===
===Logbook===
{| class="wikitable mw-collapsible mw-collapsed"
{| class="wikitable mw-collapsible mw-collapsed"
Line 700: Line 717:
| rowspan="6" |1
| rowspan="6" |1
|Geert Touw
|Geert Touw
|Group setup etc (2h)
|Group setup etc (2h), Subject brainstorm (2h)
|
|4h
|-
|-
|Luc van Burik
|Luc van Burik
Line 708: Line 725:
|-
|-
|Victor le Fevre
|Victor le Fevre
|Group setup etc (2h)
|Group setup etc (2h), Subject brainstorm (2h)
|
|4h
|-
|-
|Thijs Egbers
|Thijs Egbers
|Group setup etc (2h)
|Group setup etc (2h), Subject brainstorm (2h)
|
|
|-
|-
|Adrian Kondanari
|Adrian Kondanari
|Group setup etc (2h)
|Group setup etc (2h), Subject brainstorm (3h)
|
|5h
|-
|-
|Aron van Cauter
|Aron van Cauter
Line 725: Line 742:
| rowspan="6" |2
| rowspan="6" |2
|Geert Touw
|Geert Touw
|Meeting 1 (1h), Meeting 2(1h)
|Meeting 1 (1h), Meeting 2(1h), Subject brainstorm (1h)
|
|3h
|-
|-
|Luc van Burik
|Luc van Burik
Line 733: Line 750:
|-
|-
|Victor le Fevre
|Victor le Fevre
|Meeting 1 (1h), Meeting 2(1h)
|Meeting 1 (1h), Meeting 2(1h), Subject brainstorm (1h)
|
|3h
|-
|-
|Thijs Egbers
|Thijs Egbers
Line 741: Line 758:
|-
|-
|Adrian Kondanari
|Adrian Kondanari
|Meeting 1 (1h)
|Meeting 1 (1h), First concept elaboration (2h)
|
|3h
|-
|-
|Aron van Cauter
|Aron van Cauter
Line 750: Line 767:
| rowspan="6" |3
| rowspan="6" |3
|Geert Touw
|Geert Touw
|
|Meeting 1 (1h), meeting 2 (1h), Oceanic weather + hypothermia and water + shipping routes research (3h)
|
|5h
|-
|-
|Luc van Burik
|Luc van Burik
Line 758: Line 775:
|-
|-
|Victor le Fevre
|Victor le Fevre
|Meeting 1 (1h), meeting 2 (1h)
|Meeting 1 (1h), Meeting 2 (1h), Made the wiki look a bit more coherent (1h)
 
Made the wiki look a bit more coherent (1h)


Wrote the problem statement and objective, users, scenario and added points to the MoSCoW (1.5h)
Wrote the problem statement and objective, users, scenario and added points to the MoSCoW (1.5h)
Line 768: Line 783:
|-
|-
|Thijs Egbers
|Thijs Egbers
|
|Meeting 1 (1h), meeting 2 (1h), sea communication research (2h)
|
|4h
|-
|-
|Adrian Kondanari
|Adrian Kondanari
|
|Meeting 1 (1h), requirements (2h), MOB protocols (2h)
|
|5h
|-
|-
|Aron van Cauter
|Aron van Cauter
Line 781: Line 796:
| rowspan="6" |4
| rowspan="6" |4
|Geert Touw
|Geert Touw
|Meeting 1 (1h), Meeting 2(1h)
|Meeting 1 (1h), Meeting 2(1h), Ethics (4h)
|
|6h
|-
|-
|Luc van Burik
|Luc van Burik
|Meeting 1 (1h), Meeting 2(1h), Search area (1.5hr), Drone search speed (1.5 hr), Wiki writing (0.5 hr)
|Meeting 1 (1h), Meeting 2(1h), Search area (1.5hr), Drone search speed (1.5 hr), Wiki writing (0.5 hr)
|5.5 hr
|5.5h
|-
|-
|Victor le Fevre
|Victor le Fevre
|Meeting 1 (1h), Meeting 2(1h)
|Meeting 1 (1h), Meeting 2(1h), looked at sensors and image recognition (2h)
|
|4h
|-
|-
|Thijs Egbers
|Thijs Egbers
|Meeting 1 (1h), Meeting 2(1h)
|Meeting 1 (1h), Meeting 2(1h), drone controll research (3h)
|
|
|-
|-
|Adrian Kondanari
|Adrian Kondanari
|Meeting 1 (1h), Meeting 2(1h)
|Meeting 1 (1h), Meeting 2(1h), sensor research (2h)
|
|5h
|-
|-
|Aron van Cauter
|Aron van Cauter
Line 806: Line 821:
| rowspan="6" |5
| rowspan="6" |5
|Geert Touw
|Geert Touw
|Meeting 1(1h)
|Meeting 1(1h), Made a search pattern with Victor (2h)
|
|3h
|-
|-
|Luc van Burik
|Luc van Burik
Line 815: Line 830:
|-
|-
|Victor le Fevre
|Victor le Fevre
|Meeting 1(1h)
|Meeting 1(1h), designed and tried tried to make an xy-rig (3h), worked on the wiki (2h)
|
|6h
|-
|-
|Thijs Egbers
|Thijs Egbers
|Meeting 1(1h)
|Meeting 1(1h), meeting 2 (1h), worked on getting the raspi setup (3h),
|
try to get the camera and soldering (0.5h)
|5.5h
|-
|-
|Adrian Kondanari
|Adrian Kondanari
|Meeting 1(0.5h)
|Meeting 1(0.5h), communication methods research (4h)
|
|4.5h
|-
|-
|Aron van Cauter
|Aron van Cauter
Line 832: Line 848:
| rowspan="6" |6
| rowspan="6" |6
|Geert Touw
|Geert Touw
|
|Meeting 1 (1h), Meeting 2 (1h), Worked on wiki (2h)
|
|4h
|-
|-
|Luc van Burik
|Luc van Burik
|
|meeting 2 (1h), Concert search pattern to work inside rest of the program (3h), programming more patterns(2h)
|
|6h
|-
|-
|Victor le Fevre
|Victor le Fevre
|
|Meeting 1 (1h), Meeting 2 (1h), Worked on wiki (4h)
|
|6h
|-
|-
|Thijs Egbers
|Thijs Egbers
|
|Meeting 1 (1h), Meeting 2 (1h), get the camera setup (1.5h), try to optimise camera reading (1h),
|
convert search pattern to work inside the rest of the program (3h)
|7.5h
|-
|-
|Adrian Kondanari
|Adrian Kondanari
|
|Heating element (5h)
|
|5h
|-
|-
|Aron van Cauter
|Aron van Cauter
Line 857: Line 874:
| rowspan="6" |7
| rowspan="6" |7
|Geert Touw
|Geert Touw
|
|Meeting 2 (demo) (4h), Wiki (1h)
|
|5h
|-
|-
|Luc van Burik
|Luc van Burik
|
|Meeting 2 (demo) (4h), finishing touches test program (2h), presentation (3 h)
|
|9h
|-
|-
|Victor le Fevre
|Victor le Fevre
|
|Meeting 2 (demo) (4h), Wiki (1h)
|
|5h
|-
|-
|Thijs Egbers
|Thijs Egbers
|
|Meeting 1 (1h), meeting 2/video making (4h), finishing touches test program (2h),
make presentation (WIP)
|
|
|-
|-
|Adrian Kondanari
|Adrian Kondanari
|
|wiki (2h)
|
|2h
|-
|-
|Aron van Cauter
|Aron van Cauter
Line 882: Line 900:
| rowspan="6" |8
| rowspan="6" |8
|Geert Touw
|Geert Touw
|
|Presentation (0.5h), Short meeting to prepare for presentations (0.5h), wiki (1.5)
|
|2.5h
|-
|-
|Luc van Burik
|Luc van Burik
|
|presentations (2.5 h), Short meeting before presentations (.5h), working on wiki (6h)
|
|9h
|-
|-
|Victor le Fevre
|Victor le Fevre
|
|Finalizing wiki (5h)
|
|5h
|-
|-
|Thijs Egbers
|Thijs Egbers
|
|Presentations (2.5h), short meeting with Luc (0.25h), working on the wiki (4h)
|
|
|-
|-
|Adrian Kondanari
|Adrian Kondanari
|
|wiki (4h)
|
|4h
|-
|-
|Aron van Cauter
|Aron van Cauter
|
|
|
|
|}
|}<br />
<br />


===References===
===References===
<references />
<references /><br /><references />
 
===Things things we might need later (to be deleted)===
'''Weather''': Some waves, a breeze but still calm enough for a drone to fly, little rain, nighttime, water is 3 degrees Celsius -> 15-30 minutes until exhaustion and unconsciousness, 30-90 minutes expected survival time.
 
'''Ship''': Container ship, on the Atlantic ocean, speed: 25 knots (~46 km/h, had to take a detour, were behind schedule, faster than average)
 
'''Reason off fall:''' Dark outside, slippery because some rain, person is alone, person drifts away without major injuries

Latest revision as of 21:39, 10 April 2023


Group members
Name Student number Major
Geert Touw 1579916 BAP
Luc van Burik 1549030 BAP
Victor le Fevre 1603612 BAP
Thijs Egbers 1692186 BCS
Adrian Kondanari 1624539 BW
Aron van Cauter 1582917 BBT


Abstract

In this wiki our group examined the issue of man overboard (MOB) incidents and proposes a solution to increase the survival chances of victims. A state-of-the-art literature review was conducted, which explored various aspects of MOB incidents such as search area, search patterns, oceanic currents and temperatures, and current person in water detection technology such as drones. A proof of concept was then developed using an XY-rig, which incorporated a thermal camera connected to a Raspberry Pi. The results showed that the thermal camera was able to successfully locate a heat element in a bland environment, representing the ocean. In future work, the solution will be extended to a drone swarm, where multiple drones can be deployed to search for a victim. This solution has the potential to improve the effectiveness of search and rescue operations, and ultimately increase the survival rate of MOB victims.

Introduction

Working on board a cargo ship is a dangerous occupation, with man overboard (MOB) incidents being a particularly serious concern. 91 fatalities were reported as a result of MOB events aboard cargo ships between 2014 and 2021[1]. Numerous factors, including; bad weather, unstable loads, and a lack of adequate safety equipment, are the main cause of such accidents. Unfortunately, locating a MOB victim in the ocean can be a challenging and complicated task. The search procedure can be affected by all sorts of elements such as the climate, ocean currents, and visibility.

To better understand the scope of the issue, a 2020 study analyzed 100 MOB incidents using 114 parameters to generate a MOB event profile[2]. 53 of the 100 events that were looked at occurred on cargo ships. In 88 cases, the casualty was deceased as a result of the MOB incident, from those 88 cases 34 were assumed dead and 54 were witnessed dead. From the witness deaths 18 died before the rescue and 31 after the rescue, in 5 cases it is unknown. The cause of death was indicated for 42 cases, the most common cause is identified as drowning (26), followed by trauma (9), cardiac arrest (4) and hypothermia (3).

Despite the challenges involved in locating MOB victims in the ocean, we believe that technology can play a crucial role in increasing the survival chances of those involved in such incidents. With this in mind, we aim to develop a device that can improve the efficiency of locating MOB victims on cargo ships. Specifically, we will focus on MOB incidents that occur on cargo ships, as the 2020 study indicated that over half of such incidents occur on these vessels. The end users of our product are personnel of cargo ships, shipping companies that own a fleet of cargo ships, ports, and possibly the military.

It's worth noting that searching for a victim in the ocean can be a challenging and complex task, we therefore make several assumptions. First, we presume that both at the time of the accident and during the search and rescue effort, the weather conditions are mild. This means mild waves, a mild breeze, little rain, and average sea and air temperatures. We also assume that it is night-time at the time of the incident and that there is only one victim who is capable of maintaining afloat and does not have any major injuries. Additionally, we assume that the time of the incident is known, allowing for a general understanding of the search area.

To develop our device, we will conduct a state-of-the-art literature study to investigate various factors that can impact the search process. This will include examining search areas, search patterns, oceanic currents and temperatures, and current person-in-water detection technology.

We hope to increase the efficiency of the search process and further improve the survival chances of those involved in MOB incidents on big (cargo) ships. By developing this device, we believe that we can make a meaningful contribution to the safety of those who work at sea.

State-of-the-art literature study

Before we can get started with the project it is important to gather knowledge about the state-of-the-art (SotA) concerning safety measures and pre-existing technology that aids victims of a MOB incident. We therefore performed a SotA literature study.

Current search methods

Search procedures for when a man goes overboard already exist. In the IAMSAR manual some of these procedures are explained. In this paragraph we discuss and analyze these procedures and go over some different search patterns used.

Current Procedures[3]

The existing procedures for people overboard on big cargo ships are quite elaborate. The procedure basically consists of a few key steps: it all starts when a victim goes overboard, from there the following will happen:

Step 1: Notify staff on board

Step 2: Mark the position the victim went overboard / release a marker

Step 3: Turn the ship around

Step 4: Final search (finding the marker and victim)

Step 5: Rescue

At first this process seems quite efficient, but what should be taken into account is that Step 3, can take a long time for big ships[4] . For some ships this can take over 20 minutes while traveling a few kilometers. Then there are also some factors going into the step 4: For different environments and conditions different patterns are recommended. Some of these will be explained below.[5]

Expanding Square Search

The expanding square search is a method that can be used to systematically inspect a body of water for signs of an MOB victim. The search should start at the last known location of the victim, from that point the search starts and expands outwards with course alterations of 90°, as can be seen in the image provided. The course of expansion heavily depends on the specification of the device that will be used. Thermal imaging can be used to scan the area, looking for any signs of the victim such as floating debris or the person themselves.

Sector Search

Sector search is another technique that can be used to search for a victim in the ocean. The sector search technique involves dividing the search area into sectors or pie-shaped sections and systematically searching each sector. It again starts at the last known location of the victim. From there the search area is divided and each sector is carefully searched in a clockwise fashion. Sector search can be effective in searching large areas, especially in cases where the search area is relatively circular or symmetric. However, sector search may not be the most efficient or effective method in all situations, especially if the search area is irregular or if weather and ocean conditions make it difficult to maintain a consistent search pattern.

Sweep search

The sweep search pattern is another technique that can be used to search for a victim in the ocean. It involves moving the search vessel or search device in a back-and-forth sweeping motion across the search area.

Expanding Sqaure, IAMSAR search patterns[6]
Sweep Search, IAMSAR search patterns[6]
Sector Search, IAMSAR search pattern[7]


When the location of the MOB victim is known well, the Expanding Square or Sector Search is recommended. If the location of the accident is not accurately known different patterns are recommended such as Sweep Search. IAMSAR also mentions other factors that need to be taken into account during the search. If a person falls into the water they will for example be moved away by currents.

Search area

A person floating in the ocean can drift up to 10 nautical miles per day, this is equal to 0.2 meters per second. If the current is unknown this will mean the search area would expand in a circle if the starting location would be known. If we define the search area as a circle with a radius r, the radius of the circle will depend on time, we describe the search area with the following formula: r = 0.2 t. Here t is the time since the person fell overboard. The following table shows the search area size compared to time.

Time overboard (minutes) Search area size (m²)
1 452
5 11310
15 101788

The table clearly shows the search problem can become quite big. Especially when the search takes a long time. This shows the importance of finding the victim quickly.

Image showing the world's shipping routes on the Atlantic ocean[8]

Influence of cold water and oceanic temperatures

Having a good understanding on how factors such as water temperature influence the rescue mission is crucial in ensuring that the rescue mission is successful. Depending on the water temperature, the time frame of the rescue mission changes and different measures need to be taken into account. Ideally the victim should be rescued as soon as possible, but having a known time frame of within which the mission needs to be executed gives the crew members a good idea on how to approach the situation.

Since we're dealing with people who have fallen in water, we need to consider their survival time. When being submerged in water, your greatest risk is developing hypothermia as your body cools down 25 times faster in water than in air of the same temperature. The effects of hypothermia go from confusion and exhaustion, to unconsciousness and coma, and in the end lead to death. It is thus of utmost importance to detect and rescue a person in the water before the severe effects of hypothermia set in. The following table shows the relation between the temperature of the water and the time frame of the effects of hypothermia. The time windows are wide as the time when the effects of hypothermia begin depend on several factors, like age and percentage of body fat.[9]

Relation between water temperature and hypothermia[9]
Water temperature (C°) Time until unconsciousness Survival time
0 <15 min 15 - 45 min
0 - 5 15 - 30 min 30 - 90 min
5 - 10 30 - 60 min 1 - 3 hours
10 - 15 1 - 2 hours 1 - 6 hours
15 - 20 2 - 7 hours 2 - 40 hours
20 - 25 3 - 12 hours 3 - indefinite
Image showing oceanic temperatures on 22 March 2023[10]

Next, oceanic temperatures need to be considered. The two figures on the right show the world's shipping routes on the Atlantic ocean, and oceanic temperatures respectively. From this we can deduce that there are busy shipping routes in the colder regions of the Atlantic ocean, ranging from 3 to 7 degrees Celsius, and also the occasional ship in water at 1 degrees Celsius. The temperatures were taken in late March so it might even be colder in the middle of winter. Since these ships will benefit most from a technology that might result in faster rescue times, it can be concluded at the most desirable performance of the system is to find a person in the water in less than 10 minutes, leaving enough time for the rescue operation to start.[10][8]


Possible detection methods

There already exists technology that can detected a person in a body of water. Radars are frequently used to detect the distance and velocity of an object by sending electromagnetic waves out and detecting the echo that results from objects. In water environments, a radome is needed to protect the radar but let the electromagnetic waves through. However, even with a radome the signal strength already falls to a third in a wet environment compared to dry. Multiple radar technologies exist with each their different advantages and disadvantages. Pulse Coherent Radar (PCR) can pulse the transmitting signal so it only uses 1% of the energy. Another aspect of this radar is coherence. This means that the signal has a consistent time and phase so the measurements can be incredibly precise. It can also separate the amplitude, time and phase of the received signal to identify different materials. This is not necessarily needed in our use case. The 'Sparse' service is the best service for detecting objects since it samples waves every 6 cm. You don't want millimeter precise measurements in a rough environment like the ocean so these robust measurements are ideal.[11]

Another detection method that does not yet exist is to send out a swarm of drones to search for victim. The research paper "Intelligent Drone Swarm for Search and Rescue Operations at Sea"[12] presents a novel approach for using a swarm of autonomous drones to assist in search and rescue operations at sea. The authors propose a system that uses machine learning algorithms and computer vision techniques to enable the drones to detect and classify objects in real-time, such as a person in distress or a life raft. The paper describes the hardware and software architecture of the drone swarm system, which consists of a ground station, multiple drones equipped with cameras, and a cloud-based server for data processing and communication. The drones are programmed to fly in a coordinated manner and share information with each other to optimize the search process and ensure efficient coverage of the search area. The authors evaluate the performance of their system in various simulated scenarios, including detection of a drifting boat, a person in the water, and a life raft. The results show that the drone swarm system is capable of detecting and identifying objects accurately and efficiently, with a success rate of over 90%. The research paper presents a promising solution for improving search and rescue operations at sea, leveraging the capabilities of autonomous drones and artificial intelligence. For larger scale operations, drone swarm can be used to cover greater areas for search and rescue. This method is useful when other factors such as the wind, current etc. make finding the victim more difficult. The drone swarm would be implemented in such a way so that areas around the ship, or any search area for that matter, can be searched in a shorter time span, increasing the chances of finding the victim and increasing the chances of survival. When multiple drones are deployed, they all take up a specific search pattern in unison. Depending on the situation different search patterns will be used. If a man overboard is detected rather quickly, one search pattern may be implemented in order to optimize the time it takes in order to locate the victim. When other factors influence the case, such as wind or current, different search patterns will be used in order to adapt to the situation.

There are multiple papers that discuss drone swarms as a method of detecting people on the ocean. The research paper "AutoSOS: Towards Multi-UAV Systems Supporting Maritime Search and Rescue with Lightweight AI and Edge Computing"[13] proposes a similar approach for using multiple unmanned aerial vehicles (UAVs) equipped with lightweight AI and edge computing to support maritime search and rescue (SAR) operations. The authors describe the AutoSOS system, which consists of a ground station, multiple UAVs, and a cloud-based platform for data processing and communication. The UAVs are equipped with cameras and other sensors to detect and classify objects in real-time, such as a person in the water or a life raft. The lightweight AI algorithms are designed to operate on the UAVs, minimizing the need for high-bandwidth communication with the cloud-based platform. The paper highlights the advantages of using a multi-UAV system for SAR operations, such as increased coverage area, improved situational awareness, and faster response times. The authors also discuss the technical challenges of designing and implementing the AutoSOS system, such as optimizing the UAV flight paths and ensuring the reliability of the communication and data processing systems. To demonstrate the effectiveness of the AutoSOS system, the authors present a case study of a simulated SAR mission in a coastal area. The results show that the system is capable of detecting and identifying objects accurately and efficiently, with a success rate of over 90%.

Another paper that is of relevance is the "A Review on Marine Search and Rescue Operations Using Unmanned Aerial Vehicles"[14]. It provides a comprehensive review of the current state of research on using unmanned aerial vehicles (UAVs) for search and rescue (SAR) operations in marine environments. The paper discusses the main challenges of conducting SAR operations in marine environments, such as limited visibility, harsh weather conditions, and the vast and complex nature of the search areas. The authors then review the different types of UAVs and their applications in SAR operations, such as fixed-wing UAVs for long-range surveillance and multirotor UAVs for close-range inspection and search missions. The paper highlights the advantages of using UAVs for SAR operations, such as increased coverage area, improved situational awareness, and reduced risk to human rescuers. The authors also discuss the technical challenges of designing and operating UAVs in marine environments, such as optimizing the flight paths, ensuring reliable communication and data transmission, and complying with regulatory and ethical guidelines. To demonstrate the effectiveness of UAVs for SAR operations, the authors review several case studies of real-world applications, such as detecting and rescuing distressed vessels and locating missing persons in coastal areas. The results show that UAVs can provide valuable support to SAR operations, improving response times, and increasing the chances of successful rescue.

Another paper, "Requirements and Limitations of Thermal Drones for Effective Search and Rescue in Marine and Coastal Areas"[15], explores the potential of thermal drones for search and rescue (SAR) operations in marine and coastal areas. The authors review the main features and capabilities of thermal drones, including their ability to detect heat signatures and identify objects in low light or night conditions. The paper highlights the key requirements and limitations of using thermal drones for SAR operations, such as the need for high-resolution thermal cameras, long flight time, and reliable communication systems. The authors also discuss the challenges of operating drones in harsh weather conditions and the importance of complying with regulatory and ethical guidelines. To demonstrate the effectiveness of thermal drones for SAR operations, the authors present a case study of a simulated rescue mission in a coastal area. The results show that thermal drones can provide valuable assistance in locating and identifying missing persons or distressed vessels, particularly in remote or inaccessible areas. The research paper provides a comprehensive overview of the potential of thermal drones for SAR operations in marine and coastal areas, while also highlighting the technical and operational challenges that need to be addressed to ensure their effective use.

From these studies we conclude that the most efficient detection method is a drone swarm with thermal cameras attached to the drones.

Sensors

The main sensor that will be used to locate a victim is a thermal imaging camera (TIC). Since the general search area is small relative to the complete ocean, temperatures of the water will all be relatively the same. The victim will have a much higher temperature than their surroundings, making it easy to detect them. There are a couple important requirements for the TIC. The camera needs a sufficiently high resolution so that we can clearly distinguish the victim from other warm object (think of the marine life). It also needs to have a high refresh rate. Our goal is to detect and locate a victim as soon as possible so that the survival chance is the highest, we therefore need to 'scan' the search area fast and that requires a sufficient refresh rate. For the same reason the thermal camera is required to have a large field of view (FOV). It also needs the right temperature sensitivity.

Other sensors and actuators that we might want to include are a microphone and a loudspeaker. This would allow us to make contact with the victim. However, the issue with a microphone is that the sound of the ocean is most likely so loud that it is unlikely to understand the victim, trying to conversate with the victim might also fatigue them. A loudspeaker can be used to notify the victim rescue is on the way and help them calm down.

Thermal image recognition

For the image recognition we can make use of TensorFlow, Google's Neural Network library. We can, as mentioned before, also choose to not do advanced image detection and instead as soon as a heat source that could reasonably be the victim is detected, send the camera images to the ship where a human operator can verify whether the heat source is actually the victim.

Another option that could be looked into more would be using a hybrid of these two strategies. Namely, using the fast simple heat source detection until a heat source is found and then activating the more advanced AI image detection to verify if it is a human. This would then solve a part of the efficiency problem and would eliminate some need for human operators and higher data transfer requirements.

Communication Systems

In a scenario where a drone needs to send data, such as thermal images, to a container ship on the open sea, there are several reliable methods of communication that can be used, including:

  1. Satellite Communication: One of the most reliable methods of communication in remote areas, such as the open sea, is through satellite communication. The drone can use a satellite modem to transmit the data to a satellite in orbit, which then relays the data to a ground station or directly to the container ship. Satellite communication can support high data rates and can be used to transmit data over long distances.
  2. Long-Range Wi-Fi: Another option for communication between the drone and the container ship is through long-range Wi-Fi. The drone can be equipped with a Wi-Fi transmitter that is capable of transmitting data over a long distance, and the container ship can be equipped with a Wi-Fi receiver that is capable of receiving the data. Long-range Wi-Fi can support high data rates and can be used to transmit data over several kilometers, depending on the power of the transmitter and receiver.
  3. Cellular Communication: Depending on the location of the container ship and the availability of cellular coverage, the drone may be able to use cellular communication to transmit data to the ship. The drone can be equipped with a cellular modem that is capable of connecting to a cellular network, and the container ship can be equipped with a cellular receiver that is capable of receiving the data. Cellular communication can support moderate data rates and can be used to transmit data over several kilometers, depending on the strength of the cellular signal.
  4. Radio Communication: Another option for communication between the drone and the container ship is through radio communication. The drone can be equipped with a radio transmitter that is capable of transmitting data over a long distance, and the container ship can be equipped with a radio receiver that is capable of receiving the data. Radio communication can support low to moderate data rates and can be used to transmit data over several kilometers, depending on the power of the transmitter and receiver.

Overall, the choice of communication method will depend on the specific requirements of the scenario, such as the distance between the drone and the container ship, the data rate requirements, and the availability of communication infrastructure. Satellite communication is typically the most reliable and versatile option in remote areas, but it may also be the most expensive.[16]

Design

Requirements

MoSCoW method
Must Should Could Won't
Have sensors to detect a person within 10 minutes Have life assist systems Have communication between victim and ship Have the ability to take the person to safety
Have a communication system to communicate with the involved ship Resist (bad) weather to a degree
Be able to withstand an air temperature range of -3°C to 36°C. A person recognition algorithm
Be able to operate in the dark
Be able to operate in the rain
Be able to withstand wind forces of 4 beaufort.

Drone requirements

A couple of requirements need to be met in order for the drone to be functioning as intended. In this section we will discuss the possibilities of remote drone control and how it ties into our case. Different communication methods will be discussed and we have to choose the best one for our scenario. And the detection method using thermal imaging will also be discussed. This is also tied with the drone search speed, which also influences our approach.

Remote Drone Control

A lot of research has already gone into autonomous drone flight. Take a look at autonomous drone racing for example[17] . A lot of consumer drones also are capable of autonomous flight, for example DJI has a waypoint system based on GPS coordinates and their drones are also capable of tracking a moving person/object. DJI's waypoint system works by loading a set of GPS coordinates onto the drone, the drone will navigate itself to the first set coordinates. From there it goes on to the next. Until it has reached the final set of coordinates. The moving object tracking is a bit more complicated. This involves image recognition.[18]

However, the firmware on most of these commercial drones is not open to being interfaced with. This makes is much more difficult to make a more autonomous drone without building one from scratch. Another point is that drones, and especially the autonomous kind, are quite expensive. And due to the aforementioned lack of interfacing options with the flight controllers of commercial drones it is also not really possible to take a cheaper non-autonomous drone and hook it up to a computer that will act as a module to make the drone fly autonomously.

Drone search speed
X(axis) = time in seconds, Y(axis) = area, the blue line is area to be searched and the yellow line is area searched.

The drone search speed will be limited by multiple things such as flight speed, sensor resolution and quality/speed of the detection algorithm. Since we still have some unknowns we would like to make some assumptions. Lets assume the algorithm is able to detect a human if they take up at least a space of 25 by 25 pixels. Lets also assume a human at sea takes up at least 25 by 25 cm. A good sensor to use for the detection of humans at sea in the night would be a heat camera. These cameras take images in the IR spectrum and thus aren't hindered by low light conditions. They have a field of view (FOV), a resolution and a refresh rate. The height the drone can fly will be determined by the FOV, and resolution. A typical FOV for Raspberry Pi cameras is about 60 degrees of horizontal[19] and a typical resolution would be 1080*1920 pixels. Here 1920 is the horizontal direction. If the drone would have a sensor of these specs it would be able to fly about 15 meters high, here it would be able to scan a line of 19.2 meters wide. Lets assume the algorithm needs to have the human in shot for 5 seconds for it to be detected by the algorithm.[20] The vertical FOV of Raspberry Pi sensors is about 40 degrees, this would result in a distance of about 10 meters vertical to be seen by the drone. This means it can fly at about 2 m/s. The area the drone can covers is thus 20m*2m/s*t where t is the time since dispatched. Another solution to the detection could be to not look for humans specifically, but for relatively hot areas in the image. This might be an option since out at sea there is a significant heat difference between a human with a body temprature of approximately 37 degrees celcius and the environment that consists mostly of cold water. When using this strategy it is however necessary to include some kind of verification in the system so that the drone will not signal that the victim is found when it, for example, finds a maritime animal close to the surface of the water.

However, for our estimation of the search speed we will still use the option where an algorithm is used that actually detects a human since this might still be necessary and we prefer to work from the worst case scenario with regards to bounding the minimum required efficiency. In the graph on the right we see that after about 5 minutes the drone is not able to keep up with the search area anymore (even with a perfect search pattern). If we start taking dispatch time into account it would mean these 5 minutes would be about result in about 2.5 of effective search time) this would move the yellow line to the left, this would also mean that one drone is not quick enough. and we would likely need to use multiple drones in order to search fast enough..

Components

Sensor

  • Thermal imaging camera

Communication Equipment

  • Wireless communication module

Drone with autonomous/waypoint flight capabilities

Small computer capable of:

  • Multithreading/multiprocessing
  • Running TensorFlow or other image recognition AI software
  • Interfacing with electronics (reading the camera image and possibly interfacing with servo motors)
  • Running autonomous flight control software

Drone

For our real design we would like to use a drone that is able to fly accurately in mild wind and is able to carry some weight. Since the drone has to fly autonomously it will most likely be necessary to make a custom drone for this project since almost all commercially available drones don't allow interfacing with the flight controller in such a way that an external device/computer can control the drone. We have not really researched into how to make a drone from scratch since this is not really in the scope of the project, however for the autonomous flight a pixhawk flight controller can be used. The pixhawk is currently the most popular option for autonomous flight as far as we can tell. The pixhawk can then be interfaced with with a Raspberry Pi to controll the autonomous movement of the drone.

We have also not researched into the details of the control of autonomous drones since we already found out pretty early that using an actual drone like this for the proof of concept would not be feasable. The details of the control software would not really contribute to the rest of the design so for our demo we assumed it was possible to transform carthesian waypoints to flight instructions for the drone.

Computer

We have decided to use a Raspberry Pi as our computer due to it's computing power to size/weight ratio. The Raspberry Pi can do all the things we wish our computer to do so it is a great choice.

Search pattern

Expanding Square Search
Adapted Square Search
Cone Search
Multi drone Cone Search
Wait And Sweep Search


A critical factor that can determine the effectiveness of a search operation and its ability to meet our requirements is the search pattern employed. We have identified three potential search patterns that could be utilized, and we plan to test each of these methods using a model to evaluate their effectiveness. The selection of an appropriate search pattern is crucial as it can have a significant impact on the chances of success in locating the person who has fallen overboard. By testing each of the proposed search patterns using a model, we can determine which method is best suited for the specific circumstances of the search and optimize our search efforts accordingly.

One already existing search pattern is the expanding square search. This involves the drone moving outward from the last known location of the person in a square pattern, with each pass covering an increasing amount of area. This method is efficient and can cover a lot of ground quickly, but it does not take into account the direction of the current, so could in theory only be efficiently used in situations where the victim doesn’t drift.

To address this issue, an improvement on the expanding square search is to incorporate the direction of the current. We call this Adapted Square Search. This involves adding a few repetitions in the direction of the current to increase the chances of finding the missing person. The amount of repetitions is determined by the strength of the current; No current would mean no repetitions (regular expanding square), a weak current one extra repetition in that direction, a bit stronger two repetitions, etc. This method can be especially useful when there is a strong current that could have carried the person further away from their original location.

Another possible search pattern is the cone pattern. This involves the drone moving outward from a small distance before the last known location of the person in a cone-shaped pattern, with the width of the cone expanding in the direction of the current. This method could be the most efficient as it doesn’t return back to the initial position and doesn’t search against the current.

Then there is also a different approach, since the current moves the victim will move, we can also wait for it to come to us. This method is called wait and sweep, here the drone does a sweep search pattern, but it sweep the same area over and over.

Programming

All of the code used in this project is uploaded on the following GitHub repository: https://github.com/ThijsEgberts/ProjectRobotsEverywhere.

Proof of concept

In order to prove the concept not all Moscow requirements need to be fulfilled. Since the most important part of our system is the detection, this needs to be shown working. The algorithms for both search and detection need to be proven and the hardware also needs to be shown functioning. However weather resistibility, does not have to be taken into account yet. The test also does not have to be done at full scale.

Designing and building

Materials:

  • Plastic container 40cm x 60cm
  • 2 Metal rods 8 mm x 100cm
  • Linear ball bearing
  • ABS (to print pieces)
3D model of slider

The design process of the prototype was done with Fusion 360. This program allows for detailed 3D-modeling and easy conversion to Prusaslicer which is the slicer program. After that the pieces that were designed are ready to be printed. The 3D-printer that was used is the Voron V0. We made a total of 4 corner attachment pieces, 2 sliders and 1 slider platform. The whole process required a lot of little design changes to properly fit and assemble the entire setup. Eventually the final assembly process was easily done with some screws and bolts. The search pattern was drawn on the bottom of the container to help the people moving the camera with following the movement instructions and make it more clear for the demonstration video what was going on.

3D model of corner attachment piece
3D-model of entire setup
3D model of slider platform










Demo

A Raspberry Pi and a thermal camera were mounted on a rig to demonstrate the proof of concept. Ropes were used to manually modify the camera's position. To make it easier to follow the search path, a map of the search pattern was drawn on paper. A tiny tea light was used as a heat source to imitate the relevant scenario and provide a distinct heat signature. Also, a rope that was tied to the lamp served as an ocean current replica. The heat source was identified by following the search pattern according to instructions being output by the Raspberry Pi.

Thermal camera

For our demo we used the MLX90640 thermal camera. This camera has a resolution of 32x24 pixels and a 55° field of view. The camera itself has a refresh rate of 60fps, however in our application this is limited due to the speed at which the I2C interface works. Therefore in our application we could process images for heat sources at about 7fps when not displaying the image and at about 1fps when displaying the image. This speed may in part be due the use of python but python did make the image manipulation and output much easier.

Heating element

For the experiment we needed to have a detectable heat signature. For this we chose the simplest possible solution, which was just having a candle. This gave us a detectable heat signature that we can work with.

Design of the Model

In order to investigate how efficient the drone is at finding the victim a model was developed. The workings of the model will be explained here:

The model simulates the victim moving in the water, for a few minutes before the drones arrive at the scene. After this dispatch time, it starts moving the drone(s) simultaneously to the victim. From here it checks if the victim is in view of the drone’s view every time the drone is moved to a new location. If this is the case, the victim is found and the search ends.

Victim movement is simulated using some randomness. The victim is moved every second at a random angle, this angle is taken using a normal distribution (centered at the angle the current is at). The victim is also moved forward every second, this distance is uniformly varied between the a flow rate of 5cm per second below and above the current. The randomness is included into the victim movement in order to have every search be a unique one, and sufficiently test the search algorithms.

Drone movement is a bit different from victim movement, this is totally not random. The drones are moved along a pre-determined search pattern.

The full code of the model can be found on our programming Github




Results

In this section we will talk about our results surrounding tests with our design, namely our physical demo test and our model tests.

Demo results

Model of Adapted Square Search

Our main demo results are that it is possible to detect heat sources and follow a search pattern with a Raspberry Pi drone model. For the demo we only executed the adapted expanding square search pattern due to practicalities with our test setup, namely that our test rig was relatively small and rectangular.

Model results

Different search patterns also influence the speed it takes to find the target. Multiple different patterns were investigated using a model made in Python. Using this model different search patterns were each tested for 10000 trials. A current of about 25 cm/s is used in the positive y direction. The drone has a field of view of 3 m by 2.4 m (this matches the FOV of the camera we used for the demo), and is moved at a max speed of 3m/s. 4 different patterns were investigated. These are Square Search, Adapted Square Search, Wait and Sweep, and a Cone search. The full results can be seen in the Appendix


Single drone

Square search is one of the most simple patterns. The patterns simply spiral out, in a square. This pattern is however not very applicable to our case. Since the target moves in (mostly) one direction, and is not standing still. This can also be seen in the simulation results. Of the 10000 trials done, only 1.4 % succeeded. With an average total time of 313 seconds including 4 minutes of dispatch time.

Since the Square search did not seem to work very well, an adapted version of the algorithm was set up, this Adapted Square search puts more focus on the flow direction, by looping back on itself . This pattern saw serious improvement over the ‘normal’ square search. The success rate went up to 22.4% with a total time of 326 seconds including the dispatch time.

Model of single drone Wait And Sweep

Cone search is different from the previous two patterns. It only searches in one direction and sweeps with increasing width. For this pattern multiple different cones were used, one with an angle of ¼ pi radians, one with an angle of ½ pi radians, and finally one with an angle of ¾ pi radians. The success rate varied between these, with the highest of 57.7% corresponding to the ½ pi radians cone. The other success rates were 16.9% and 51.3% belonging to the ¼ pi radian and ¾ pi radian cones respectively. The average total time of success for the ½ pi cone was 343 seconds including the dispatch time.

Wait and Sweep is different from the other patterns, this pattern does not focus on covering new areas it instead focuses on searching the same area over and over again, waiting for the victim to drift into the frame. The parameters going into this search are mainly the distance from the spot the victim dropped into the water and the sweep width. Multiple different distances (2.4 m, 7.2 m, 12 m, 16.8 m, and 24 m) were tried with both a 15 m and 30 m sweep width. A distance of 16.8 m with a sweep with of 15 meters resulted in the highest success rate of 70.0% with a success time of 428 seconds including dispatch time.


Model of multi drone Cone Search

One note here is that both Square search and Adapted Square search are a lot more efficient at lower flow rates. reducing the flow rates from 25 cm/s to 10 cm/s increases the success rates to 49.5% for normal square search and 92.5% for the adapted version. this is a very big improvement. While cone performs similarly and decrease in efficiency is seen for Wait and Sweep.

Multi drone

Both Square Search algorithms were not tested for multiple drones since the ‘normal’ square search was not very efficient and the adapted version is difficult to scale for multiple drones. The cone and sweep and wait patterns were however tested for multiple drones.

For both the cone and sweep and wait an increase in efficiency was found when implementing more drones. For both we saw more drones equal to a higher efficiency, however, for the sweep and wait the efficiency reaches a limit of about 95% (distance 15m, width 30m) while the cone also slowly gets more efficient, and reaches about the same limit of 95% (cone angle ½ pi radians). The cone has a 260-second average detection time while the sweep and wait takes a bit longer (288 seconds). These include the dispatch time. This means the cone will be a bit better on average.

Ethical analysis

The paper "Ethical concerns in rescue robotics: a scoping review"[21] describes seven ethical concerns regarding rescue robots. In this chapter, each ethical concern from the paper is summarized and then applied to our case.

Fairness and discrimination

Hazards and benefits should be fairly distributed to avoid the possibility of some subjects incurring only costs while other subjects enjoy only benefits. This condition is particularly critical for search and rescue robot systems, e.g., when a robot makes decisions about prioritizing the order in which the detected victims are reported to the human rescuers or about which detected victim it should try to transport first. For our scenario, when there is a situation where multiple people have fallen over board, discrimination is something to look out for. When searching for people, discrimination can not take place because if the robot has found one person it will continue looking for the others and since it does not take people out of the water it does not need to make decisions about prioritizing people. However, we are also considering dropping lifebuoys and here prioritizing order does come into account. In the situation where the robot can only carry one life vest, we propose that it drops the lifebuoy one the first person it finds, then comes back to the ship to restock, and continues the search. This will be the most efficient and also prevents discrimination between multiple victims.

False or excessive expectations

Stakeholders generally have difficulties with accurately assessing the capabilities of a rescue robot. May cause them to overestimate and rely too much on the robot and give false hopes for the robot to safe a victim. On the other hand, if a robot is underestimated, it may be underutilized in cases where it could have saved a person. For our scenario, it is important to accurately inform stakeholders about the capabilities and limitations of the rescue robot.

Labor replacement

Robots replacing human rescuers might reduce the performance of human contact, situational awareness and manipulation capabilities. Robots might also interfere with rescuers attempt to provide medical advice and support. In our case it is important to not fully take the human aspect out of the equation, the robot should be equipped with speakers for people on the ship to talk to the person. They can check how he is responding and reassure the victim that they’re coming to help. This will most likely help to calm the person. When the rescue attempt is being made by the crew or other rescuers, the drone should fly away since it’s no longer useful and as to not interfere with the rescue attempt.

Privacy

The use of robots generally leads to an increase in information gathering, which can jeopardize the privacy of personal information. This may be personal information about rescue workers, such as images or data about their physical and mental stress levels, but also about victims or people living or working in the disaster area. What this means for our scenario is that The information gathered by the robots is not shared with anyone outside professional rescue organizations and is exclusively used for rescue purposes. We should also try to limit the gathering of irrelevant data as much as possible.

Responsibility

The use of (autonomous) rescue robots can lead to responsibility ascription issues in the case when for example accidents happen. What this means for us is that we clearly need to state who is (legally) responsible in the case accidents happen during rescue (like operator, manufacturer etc.) Proposal: To ensure trust in our product, we should claim full (legal) responsibility as a manufacturer when accidents happen caused by design flaws or decisions made by the autonomous system.

Safety

Rescue missions necessarily involve safety risks. Certain of these risks can be mitigated by replacing operators with robots, but robots themselves, in turn, may determine other safety risks, mainly because they can malfunction. Even when they perform correctly, robots can still be harmful: they may, for instance, fail to identify and collide into a human being. In addition, robots can hinder the well-being of victims in subtler ways. For example, the authors argue, being trapped under a collapsed building, wounded and lost, and suddenly being confronted with a robot, especially if there are no humans around, can in itself be a shocking experience. Risks should be contained as much as possible, but we do acknowledge that rescue missions are never completely risk free.

Trust

Trust depends on several factors; reputation and confidence being the most important once. Humans often lack confidence in autonomous robots as they find them unpredictable. What this means for us is that we need to ensure that the robot has a good reputation by informing people about successful rescue attempts and/or tests, and limit unpredictable behavior as much as possible. We can also ensure trust by claiming full (legal) responsibility as a manufacturer when accidents happen caused by design flaws or decisions made by the autonomous system.


Conclusion

For the chosen scenario of search and rescue, the drones prove to be efficient tools in locating and helping victims. Specifically for man overboard situations, where the drone is able to cover large areas. These sorts of technologies prove to be efficient in increasing the rescue chances of such scenarios and can be expanded further than only maritime operations.

As we assumed in the beginning of the wiki that the chosen scenario excludes extreme weather conditions. In our case, mild wind and waves are tolerable as long as these sorts of conditions don't affect the operations of the drone. Certain cargo ship pathways were also chosen such that we avoid extremely cold waters (which are not common paths for ships anyways as seen in the state-of-the-art literature study).

When defining the search area, the current was also taken into account. Assuming that a person can drift up to 0.2 m/s, we calculated that the search area would be around 450 m². As for the search patterns, we can conclude that in the case where we know the exact direction a victim is drifting in, wait and sweep search is the most effective. However for a scenario where the exact drifting direction is less certain, the cone search is probably the best option. For single drone applications with a low drift rate the adapted square search pattern can also be useful.

For the construction of the drone we can conclude that it is a good idea to take a custom drone with a flight controller like the Pixhawk and then control the autonomous flight with Raspberry Pi. The Raspberry Pi will be connected to a thermal camera with a decently good resolution and framerate. The images captured by the camera will then be analyzed with TensorFlow to find victims in the image. When victims are found the ship will be notified of the location over radio communication.

Discussion

During this project there were quite some things that could have been done better. In the beginning of the course the biggest issue was defining the goal of the project and narrowing down the case. The first couple of weeks were spent on deciding on what we would do for this project and defining the specific preferences, requirements and constraints. This left us with a lot less time to go through all of the necessary design phases. The scale of the project was also changing as the weeks were going by. Some things that we wanted to do just weren't feasible in the amount of time that was left. The main example here was that we found out that actually getting an autonomous drone to work turned out to be very hard and expensive given how specific that use case is. The main problem was a combination of us either having to find a very expensive (in the thousands of euros) drone that can be made autonomous or built our own drone from scratch, which would still be pretty expensive and take a lot of effort.

Looking back on the course, the biggest issue that should have been tackled was the planning and making the decision to go for the vision that we had in the beginning. The weekly goal had to be more ambitious as we were left with little time for the project. The vision of the project should have been narrowed down for a specific case like the one that we have right now somewhere around the first week. After this a concrete plan on the whole approach should have been made rather than making up on the spot what needed to be done.
It might also have been a good idea to look into the feasibility of making a physical prototype earlier on in the project so that we could make better decisions surrounding this given that in the way it turned out, our physical demo was more a way of showing that the control software and automatic detection is possible instead of also testing our search patterns in real life.

As for our simulations, they are definitely useful, but do not account for more complex movement of the victim or the influence of the weather on the drone. Therefore it might be useful for future research to try to more accurately model oceanic current and the influence of wind and rain on the movement and detection accuracy of the drone.

Appendix

Full model results

Model of Square Search
Model of multi drone Wait And Sweep
Model of single drone Cone Search
Source.Name Count Mean STD Median Min Max Succes Ratio Succes Speed
Sweep5_d5_w10.txt 9006 306.4168332 66.22582738 280 240 590 0.9006 66.41683
Sweep4_d5_w10.txt 8840 313.3597285 69.5844632 290 240 590 0.884 73.35973
Sweep3_d5_w10.txt 8448 321.1316288 71.79807856 310 240 590 0.8448 81.13163
cone5_halfpi.txt 8266 261.1154125 10.50518206 260 240 310 0.8266 21.11541
Cone_2_halfpi.txt 7900 282.4898734 40.08713795 270 240 590 0.79 42.48987
cone_3_halfpi.txt 7684 267.8227486 21.28586226 260 240 560 0.7684 27.82275
cone4_halfpi.txt 7494 261.1956232 11.1747414 260 240 350 0.7494 21.19562
Sweep2_d5_w10.txt 7417 328.5371444 71.95174355 310 240 590 0.7417 88.53714
Cone_2_3quarterpi.txt 7303 316.4042174 73.25601744 280 240 570 0.7303 76.40422
Sweep1_d7_w5.txt 7007 428.0034251 85.79344366 420 240 590 0.7007 188.0034
Sweep1_d5_w5.txt 6940 335.2247839 73.24696142 320 240 590 0.694 95.22478
Cone3_3quarterpi.txt 6801 284.2067343 42.58235158 270 240 590 0.6801 44.20673
cone4_3quarterpi.txt 6644 268.385009 22.70430898 260 240 580 0.6644 28.38501
cone5_3quarterpi.txt 6350 262.5905512 15.23111931 260 240 430 0.635 22.59055
Cone_1_ halfpi.txt 5777 342.7367146 83.92058695 320 240 590 0.5777 102.7367
Cone_1_3quarterpi.txt 5125 357.8887805 85.33027697 330 240 590 0.5125 117.8888
Sweep1_d7_w10.txt 4534 430.8557565 87.55154123 440 240 590 0.4534 190.8558
Sweep1_d5_w10.txt 3661 364.1873805 73.35725043 340 240 590 0.3661 124.1874
Sweep1_d3_w5.txt 2635 288.2049336 46.36198948 280 240 540 0.2635 48.20493
cone5_quarterpi.txt 2396 258.6978297 10.86522538 260 240 300 0.2396 18.69783
cone4_quarterpi.txt 2242 258.8492417 11.08224585 260 240 300 0.2242 18.84924
Adapted_SQ.txt 2236 325.8184258 71.30174178 350 240 590 0.2236 85.81843
Cone3_quarterpi.txt 2060 257.7669903 10.84412396 260 240 290 0.206 17.76699
Cone_1_quarterpi.txt 1689 279.2243931 37.61462698 270 240 580 0.1689 39.22439
Cone_2_quarterpi.txt 1514 257.3051519 11.50892696 260 240 330 0.1514 17.30515
Sweep1_d3_w10.txt 1436 288.6699164 70.27055499 240 240 590 0.1436 48.66992
Sweep1_d10_w5.txt 968 528.7086777 40.5658764 530 340 590 0.0968 288.7087
Sweep1_d10_w10.txt 530 533.1320755 50.86772116 560 430 590 0.053 293.1321
Sweep1_d1_w5.txt 194 271.4948454 45.12885953 240 240 440 0.0194 31.49485
Square.txt 141 313.9007092 52.69552773 300 240 460 0.0141 73.90071


Logbook

Week Name Work done & hours spent Total hours
1 Geert Touw Group setup etc (2h), Subject brainstorm (2h) 4h
Luc van Burik Group setup etc (2h), Subject brainstorm (2 h) 4h
Victor le Fevre Group setup etc (2h), Subject brainstorm (2h) 4h
Thijs Egbers Group setup etc (2h), Subject brainstorm (2h)
Adrian Kondanari Group setup etc (2h), Subject brainstorm (3h) 5h
Aron van Cauter Group setup etc (2h)
2 Geert Touw Meeting 1 (1h), Meeting 2(1h), Subject brainstorm (1h) 3h
Luc van Burik Meeting 1 (1h), Meeting 2(1h), Subject brainstorm (2h) 4h
Victor le Fevre Meeting 1 (1h), Meeting 2(1h), Subject brainstorm (1h) 3h
Thijs Egbers Meeting 1 (1h), Meeting 2(1h), Subject brainstorm (2h) 4h
Adrian Kondanari Meeting 1 (1h), First concept elaboration (2h) 3h
Aron van Cauter Meeting 1 (1h)
3 Geert Touw Meeting 1 (1h), meeting 2 (1h), Oceanic weather + hypothermia and water + shipping routes research (3h) 5h
Luc van Burik Meeting 2 (1h), IAMSAR reading (3.5h), Drone navigation research (1.5h) 6h
Victor le Fevre Meeting 1 (1h), Meeting 2 (1h), Made the wiki look a bit more coherent (1h)

Wrote the problem statement and objective, users, scenario and added points to the MoSCoW (1.5h)

Read Sensor and Image Recognition papers (1h)

5.5h
Thijs Egbers Meeting 1 (1h), meeting 2 (1h), sea communication research (2h) 4h
Adrian Kondanari Meeting 1 (1h), requirements (2h), MOB protocols (2h) 5h
Aron van Cauter
4 Geert Touw Meeting 1 (1h), Meeting 2(1h), Ethics (4h) 6h
Luc van Burik Meeting 1 (1h), Meeting 2(1h), Search area (1.5hr), Drone search speed (1.5 hr), Wiki writing (0.5 hr) 5.5h
Victor le Fevre Meeting 1 (1h), Meeting 2(1h), looked at sensors and image recognition (2h) 4h
Thijs Egbers Meeting 1 (1h), Meeting 2(1h), drone controll research (3h)
Adrian Kondanari Meeting 1 (1h), Meeting 2(1h), sensor research (2h) 5h
Aron van Cauter
5 Geert Touw Meeting 1(1h), Made a search pattern with Victor (2h) 3h
Luc van Burik Meeting 1(1h), Working on Image detection(2.5h), meeting 2 (1h), Square search (2.5h),

adapted square search(WIP) (1h)

8 h
Victor le Fevre Meeting 1(1h), designed and tried tried to make an xy-rig (3h), worked on the wiki (2h) 6h
Thijs Egbers Meeting 1(1h), meeting 2 (1h), worked on getting the raspi setup (3h),

try to get the camera and soldering (0.5h)

5.5h
Adrian Kondanari Meeting 1(0.5h), communication methods research (4h) 4.5h
Aron van Cauter Meeting 1(1h)
6 Geert Touw Meeting 1 (1h), Meeting 2 (1h), Worked on wiki (2h) 4h
Luc van Burik meeting 2 (1h), Concert search pattern to work inside rest of the program (3h), programming more patterns(2h) 6h
Victor le Fevre Meeting 1 (1h), Meeting 2 (1h), Worked on wiki (4h) 6h
Thijs Egbers Meeting 1 (1h), Meeting 2 (1h), get the camera setup (1.5h), try to optimise camera reading (1h),

convert search pattern to work inside the rest of the program (3h)

7.5h
Adrian Kondanari Heating element (5h) 5h
Aron van Cauter
7 Geert Touw Meeting 2 (demo) (4h), Wiki (1h) 5h
Luc van Burik Meeting 2 (demo) (4h), finishing touches test program (2h), presentation (3 h) 9h
Victor le Fevre Meeting 2 (demo) (4h), Wiki (1h) 5h
Thijs Egbers Meeting 1 (1h), meeting 2/video making (4h), finishing touches test program (2h),

make presentation (WIP)

Adrian Kondanari wiki (2h) 2h
Aron van Cauter
8 Geert Touw Presentation (0.5h), Short meeting to prepare for presentations (0.5h), wiki (1.5) 2.5h
Luc van Burik presentations (2.5 h), Short meeting before presentations (.5h), working on wiki (6h) 9h
Victor le Fevre Finalizing wiki (5h) 5h
Thijs Egbers Presentations (2.5h), short meeting with Luc (0.25h), working on the wiki (4h)
Adrian Kondanari wiki (4h) 4h
Aron van Cauter



References

  1. ANNUAL OVERVIEW OF MARINE CASUALTIES AND INCIDENTS 2022. (2022). In EMSA (Ares(2022)8241169). European Maritime Safety Agency. from https://www.emsa.europa.eu/newsroom/latest-news/download/7362/4867/23.html
  2. Gonel, O., & Çiçek, I. (2020, 25 december). STATISTICAL ANALYSIS OF MAN OVER BOARD (MOB) INCIDENTS. ResearchGate. https://www.researchgate.net/publication/348266442_STATISTICAL_ANALYSIS_OF_MAN_OVER_BOARD_MOB_INCIDENTS
  3. https://www.marineinsight.com/marine-safety/man-overboard-situation-on-ship-and-ways-to-tackle-it/#:~:text=1
  4. https://content.iospress.com/download/international-shipbuilding-progress/isp23-260-02?id=international-shipbuilding-progress%2Fisp23-260-02
  5. https://owaysonline.com/iamsar-search-patterns/
  6. 6.0 6.1 O. (2021, 5 april). IAMSAR Search Patterns Explained with Sketches - Oways Online. Oways Online. https://owaysonline.com/iamsar-search-patterns/
  7. O. (2021, 5 april). IAMSAR Search Patterns Explained with Sketches - Oways Online. Oways Online. https://owaysonline.com/iamsar-search-patterns/
  8. 8.0 8.1 "Shipping routes are changing with the climate" by Brad Plumer, published on Vox on April 25, 2016 (https://www.vox.com/2016/4/25/11503152/shipping-routes-map).
  9. 9.0 9.1 "Cold Water Immersion and Hypothermia" by Aceboater, published on their website on an unknown date (https://aceboater.com/en-us/cold-water-immersion-and-hypothermia).
  10. 10.0 10.1 "SeaTemperature.org - Sea Surface Temperature (SST) map for the world's oceans and seas" by SeaTemperature.org, accessed on April 9, 2023 (https://www.seatemperature.org/).
  11. Frotan, D. F., & Moths, J. M. (2022). Human Body Presence Detection in Water Environments Using Pulse Coherent Radar [Bachelor Thesis]. Malmo University.
  12. Lomonaco, V., Trotta, A., Ziosi, M., De Dios Yáñez Ávila, J., & Díaz-Rodríguez, N. (2018b). Intelligent Drone Swarm for Search and Rescue Operations at Sea. HAL (Le Centre pour la Communication Scientifique Directe).
  13. Queralta, J. P., Raitoharju, J., Gia, T. N., Passalis, N., & Westerlund, T. (2020). AutoSOS: Towards Multi-UAV Systems Supporting Maritime Search and Rescue with Lightweight AI and Edge Computing. arXiv (Cornell University). https://arxiv.org/pdf/2005.03409.pdf
  14. Yeong, S. P., King, L. M., & Dol, S. S. (2015). A Review on Marine Search and Rescue Operations Using Unmanned Aerial Vehicles. Zenodo (CERN European Organization for Nuclear Research). https://doi.org/10.5281/zenodo.1107672
  15. Burke, C., McWhirter, P. R., Veitch-Michaelis, J., McAree, O., Pointon, H. A. G., Wich, S. A., & Longmore, S. N. (2019). Requirements and Limitations of Thermal Drones for Effective Search and Rescue in Marine and Coastal Areas. Drones, 3(4), 78. https://doi.org/10.3390/drones3040078
  16. Long-distance Video Streaming and Telemetry via Raw WiFi Radiohttps://dev.px4.io/v1.11_noredirect/en/qgc/video_streaming_wifi_broadcast.html
  17. AlphaPilot: autonomous drone racing, Philipp Foehn, Dario Brescianini, Elia Kaufmann, Titus Cieslewski, Mathias Gehrig, Manasi Muglikar & Davide Scaramuzza https://link.springer.com/article/10.1007/s10514-021-10011-y
  18. Autonomous Waypoint-based Guidance Methods for Small Size Unmanned Aerial Vehicles, Dániel Stojcsics http://acta.uni-obuda.hu/Stojcsics_56.pdf
  19. https://www.raspberrypi.com/documentation/accessories/camera.htm
  20. https://www.researchgate.net/publication/350981551_Real-Time_Human_Detection_in_Thermal_Infrared_Images
  21. Battistuzzi, L., Recchiuto, C.T. & Sgorbissa, A. Ethical concerns in rescue robotics: a scoping review. Ethics Inf Technol 23, 863–875 (2021). https://doi.org/10.1007/s10676-021-09603-0