# One page - Group 4 - 2018/2019, Semester B, Quartile 3

(Difference between revisions)
 Revision as of 14:27, 4 April 2019 (view source) (→Conclusion of the validation for this airport)← Older edit Revision as of 18:29, 5 April 2019 (view source) (→Decision Model validation)Newer edit → Line 2,060: Line 2,060: === Picking an airport to use for validation === === Picking an airport to use for validation === - So, as described, our approach was as follows. We picked a specific airport and filled in the questionnaire on their behalf. Since we do not know all the needed information for a certain existing airport in the Netherlands, we decided to come up with our own mock-up airport. We know all the important attributes, beliefs and wants of this airport, which allows us to fill in the questionnaire on their behald. We used the information from our [[Airports under a microscope - Group 4 - 2018/2019, Semester B, Quartile 3| airport analysis]] to come up with this airport and the reasoning behind what to answer to which proposition. Examples of important attributes we took into account were among other the size of the airport and the amount of daily departures and arrivals. + So, as described, our approach was as follows. We picked a specific airport and filled in the questionnaire on their behalf. Since we do not know all the needed information for a certain existing airport in the Netherlands, we decided to come up with our own mock-up airport. We know all the important attributes, beliefs and wants of this airport, which allows us to fill in the questionnaire on their behalf. We used the information from our [[Airports under a microscope - Group 4 - 2018/2019, Semester B, Quartile 3| airport analysis]] to come up with this airport and the reasoning behind what to answer to which proposition. Examples of important attributes we took into account were among other the size of the airport and the amount of daily departures and arrivals. === Filled in proposition on behalf of this airport === === Filled in proposition on behalf of this airport === Line 2,345: Line 2,345: === Discussion of the results for this airport === === Discussion of the results for this airport === - Now, we discussed with all the group members whether these outcomes did make sense. After looking through all the solutions that we have gathered, we indeed concluded that these solutions would be the best fit for our airport. First and foremost were the arguments that all these solutions were easily scalable and all had a neutralization range of 1000m. These were the some of the most important aspects for this airport and all solutions with a matching higher than 60% also agreed to these specifications. + Now, we discussed with all the group members whether these outcomes did make sense. It was interesting to see that there are no solutions that have a higher matching percentage than 77% with our mock-up airport. However, after looking through all the solutions that we have gathered, we indeed concluded that these top 7 solutions would be the best fit for our airport. So apparently there is no solution in the list that almost perfectly matches the expectations of the mock-up airport. After close investigation of the highest matched solution attributes against the needs of our mock-up airport, we indeed see that the top solutions mostly cover the airport's needs for a large part. First and foremost were the arguments that all these solutions were easily scalable and all had a neutralization range of 1000m. These were the some of the most important aspects for this airport and all solutions with a matching higher than 60% also agreed to these specifications. We could also cover all other propositions and compare these with the attributes of the solutions but that would simply not be very interesting and also would only just validate the logic behind the code of the web app. - What is also good to see is that some solutions that we included, of which we knew would not be ideal solutions, were not included in the top matches. For example, we also included human sight as a detection system, or eagles as a neutralization system, but all these solutions had a very low match compared to the others. For example, human sight only had a match of 20.5%, which makes sense since this is not a very good solution in the first place. We did however include these solutions to give as much freedom for the users, and also to validate that solutions which are less good are not picked that often. After all, good solutions having a good match is important, but it is also important that solutions that are not so good also get a match that is not so good. + What is also good to see is that some solutions that we included, of which we knew would not be ideal solutions, were not included in the top matches. For example, we also included human sight as a detection system, or eagles as a neutralization system, but all these solutions had a very low match compared to the others. For example, human sight only had a match of 20.5%, which makes sense since this is not a very good solution in the first place. This is also not a solution that any airport would not have come up with themselves. We did, however, include these solutions to give as much freedom for the users, and also to validate that solutions which are less good are not picked that often. As well as giving the users a comparison between such simple solutions and more sophisticated solutions, such that the users can decide for themselves whether these differences are worth investigating in. After all, good solutions having a good match is important, but it is also important that solutions that are not so good also get a lower match rating. After performing our internal validation, we indeed found this to be the case. It was unfortunate that our decision model could not be validated by external users, however, we hope that this validation, whilst far from perfect, gives some impression of the performance/accuracy of the decision model. === Conclusion of the validation for this airport === === Conclusion of the validation for this airport ===

# Group members

Name Student ID Major
Jort de Bokx 1050214 Software Science
Sander de Bruin 1006147 Software Science
Stijn Derks 1008002 Software Science
Martin de Quincey 1007047 Software Science and Applied Mathematics
Nick van de Waterlaat 1009357 Software Science

# Introduction

The goal of this wiki page is to show a study and analysis of a robotic subject. This research is an assignment of the course Robots Everywhere (0LAUK0). For this project, students work in a group of five people choosing a subject in the core of robotics to work on, thereby making sure the USE aspects are leading. As is usual in a Wiki, multiple pages will be used rather than considering one extremely long page. Make sure to explore all subpages contained under this page.

# Initial ideas

In this section, we present some initial ideas that were formed at the start of the project. Each of the initial ideas includes a short description of the problem related to the idea. Ultimately, the chosen idea is given.

## Robotic surgery

With all the progress in robotics, we have now reached a stage in time where it is (almost) possible to let robots do surgery. There have been quite some recent breakthroughs, and it is also already applied to some scale in hospitals. However, there are some aspects to this robotic technology that need closer investigation.

## Medical rehabilitation with the help of robots

Many people suffer from injuries that may require long-term medical rehabilitation. This rehabilitation is typically complex and takes a lot of staff to help guide the patients through the process. Then there might be benefits for both the patients and the staff helping the patients with rehabilitation if robotics were to help the rehabilitation process.

## Drone interception

Between 19 and 21 December 2018, hundreds of flights were cancelled at Gatwick Airport, following reports of drone sightings close to the runway. The airport did not have any measures to prevent this issue. Many users of airlines were stranded, and airlines (enterprises) lost. The airport only had detection and tracking devices, but no counter-drone mechanism. Just like birds, drones can cause enormous damage to aeroplane engines and are therefore illegal around airports. However, no airport yet has a fully working anti-drone defence mechanism, while most airports do have anti-bird systems, consisting of noise mechanisms to scare birds away.

## Drone pesticides

An important consequence of the increased global population is the demand for food. In order to meet these demands, farmers require the use of pesticides to ensure enough yield from their crops. However, the overuse of pesticides and fertiliser can have huge negative impacts on society. Hence we the use of drones to analyse the state of farmland and automatically apply fertiliser and pesticides as needed could make a farmer’s job easier, making the production more eco-friendly.

Trading bots have been used on the stock market for quite some time already, but ever after the boom of cryptocurrencies, the usage of these bots has become ever more increasing. The stakeholders of these bots are people that are active in, for example, the stock market and cryptocurrency market. People could use such a bot in order to achieve a passive income. Designing such a bot for interested parties would be interesting. Furthermore, it would be interesting to consider the ethical discussion regarding the permission to use such trading bots in the stock market.

## Networking AI

Gridlock is problematic in large western cities, but also many large cities with underdeveloped infrastructure in countries like Asia. It massively hinders any form of transport, and also unnecessarily increases pollution. Forms of AI in private cars or forms of public transport such as buses or trains might help reduce this problem. On an abstract level, buses or trains could adjust their schedule or route such that they are deployed at places where passengers are waiting in real time, not where they are expected to be waiting. This way, one might prevent the case where two half-full buses are driving on similar routes. By sharing information and adapting to real-time information, in this case only one bus would be necessary.

## Use drones to monitor and improve marine life

Due to climate change, many problems arise. A large part of these problems emerges in the seas and underwater. Examples include changes to the habitat of marine mammals, irreversible damage to coral reefs, and already endangered species being threatened quicker by their changing environment. Current use for them is flying through and capturing fluid samples of the exhaled fluids of whales, in order to monitor their health. Specific autonomous robots designed for underwater operation might help monitor the state of coral reefs, and introduce new coral to a reef to support its growth.

## Chosen concept

The concept we have chosen considers Drone interception but applied in the setting of airports. Initially, we consider this concept in a more general context, but after the feedback provided by the professors and internal discussion, the group has decided to limit itself to the context of airports.

# Project setup

In this section, we put the project setup under a microscope. We take a look at the objectives of the project; the approach is taken, the planning during the project, milestones of the project, and deliverables that will exist at the end of the project.

## Objectives

The objective of our project is designing a decision model for airports to decide what anti-drone solution is the best. This model will be delivered by means of a report that can indicate which solutions are better for which types of airports. This report will also give extensive argumentation for why certain solutions outperform others in certain scenarios.

Objectives of the project as a whole include:

• Gaining insight into accidents and incidents involving various forms of drones.
• Identify and specify the currently existing countermeasures and counter mechanisms against drones and UAVs in general.
• Identify and specify the USE stakeholders of the problem space and their interests regarding possible solutions.
• Propose multiple possible solutions to the problem.
• Identify the advantages and the disadvantages centred around user interests for each provided solution.
• Validate and verify that our proposed solutions solve the discussed problems with respect to the USE stakeholders and their interests.
• Design a basic decision model around providing solutions for airports against UAVs.

## Approach

We now take a look at how we will approach this project. We will start our approach by doing an extensive study into the current state of the problem. We will do this by studying the literature of different forms. We will look at papers where this problem has been discussed before, but also at what the current solutions are at the moment and what their flaws are. Furthermore, we also look at studies and research of institutes that have made investigations into this phenomenon.

After we as a group have a good grasp on the problem, we analyse the problem ourselves from a USE (User, Society, Enterprise) point of view. These three components will be central in our study, and the development of our design as the users of the technology always need to be the main focus. Following this study into the USE aspects surrounding our problem space, we expect different categories of subproblems to arise. For example, when considering two distinct incidents involving UAVs, they might be categorised by the type of failure that occurred, be it a human failure or technical failure. We expect that many of these distinctions can be made, and as different categories of subproblem might involve different USE aspects, they might require different solutions. Furthermore, we take a look at different types of airports and see what each of these types need for an anti-drone solution.

After this, we will provide possible solutions for a number of distinct problem categories. For each discussed problem category, we will then present multiple implementations of these requirements and functionalities, which will be our first drafts. These draft solutions will be further discussed and analysed based on their advantages and disadvantages. We will also provide research into the feasibility of these proposed solutions.

As the discussed subproblems heavily integrate with various aspects of society, we are also interested in the ethical aspects of the evolution of the proposed technologies of counter-drones. We will investigate the ethical and regulatory consequences of these developments. We might also provide insight into which areas of the problem space have not been sufficiently discussed by previous research and falls outside the scope of ours. For future reference, we also look ahead and shortly discuss improvements or otherwise changes to our proposed solutions, that are not currently possible due to technical or other limitations. Finally, we will wrap up with completing the wiki and our documentation of the project.

Once we have a good comparison fo the various existing solutions, we will implement them in a concise decision model, aimed at informing Airports of the status quo regarding drone defences, and provide a decision model to allow them to choose the most suitable option for them. The model will be presented through the means of a Web Application.

## Planning

We now take a look at the planning of the project. The planning is presented in the form of an excel sheet that clearly states the tasks that need to be carried out, by whom these tasks will be carried out, an estimation of the time that it takes to carry this task out, if the task has been completed or not, and when it needs to be completed. Furthermore, an orange cell indicates that this will be done during a group meeting, and a blue cell indicates that this will be done outside of a meeting. Note that this planning also considers the division of work in no small degree.

## Milestones

We now consider the milestones within the project. Here, we consider Table 1 that displays the accomplishments on a specific date. Furthermore, if there were any learning moments during each of these accomplishments, they will be written down in the `Aditional notes' column and taken into consideration for the next accomplishment. Note that this table will be regularly updated throughout the course.

Table 1: Milestones
Date Accomplishment
06/02/2019 Finalise the decision of the subject
17/02/2019 Finalise research into State of The Art
19/02/2019 Formulate analysis of problem space
6/03/2019 Formulate possible solutions to identified problems
27/03/2019 Formulate possible further improvements
29/03/2019 Formulate conclusions regarding proposed solutions
26/03/2019 Create a presentation format of our research
01/04/2019 Present complete research
05/04/2019 Finalise the Wiki and documentation

## Deliverables

We now cover the deliverables of this project. The deliverables focus on the problem introduced in the problem description. These deliverables for this project will be as follows: In this section, we refer to the present situation, which consider the present situation regarding the specific problem description. We discuss the current rules and regulations, current solutions, and the limitations of the current rules, regulations, and limitations.

• A presentation regarding the problem and possible solutions

This presentation will be held in the final week of the course. In this presentation, we start by introducing a problem through a summary of the problem description. Then, the finding regarding the problem will be presented. This is followed by multiple solutions to the problem with their advantages and disadvantages. Then, we zoom into the `best' solution and provide a design regarding this solution. If possible, a demonstration will also be given.

• A literature study in the form of coherent Wiki pages in a hierarchical manner

This Wiki page contains an in-depth study regarding the problem introduced in the problem description. An extensive literature study will be presented, which offers multiple solutions with both their advantages and disadvantages. Furthermore, it will be argued that what solution would be the `best' through means of a decision model. This is followed by areas that are still undiscovered and improvements that can be made to our design.

• A Web App that implements a decision model

This Web app should implement a form of a decision model. The result of this decision model should be a possible solution against unwanted UAVs near airports based on input obtained from an individual. This input should consider the needs, beliefs, and wants of an airport when it comes to defending themselves against illegal UAV activity.

# General problem

## General problem description

Between 19 and 21 December 2018, hundreds of flights were cancelled at Gatwick Airport, following reports of drone sightings close to the runway[1]. A total of 760 flights were disrupted on the 20th of December due to the drone. Naturally, this angered many people whose flight was delayed. Not only does it anger people, but it is also a financial worry for the airport organisation as all of these people with delayed flights have to be compensated. The airport did not have any `good' measures to prevent this issue. Gatwick chief operating officer Chris Woodroofe said: `The police are looking for the operator and that is the way to disable the drone'[1]. Woodroofe further elaborates that the police had not wanted to shoot the devices down because of the risk from stray bullets. This is, of course, not something that is to be repeated as this caused much inconvenience for many travellers. The airport itself only had detection and tracking devices, but no real effective counter mechanisms available. This issue is not limited to the setting of airports, but it can be further extended to any hot spot, such as the centre of cities, special events that involve essential figures, and more. With the ever-increasing possibilities of technology, it should in the future not be unexpected for an unmanned aerial vehicle (UAV) to suddenly show up and wreak havoc. This havoc can range from taking pictures of people in public places to spy or stalk them to terrorists that use UAVs to drop bombs in highly populated areas. These occurrences are more likely to appear as the technology we possess increases.

We think that we should not sit idle and passively wait for the worst-case scenarios to occur before starting to think about countermeasures. The recent incident between 19 and 21 December 2018 at Gatwick Airport should already sound an alarm that we should take an active attitude and develop mechanisms that counter UAVs in productive ways. These mechanisms should be able to deal with much more than mere birds and should consider any form of terrorism that can be caused through the airspace.

## State of the Art

In order to gain more insights regarding the topic, we do extensive research. The results of this research can be found on the State of the Art page.

## General USE aspects

In this section, we consider the users, society, and the enterprise based on the general problem description.

### Users

There are multiple stakeholders involved in a solution to the problem introduced in the problem description. By far the largest and most diverse category of stakeholders consists of those parties that are (majorly) disadvantaged by unauthorised or unwanted UAV operations, or malicious events that could follow. We can identify several subcategories of stakeholders whose material possessions or immaterial values are at stake.

The most important users are as follows:

• The Government

As the governing body, a collection of many large instances with national interests, the government of a nation aims to preserve the material and immaterial assets of its citizens. This goal might be obstructed by the unwanted presence of UAVs or events caused by them.

• Non-governmental organisations

Includes companies or other privately owned bodies that want to protect their material assets against damage from UAV incidents, or protect immaterial assets such as privacy or intellectual property that could be violated by the unwanted presence of UAVs.

• Civilians or individuals in general

Civilians or individuals, in general, are also stakeholders in the problem space that we consider. They might have their assets violated in some way by UAVs or UAV related events, such as civilian espionage empowered by UAVs. Since drone flight is growing as a commercial pass-time, more specifically the operation of drones by civilians for fun is becoming more popular, this user group must also be considered as a stakeholder from this perspective.

### Society

Society as a whole is affected by the already existing and upcoming dangers of drones. First of all, drones can be a massive hindrance at for example airports (Gatwick airport), football stadiums or other public places. However, apart from simple hindrance, drones can also be extremely dangerous for society, as they can be weaponised and used by terrorists, the military or any other person with bad intentions. Furthermore, more and more people are using drones privately, causing privacy issues for society, as these drones are equipped with a camera most of the times and can easily reach private places. This is why drone interception is so vital to society. If there were to be a tool that could detect, identify and neutralise drones, this could help in a decrease of hostile drones and thereby also decrease the dangers above that drones bring to society.

### Enterprise

Enterprise is greatly affected by the illegal use of drones; for example, the airlines at Gatwick lost much revenue due to delayed flights and passenger compensation. The airport itself also suffered from the forced shutdown. Drones also threaten other industries, espionage via drones can be done remotely, where attackers may steal a company secrets through aerial photography or by taking pictures through windows. Hence the development of anti-drone will be of enormous benefit to existing enterprises but also spark new business opportunities for security contractors and UAV oriented startups.

## Zoom in

After a lot of internal discussions, we found out that we have to zoom in more due to otherwise undertaking too broad of a problem space. Therefore, we zoom into the problem description and define a more specific problem description on the Specific problem description page.

# State of the Art

We now consider the State of the Art regarding the general problem description. We take a look at numerous distinct papers and patents.

## Game of drones: defending against drone terrorism[2]

This article discusses the threat of weaponised drone warfare. Not only are drones UAVs that may hinder people at places like for example airports, but they can also be equipped with weaponry, and this potentially makes them extremely dangerous. Weaponised drones could be used in terrorism as they are unmanned and can be operated from a distance, meaning that no people are put at risk. However, for this same reason, it can also be used for military purposes. On November 3, 2002, the era of weaponised drone warfare began when an American drone blasted a car with a missile, killing all six occupants. Since weaponised drones form such a threat to potentially innocent people, the article lays out the three challenges to defending against drone terrorism: detecting potentially hostile drones, identifying them, and destroying or neutralising them.

The detection of drones could be done by using a radar detection system, where the location and height of a detected object in the air can be calculated. However, the critical challenge of the radar system is to determine whether such a detected object is actually a drone. Furthermore, due to the way a radar detector works, a drone can be created using materials such that it will not be detected. Therefore, there is a need for a proper identification system to classify a detected object as a potentially dangerous drone. Identification of any specific aircraft, at present, relies upon broadcasting a coded signal, which is decoded by air traffic control towers. Such that allies and enemies can be identified and to avoid targeting a friendly aircraft. As a result, all aircraft where radar service is provided should require systems that can broadcast coded signals for identification. Once a drone has been detected and identified as potentially hostile, it needs to be neutralised. Drones can be shot down, have their guidance systems damaged, or their control signals can be jammed or interfered with. Air-to-air missiles and gunfire, lasers can be an effective weapon against drones. Lastly, electromagnetic attacks that consist of interfering with the GPS signals would make the drone uncontrollable to the pilot and using 'spoofing' could enable an attacker to take control of the drone.

## Investigating Cost-effective RF-based Detection of Drones[3]

The focus of the article is on the detection of a drone, such that it can be dealt with. More specifically, a drone detection system that autonomously detects and characterises drones using radio frequency wireless signals. Where two approaches are proposed, both using inexpensive technology, e.g., WiFi and inexpensive software-defined radios, to automatically detect drones. One effective method that detects drones by observing the reflected wireless signal, and a second passive method that listens to the communication between the drone and its controller. In the active method, a Wi-Fi receiver can be used to detect a drone based on the signature of the signal reflected from the propellers of a drone. Similar to radar, a transmitter broadcasts signals and a receiver captures reflected signals that bounce of a drone. The passive method detects a drone by listening to the communication channel between the drone and its controller using a wireless receiver. Usually, drones communicate with their controllers a few times per second to update their status and to receive commands from the controller. A system could collect wireless samples and observes the signal, analyse them and can then detect a drone's presence.

## Clash of the drones[4]

The motivation for this article was the trouble at Gatwick Airport in London, where flights had to be diverted because a drone was spotted nearby. They stated that in the year 2017, in the UK alone, it has happened over 100 times that a drone was too close to an Airport. These events are undesirable, and thus authorities are trying to find reliable and safe strategies to take down these drones. They state that current countermeasures of taking out drones cause too much collateral damage. One option would be “Geo-fencing”, where drones would simply be fenced out due to software. However, this requires the manufacturers to implement this and the users to not tamper with this, which is considered too risky. The Dutch Ministry of Justice and Security even gave away \$30.000 for the best idea to take out drones, so the desire for such technology is high.

Ideas were among others using other drones to take out the undesired drone. Other examples were using airguns to bring the drones to the ground, and training animals such as eagles to take down the drones. The consequences of drones on airports are catastrophic. Even a small drone could severely damage the windshield of an aeroplane, so there need to be forbidden zones for drones to guarantee public safety.

## Small Remotely Piloted Aircraft Systems (drones), Mid-Air Collision Study[5]

The Department for Transport, the Military Aviation Authority and British Airline Pilots’ Association commissioned a study about what the consequences are of collision mid-air between a crewed aircraft and a drone. The goal of the study was to find the minimum speed at which such a collision would cause critical damage to the aircraft. An important note is that they only focused on windscreen collisions, and did not take, e.g. the motors into account. The main results of the study were that for aviation aeroplanes with windshields that were not birdstrike-certified, the damage done was critical at speeds well below the regular cruise speeds.

For airliners, their windscreens are much more resistant. For drones in the 1.2kg class, no critical damage occurred, but for drones in the 4kg class, the damage did undoubtedly occur. Another interesting remark is also that how the drone was built has a significant influence on the damage done, for example, if the motors are covered in plastic or not. Their study also concluded that drones do much more damage than regular birds at equal speeds and with equal weight. This is due to the fact that birds act more like a fluid when colliding at such speeds, whereas the drones do not act like this due to their hard materials.

## Drone Safety Risk: An assessment[6]

This study, published by the Civil Aviation Authority, 2018, has investigated the likeliness of a collision between a drone and an aeroplane, as well as the consequent damage. In January 2018, there have been seven confirmed cases of a direct collision between a drone and a civil or military aircraft. Furthermore, they have estimated that the probability of a drone being in the proximity of an aircraft going at speeds high enough such that a collision could cause damage, is about 2 in a million. Furthermore, the probability of consequently causing critical damage is even lower than this probability.

They have also investigated the consequences of a drone colliding with a turbo-fat jet engine. They have concluded that a small drone would not do any significant damage. On top of that, even if it did damage, a multi-engine aircraft should still be able to land most likely. However, they also stated that helicopters are much more susceptible to drone collisions.

## How do you catch a drone? With an even BIGGER drone and a giant net: Tokyo police reveal bizarre 'UAV catcher' [7]

In this article, emerging technology is discussed to take out unwanted drones. They do not only discuss the technology but also report on the fact that police have officially employed this technology in Tokyo, Japan. The technology that they use is a drone with a net attached to it, making it able to catch the unwanted drones. The primary motivation for this deployment of technology was a security breach from 2015. A man called Yasuo Yamamoto controlled a drone that contained dangerous concentrations of radioactive caesium and landed it on the roof of the Japanese Prime Minister’s Official Residence. It managed to stay there undetected for 14 days after it was accidentally discovered during a tour around the building for new employees. The goal was to raise awareness to close all nuclear reactors in Japan.

The developed counter-drones will be used to find and capture malicious drones who fly dangerously close near public officials, in fear of, e.g. a drone containing explosives. The deployment of these drones was part of a more massive project of Japan in order to strengthen airspace security. Masahiro Kobayashi, an Osaka-based lawyer, mentioned that the biggest fear raised by experts is still the possibility of uncrewed aircraft coming too close to commercial aeroplanes.

## The SkyWall 100 bazooka captures drones with a giant net[8]

This article discusses a new type of technology in order to take unwanted drones out mid-air. In a nutshell, it is a bazooka which can shoot nets as far as 100 meters away. The bazooka is portable and can be operated by a single individual, meaning it is not a stationary weapon and can thus be moved from place to place. Important to note is that after the net is shot and the drone has been successfully shot, a parachute on top of the net is deployed to avoid any dangerously falling debris. The product is not meant for regular people, but the device is marketed to be deployed at sensitive events and near buildings such as an airport.

The bazooka is also equipped with an intelligent locking system to aid the controller to hit the drone successfully. They also announced the SkyWall 300, which is a remotely controlled mounted tripod with the same effect but with a further range. The SkyWall 100 was not yet available at the time of release, but nowadays it has been used multiple times, for instance at a Berlin air show in April 2018.

## A literature review on new robotics: automation from love to war[9]

In this literature review, Royakkers and Est investigate the social significance of robotics for the coming years in both Europe and the US by studying robotics developments in five different areas: the home, health care, traffic, the police force, and the army. Royakkers and Est argue that our society currently accepts the use of robots to perform dull, dangerous, and dirty industrial jobs, but wonder how this will be in the future as robotics is moving more and more out of the factory. Royakkers and Est provide a literature review that `attempts to provide an engaged but sober (non-speculative) insight into the societal issues raised by the new robotics: which robot technologies are coming; what are they capable of; and which ethical and regulatory questions will they consequently raise?' Especially the areas that concern the police force and the army are useful for the problem definition we provided.

Royakkers and Est argue that police robots are still in an experimental, exploratory phase, but that the USA and Japan are way in front of Europe when it comes to the development of these robots. The two central applications are carrying out surveillance and disarming explosives. One ethical issue that Royakkers and Est bring up is a discussion about privacy versus safety. They argue that a tricky issue with robots is the violation of privacy. Moreover, there is a risk of manipulation of sound and recordings, which would be a considerable disadvantage. Furthermore, what happens if malicious attackers steal essential data stored on such a robot? The increasing deployment of police robots would also mean that police officers must acquire new skills, which costs time and money. It could also eventually lead to the loss of essential police skills as the police officers will be trained in different ways.

Furthermore, there is an essential legal complication regarding the deployment of airborne robots for police purposes. That is, it is not yet clear how they can be deployed following existing laws and regulations. Abuse and proliferation are important factors that have to be kept in mind as well. Specific safety rules will have to be met, and these robots must not pose any danger to civilians at all. What would happen if one of these robots were hacked? There could be disastrous consequences, which could then lead to even stricter legislation concerning employing these robots. That is not all. Armed police robots will raise critical ethical questions on the usage of these robots.

## Developing tools to counteract and prevent suicide bomber incidents[10]

In this paper, Royakkers and Steen describe how teams of developers and designers engaged with ethics in the early phases of innovation based on case studies in the SUicide Bomber COunteraction and Prevention (SUBCOP) project. In order to achieve that goal, Value Sensitive Design (VSD) is used as a reference. The most important ideas presented in this scenario are the focus on the effectiveness, the safety, and the utility of the tool developed. That is, their ability to remove the threat, the ability to survive the threat, and the ability to properly utilise the tool. Five selected tools were developed by different teams of researchers and developers of different organisations: An Acoustic Warning Signal Projector (A-WASP), electronic countermeasures to prevent remote detonation, procedures for using electroshock devices, a system that produces a Water Mist, and a protective shield. Here, these last two are both for blast and fragmentation mitigation. These tools are aimed at various things. Some tools such as the Water Mist focuses on protecting bystanders, whereas the electroshock devices are intended to approach and engage suspects. At the end of their paper, Royakkers and Steen conclude that the researchers/developers involved are able to do something similar to VSD, supported by relatively simple exercises in the project, such as meetings with potential end-users and discussions with members of the Ethical Advisory Board of the project.

## Persistence Surveillance of Difficult to Detect microdrones with L-band 3-D Holographic RadarTM[11]

This paper focusses on the detection of small, difficult to detect, microdrones and how to discriminate drones from other moving objects. Since scanning radars have to find a compromise between time on target and update rate, this can negatively impact the radar from reliably detecting very weak signatures targets in another clutter of objects. What this means is that the scanning radar cannot see a difference between drones and birds, when for example a drone is flying between a group of birds. Then employing a 2D antenna and appropriate signal processing to create a multibeam, 3D, wide area overcomes the weakness of scanning radars and achieves high detection sensitivity. A decision tree based classifier can be used to identify the difference between drones and other moving objects. Where it rejects non-drone targets, decreasing the number of false positives and increases true positives. Such that when neutralising such a moving object in the air, with high probability, it will be a drone instead of for example a flying bird.

## Radar-Based Detection and Identification for Miniature Air Vehicles[12]

This paper discusses a radar-based detection and identification method for drones. More specifically the paper describes the design of a lightweight, X-Band (10.5GHz) radar system for use on a small-scale (less than 25 kg) rotorcraft. The prototype implementation of the radar is small enough to be carried by a drone and is able to differentiate other 'miniature rotorcrafts' (drones) by their doppler signature. The prototype uses a radar system which utilises electromagnetic energy to gain information on objects by analysing the reflected energy. The types of radars used are continuous wave radars, and a frequency modulated continuous wave radar and a Doppler radar. While in the paper the cause of the radar system is to avoid aerial collisions between uncrewed vehicles, it might still be useful to us as it is a method of detecting and identifying moving objects in the air. Therefore it this prototype can also be used/expanded for neutralising such aerial objects, might a drone be detected and identified.

## Privacy, data protection and ethics for civilian drone practice: A survey of the industry, regulators and civil society organisations[13]

In this article, Finn and Wright present results of a survey of primarily European drone industry representatives, regulators, and civil society organisations that examined privacy, data protection, and ethics concerning civilian drone operations. The article also demonstrates, using self-reported information from industry representatives, that these stakeholders do not have a clear understanding of European privacy and data protection law. Finn and Wright argue that this can impact their levels of liability and protections for individuals on the ground. The findings in this article demonstrate that law enforcement, commercial, and private or recreational drone operators are all thought to be associated with significant privacy, data protection, and ethical risks. Here, the recreational operators are thought to carry the highest risks. The article concludes with a consideration of the implications of these findings for the regulation of privacy, data protection and ethics for civilian drone operations.

## Robot ethics: Mapping the issues for a mechanised world[14]

In this article, Lin et al. describe what kind of new ethical and policy challenges are introduced to society due to the emerging technology of advanced robotics. They point towards the flourishing role of robots in society - from security to sex - and survey numerous ethical and social issues. These issues are divided into three categories; safety and errors, law and ethics, and social impact. Lin et al. argue that these future robotic technologies, first and foremost, need to be safe, while they point towards examples of where this went wrong in the past. They argue that with robotics, the safety issue is with their software and design. Errors and vulnerabilities are likely to exist. These errors and vulnerabilities could lead to fatal results when it comes to robotics. Furthermore, linked to the risk of robotic errors, it may be unclear who is responsible for any resulting harm. Product liability laws are primarily untested in robotics and, continue to evolve in a direction that releases manufacturers from responsibility, e.g., end-user license agreements in software.

It is argued that one way of minimising the risk of harm from robots is to program them to obey our laws or follow a code of ethics. That is, however, easier said than done as laws can be vague and context-sensitive. It is further argued that `even the three (or four) laws of robotics in Asimov's stories, as elegant and sufficient as they appear to be, create loopholes that result in harm'. The importance of privacy and laws concerning this privacy are touched upon. To make things worse, ethical and cultural norms, and therefore law, vary around the world, so it is unclear whose ethics and the law ought to be the standard when it comes to robotics. Such challenges could require international policies, treaties, and even laws. Other questions regarding the social impact are: `What is the predicted economic impact of robotics?', `How do we estimate the expected costs and benefits?', and `Are some jobs too important or too dangerous for machines to take over?'. The article presents many questions on which the answers can vary a lot. One thing the article makes clear, however, is that we have to start thinking about these challenges already.

## Policing Police Robots[15]

Joh argues that as there will be changes in healthcare, manufacturing, and the military due to robots, these robots also have the potential to produce tremendous changes in policing. She argues that we can expect that at least some robots used by the police in the future will be artificially intelligent machines capable of using legitimate coercive force against human beings. She does not explicitly state whether she thinks this is a good thing or not. She continues by bringing up the assumption that police robots may decrease dangers for police officers by completely removing these officers from situations that have the potential to be dangerous. Moreover, those suspected of crimes may risk less injury if robots can assist the police in conducting safer detentions, arrests, and searches. On the flip side, however, the use of robots also introduces new questions and challenges about how democratic norms and laws should guide decisions made by the police. Joh argues that these questions have yet to be addressed systematically. Furthermore, she states that how we design and regulate some uses of police robots requires a regulatory agenda right now in order to address the foreseeable problems of the future.

## Privacy and drones: Unmanned aerial vehicles[16]

In the paper, Cavoukian discusses, amongst other things, privacy concerns associated with the deployment of UAV technology. Furthermore, the paper addresses the privacy concerns by showing how privacy by design approach can assist in ensuring that the benefits of drones are facilitated while reducing privacy issues. Due to the manner in which drones may collect information, they pose privacy issues. The sensor equipment on board of drones may be commonplace in the consumer marketplace. However, drones have the ability to gather information dynamically from vantage points where for example regular video surveillance cameras or the camera of peoples phones could not reach. Since these drones can gather information so dynamically, on private property, for example, it creates these privacy concerns. Especially since the drone market is growing so much for the consumer market, now, if drones were to be designed with privacy in mind, the privacy concerns of the drone could be addressed appropriately. That is, drones should have privacy built into the system, the equipment on the drone should not monitor any private areas — for example, the insides of public washrooms, or peoples homes/backyards, and so forth.

## Anti-drone flight protection systems and methods[17]

This patent, owned by Etak Systems LLC, a telecommunications company in the US, describes the user of Geo-fencing to avert uncrewed aerial vehicles. It describes the use of transmitting geo-fence coordinates, avoidance commands or disruption of radio communication in order to avoid UAV's entering no-fly zone. The patent describes various flow diagrams dictating how a UAV should receive, process and respond to avoidance commands transmitted over cellular networks, or between other UAVs. It assumes flowcharts for cooperating drones, where the main focus of the technology is collision and object avoidance, as well as procedures for the save removal of distressed (low battery, mechanically malfunctioning) or rogue UAVs. The following flowcharts consider mandatory "kill commands" the UAV is expected to follow, leading to a safe and immediate emergency landing, or, in the case of rogue drones, transmission of a signal interrupting the communication associated with the UAV.

## Exploring civil drone accidents and incidents to help prevent potential air disasters[18]

Following an alleged drone collision with an Airbus A320 owned by British Airways at Heathrow Airport, the need to understand accidents and incidents involving drones arose. In this paper, Wild, Murray, and Baxter analyse and discuss one hundred and fifty-two events involving drones, or Remotely Piloted Aircraft Systems (RPAS). Differences were found between events involving these RPAS and events involving Commercial Air Transportation (CAT), where these events were categorised by their type, the specific safety issue, and the phase of flight. It was found that, compared to CAT, events involving RPAS more frequently involved a loss of control during flight, events occurring during takeoff, and general issues with the associated equipment. In the analysed events, technology factors, rather than human factors, contribute the most to these accidents and incidents involving RPAS. This article is part of our literature study as it provides more context on various events involving RPAS.

## Determination and Evaluation of UAV Safety Objectives [19]

As the integration and acceptance of UAVs in society grows, so does the need for appropriate security measures when these UAVs carry out operations in civilian airspace. In this article, Clothier and Walker discuss the safety measures surrounding UAVs and the need for developers, operators, and regulators of UAVs to prove that they have at least the same level of safety standards as human-piloted aerial vehicles. The paper defines various safety objectives of UAVs, the impact of these safety objectives and their applications on the design and operation of UAVs, and the societal acceptance of the risk factors surrounding UAVs. It is of value to our research as it provides a base analysis of safety objectives of UAVs, and the findings of this work can be used to define appropriate countermeasures for (civilian operated) UAVs.

## An innovative response to commercial UAV menace - Anti-UAV falconry [20]

This paper from the Educons University in Serbia talks about how UAV Falconry, i.e. the use of Birds like Eagles to attack undesired UAVs. It shows that the use of animals in security-related tasks has been done for thousands of years. Recently, the Dutch national police have partnered with private enterprises to train eagles to track and hunt down drones. The paper then discusses many drone-related incidents that would have been prevented by the use of this falconry. The enterprise responsible for training the eagles is Guard From Above, describes their method as “a low-tech solution for a high-tech problem”. The use of Eagles is pretty compelling due to their natural talent for mid-air combat, their massive speed advantage compared to drones and the ease of training. However, the solution was also criticised in the paper as being expensive, and limited to situations where birds could safely fly. The competition from other systems would deem this technology hard to justify in the future.

## Taking Flight: The Future of Drones in the UK [21]

The UK is one of the countries at the forefront of the rapidly developing market of commercial UAVs. The public sector employs UAVs to significant effect, for example in emergency search and rescue operations, and assist people working in a hazardous sector to reduce the risk their job exposes them to. However, sparked by the recent disruptions of operation at Gatwick airport, among others, the government of the United Kingdom presented this document in January 2019 outlining the following regulations in the drone sector. Existing regulations prohibit drone use near people or property in the UK, as well as requiring the drones to follow a flight path where it stays within line of sight of the operator. The government of the UK outlines in more detail the following regulations, where they aim to work together with the Civil Aviation Authority (CAA) as well as drone manufacturers, in an attempt to ensure safety and security in the airspace while civil aircraft become more popular.

## Defense against drones [22]

As a company specialising in X, Battelle has expertise in the areas of communications, electronic warfare and its countermeasures. They recognise that the growing popularity of UAVs poses real dangers to government and privately owned agencies, officials and assets. To serve the growing need for countermeasures against unwanted UAV presence, Battelle has created a focus area for counter unmanned aerial systems (cUAS). Their current top of the line product is aptly named the DroneDefender and disrupts the remote control systems and GPS systems of unwanted UAS. Their product presents one possible solution for the problem presented in our study and is therefore of great use to our work.

## Counter-unmanned aerial vehicle system and method[23]

This patent, owned by Lockheed Martin Corp, an aerospace and defence company in the US, describes the use of nets to capture and eliminate uncrewed aerial vehicles. The patent shows a variety of methods that these nets can be deployed, from small nets attached to other UAVs to large parachute-like nets attached to larger UAVs or small aerocrafts. The patent also discusses the use of passive capturing methods, where a net is suspended from a parachute and deployed from a UAV, using trajectory calculations an enemy UAV could be captured from above.

## Deterent for unmanned aerial systems[24]

This patent of a joint invention by three inventors describes an invention meant to fill the need for an integrated system and method of detecting, tracking, identifying and deterring the approach of unwanted UAVs. The patent further describes various systems, specifically for drone detection, classification, interdiction and countermeasures. It describes the differences between Human-in-the-loop (HIL) countermeasures and electronic countermeasures.

## EU aviation agency publishes new drone framework[25]

This opinion from the European Union Aviation Safety Administration states their opinion on the widespread use of UAVs. They believe that the use of unmanned aircraft systems beyond the visual line of sight is of danger to airlines and other uses of airspace. Hence they propose that all hobbyists should register for an official flight plan in advance. The agency further wishes to distinguish two categories for drone usage, namely the open category covers drones of a mass between 250 grams and 25kg. Their maximum permitted EASA gives operating height as 120m or 394ft. They are free to be used as long as the vehicle remains in the line of sight. The principle behind specific, or specified, drone flights is that the operator must `declare' them in advance to a regulator. At the time of writing the opinion is still pending.

## Gatwick spends 5 million pounds on anti-drone measures [26]

As a response to the drone incident in December 2018, Gatwick airport has decided to invest in the use of anti-drone measures. The airport has partnered with US airports to prepare against potential future attacks. Although the article does not show what exact instruments are to be used, it does show that the need for this technology is imminent. The article shows that anti-drone technology is now more critical than ever and companies like Gatwick airport are willing to invest a lot into current technology, as to avoid inconveniences to its passengers and the fines resulting from the 140000 stranded passengers.

# Specific problem

In this section, we elaborate on the specific problem we want to consider. Comparing the specific problem to the general problem, the specific problem considers a particular context and environment. This was done in order to limit the scope of the problem.

## Specific problem description

As described in the `Approach' Section, we expected the societal issue of unwanted UAV presence to be divisible into multiple subcategories. Following our initial literature study, we indeed found this to be the case. There are many axes along which the problem space can be divided. For example, we might consider a division based on the nature of the cause of a drone incident, and as such whether it was caused by human failure or technical failure. Another possible distinction can be given based on the specific part of society that is impacted, whether it be the privacy of individuals when a camera-equipped UAV flies over their backyard, or the safety of a group of users when there are UAVs present around an airfield. When we consider the existing legal regulations, another commonly occurring division is that between human-controlled and autonomous UAVs.

This realisation leads us to formulate a more specific problem definition with a smaller scope. In our study, we consider possible deterrents against unwanted UAV presence around airports. This includes studying the current legal regulations considering UAVs, both in general and more explicitly considering airports. The term UAV is also divisible into multiple subcategories; for this study, we take all sub-types of UAV into account. These specific sub-types will be further discussed in the following Section.

As can be observed in the image below, the number of drone-related incidents has risen dramatically over the last few years. The reason for this is that technology and its evolution moves faster than regulators, whose job is to maintain safety standard when confronted with ever-evolving aspects of technology. Regulations require extensive research into the technology it should apply to, and these regulations also take time to roll out. In the meantime, the technology in question does not stop evolving and by the time regulations take effect, the technology in question has often already evolved beyond the scope of the regulations. Different categories of drones, which will be further discussed later, pose different threats to aeroplanes. One might hit the windscreen of an aeroplane, posing a direct danger to the pilots and therefore the plane's passengers, or a drone might get sucked into the air stream entering the plane's engines and ultimately destroy a turbojet motor propelling the plane.

Airprox reports involving drones and other objects.

Because different types of drones and different types of airports exist, there are multiple types of incidents to consider. All of these are factors that should be taken into consideration when deciding which drone countermeasure should be applied in a particular situation. This means that anyone drone countermeasure is doubtful to work in a majority of situations, and care should be taken when choosing which countermeasure to invest in. A 'geo-fencing' system, which prevents commercial drones from entering certain no-fly zones, might be bypassed or disabled, not correctly implemented by the third party drone manufacturer, or include a multitude of other problems. A drone that launches a net to disable other drones might be difficult to operate and has many downtimes after it fails to take out another drone. When the safety standard is high, the ones at airports are exceptionally so; this becomes a difficult problem. It is not a simple decision to choose which drone countermeasure should be applied.

## Examples of financial consequences

It is essential to consider the financial consequences of the problems proposed in the specific problem description. Let us review a recent example that sparked the controversy regarding drone interception even more. The drone activity that obstructed flights in and out of London’s Gatwick airport for 33 hours cost airlines an estimated £50 million (\$64.5 million)[27]. This estimate of £50 million is based on EasyJet’s disclosure that it lost £15 million (\$19.3 million) in revenue and customer welfare combined during the 33 hours long illegal drone activity. Easyjet further stated that the drone incident was a wake-up call for airports. Not only Gatwick but also other airports are now plotting to try to enhance its response to any similar threats that may occur in the future. Gatwick’s flight interruptions affected about 140,000 people, where 82,000 of them were EasyJet customers.

In July 2017, Dubai International Airport was shut down temporarily due to illegal drone activity. The costs of the shutdown were roughly \$100,000 a minute according to Emirates Authority for Standardisation and Metrology (ESMA) estimates[28]. ESMA has introduced new regulatory standards for commercial and recreational use of drones, which includes a monitoring system for detecting UAVs in the country. In 2017, the Dubai Civil Aviation Authority (DCAA) head of airspace safety, Michael Rudolph, told Arabian Business that they were planning on testing their indigenously developed spectrum analysis technology to track threats of rogue drones and pinpoint their locations. With this, it would be possible to attack the rogue drones. In Dubai, it is now obligatory for all drone operators to apply for a license and undergo a training program. Since the illegal drone incidents, several new no-fly zones have been introduced by the General Civil Aviation Authority.

In January 2019, Heathrow Airport had closed its runway after a possible drone sighting. This happened three weeks after the Gatwick fiasco, in which the airport was closed for 36 hours after multiple drone sightings had occurred. Around 140,000 travellers were impacted after 1,000 flights were cancelled or diverted. No numbers regarding the costs have been given, but we can only assume that these costs were relatively high but not as high as the estimated cost of Gatwick's incident. All in all, these are just a few examples of how tremendous the financial consequences of illegal drone activity around airports can be.

## State of the Art

Again, in order to gain more insights regarding the specific problem, we add additional information to the State of the Art that was started for the general problem description. We further build upon it in this section.

## Specific USE aspects

In this section, we consider the users, society, and enterprise when considering the specific problem description.

### Users

When we take a look at the users in the more specific context of considering solutions against unwanted UAVs at and around airports, we see a shift in the types of users compared to the previously defined users in the more general setting. We see that mainly non-governmental organisations such as airports and airlines are the users of this more specific setting instead of the government, which was one of the users in our general setting. These types of users are also part of the use aspect `enterprise' and will be elaborated on in more detail in the corresponding section below. Indirectly, also passengers of flights are users of solutions against unwanted UAVs at and around airports, as passengers benefit from such solutions since they just want to be able to travel without any hindrance. Again passengers of flights is a type of users that overlaps with the use aspect 'society' and will be elaborated on in more detail in the section below. Lastly, companies with the intent/goal to intercept or detect flying objects such as UAVs are, to some extent, users of solutions against unwanted UAVs.

### Society

Let us take a closer look at how this specific problem description is relevant when we consider a society concerning airports. If a UAVs enter the air space of an airport, aeroplanes are not allowed to land at the airport nor are they allowed to leave the airport. Therefore, airports have placed a ban on the usage of UAVs around the airport in order to make sure that aeroplanes can still land and leave the airport. As one might already know, a tremendous number of people visit airports every day. In 2017, Schiphol airport, located in Amsterdam, already counted 68 515 425 passengers[29]. One can already imagine how enormous the consequences can be if this airport cannot be used for a few days. This means that if a UAVs flies by for whatever reason, a large number of people will not be able to travel. This results not only in many angry travellers but also in airport companies that have to compensate these travellers for the delays introduced by these UAVs.

From a societal perspective, this would mean that all travellers have a risk of their flight being delayed, which is undesirable due to many reasons as elaborated on previously. As of now, we have only considered the situation where a UAV simply flies by but what if this UAV has malicious intentions. For example, what if this UAV has been weaponised and is used by terrorists or a specific individual with malicious intentions and is used to wreak havoc at the airport. Then, these weaponised UAVs could be extremely dangerous as they could result in mass-killings. This would be a colossal disaster, and this should be avoided at all cost. A disaster is not only bound to happen when we consider weaponised UAVs. A disaster could also occur when the systems of the airport do not detect one of these UAVs. This UAV could then end up damaging the aeroplane, which could result in perilous situations. For example, we can expect disastrous situations when a UAV gets stuck in the motor of an aeroplane. All things considered, UAVs cannot only be hazardous to society when operated by malicious attackers, but they can also introduce many annoyances.

### Enterprise

When we restrict the USE case analysis to only deterrents against unwanted UAVs or drones around airports, the enterprise aspect of the use analysis becomes more concrete. The main type of enterprise that is under risk is the airport branch. The total revenue of the aviation industry in 2018 alone is a staggering 821 billion USD[30], so there are huge amounts of money at risk here. The current protocol is to suspend all flights of the airport by 30 minutes, the average lifespan of a drone. This means that, should drones occur often enough, all flights will be suspended for an indefinite amount of time[31]. This will, of course, have huge costs for multiple branches of the aviation industry. The three most notable branches in our opinion are the airports, the airlines and the companies who use aviation to transport goods. For these three enterprise branches, we will analyse the consequences of such an unwanted drone near an airport.

#### Airports

Airports suffer the largest loss in the case of such a drone in the airspace, which makes sense since it is where the problem is located. The airports suffer huge financial losses mainly through three different ways. First and foremost, no profits can be made when no planes fly. For example, the drone incident at Gatwick Airport caused a loss of over 50 million English Pounds[32]. This was caused by suspending all flights, which were just over 400, over the total duration of 33 hours. The airport suffers a huge blow to their reputation. So the hourly financial losses are huge for airports when flights must be suspended due to drones in the area. Furthermore, should one airport consistently suffer from the presence of unwanted drones, both airlines and travellers might opt to choose a different airport. The other airport might be farther away, but it will be a more reliable airport. This would result in a drop in total passengers at the airport.

#### Airlines

Airlines such as RyanAir, KLM or EasyJet also suffer huge losses during the event of a drone suspending or cancelling flights. The airlines are the companies actually offering flights to travellers. What also causes more losses for airlines is that they have to compensate the travellers for the delay or cancellation of their flights. By European Law, airlines are required to provide travellers with enough food, drinks, and nightly accommodations for as long as necessary[33]. For a long, sudden suspension of flights at one place, which is the case in our problem, this also becomes an enormously difficult task for an airline. If the airline would not act accordingly, the airline could also suffer from a huge reputation loss, resulting in travellers not flying with that airline anymore. Furthermore, the travellers are also eligible for financial compensation by European Law[33]. Airports.

Another aspect of this branch are the employees of the airlines. Apart from the company as a whole, the employees, such as flight attendants and pilots, suffer significantly from such an airline 'shutdown'. This is because of the way that the employees get paid. They do get a baseline salary, but the most salary they receive come from the hours that they are actually in the plane either flying or aiding passengers[34]. If the planes do not fly, they suffer from a huge salary cut, which means that the whole branch of airline employees have financial losses.

#### Companies who transport goods via aviation

Another enterprise that suffers from delaying and cancelling of flights at an airport are transport companies. In general, these goods are transported with different aeroplanes than passenger aeroplanes, but cargo is usually transported with passengers in the same aeroplane [35]. Besides, cargo aeroplanes also frequently fly to airports just for packages, since restaurants, shops, e.g. are not necessary there, decreasing airport costs. However, these airports can also be subjected to an unwanted drone in the airspace. Again, the protocol is that the aeroplane will not land and thus will be either delayed or cancelled. This can have dire consequences for such companies. The delay of their goods usually set off a chain reaction of consequent delays, which can be devastating if the timing is crucial. In conclusion, the consequence of these delays for these companies is huge financial losses and huge logistic issues to fix the delays of their goods.

# Present situation

In this section, we consider the present situation regarding the specific problem description. We interview an airport and look at current solutions.

## Airport Interview

In order to get a more unobstructed view of the issues our users (airports) face today we decided to ask them a couple of questions. We want to obtain a clear picture of their current approach to airport security regarding drones, what the consequences would be if a drone were to fly in their airspace right now, and what the consequences were of the 19th of December Gatwick incident. We will then ask them what their requirements would be for a drone defence mechanism.

• What is the airport's current mechanism for detecting drones?
• How will the airport respond when the drone is sighted in restricted aerospace?
• Roughly how much damage will the airport take if a drone were to restrict air traffic for 1 hour?
• The 19th of December and 21st of December drone attack at Gatwick airport caused over 1000 flights to be affected, did your airport get affected by the knock-on effects?
• What would be the maximum budget for an automated anti-drone mechanism?
• What kind of system would you imagine when thinking of anti-drone mechanisms?

We contacted most major Dutch airfields; Eindhoven, Schiphol, Maastricht Aaken, Groningen, Twente, Den Helder, Rotterdam the Hague and Bergen op Zoom.

Eindhoven airport responded to the questions, firstly stating that Eindhoven airport uses the runway and infrastructure provided by the Military airbase Eindhoven. This means that the Dutch Royal Airforce is responsible for air traffic control and hence the safety in the airport's airspace. We had the following answers to the aforementioned questions:

What is the airport's current mechanism for detecting drones?

At the moment the airport has no automated system to detect drones. At the moment this done by sight from the air traffic control tower.

How will the airport respond when the drone is sighted in restricted aerospace?

This depends on the location of the drone. At the moment an incident affecting air traffic has not yet occurred. When a drone is spotted, we will suspend all traffic.

Roughly how much damage will the airport take if a drone were to restrict air traffic for 1 hour?

I cannot answer this question [in detail], for the military activities, the impact will be limited. However, the impact on Eindhoven Airport will be much more significant.

The 19th of December and 21st of December drone attack at Gatwick airport caused over 1000 flights to be affected, did your airport get affected by the knock-on effects?

We were not affected as there are no flights to Gatwick from Eindhoven.

What would be the maximum budget for an automated anti-drone mechanism?

None, for safety there will always be a budget available.

What kind of system would you imagine when thinking of anti-drone mechanisms?

The location, altitude and flight-profile are crucial. The weight of a drone is also very important.

The correspondent also told us he was very interested in our research, offering the opportunity for further collaboration.

## Solutions

In this section, we will take a look at solutions against unwanted UAVs at and around airports that are currently/in the near future being used by airports/authorities. These solutions might exclude many solutions that might be useful but are simply not in use due to for example the jurisdiction not being up to date with the current technology. However, a list of all possible solutions including solutions that might not even be feasible right now, but maybe within the next few years will be discussed in the section solutions. It is important to note that there are different rules for different types of drones.

• There will be European rules and regulations in the near future, expected around June 2019, obligating operators wanting to fly with a drone that is heavier than 250 gram to be registered. Drones will be obligated to send out identification signals such that authorities, for example, the police, can trace and identify the operator of the drone[36].
• With these same rules and regulations drones will be obliged to be equipped with geofencing software. This will restrict the operator to be able to fly close to an airport[36].
• Anti-drone systems deployed at two London airports are capable of tracking the devices from as far as six miles away. As well as being able to sever communications with the operator, some models can also destroy the drones using a laser beam. However, it is not exactly been released to the public as to what equipment is used and how it works[37].
• The police trains eagles to make them consider unwanted UAVs as preys, such that they would catch the UAVs and place them in a safe area. However, the Dutch police have already stopped using this solution because training the eagles is more expensive and complicated than they anticipated[38].
• In May of 2018, London Southend Airport successfully tested an anti-drone system that combines optical sensor and radio frequency to detect drones[39].
• The US Federal Aviation Authority trialled the Anti-UAV Defense System (Auds) system in 2016. It uses high powered radio waves to disable drones, it blocks their communication with the controller and switches them off mid-air[39].

## Limitations

### The jurisdiction regarding drones is not up to date with current technology

As is often the case, the laws we have are not able to keep up with the tremendous advancements of technology [40]. This has happened many times already in history, for example with the rise of copyright laws at the end of the 19th century. Due to the huge advancements in copying and spreading literature, originals authors lost lots of money to people selling the author's work without proper permission. This was facilitated due to the rise in printing technologies. Under the pressure of this growing technology, the copyright laws had been created, albeit years and years later after the problem had occurred [41]. This example is just one of the many examples where the laws come much too late after the technology has been fully developed.

The same problem is currently happening to drone regulations. Over the last decade, the technological advancements in drones have been enormous, and as a consequence, the accessibility of drones for normal people has increased as well. Nowadays, anyone can buy a drone without any license and fly the drone with a camera to any house in his or her neighbourhood for under 100€ [42]. This seems like an obvious illegal intrusion of privacy by laws such as personality rights ("portretrecht"). However, these rules are not properly enforced concerning drones. In Europe, new drone regulations will be enforced, starting halfway through the year [43]. However, there have been huge debates about how the regulations should be changed, with no concrete answers. Just recently, on January 21 2019, the Dutch House of Representatives ("Tweede Kamer") organised a "rondetafelgesprek", where experts discussed what should be done in terms of regulations[44]. These examples show that the regulations of drones are not up to date with the current technological advances of drones.

### Limitation of current solutions

As we have described before, current solutions such as the eagle experiment, are simply not good enough to efficiently provide a solution to the problem. For this exact reason, airports and governments all over the world are investing vast amounts of money in the development of technologies to counter drones. Heathrow and Gatwick airport are two examples of airports that are investing millions of dollars in this technology [37].

Apart from the fact that some solutions simply do not work, other proposed solutions have negative side results. For example, shutting the unwanted UAVs down with radiowaves means that they will crash straight down to the ground. If such a drone falls on someone's head, he or she could get seriously injured. Furthermore, the crashing drone can also break specific equipment when falling. Lastly, if the drone, e.g. falls and breaks on the runway, this could also be dangerous. These consequences also apply to the current solution where the drones are shot down with a laser for example.

Other solutions such as geofencing and identification signals also have the flaw that they can be bypassed easily. If someone intentionally wants to fly a drone to the airport, it is not that difficult to make sure that the drone does not broadcast identification signals anymore. The drone operator could also make sure that the drone does not send signals that the geofencing uses, such that the geofence is, in fact, useless for deterring this drone. Furthermore, someone could also build a drone themselves, and choose not to send these required signals. This would indeed be against the law shortly, but since the drone operator is already engaged in criminal activities, these regulations would most likely not stop him. Thus, the technologies can easily be bypassed, rendering them as useless.

# Drone analysis

## Introduction

There exist various categories of drones. For example, not only small drones for recreational use exist but also large weaponised drones. It is important to categorise the various types of drones that exist in order to deal with each category appropriately. Furthermore, the current rules and regulations when it comes to commercial and recreational use of drones are essential to consider as well. We do this in order to get a better understanding of what is allowed and what is not allowed. This better understanding will make it easier to provide solutions for the various categories of drones and their usage.

## Current regulation

When considering the current rules and regulations, we do not merely restrict ourselves to the rules and regulations surrounding airports, but we increase the width of our view to a more general perspective as we think we might do interesting findings this way. Different countries have different rules and regulations when it comes to UAVs. Furthermore, distinctions are made between recreational use and commercial use. The United States of America (U.S.A.), for example, considers different rules when it comes to recreational use and commercial use. The requirements when flying a drone under commercial use are much stricter than flying a drone under recreational use. If one wants to fly under commercial use, one has to pass an FAA test and receive Part 107[45] certification. Furthermore, a drone needs to be registered so that the owner of the drone can be traced back in case this is needed.

A few guidelines to follow when flying a drone in the U.S.A are as follows[46]:

• Fly at or below 400 feet
• Keep your drone within sight
• Never fly near other aircraft, especially near airports
• Never fly over groups of people
• Never fly over stadiums or sports events
• Never fly near emergency response efforts such as fires
• Never fly under the influence
• Be aware of airspace requirements

There exist applications, available for smartphones and on the web, that display where a drone is allowed to fly. One example is AirMap that shows users that they should be at least five miles away from an airport to operate the drone without notifying the control tower of the airport. As you might have realised by now, the rules and regulations regarding drones are still a work in progress. As the rules and regulations per country differ significantly, we will solely focus on the rules and regulations considered in the Netherlands. This is only natural as the project is carried out in the Netherlands as well. The Netherlands considers different rules and regulations based on the type of usage of the drone. The main categories specified by the Dutch Government consider recreational use and commercial use.

### Recreational use

When one flies a drone for personal purposes, one must abide by the Model Aeroplanes Regulations[47]. This means that one is not permitted to fly over groups of people or connected buildings. Furthermore, the drone needs to be in sight at all times. As soon as one sees an aeroplane or helicopter approaching, one must land as quickly as possible.

For reasons of safety, it is not allowed to fly a drone just anywhere. As mentioned earlier, it is not allowed to fly over groups of people. The Dutch Government has also set down requirements regarding the conditions under which it is allowed to fly[48]. This includes but is not limited to:

• You must be able to see the drone at all times.
• You may not fly in the dark.
• You must always give priority to all other aircraft, such as aeroplanes, helicopters, gliders, et cetera. This means that you must land immediately once you see an aircraft approaching.

An overview map for the recreational use of drones has been depicted in the figure below. This image accurately presents where one is allowed to fly their drone for recreational use and where it is forbidden to fly a drone. An interesting observation that can be done from this image is that in many significantly sized cities, it is forbidden to fly a drone at all. Furthermore, for uncontrolled airports, flights within a distance of 3 kilometres are permitted, provided that there is no objection from the airport operator. Additionally, there may always be temporary bans and restricted areas for a limited time due to, for example, events.

Overview map for recreational flying with drones. Note that this map will change in March 2019.

There also exists a maximum weight for private drones of 25 kilograms. Making films and photographs with a drone may only be done for personal use. Here, the privacy right of others must be kept in mind. It is, for example, not allowed to film someone secretly. If pictures are being taken of a specific person or that person is being recorded, the person concerned must be informed. This leads us to the following point. The owner of a drone is responsible for any damage caused by their drone. This means that the owner of a drone is liable for any damages or injuries caused by their drone. Therefore, it is vital for the owner of a drone to verify whether their liability insurance covers any damage to drone incidents. In some cases, it is possible for the damage to run up to a substantial sum up to thousands of euros. We can further extend this by considering fines that can be given to drone pilots. Failing to abide by the rules mentioned above can result in either a warning or a fine. It is also possible for the controlled drone to be confiscated. The amount of the fine or the punishment is given depends a lot on the type of violation caused by drone usage. It will be considered if the drone was used in a professional setting or for hobby purposes. Furthermore, it will be considered if people were endangered or not.

The Dutch Government provides a summary in a visual form of what guidelines to follow during recreational usage of drones[49]. This visual can be observed in the figure below. This figure accurately presents the most important rules to follow when using drones in a recreational setting. Note that the text on this figure is in Dutch.

A summary considering recreational usage of drones in Dutch.

### Commercial use

On the other hand, we can also consider the commercial use of drones. Examples include but are not limited to people that use the drone to earn money or people that use drones for business purposes. For these commercial users, different rules and regulations apply than for recreational users. A commercial user needs, for example, a license. The additional rules and regulations for commercial users must minimise the risk of accidents, both in the air and on the ground.

Examples of commercial uses include:

• Video production companies that make aerial shots.
• Making promotional films for a company.
• Using a drone for a business, such as companies that want to view hard-to-reach places for specific reasons.

For using a drone in a commercial setting, the owner of this drone needs an RPAS Operator Certificate (ROC). One can be requested from the `Inspectie Leefomgeving en Transport' (ILT). If someone is piloting the drone, then this person also needs a pilot's license (vliegbrevet in Dutch). Furthermore, a certificate of airworthiness and proof of enrollment in the aviation register is needed[50]. There exist two sorts of ROC licenses, namely a regular ROC and a ROC Light. If a drone is heavier than 4 kilograms, then a ROC is needed. Otherwise, a ROC-light will be fine in most cases. Additionally, it is not allowed to fly as high with a ROC Light compared to a regular ROC.

Other differences are displayed in Table 2 below. Here, one can more clearly observe the differences between a ROC and ROC Light. We will not display all difference here as we save this for the next section where we also compare the commercial use to the recreational use.

Table 2: Differences ROC and ROC-light
Rules license Drone heavier than 4 kilograms Drone lighter than 4 kilograms
Maximal weight of drone allowed 150 kg 40 kg
Maximal flight height 120 metres 50 metres
Maximal distance between drone and owner 500 metres 100 metres
Minimal distance towards crowds 150 metres 50 metres
Minimal distance to buildings 150 metres 50 metres
Minimal distance to highways 150 metres 150 metres

If one does not abide by the rules, it is possible to obtain a fine and for the drone to be confiscated. People who do wrong more often can also get a prison sentence. The National Coordinator for Counterterrorism and Security (NCTV) focuses on the abuse of drones. The NCTV cooperates with national and international government organisations. Given fines can be around +/- 400 euros for commercial usage with a ROC Light and +/- 10 000 euros for commercial usage with a ROC.

### Summary

In this section, we provide a summary when considering recreational, commercial (ROC), and commercial (ROC Light) usage of drones. These rules and guidelines are from the most up-to-date version provided by the Dutch Government[51] (20-09-2016).

Table 3: Comparison of recreational and commercial (ROC and ROC Light) drone usage
Reacreational flying Commercial flying (ROC) Commercial flying (ROC Light)
Use of a drone Hobbyism, recreational use Commercial use Commercial use
Weight drone (total starting mass) Max. 25 kg Max. 150 kg Max. 4 kg
Priority for other air traffic Gives priority to all other air traffic and lands immediately when other traffic is approaching. Gives priority to all other air traffic and lands immediately when other traffic is approaching. Gives priority to all other air traffic and lands immediately when other traffic is approaching.
Visual Flight Rules Always in sight of the pilot Always in sight of the pilot Always in sight of the pilot
Distance to pilot or observer N/A Max. 500 metres Max. 100 metres
Daylight Only daylight Only daylight Only daylight
Height (from ground/water) Max. 120 metres. Some exceptions (KNVvL or FLRVC members): max. 300 metres Max. 120 meters (exemption possible in ROC) Max. 50 metres
Distance criteria: Exemption possible Exemption impossible
Distance to crowds Not above Min. 150 metres Min. 50 metres
Distance to buildings Not above Min. 150 metres Min. 50 metres
Distance to works of art, port, and industrial areas Not above Min. 50 metres Min. 50 metres
Distance to railway lines Not above Min. 50 metres Min. 50 metres
Distance to public roads and motorways Not above except for roads in 30 km zones within the built-up area and roads in 60 km areas outside the built-up area Min. 50 metres Min. 50 metres
Distance to vessels and vehicles N/A Min. 150 metres Min. 50 metres
Where are you allowed to fly? Not in controlled airspace Not in controlled airspace Not in controlled airspace
Not within 3 km of uncontrolled airports, unless there is no objection from the operator N/A Not within 3 km of uncontrolled airports, unless there is no objection from the operator
Not in the military and civilian low-flying areas, unless with an observer N/A Not in the military and civilian low-flying areas, unless with an observer
Proof of Authority for the pilot / driver ('brevet') N/A Certificate of Competence (RPA-L)(medical examination compulsory, at least LAP-L) Exemption Certificate of Competence Well: pilot can demonstrate sufficient competence, e.g., with a KEI diploma or a recognized pilot's license(this requirement does not apply if the drone weighs less than 1 kg) no medical examination
Certificate of Airworthiness for the drone N/A Certificate of Airworthiness (technical inspection required) Exemption Certificate of Airworthiness (no technical inspection)
Registration in aircraft register N/A Proof of registration Proof of registration
Minimum age N/A 18 years old 18 years old
Operational manual N/A Handbook necessary N/A
Insurance Not required WA insurance required WA insurance required
Notification obligation N/A 24 hours before the flight with Minister and mayor NOTAM N/A
Fines N/A +/- 10 000 euro +/- 400 euro

## Future regulation

Before reading the next text, keep in mind that most of the things elaborated on below hold as of now, but that they can change in the future when the final rules and regulations regarding drones are published. The introduction of mandatory knowledge requirements for recreational drone users will no longer take place at a national level. The primary regulation for EASA came into effect on the 11th of September, 2018[52]. This means that the responsibility for civilian drone regulations is transferred from the Netherlands to the European Union once the European regulations regarding drones come into effect. The Netherlands is, however, still responsible for the national implementation and execution of these regulations. It is expected that this regulation will be published in the June 2019[53] and will take effect early 2020[54]. Furthermore, there will most likely be a transition phase of roughly two years where national documents (licenses, et cetera) can still be used. The European drone regulations will replace the national regulations regarding airworthiness of drones, pilots of drones, and flight operations concerning drones. The moment these new regulations are active, the air traffic regulations for civilian drones laid down in the `Regeling Modelvliegen' and `Regeling op afstand bestuurde luchtvaartuigen' will be withdrawn.

The European rules are based on the risk that flying with a drone entails. There will be no more distinction between professional and recreational flying. The regulations will focus on different categories based on the risk involved for third parties both in the air and on the grond[52]. These categories consist of:

• the open category for low risk,
• the specific category for higher risk, and
• the certified category for the riskiest operations.

The operational conditions under which the flight is carried out, such as the flight altitude, determine the category in which a flight falls.

Note that there is a summary section below that summarises the different types of operations together with the different types of drones in a very concise and understandable way. One should go to these if one does not want to go into all the details regarding the types of operations and the types of drones.

### The EASA

While the preceding description provides a rundown of what to expect, it is possible to consider subcategories within the open category. It was decided by the European Aviation Safety Agency (EASA) to further partition operations in the open category into three subcategories to allow different types of operations without the need for authorisation[55]. This subdivision was made in Opinion No 01/2018 made by the EASA. The objective of this Opinion, as described in Opinion No 01/2018[55], is to create a new supervisory framework that defines means to alleviate the risk of operations in the:

• open category; through a mixture of limitations, operational rules, and requirements for the competency of the operator, as well as technical requirements for UAVs, such that the UAV operator may conduct the operation without prior authorisation by the competent authority, or without submitting a declaration, and
• specific category; through a system that includes a risk assessment being conducted by the UAVs operator before starting an operation, or an operator complying with a standard scenario, or an operator holding a certificate with privileges.

Additionally, the Opinion provided by the EASA intends to:

• implement an operation-centric, proportionate, risk- and performance-based regulatory structure for all UAV operations conducted in the open and specific categories,
• ensure a vast and uniform level of safety for UAV operations,
• foster the development of the UAV market, and
• contribute to addressing citizens’ concerns concerning security, privacy, data protection, and environmental protection.

The proposed Opinion will harmonise operations regulations in Europe and create a general EU market for drones. It will allow everyone to buy and operate a drone ensuring the following attributes:

• safety; by keeping drones away from crewed aircraft, protecting people and critical and sensitive infrastructure,
• security; by keeping drones at an appropriate distance from nuclear reactors; military bases or oil pipelines,
• privacy; using a proper separation from residential areas;
• environmental protection, by reducing the noise level.

### Consumer information

One of the novelties EASA has is the combination of product and aviation legislation in these new rules. In particular, design requirements for small drones (up to 25kg) will be implemented by using the well-known Conformité Européenne (CE) marking for products brought on the market in Europe. All European drones are assigned a CE-Marking. This marking consists of a number between 0 and 4, which specifies the class of the drone. We can then consider the following classes: C0, C1, C2, C3, and C4. Operators of drones will then find in each drone package a digital consumer information with the “do’s and don’ts” related to each class on how to fly a drone safely. These “do’s and don’ts” can be seen here. We provide an excellent overview of these classes together with the subcategories of the drones in the open category below.

### The open category

One of the conditions for the open category concerns a maximum altitude of 120 meters. For all drones more massive than 250 grams, requirements concerning the knowledge of the pilot are imposed on the pilot. Furthermore, product requirements apply to drones this category, such that it is reliable enough and has the right functionality, such as geofencing, on board. The open category probably includes recreational or professionally controlled drones with a mass of heaver than 250 grams and lighter than 4 kg. The operators of these drones will probably need a 'theory certificate'[54]. This certificate is comparable with the current Dutch theory certificate needed for mini-drones. It is expected that the (unqualified) Dutch mini-drone theory certificate can be converted into a European document or considered as valid without conversion. This also applies to those who fly a microdrone (max. 1 kg) and have an exemption for this (a ROC light permit). These operators still have more than two years to achieve that theory certificate. This certificate is only necessary if one wants to fly outside a model airfield.

In the Opinion mentioned above, which has been designed by the EASA, it was decided to subdivide operations in the open category into three subcategories further to allow different types of operations without the need for authorisation. The subcategories were defined according to the risks posed to persons and objects on the ground. Furthermore, these operations are all below 120m in height and far from aerodromes. The subcategories of the open category are:

• A1: flights over people but not over open-air assemblies of persons;
• A2: flights close to people, while keeping a safe distance from them;
• A3: flights far from people.

An overview of the requirements put on these subcategories together with the CE markings can be seen below in the figure below.

An overview of the CE markings and the subcategories within the open category.

### The specific category

The specific category has no restrictions in advance such as a maximum height or not being allowed to fly over people. In the specific category, professionally controlled drones are considered that have a risk that is greater than that in the open category. This can, for example, be the flights for which a ROC is now required, which is the case when a drone is heavier than 4 kg[54]. If the operation carried out does not meet the requirements that apply to the open category, it will, in most cases, fall into the specific category. This holds unless there is a risky operation requiring a certificate (see below). For an operation in the specific category, the operator has to carry out risk analysis. Based on the results of this risk analysis, the operator must propose mitigating measures to keep the risk of the operation low enough. This analysis is then handed over to `Inspectie Leefomgeving en transport' (ILT) which, after testing the analysis, can grant a license to the operator. Because there are no restrictions in the regulations in advance, this category offers considerably more opportunities for operators than the current national regulations. Flying above buildings, in the dark, out of sight of the pilot and even flying with an autonomous drone belongs in this category to the possibilities.

### The certified category

In the current proposals for European legislation (1 Nov. 2018), an operation falls into the certified category if the UAV is more significant than 3 meters and is flown over a crowd, when people are being transported by the UAV, and if hazardous substances are transported that can create dangerous situation in the event of a collision. We provide a concrete list below. Besides, if the risk analysis in the specific category shows that the risk of the operation is too high for the operation to be classified as the specific category, it is classified as the certified category.

Drones fall within the certified category if the risk of the intended flight is probably only to be controlled by setting strict requirements (through certification) on both the operator and the drone. This is the case, for example, with:

• flights with drone
• with which people are transported, or
• dangerous goods are transported where a high risk may arise for third parties in the event of a crash, or
• flights with large or complex drones:
• almost constantly above crowds,
• outside view distance, or
• in a part of the airspace where much other air traffic is present.

### What does this mean for us?

Drs. C. van Nieuwenhuizen Wijbenga, `Minister van Infrastructuur en Waterstaat', stated that she is currently working on a plan for the national implementation of the European drones rules[53]. She further elaborates that she can finalise this plan and make it public once the rules are fixed at EU level. At that time, she will also organise a large information meeting to inform all national stakeholders in the drones dossier on the European rules and the national implementation plan. Her current estimate is that this meeting can take place in the first quarter of 2019. One of the components of this plan she proposes is the implementation at a national level of zones in which drones are not allowed to fly or are restricted. Before the establishment of these zones, the various stakeholders such as municipalities and the drones sector will discuss the criteria for zoning. The plan itself takes into account the transitional periods that will be included in the European regulations.

### Summary

In case one does not want to read all of the above text, we provide a concise summary here that considers all of the major elements that should be kept in mind when it comes to different types of operations and drones in the close future.

When it comes to drones use, we will (most likely) consider the following categories of operations:

An overview of the different categories of operations.

In-depth, these categories are as follows:

An in-depth overview of the different categories of operations.

When it comes to the open category, the various subcategories within the open category, and the various C markings, we consider the following:

An in-depth overview of the open category.

The figure above accurately depicts what a typical customer that is wanting to buy a commercial drone is interested in.

## Dangers

Let us consider a few of the dangers that various types of drones can impose on airports. We limit ourselves to two broad types of dangers that drones can introduce near airports. When we consider different types of airports on the Airports page, we take a deeper dive into the most considerable dangers that are introduced for each specific type of airport.

### Collisions with aeroplanes

It is possible for drones to collide with aeroplanes. Of course, smaller drones are more likely to inflict less damage upon the aeroplane than larger drones. In this section, we consider the possible types of damages drones, in general, can inflict upon aeroplanes as going in-depth about which drones can inflict what kind of damage falls outside of the scope of this project. The previous figure provides a visualisation of the types of damage that can be inflicted upon aeroplanes by all sorts of drones.

Example of potential events following from a collision of a drone with an aeroplane. Retrieved from https://publicapps.caa.co.uk/docs/33/CAP1627_Jan2018.pdf.

If a collision between a drone and an aeroplane takes place, the damage inflicted upon the aeroplane can differ. There is a high probability that the drone in question will be destroyed. Then, it is possible for this drone to become unable to operate, which in turn, can result in that drone falling. Then, it is possible that someone or something gets hit by this drone, which could lead to the death of people and the destruction of things hit. Furthermore, a collision between an aeroplane and a drone can cause no damage at all, superficial damage, or severe damage to the aeroplane in question.

Below, one can observe a demonstration of a drone striking the nose cone of an airliner from the Crashworthiness for Aerospace Structure and Hybrids (CRASH) Lab at Virginia Tech. Note that the aeroplane is standing still in this scenario and that real collisions are bound to take place at higher speeds, which often results in more power being involved in the equation.

A demonstration of a drone striking the nose cone of an airliner from the Crashworthiness for Aerospace Structure and 3Hybrids (CRASH) Lab at Virginia Tech.

Let us ignore when no damage or superficial damage is inflicted as these types of damage do not put the aeroplane at significant risk. If, however, severe damage is inflicted upon the aeroplane, it is possible that component failures take place or that persons on board are injured. Component failure and injuries of people on board can lead to an unsafe landing. Additionally, if the pilots of the aeroplane get injured, it is possible for loss of control and an eventual crash of the aeroplane. Note that this is the worst case scenario, but that it is vital to consider these worst cases. In turn, an unsafe landing or loss of control can lead to more injuries of persons on board due to people walking over one another trying to escape the aeroplane or due to the crash.

All in all, we observe that drones can cause disastrous consequences when it comes to collisions with aeroplanes.

## Armed drones

An example of a weaponised drone.

It is possible that armed drones are used for planned attacks on airports. While this is a worst-case scenario to happen, it should still be considered, and measures should be taken against this scenario. These type of drones could be equipped with long-range weapons such that airport detection systems cannot detect them or with explosives. In the case that drones are equipped with explosives, it is possible for the drone to `simply' infiltrate the airport and explode, which can put the lives of many people at risk. On the other hand, when long-range weapons are used, it is possible that the airport detection systems do not detect these drones. Then, these drones could freely attack the aeroplanes and also the airport. This, also, puts the lives of many people at risk. One example of a weaponised drone can be seen on the right-hand side.

# Solution analysis

In this section, we consider the requirements of solutions for the problem proposed in the specific problem description, all possible solutions, and both the advantages and disadvantages of each solution.

## Categories

When considering the state of the art research presented in the relevant Section, we can distinguish multiple categories in which the presented solutions might fall. In this Section, we further elaborate on these different categories, and as such provide a better overview and allow for more a more specific formulation of requirements. Firstly, different anti-UAV systems serve different purposes. For our study, we differentiate between the following purposes:

### Purposes

UAV Detection
These systems serve to detect the presence of UAVs in unwanted airspaces. They often also locate the UAV in question and sometimes include the possibility of continuous location tracking to assist systems categorised under the other purposes.
UAV Identification
Systems from this category serve to identify UAVs, obtaining more information about the UAV than simply its location. This information might include simple statistics, such as the average size of the drone which can often be observed by a human, given that the UAV is present in their field of view. More complicated statistics might also be obtained, such as a serial tracking number to identify commercial UAVs.
UAV Neutralisation
Drone neutralisation systems serve to neutralise a drone. This is the main topic of our study since UAV presence in the airspace above an airport introduces various risks, discussed in other Sections, that have to be neutralised in order to maintain public and societal security.

Now that the scope of the purpose of the anti UAV systems for airport security that we consider has become clear, we might further distinguish the main purpose considered in this study. As such, we differentiate between 3 different subcategories, all part of the drone neutralisation purpose. These categories are as follows:

### Categories

Preventative solutions
This category encompasses all solutions that serve to prevent the problem from occurring. More specifically, entries of this category focus on keeping UAVs away from airspace belonging to airports. An example might include the geofencing system that was described previously and will be elaborated on further in the following sections.
Corrective solutions
Solutions from this category focus on solving the problem of UAV presence in the airspace over airports, especially when said UAV is already present in that airspace. These solutions attempt to do so with minimal damage to the parties involved, an example might consist of a procedure where the control of the drone is overridden, either automatically or by a human, before the drone is removed from the airspace by landing or flight and after which control could be passed back to the pilot.
Destructive solutions
These solutions have the same area of focus as the previous category of corrective solutions, namely the minimising of further risk to air traffic above airports after a UAV has entered the airspace. The main difference is that, while corrective solutions attempt to do so in a non-destructive way, this limitation does not apply to destructive solutions. Sub-systems of a UAV or the UAV as a whole may be destroyed or permanently disabled. A coarse example consists of taking down unwanted UAVs with firearms, causing damage to the UAV and rendering it unable to continue operations.

This division into categories is not entirely black on white, however. Consider an abstract example system that temporarily incapacitates a UAV in flight, causing it to cease operation and enter a free fall towards the ground. This might result in the destruction of the drone, given the collision with the ground. We have found a grey area in our division into subcategories, and as such, we further define destructive solutions as those solutions, where the incapacitation of the drone follows from the destruction, and not the other way around. We also require the destruction to be an integral part of the solution, if we want it to count as a destructive solution. In this example, the destruction is not guaranteed nor does the incapacitation follow from the destruction. Instead, the destruction might follow from the incapacitation, dependent on other circumstances. Therefore, this specific example counts as a preventative or corrective solution, based on where the UAV in question is located. Note, however, that this is based on the keywords `temporarily incapacitate'. If the incapacitation of the UAV or one of its subsystems were permanent, the destruction would be guaranteed since it does not depend on how hard the UAV hits the ground anymore. In this case, it would count as a destructive solution.

## Requirements

A solution to the specific problem described will have to adhere to requirements. These requirements are not simply capabilities the solution has to provide in the form of functional requirements, but they should also cover constraints posed on the solution. The constraints can be on the design of the solution in order to meet specified levels of quality, on the environment and technology of the system, and the project plan and development methods. Note that there can exist multiple different types of solutions and that we, therefore, have to keep the requirements of a solution as abstract as possible. We should not limit the solution space with these requirements. Instead, we should provide a general outline of what capabilities (functional requirements) the solution should provide and under what constraints (non-functional requirements).

Furthermore, these requirements might serve as a basic framework for further development of solutions to similar problems, thereby widening the scope to other problem spaces involving UAVs as well.

In this case, it is rather simple to provide some basic requirements and let the input of the `airports' decide on further requirements.

For detection, the solution should be able to detect UAVs. For identification, the solution should be able to identify a detected UAV. For Neutralisation, the solution should be able to neutralise the detected and identified UAV. Of course, there are many other requirements, but we let the airport place these requirements as certain airports might argue that the safety of bystanders is more critical than other airports, such as recreational airfields.

## Possible solutions

As we have already elaborated on, a possible solution can be categorised into the purpose it fulfils with respect to anti-UAV systems at and around airports. Since a full anti-UAV system should be able to do three things: detect flying objects, identify that this object is an (unwanted) UAV, and lastly neutralisation of the UAV. However, the identification of the object might be something that is up for discussion, since it might be safer to neutralise every flying object, we will discuss this later on. As most possible (partial) solutions only cover one or two of the three things it should be able to do, before it can be considered at a full anti-UAV system, for each of the (partial) solutions listed below, they are divided up into categories of its purposes it fulfils. Such that, later on, we can compare and afterwards combine multiple of these partial solutions into one system that meets the needs of the users.

#### UAV Detection

• Radar system for detecting the location and height of an object in the air. The radar makes use of a transmitter which produces an electromagnetic signal which is radiated into airspace with an antenna. If this signal hits an areal object, it will get reflected in many directions. This reflected signal is received by the radar antenna then it is processed to determine the geographical data of the object.[2]
• A Wi-Fi receiver can be used to detect a UAV based on the signature of the signal reflected from the propellers of a UAV. Similar to radar, a transmitter broadcasts signals and a receiver captures reflected signals that bounce of a UAV. [3]
• Detect a UAV by listening to the communication channel between the UAV and its controller using a wireless receiver. Usually, UAVs communicate with their controllers a few times per second to update their status and to receive commands from the controller. A system could collect wireless samples and observes the signal, analyse them and can then detect a UAV's presence. [3]
• Detection of UAVs with the use of other UAVs that fly around the airports, carrying lightweight radar systems or cameras to scan their environment.
• Echodyne's 3D Security radar that offers superior sensor performance in a compact, solid-state, all-weather product. A recent winner in the SOFWERX Game of Drones competition.
• Human detection, for example by using watchtowers or pilots in the aeroplanes to spot UAVs. (Currently what Eindhoven Airport uses to detect UAVs)
• 3D Radio frequency antenna (https://drone-detection-system.com/the-system/)

#### UAV Identification

• Identification of any specific aircraft can be made by broadcasting a coded signal, which is decoded by air traffic control towers. Such that allies and enemies can be identified and to avoid targeting a friendly aircraft. As a result, all aircraft where radar service is provided should require systems that are able to broadcast coded signals for identification, for this solution to work. [2]
• For identification of UAVs, employing a 2D antenna and appropriate signal processing to create a multibeam, 3D, wide area overcomes the weakness of scanning radars and achieves high detection sensitivity. A decision tree based classifier can be used to identify the difference between UAVs and other moving objects. Where it rejects non-UAV targets, decreasing the number of false positives and increases true positives. Such that when neutralising such a moving object in the air, with high probability, it will be a drone instead of for example a flying bird. [11]
• A lightweight, X-Band (10.5GHz) radar system for use on a small-scale (less than 25 kg) rotorcraft. The prototype implementation of the radar is small enough to be carried by a drone and is able to differentiate other 'miniature rotorcrafts' (drones) by their doppler signature. The prototype uses a radar system which utilises electromagnetic energy to gain information on objects by analysing the reflected energy. [12]

#### UAV Neutralization

• Taking out UAVs by using air to air missiles, where these air missiles could be launched from other UAVs used by the airport or possibly any other aerial vehicle.[2]
• Taking out UAVs or disabling specific subsystems might be achievable by using lasers. Different kinds of lasers can be used for different purposes, either permanently or temporarily disabling a UAV. [2]
• Electromagnetic attacks to interfere with the GPS signals of the UAV, that the UAV uses to position itself. Jamming the GPS signals causes the UAV not to be able to follow the pilot's navigation commands accurately.[2]
• Taking control of a UAV by spoofing the GPS signals of the UAV, such that the UAV thinks that it is still talking to the original pilot when it is actually being taken over. This way the drone can easily and safely be landed somewhere out of danger.[2]
• Capturing a UAV using another UAV carrying a net, which drops the net over the unwanted UAV. Thereby taking control of the UAV as the net makes sure the UAVs rotors get tangled in the net making sure it is unusable for the pilot. Then with a parachute on the net, it can be made sure that the UAV lands safely on the ground[7]
• A bazooka with an intelligent locking system to aid the controller to hit the UAV successfully, that shoots a net to capture a UAV. The rotors of the UAV will then get tangled in the net, making sure it cannot cause any harm anymore. Then a parachute that is attached to the net will make sure that the UAV will land safely on the ground. [8]
• Transmitting geo-fence coordinates, avoidance commands or disruption of radio communication in order to avoid UAV's entering no-fly zone. [17]
• Using trained eagles to neutralise UAVs. These eagles would be trained into considering UAVs as preys so that they could catch these drones and place them in a safe area. [38]
• Geo-fencing software built into the UAVs restricts consumer UAVs even to be able to fly within a certain range of unwanted areas such as airports. [56]
• Using high powered radio waves to disable drones, it blocks their communication with the controller and switches them off mid-air. [39]

### UAV Detection

Echodyne designs and manufactures radars with unparalleled price-performance. MESA technology is used, which is a fundamental breakthrough in high-performance radar with game-changing benefits in many markets. Acuity is an intelligent radar control software suite to enable user configurability. At an order of magnitude lower cost, Echodyne radar radically outperforms all other radar sensors in its class[57]. Their 3D Security radar offers superior sensor performance in a compact, solid-state, all-weather product. A recent winner in the SOFWERX Game of Drones competition, EchoGuard is the `perfect' radar for a multilayered perimeter defence solution. Furthermore, Acuity API integrates seamlessly with existing security ecosystems to provide situational awareness.

Their radar can be seen in usage in the following video. Echodyne provides another video that depicts a visualisation of the working of their radar.

The specifications of the 3D Security radar are as follows:

• Size: 8.0in x 6.4in x 1.57in (20.3cm x 16.3cm x 4cm)
• Weight: 1.25kg
• Power: DC +15V to +28V
• Operating: <50W
• Hot standby: <15W
• Hibernate TBR: <100mW
• Field of View: 120° Azimuth x 80° Elevation

This radar reliably detects and tracks aircraft and cars at 3km, people walking at 2km, and sUAS at 1km.

The Guardian reported that one such system could cast around 150 000 dollars[58].

WiFi receivers can accurately determine the position of drones. They are, however, very susceptible to interference. For example, WiFi signals can be blocked by obstacles. While it might be sufficient for right now, it is possible for malicious attackers to attach WiFi interfering tools to their UAVs. Then, WiFi receivers might not be as reliable in the future. When it comes to solutions, reliability is one of the main concerns as the solutions has to work in all cases.

Listening on communication between drone and ground

Listening to communication between a drone and its operator can be an easy way to detect the presence of both the drone and the operator. Often, this type of communication is not encrypted. The U.S. government displayed how easy it is to hack drones made by Parrot, DBPower, and Cheerson[59]. One significant disadvantage, however, is that custom built drones might use significantly different communication standards which do make use of encryption. So this solution is only useful when the communication is not encrypted, which still happens quite often as of now. The number of communications that do make use of encryption is suspected of increasing with the years as the technology gets more established. Then, this solution does not provide a way of detecting drones.

Detecting drones with other drones

When we use drones to detect other drones, we do not depend on insecure channels. With this solution, we are not just limited to drones in the line of sight as the drone-detecting drone can fly around. A disadvantage, however, is that flying around with a drone at a busy airport can be quite dangerous. Furthermore, these drones can only stay in the air for a limited time due to battery-related constraints. This can be mitigated by simply using larger batteries, but this increases the weight of the drone, which leads to some negatives again. This might, however, still provide to be a sound solution as it makes dealing with the illegal drone activity easier as the drone itself can, for example, be weaponised.

### UAV Identification

Identification by coded signal

UAV identification through coded signals can quickly identify activity. A disadvantage, however, is that other areal entities, such as bird, might also be targeted. This is due to birds being able to be roughly similarly sized as drones. Thus, this method can identify the aerial activity, but there is no guarantee that only illegal drone activity is identified. Then, this could lead to negative results when we consider, for example, birds. It is possible they are targeted by the drone interception system.

3D radar system with machine learning

UAV identification through a 3D radar system that uses machine learning can eventually lead to a precise system. The issue with it is that it first needs data to learn from. The gathering of this data can provide to be difficult. Furthermore, even if it learns from this data, it does not always have to lead to correct results as there are, often, biases in data.

UAV identification through an X-band radar system can perform accurate shape analysis of flying objects using doppler and high-frequency radar signals. Attaching a radar system to a drone, however, can be an issue around airports as this might result in interference with already existing systems.

### UAV Neutralisation

Missiles

The use of missiles might, on the one hand, be a rapid method and affordable method to take out hostile drones. However, there are a few downsides to this method. First of all, missiles are very dangerous, especially in an area where, apart from hostile UAVs, many aeroplanes with innocent passengers fly. The chance exists that a missile might miss an unwanted UAV and, instead, hit an aeroplane. Which would be disastrous and would only make the situation worse, especially if the unwanted drone was just a hindrance to the airport. UAVs are often quite small and can move/switch directions pretty quickly it is actually quite hard for a missile to correctly take out a UAV. Furthermore, might the unwanted UAV actually be taken out by a missile, then it will most definitely be destroyed meaning that police investigation will be more difficult. All in all, the use of missiles at and around airports is most likely a bad idea.

Lasers

Lasers are very precise and can be used in multiple ways to deal with drones. On the one hand, a very narrow laser beam can be pointed at an unwanted UAV to melt the body of the UAV causing structural failure and crashing of the UAV. On the other hand, a wide laser beam can be used to target multiple unwanted UAVs at the same time, taking out their control systems causing them to crash. Both methods require pretty close range to a target, the exact range depends on the type of laser that is used, and clear sight to the targetted UAV(s). However such systems could be attached to moving vehicles making such systems very mobile. Another advantage is, compared to the use of missiles, is that there is no need to reload as it uses the energy of a generator or the vehicle it is attached to. However, this means that the energy could deplete might there be too many targets, or might a target take too long to take out. Also, one might not always have clear sight to a target, or the range might be too long, making this method ineffective.

Interfering with GPS

Interfering with the GPS of a UAV, will not cause any harm to the drone. Then the drone can be inspected, once landed, to find out who is responsible for the UAV. However, when interfering with the GPS of a UAV, the pilot will be unable to send commands to the UAV making the UAV uncontrollable. This might cause the UAV to crash into aeroplanes, buildings or crowds of people. Hence this method can only be used when a UAV is in a so-called 'safe space' where it cannot harm anyone/anything. Furthermore, GPS interference might also affect the GPS systems of the aeroplanes at the airport. It might be the case that this method of neutralisation will not work against every UAV, as some UAVs might not use GPS to communicate with its pilot. Also, if a UAV is autonomous, it does not even need communication with a pilot necessarily. Lastly, it might also be difficult to get regulatory approval for the use of jamming devices due to jurisdictions.

GPS spoofing

This method is similar to the method discussed above (Interfering with GPS) and thus also shares the most advantages and disadvantages. However, an advantage of this method compared to GPS interference is that the targetted UAV will not be uncontrollable, but instead, the UAV can just be safely landed on the desired location. Apart from that, it shares the same disadvantages as GPS interference.

Capturing UAVs using nets underneath other UAVs

This method is a method that is not harmful to UAVs/the surroundings and does use interference which might cause problems for aeroplanes at the airport; those are two significant advantages. Another advantage is that it is a very affordable method to deploy to counter unwanted UAVs and allows for safe retrieval of UAVs. However, UAVs carrying a net to capture hostile UAVs, mostly just have room for carrying a single net to capture a hostile UAV. Meaning that if the UAV misses the net, it needs to return and completely be reset. Another thing is that these UAVs need to be able to follow small and very fast hostile UAVs to be able to place a net over them. This might be quite an issue, since carrying a net might be pretty heavy for a UAV causing it to move more slowly. Lastly, using this method, there will be even more UAVs in the air space, meaning even more interference for the airport.

Bazooka net system

This method is quite similar to previously mentioned method (Capturing UAVs using nets underneath other UAVs), however, in this case instead of using a UAV carrying a net, it uses a bazooka to capture a hostile UAV. It will also be quite affordable and straightforward to implement. Another advantage of this method is that there will not be any extra UAVs in the air space which might cause more interference. However, this method might be more inaccurate since a net needs to travel in the air for a more extended period before reaching an unwanted UAV, giving the UAV more time to evade the net. Furthermore, a bazooka firing a net will have more limitations in its range compared to UAVs carrying a net as it might not be able to reach unwanted UAVs which are high up in the air space.

Geo-fence coordinates

This method is the most straightforward solutions for the airports, as they will not need to implement a system to neutralise UAVs since UAVs simply will not be able to enter the air space of airports. However, this method is dependent on drones being programmed not to enter certain areas and will therefore not always avoid unwanted UAVs at airports. This might help against unwanted consumer UAVs that accidentally enter the air space above airports due to ignorant pilots. However, if someone really has terrible intentions with an airport, it will be straightforward to either turn off the geo-fencing software on a UAV or simply design their drone which will not have any geo-fencing software either. Furthermore, the software of the UAVs must continuously be updated according to new areas that might not be entered by UAVs, which might not be possible. Lastly, this solution also does not help against a large number of UAVs that already exist and can still be used for interference at airports. So in summary, this method might be useful for taking on a large part of newly bought UAVs, but can easily be avoided and should not be relied on by airports as the only method against UAVs.

Eagles

Using eagles to intercept drones is an economically friendly solution. Furthermore, the chances of a technical malfunction are non-existing. It is, however, still possible for the birds to deviate from the standard procedure when intercepting a drone even after extensive training. Moreover, flying birds near and around airports can be dangerous as they can get damaged by aeroplanes and other obstacles.

The Dutch police started using eagles to intercept drones back in 2016 already[60]. This, initially, seemed like a successful approach to seize drones mid-air. Not long after their initial usage, the Dutch political part `Partij voor de dieren' expressed their concerns regarding the safety and the wellbeing of the eagles.

After a year of training the birds, the police have concluded that the Eagles were barely used. Furthermore, the NOS reports, the training of the eagles is more complex and more expensive than the police expected. Additionally, there was little to no return in training these birds. Moreover, the birds did not always follow the procedures they were instructed to follow and therefore, the police was not convinced the birds would follow these procedures in real use[60].

Anti-Drone Eagles - title text: It's cool, it's totally ethical--they're all programmed to hunt whichever bird of prey is most numerous at the moment, so they leave the endangered ones alone until near the end.

Let us consider the following comic created by Randall Munroe on May 26, 2017. This comic raises a crucial ethical argument against the use of eagles in anti-drone mechanisms. While eagles, the predators they are, have natural inclinations to attack central parts of drones while evading sharp bits, their lives are still put at risk.

Cueball (person in the middle) argues that using eagles as anti-drone mechanisms is unethical as it forces a rare animal to put their lives at risk. Cueball compares it to using police dogs for traffic control, which is something that most people would frown upon after giving it some thought.

The effectiveness of eagles depends a lot on the conditions of how they are used. Naturally, eagles cannot be used everywhere, but they are often effectively used where some form of ground security is present that can be used to identify and arrest those illegally flying their drones. This is partly due to these people not being able to replenish their hardware indefinitely.

Not only would the use of eagles be unethical, but also ineffective. That is partly due to the supply of eagles being somewhat limited. Furthermore, there are natural boundaries to how fast they can be replenished, whereas more drones can easily be created to replace those that have been destroyed. Of course, this will involve more money, but we should be prepared for the worst. As brought up in the third part of the comic, traffic control dogs would be similarly ineffective, as dogs would struggle to run equally fast as racing motorcycles. Moreover, they would, in most cases, be too powerless to stop the motorcycle even if they could keep up them.

Megan (the girl on the right) states that both ideas, the usage of eagles and dogs, sound `cool'. She does, however, understand the ethical argument that Cueball raises against their use for traffic control. On the other hand, Black hat (the man on the left) goes a step further and states that he has created a drone that hunts eagles. This flips the premise from `anti-drone eagles' to `anti-eagle drones'. In the title text, which represents a statement from Black hat, he continues that it is ethical because they - the `anti-eagle drones' - only target the most populous species first, although they will eventually eradicate the endangered ones once they bring down the number of all birds of prey. Here, Black hat seems to miss the point that it is not merely the relative number of birds that creates an ethical problem, but the fact that animals' lives are being put at direct risk by humans. This is especially negative when other mechanisms can be used that are similarly effective for a bit more money, if not equal or less. The construction of his anti-eagle drone may simply be for the point of making the eagles' goals not only dangerous but also entirely ineffective.

All things considered, the comic above raises a critical ethical argument against the use of eagles in anti-drone mechanisms.

This method is actually really similar to the previously discussed GPS interference method and shares the same advantages and disadvantages as that method.

## Discussion

After finishing the collection of solutions (for now), it quickly became clear that the split into the three subcategories, namely detection, identification, and neutralisation made sense in quite a few cases. There were, however, solutions where identification and detection were merged. Furthermore, in order for identification to be something tangible, there needs to be detection. Many solutions already provided forms of identification, which is (by the previously mentioned prerequisite) also included detection. Therefore, we have decided to merge detection and identification. This would make the execution of the decision model a lot cleaner and also makes sense from a logical point of view. We want to group things together as much as we can as long as this does not degrade the quality of the result. From this point onwards, we only consider the detection (and identification) and neutralisation subcategories for all solutions.

## Comparison chart

In order to summarise the differences between the above solutions, we made a comparison chart. In this chart, we quantify the various attributes of each solution to allow us to see what solution is better for which type of airport.

We have chosen to look at the following attributes:

• Range, differently sized airports need different ranges of effect
• Speed of operation, for some airports a threat needs to be detected and neutralised in seconds, other airports do not have that much time pressure
• Disturbance to the environment, often airports closed to urban areas have noise restrictions, while others may not
• Effect on different types of drones, some solutions may only work on a targeted subgroup of drones, which can differ per airport
• Scalability, some airports may expand in the future and do not want to be limited by the investment in this technology
• Number of drones it can concurrently handle
• Emission, sustainability is essential for some airports. Other airports find this to be less of a concern
• Size, some solutions are way larger than others
• Identification, some solutions have a way also to identify
• Level of autonomy, the autonomy of the solution can play an important role as well
• Power Outage risks, it is possible to argue that the solution should always work, even when there is a power outage
• Weather, certain solutions might not work when it is really bad weather
• Uptime, some solutions have to be active 24/7, whereas others do not
• Portability, some solutions should be portable
• Danger to humans, at some airports many civilians may be present, for other airports this is not that much of a concern
• Destructivity, it can be possible that there should not be any destructivity be involved
• Level of training needed, sometimes it is possible that individuals operating the solution need a form of training

Rendering the attribute - solution table does not result in a clear overview. Therefore, we choose to only redirect the user to the tables rather than posting the tables here. These tables can be found here and here.

# Airport analysis

## Commercial airports

### Introduction

In this section, we will take a look at commercial airports and inspect the aspects of such airports that might affect our decision model for unwanted UAVs. Here, we define commercial airports as airports where the main purpose is to earn money by transporting passengers, transporting cargo loads or selling goods/services on the airport itself. We will first be looking at some general commercial airports in the Netherlands and compare these airports to find differences between them that might be critical for the decision model. After that, the USE stakeholders of commercial airports will be discussed. Furthermore, we will do a risk analysis of the types of drones that might be expected for this type of airport, which might be different for any type or airport. Lastly, we will look at the requirements that an anti-UAV system for commercial airports should adhere to.

### General

In the Netherlands, there are quite a few commercial airports ranging from very large airports with almost 71 million passengers a year to smaller airports of 250 thousand passengers, or less, yearly [61]. Larger airports might have a bigger budget to invest in an anti-UAV system. However, such airports might also be more prone to UAV hindrance or attacks. Next, to that, the requirements of an anti-UAV system also might be very dependent on the attributes of the airports, which could vary a lot between certain airports. Apart from moving around passengers, some commercial airports also move around huge amounts of cargo loads yearly [62], while others might not work at all. Amongst other things, the area of an airport, the number of runways at an airport or certain rules or regulations of an airport can all be a factor on deciding the right type of anti-UAV system for a certain airport. This is why it is so crucial to investigate different (commercial) airports that are located in the Netherlands.

In the section, five of the commercial airports located in the Netherlands will be inspected briefly. We will not do a full analysis of these airports as we think this will not be of any help our decision model, but we will be looking at the attributes of the airports that might be interesting or be of any impact to our decision model. The commercial airports that will be looked into are:

• Amsterdam Airport Schiphol
• Eindhoven Airport
• Rotterdam The Hague Airport
• Rotterdam The Hague Airport
• Maastricht Aachen Airport

The reason for choosing these five particular airports is because they differ quite a lot from each other on different attributes, such as the size of the airport, the number of passengers transported yearly, the amount of cargo load being transported yearly, the layout of the airport et cetera. Furthermore, we feel like these five airports cover/generalise to most of the commercial airports located in the Netherlands and cover most of the different attributes that might impact the type of anti-UAV system being used.

#### Amsterdam Airport Schiphol

Amsterdam Airport Schiphol is the largest airport of the Netherlands and was the third busiest airport of Europe in 2017. [62] Schiphol is home to Dutch airlines KLM, Martinair, Corendon, TUI fly and Transavia. The airport has 6 runways with a total length of 19467 metres. [63] In the figure below an overview and the locations of the runways of Schiphol can be observed with the names and lengths of each runway included.

General overview of the layout of Schiphol and overview of the runways of Schiphol. [63]

The table below contains general facts of the Amsterdam Schiphol Airport, mainly of 2018.

 Passenger amount 2018 70.956.604 passengers Amount of airplane movements (departure/arrival) in 2018 499.446 movements Total cargo load 2018 1.716.982 tonne Amount direct worldwide destinations 322 destinations Area of the airport 2.787 hectares Arrival peak hour capacity 106 movements Departure peak hour capacity 110 movements Amount runways 6 runways

In the table, we observe that Schiphol had a total of 499.446 movements in 2018, that would be an average of around 1.367 movements every day. Keep in mind that this is both arrivals from and departures on Schiphol and includes both passenger flights and cargo flights. Then with the six runways the airport has, this would mean that with an average of 1.367 movements every day, each runway would, on average, observe an aeroplane movement around every 6 minutes. Note that this is a calculated average, and this number of movements on each runway can be higher during peak hours. This, in turn, means that at any given point in time, Schiphol is extremely busy and loaded with at least thousands of passengers. Hence any form of interception with UAVs would already cause major issues. Let alone the damage that could be caused by any forms of terrorist attacks with the use of UAVs.

#### Eindhoven Airport

Eindhoven Airport is the second largest airport in the Netherlands in terms of passengers [65]. Furthermore, Eindhoven Airport is also the home base of all military transport aeroplanes of the Royal Netherlands Air Force [66]. In total, there are nine large military aeroplanes, which are usually used after a natural disaster[67]. The latest example being hurricane Irma, who caused huge amounts of damage in the Dutch colonies Sint Maarten and Curacao [68].

In the figure below, the general layout of the Eindhoven Airport is depicted, showing mainly the single runway the airport has and the parking spots of the aeroplanes currently not in use.

General overview of layout of Eindhoven Airport [69]

The table below below contains general facts of Eindhoven Airport, mainly of 2018.

 Passenger amount 2018 6.237.755 passengers Amount of airplane movements (departure/arrival) in 2018 38.642 movements Total cargo load 2018 0 tonne Amount direct worldwide destinations 81 destinations Area of the airport 639 hectares Amount runways 1 runway

As can be seen in the table, Eindhoven Airport has only one runway, which means that both the commercial aeroplanes and the military aeroplanes depart and arrive using the same runway [70]. Eindhoven Airport is also experiencing an enormous growth at the moment, with an increase in served passengers of almost 10% [71]. The military departures occur so infrequently compared to the commercial airlines that their contribution to the number of departures from Eindhoven Airport is negligible. The operating hours of Eindhoven Airport are between 07:00 in the morning and 24:00, so midnight, 365 days per year. This corresponds to an aeroplane either departing or arriving on the runway every 10 minutes on average[70].

Some critical remarks regarding the safety hazards of Eindhoven Airport are that it is, in fact, the second largest airport in the Netherlands in terms of passengers. This means that a sudden shutdown due to a drone would have enormous consequences for many passengers. Furthermore, due to the fact that the airport only has one runway, it has a single point of failure to prevent all aeroplanes from both taking off and arriving. If one drone would be flying around this runway, then the whole airport has to be shut down. On top of that, the fact that Eindhoven Airport houses military transport vehicles increases the risk of drone attacks. Imagine that there would be a sudden natural disaster and these aeroplanes need to depart to aid the people in the disaster area, then it takes only one drone to delay this much-needed help by several hours. This could be the hours that would mean life or death for multiple people in the disaster area. Lastly, the giant and continuous growth of Eindhoven Airport could also pose problems in terms of drone safety. If the growth is not adequately regulated and safety measures are not adequately investigated, the growth could increase the possibility of a drone causing problems at Eindhoven Airport.

#### Rotterdam The Hague Airport

Rotterdam The Hague Airport is the third largest airport of the Netherlands with respect to served passengers served, with a total of 1.943.733 in 2018[72][73]. As the name suggests, it is located between the Dutch cities Rotterdam and The Hague. Due to the airport being located close to the political heart of The Netherlands, Rotterdam The Hague Airport functions as the airport of the government [74]. Important international guests use this airport to go to the Netherlands, for example to the Nuclear Safety Summit in 2014 [74].

General overview of layout of Rotterdam The Hague Airport. [75]

Table 3 below contains general facts of Rotterdam The Hague Airport, mainly of 2018.

 Passenger amount 2018 1.943.733 passengers Amount of airplane movements (departure/arrival) in 2018 53.322 movements Total cargo load 2018 19 tonnes Amount direct worldwide destinations 50 destinations Area of the airport 222 hectares Amount runways 1 runway

What is unique about this airport is that it has much more departing and arriving aeroplanes than Eindhoven Airport, whereas Eindhoven Airport serves three times as many passengers. The reason for this is the type of flights that occur at the airport. Rotterdam The Hague Airport also houses many flying schools. Here, people train their flying skills to acquire a flying permit as a hobby. These flying lessons are also the main contributor to the number of movements, with a total of 13.761 [77].

From this information, we can deduct some safety hazards of Rotterdam The Hague Airport. It is quite a small airport with not a lot of passengers, which means that fewer passengers will be affected, should the airport be shut down due to a drone in the area. However, there is again only one runway. This results in a single point of failure to prevent all aeroplanes from both taking off and arriving. If one drone would be flying around this runway, then the whole airport has to be shut down. Furthermore, the airport also serves a lot of passengers who are internationally and politically important, which increases the risk of someone who wants to disrupt this politically important person by delaying his or her flight. What is also important to note is that there are a lot of flying lessons and recreational departures and arrivals. In general, these planes are a lot smaller and are flown by less talented pilots than commercial aeroplanes. This means that a drone would do more damage to the smaller aeroplane. Furthermore, the lesser talented pilot is more likely not to know the rules correctly, and would most likely handle more poorly in the event of a drone in the area. The consequences of a drone collision to a small aeroplane would, however, be smaller in terms of people affected by the incident.

#### Maastricht Aachen Airport

Maastricht Aachen Airport is one of the larger airports of the Netherlands with 274.986 passengers in 2018[61], however, this is really small compared to the number of passengers of Schiphol in 2018. The airport is located eight kilometres north of Maastricht.

In the figure below, the general layout of the Maastricht Aachen Airport is depicted, showing mainly the single runway the airport has.

General overview of layout of Maastricht Aachen Airport. [75]

The table below contains general facts of Maastricht Aachen Airport, mainly of 2018.

 Passenger amount 2018 274.986 passengers Amount of airplane movements (departure/arrival) in 2018 15.781 movements Total cargo load 2018 124.676 tonnes Amount direct worldwide destinations 25 destinations Area of the airport 450 hectares* Amount runways 1 runway

*Approximation of area calculated using a google maps area calculator [79]

#### Groningen Airport Eelde

Groningen Airport Eelde is the fifth largest airport of the Netherlands in terms of passengers with 228.698 passengers in 2018[61], however, this is again really small compared to the number of passengers of Schiphol in 2018. The airport is located close to the village Eelde in Groningen. The airport has only one runway in use, the second (smaller) one depicted also in the figure below, is not in use anymore.

General overview of layout of Groningen Airport Eelde. [75]

Table 5 below contains general facts of Groningen Airport Eelde, mainly of 2018.

 Passenger amount 2018 228.698 passengers Amount of airplane movements (departure/arrival) in 2018 31.413 movements Total cargo load 2018 0 tonnes Amount direct worldwide destinations 11 destinations Area of the airport 400 hectares* Amount runways 1 runway

*Approximation of area calculated using a google maps area calculator [79]

#### Most important attributes

The most important attributes of commercial airports. Where, with 'most important', we refer to the importance of the attributes of commercial airports concerning anti-UAV systems. If we look at the general facts of each airport, then the two main attributes that could influence the chosen type of anti-UAV system for a certain airport, are the size of the airport and the number of aeroplane movements yearly. The amount of runways an airport has is correlated to the size of an airport and the amount of passengers/cargo moved yearly is correlated to the amount of airplane movements yearly. As for size, an anti-UAV system should be able to cover the whole range of a massive airport (i.e. Schiphol airport), but for another relatively small airport (i.e. Maastricht Aachen airport) that same system might be disproportionate or too excessive and simply not affordable. On the other hand, the number of aeroplane movements, which is correlated to the number of passengers/cargo moved, might make an airport more/less susceptible to UAV attacks. Furthermore, the number of aeroplane movements of an airport might make the airport more/less susceptible for a UAV of a certain category to attack, which some anti-UAV systems could be specialised for. Since both attributes, size and amount of aeroplane movements, differ a lot between some of the commercial airports discussed above, these attributes are important to take into account for the decision model.

### USE stakeholders

There are quite a number of stakeholders involved with commercial airports. Think about passengers, air carriers, shops at the airport, residents near the airport et cetera. All of these stakeholders have a goal for the airport and might have requirements for a potential anti-UAV system that such an airport might deploy. As a simple example, residents near the airport want to minimise any hinder or do not want to be affected in any way by the airport nearby, and therefore also want to minimise any hinder of an anti-UAV system deployed at the airport nearby. There are many more such stakeholders that want to minimise any hindrance an anti-UAV system could cause for their goals, concerning the airport. Therefore, an analysis of the stakeholders and their requirements for a potential anti-UAV system is important. In table 6, a list of stakeholders for commercial airports is given, including a definition or example of the stakeholder, and whether they belong to the user, society or enterprise group.

Table 6: Stakeholders of commercial airports [81]
Stakeholder Definition/Example(s) USE
Passengers Transferring passengers User
Air carriers Passenger and cargo carriers Enterprise
General aviation users Air taxi, corporate transportation, etc. User
Airport organization Management and staff with responsibility for operation of the airport Enterprise
Investors and bond-holders Individuals/companies investing in the airport or holding bonds with the airport Enterprise
Concessionaires Providers of services to passengers such as food or retail Enterprise
Employees Employees of the airport Enterprise
Service providers Providers of services to airport/air carriers such as fuel Enterprise
Government Responsible for infrastructure, security, etc. Society
Communities affected by airport operations Residents near the airport Society
Ground transportation providers Buses, shuttles, taxi's, rental cars, off airport parking services, etc. Enterprise

Each of these stakeholders has goals for the airport, and therefore a potential anti-UAV system must not hinder this goal. By listing the goals of the stakeholders, we can use these goals to set up requirements for anti-UAV systems for commercial airports later. Since most of the goals of the stakeholders with the same USE group overlap, we will summarise a list of the goals of each of the USE groups instead of every single stakeholder, to keep it comprehensible and easy to understand. This list is given below.[81]

#### Users goals for commercial airports

• Moving passengers quickly and conveniently
• Ensure all services are on-time
• Keep fares as low as possible
• Serve as an access point and ensure good availability and high equipment capability (for general aviation)

#### Goals from enterprises for commercial airports

• Ensure all services are on-time
• Keep costs of services as low as possible
• Maximise the number of passengers and cargo loads transported
• Ensure safety of all services
• Maximise the number of destinations served
• Provide secure jobs and wages
• Maximise users satisfaction
• Minimise hindrance and noise for the area around the airport
• Maximise environmental sustainability

#### Goals from society for commercial airports

• Minimise noise and hindrance for the area around the airport
• Minimise emissions
• Ensure airports can accommodate growth
• Ensure safety and security of the airport

This list shows a lot of the goals of each of the USE groups, which a potential anti-UAV system should not affect, or at least minimise any hindrance of those goals. These goals can translate to requirements for anti-UAV systems at commercial airports, which will be done later this section.

### Risk analysis

Analysing the types of UAVs that are more likely to cause any hindrance/attacks for commercial airports can also form requirements for potential anti-UAV systems. As elaborated on in section drones, there are different categories of UAVs, based on attributes of a UAV. Some anti-UAV solutions might not be able to accurately detect, identify and neutralise UAVs of a certain category. Therefore, might this category of UAVs be present at commercial airports, then it would not be well suited as an anti-UAV solution for this type of airport. On the other hand, some anti-UAV solutions might be specialised for a certain UAV category. If the majority of UAVs that are present at commercial airports are of this category, then this anti-UAV solution might be well suited for commercial airports. Therefore, an analysis of the types of UAVs that might form a threat to commercial airports is important as well, for forming the requirements for a good anti-UAV solution for commercial airports.

Let us take the classes of UAVs C0 to C4 as discussed in section drones as the categories of UAVs we take into consideration. Starting with UAVs of class C0, which is the only class of UAVs that do not require an electronic ID or geo-awareness. This means that the pilot of the UAV cannot easily be tracked, might the UAV fly over an area it is not supposed to. Furthermore, the UAVs in this class have not been programmed with geofencing to stay away from airports within a certain range. Hence, in theory, these UAVs are free to fly over any area the pilot wants. However, most of the (consumer) UAVs (at the moment of speaking, March 2019) in this class have a maximum controllable range of around 100-200 meters and a battery that lasts a maximum of about 15 minutes.[82] So in practice, it is almost impossible for a UAV of this class to ever reach an airport and be of any hinder. Hence this class of UAV will probably not be much of a hindrance for commercial airports. Classes C1 to C4 do require an electronic ID and geo-awareness. So, for any consumer bought UAV, they will be programmed with software that limits them such that these UAVs will not be able to enter a certain radius around the airport. So if consumers do not deactivate this software, then these classes of UAVs should also not be of any hindrance to commercial airports.

Lastly, there is the class of privately build UAVs, for which the hobbyist builders themselves decide whether or not to implement any form of geo-fencing. This class of UAVs seems to be the most troublesome, as they are privately built, hence do not adhere to any rules necessarily. These drones also could be build of any weight and are not limited to reach a certain speed. Therefore it is not possible to assign them a drone class. Given that commercial airports, mostly, are hectic and crowded places, and might be a good target for anyone want to cause considerable damage to many people, these privately build UAVs could form a significant threat to commercial airports, especially if they are armed. Since a commercial airport would be such a good target for anyone that wants to cause a lot of harm/hindrance, it is also to be expected that privately build drones are very popular for these airports. So in short, the types of UAVs that can be expected at commercial airports are the ones of class C1 to C4, if their geo-fencing software were to be disabled, and privately build UAVs which could belong to any of the classes (except for C0). The drones of type C0 simply will not have enough range or enough battery life to cause hindrance to commercial airports.

### Requirements for solution

After the investigation of multiple aspects on commercial, we have enough information to form requirements that an anti-UAV system should adhere to for commercial airports. We have discussed general facts of commercial airports, the stakeholders of these airports and what their goals are for the airports, and lastly a risk analysis of commercial airports. This should form a good base of were to be expected for an anti-UAV system at a commercial airport. The list of requirements below takes into account all found results from the investigation on commercial airports and translates them into the needed requirements for commercial airports.

Requirements

• Be able to take down drones in the entire area of the airport
• Not interfere with an aeroplane at the airport
• Take down a hostile drone within 3 minutes, to keep costs as low as possible and flights on time
• Ensure safety of all services including passengers or any other humans at any time
• Should not emit/minimise emission of CO2
• Be able to be expanded to cover a bigger terrain in the future
• Not be any louder than aeroplanes
• Not hinder the user's satisfaction
• Be able to take down any UAV from class C1 to C4

## Military Airbases

As of now, there are eight military bases for the Royal Dutch Air Force in the Netherlands \cite{royal}. However, they differ quite a lot in nature due to the housing of different types of aerial vehicles. There are main operation bases, which are the biggest bases. Furthermore, we have a tactical air operations base, which is the air traffic control centre that is used by all military air traffic, air battle management and air surveillance among others. Next, we also have defence helicopter command bases, where most of the attack helicopters are stationed. Then, we also have an air transport base, which is mostly used to transport either infantry or military equipment. Lastly, we also have a common support base, which is currently mainly used for military training purposes. Thus, the eight military bases can be structured as follows:

• Main Operating Bases
• Leeuwarden Air Base
• Volkel Air Base
• Tactical Air Operations Base
• Air Operations Control Station Nieuw Milligen
• Defense Helicopter Command Bases
• Gilze-Rijen Airbase
• De Kooy Airfield
• Deelen Air Base
• Air Transport Base
• Eindhoven Air Base
• Common Support Base
• Woensdrecht Air Base

### Leeuwarden Air Base

Leeuwarden Air Base is together with the Volkel Air Base and the Gilze-Rijen Air Base on of the three biggest military air bases in the Netherlands \cite{leeuwarden_wikipedia_en}. It is one of the two bases that station the F-16s of the Royal Netherlands Air Force. It has two runways. The Leeuwarden Air Base is mostly used for homeland security when needed. Its main task is to actively monitor the airspace of the Benelux, deploying F-16s when necessary. At all times, Dutch F-16s are on stand-by. Apart from this monitoring, they are also deployed for missions all around the globe. Occasionally, they are also used for training sessions, which occur mostly above the North Sea \cite{leeuwarden_royal}.

General overview of layout of military air base Leeuwarden. [75]

### Volkel Air Base

Volkel Air Base is, as mentioned before, one of the biggest military air bases of the Rotal Netherlands Air Force \cite{volkel_wikipedia_en}. It is the other bases that stations the Dutch F-16s. With the housing of F-16s comes the task of providing air support and air defence. Just as done in Leeuwarden, the F-16s are most often deployed for monitoring the airspace of the Benelux \cite{volkel_royal}. It is located in the province of Noord-Brabant.

General overview of layout of military air base Volkel. [75]

### Air Operations Control Station Nieuw Milligen

Air Operations Control Station Nieuw Milligen is one of the smaller military bases and is used mainly for air traffic control. At the moment, its main task is air traffic control \cite{milligen_wikipedia_en}. More concrete, this means for example that this air base checks (from their command-and-control centre) whether an aircraft adheres to its flight plan. If not, radio contact will be established from Nieuw Milligen. In the worst case, the command-and-control centre is capable of deploying tactical air control by scrambling a so-called Quick Reaction Alert. This means that two fully armed F-16 fighter aircraft will be deployed \cite{nieuwmilligen}. Note that this is only a control station and hence, no aerial vehicles are located at this air base. However, for the sake of completeness, we decided to include this air base in the report.

### Gilze-Rijen Air Base

Gilze-Rijen Air Base is a helicopter command base of the Royal Netherlands Air Force. These helicopters form a part of the Defence Helicopter Command \cite{gilze_wikipedia_en}. An example when helicopters from Gilze-Rijen are deployed as support for Navy ships. They can, if necessary, also be deployed as fighter helicopters \cite{gilze_royal}. In the case of a major fire, they can also be deployed. The air base is located in the south of the Netherlands. Like all the previously mentioned airports, Gilze-Rijen Air Base has two runways. Among the military aircraft, the Royal Air Force Historic Flight Foundation is located at the air base. Here, a collection of historic military aircraft is stored and occasionally operated.

General overview of layout of Gilze-Rijen air base. [75]

### De Kooy Airfield

De Kooy Airfield is a helicopter command base as well. It is located near Den Helder, a city in the Netherlands near the North Sea \cite{kooy_scramble}. Consequently, its main task is bringing workers from and to oil rigs located on the North Sea and also houses military helicopters \cite{kooy_wikipedia_en}. It only has one runway and is not used very often.

General overview of layout of Airfield De Kooy. [75]

### Deelen Air Base

Deelen Air Base is the last of the three helicopter command bases of the Royal Netherlands Air Force. It is located in Deelen, a small place in the middle of the Netherlands with about 50 inhabitants \cite{deelen_wikipedia_nl}. Its main task is for practices with helicopters. The main type of practices done in Deelen is communication and cooperation between on one side the helicopters and on one side the military on the ground. The military basis is also often used as a refuelling point for helicopters which are going on a mission abroad. Apart from fuel, ammunition is also loaded on the helicopters. The Air Base has only one runway.

General overview of layout of Deelen air base. [75]

### Eindhoven Air Base

Eindhoven Air Base is an air transport base, located in Eindhoven \cite{eindhoven_wiki}. As the type of air base suggests, all of the military transport vehicles are stationed here. In total, there are nine large military aerial vehicles, which are usually used after a natural disaster\cite{infograph}. The latest example being hurricane Irma, who caused huge amounts of damage in the Dutch colonies Sint Maarten and Curacao \cite{irma}. From the airport, goods and military were transported to help the people in need \cite{eindhoven_royal}. As described before, the military base uses the same single runway as the commercial aeroplanes. For an overview of the runway, see the section about Eindhoven Airport in the commercial aeroplanes section.

### Woensdrecht Air Base

Woensdrecht Air Base is a common support base for the Royal Netherlands Air Force located in the south-west of the Netherlands. The airport has one runway. It is a base that is mainly used for training and logistics and does not house any combat military. As of now, the air base is not operational, meaning that no military aerial depart or arrive as of now from the air base, but for the sake of completeness, we want to mention the presence of this military base.

## General description/facts of recreational Airfields in the Netherlands

We distinguish a third main category of airport, which recreational airfields. These locations are often used by pilots-in-training, or for recreational flights. Traditionally meant for small, single prop planes like the well-known Cessna, these airfields often feature short runways. These runways are often simply flattened strips of grass or dirt since the types of aeroplanes that might take off, and land from these airfields do not have stringent requirements for takeoff. Often these airfields are home to one or more local sky-gliding organisations. It is common that also other organisations frequent these airfields, whether they are flight related organisations or not. It is noteworthy that these airfields are also frequently prime locations for drone flight. Being large, open spaces away from any residential areas, the aspects that make these places into great locations for recreational or educational flight, also make them suitable for drone flight.

### Airfield Malden

Airfield Malden is an airfield mainly used for glide flights near Malden and Nijmegen in the Netherlands. It is situated on a large open field in the forests. In existence since 1954, Malden Airfield is mainly used by two glide flying organisations. These are the NijAC (Nijmeegse Aeroclub), the local glide aero sport club, and NSA Stabilo, which is catered towards students from local educational institutes. The Airfield is relatively small and features a reasonably popular restaurant. Given the location of Malden Airfield in the forest, there are many cycling routes and hiking trails that either start, end, or pass by Malden Airfield and thus by the restaurant. This, plus the entertainment provided by watching planes take off and land, makes Malden Airfield into a fairly popular location for not only flight related visitors but also cyclists, hikers and the general population. The Airfield in Malden has two runways. A fleet consisting of 9 glide planes including a motorised one is stationed at the Airfield. These planes belong to the NijAC as the main group behind the operations at this Airfield. As this Airfield supports mainly the flights of club members of the NijAC, no real full-time personnel is present. The NijAC offers many services to non-members, such as educational glide flying, flying as recreation with one of the club members, or offering their location as a service for non-members flying their plane. It should be mentioned that the services, popularity and reachability of Malden Airfield would make this into a perfect location to fly commercial drones. Larger drones or even RC (radio controlled) model planes could make use of the runways at this airfield, granted that safety is a priority. This information, together with the mentioned fact that this Airfield is frequently visited by many people with non-flight related goals, makes Malden Airfield a unique situation for an anti-drone system. Since Malden Airfield is a prime location for the practice of recreational flight, this might also include drone flight. Organised events like drone races might also attract more business for both the flying clubs and the restaurant. Drone flight is not necessarily something we wish to discourage here permanently, and might be lucrative for all parties involved if we can guarantee safety.

### Airfield Terlet

Terlet Airfield is home to the most significant association of gliding flight clubs in the Netherlands, consisting of the following active clubs: Gelderse Zweefvlieg Club, Delftse Studenten Aeroclub, Gliding Adventures Europe, Kennemer Zweefvlieg Club, ZC Deeleen, the EZZC, ZHVC and Zweefvliegclub Ameland. The Airfield has six grass winch tracks, where winches might be used to tug glide planes up to take off speed. Small motorised planes might also be used to tug gliding planes, but only on 1 of the take-off/ landing tracks. Airfield Terlet is larger than the previously discussed Malden Airfield, averaging around 17.000 glide plane take-offs each year. Around 10% of which is done by using a motorised plane as a tug. This is important to note that since different types of planes might incur different kinds of risks for drone incidents. Unlike Malden Airfield, there are no popular cycling routes or hiking trails near the Airfield, and entering the grounds without the supervision of an experienced glide plane pilot or local instructor is strongly discouraged. Reasons stated for this are that the Airfield can be quite dangerous without proper supervision, as planes are taking off and landing and more importantly when a gliding plane is tugged using a ground-based winch, the cable used to attach the plane will fall to the ground after takeoff. This is very dangerous to the uninformed or inexperienced. On the website of the club association, it states that there is a camping ground nearby, meant for use by glide plane enthusiasts and club members. There is no mention of this Airfield being a popular location for drone enthusiasts or non-flight related activities. However, it should be noted that since motorised planes take off and land from Terlet Airfield, it is not a stretch to imagine larger UAVs using the runway as well. This could be a future expansion possibility, and then the question becomes how safety should be guaranteed.

### Possible risks

We can conclude that non-flyers and non-flight related clubs or organisations are also often present at recreational airfields. The more people are present. The more people are at risk of a possible drone incident. To get a gliding aeroplane off the ground, often a motorised aeroplane is used to propel it forward by interlinking them. These motorised aeroplanes are often more susceptible to drone incidents. As mentioned before, the attributes of recreational airfields also make them excellent locations for civilian drone flight. Parents might take their children to test out a newly bought drone with a camera and take pictures. These airfields are often located in large, open fields and away from direct residential areas. As drone regulations include that direct line of sight must be maintained with the drone, these types of locations make it easier to follow these, and other, regulations. We conclude that recreational airfields are a right place for recreational drone flight, and thus are more prone for drone incidents including recreational drones.

## Recreational Airfield

### Introduction

This Section covers recreational airfields. These are defined as airfields, where the main purpose is not to earn money from airlines by enabling them to transport people as a service. Rather, the main purpose is to enable flight for people whose enjoyment is in the flight itself. Concrete examples are airfields where mainly sky-gliding planes are flown, or where pilots make private flights for enjoyment, usually in small aeroplanes. This type of airfield is so different from the commercial type airports discussed before, and they are open to different types of drone incidents involving different categories of drones.

### General

In this Section, we look at the various recreational airfields throughout the Netherlands. We will research how these types of airfields influence the requirements for anti-drone solutions, by analysing airport-type specific risks and the perspectives of the stakeholders. Recreational airfields are a platform for recreational flight. Recreational flight is made up of various subcategories, such as flight in glide planes, pilots in training making their first flight hours in small aeroplanes, or simply hobbyist pilots flying their plane for fun. Often, the airfields that support this kind of flight are fairly small, and located in large open fields in forests, away from towns or cities. The characteristics of these locations that make them suitable for recreational flight, also make them good locations for drone flight. This will be elaborated on in the risk analysis at the end of this Section.

#### Malden airfield

Malden airfield is an airfield mainly used for glide flights near Malden and Nijmegen in the Netherlands. It is situated on a large open field in the forests. In existence since 1954, Malden Airfield is mainly used by two glide flying organisations. These are the NijAC (Nijmeegse Aeroclub), the local glide aero sport club, and NSA Stabilo, which is catered towards students from local educational institutes. The Airfield is relatively small and features a fairly popular restaurant. Given the location of Malden Airfield in the forest, there are many cycling routes and hiking trails that either start, end, or pass by Malden Airfield and thus by the restaurant. This, plus the entertainment provided by watching planes take off and land, makes Malden Airfield into a fairly popular location for not only flight related visitors but also cyclists, hikers, and the general population.

The general layout of the airfield is visible in the aerial photograph visible below.

Malden airfield

As visible in the figure above, the Airfield in Malden has two runways. A fleet consisting of 9 glide planes including a motorised one is stationed at the Airfield. These planes belong to the NijAC as the main group behind the operations at this Airfield. As this Airfield supports mainly the flights of club members of the NijAC, no real full-time personnel is present. The NijAC offers many services to non-members, such as educational glide flying, flying as recreation with one of the club members, or offering their location as a service for non-members flying their plane. It should be mentioned that the services, popularity, and reachability of Malden Airfield would make this into a perfect location to fly commercial drones. Larger drones or even RC (radio controlled) model planes could make use of the runways at this airfield, granted that safety is a priority. This information, together with the mentioned fact that this Airfield is frequently visited by many people with non-flight related goals, makes Malden Airfield a unique situation for an anti-drone system. Since Malden Airfield is a prime location for the practice of recreational flight, this might also include drone flight. Organised events like drone races might also attract more business for both the flying clubs and the restaurant. Drone flight is not necessarily something we wish to permanently discourage here and might be lucrative for all parties involved if we can guarantee safety.

#### Terlet Airfield

Terlet Airfield is home to the largest association of gliding flight clubs in the Netherlands, consisting of the following active clubs: Gelderse Zweefvlieg Club, Delftse Studenten Aeroclub, Gliding Adventures Europe, Kennemer Zweefvlieg Club, ZC Deeleen, the EZZC, ZHVC, and Zweefvliegclub Ameland. The Airfield has six grass winch tracks, where winches might be used to tug glide planes up to take off speed. Small motorised planes might also be used to tug gliding planes, but only on 1 of the takeoff/ landing tracks. A map of the layout of Terlet Airfield is given below in the figure below.

Terlet Airfield

Airfield Terlet is larger than the previously discussed Malden Airfield, averaging around 17.000 glide plane take-offs each year. Around 10% of which is done by using a motorised plane as a tug. This is important to note since different types of planes might incur different kinds of risks for drone incidents. Unlike Malden Airfield, there are no popular cycling routes or hiking trails near the Airfield, and entering the grounds without the supervision of an experienced glide plane pilot or local instructor is strongly discouraged. Reasons stated for this are that the Airfield can be quite dangerous without proper supervision, as there are planes taking off and landing and more importantly when a gliding plane is tugged using a ground-based winch, the cable used to attach the plane will fall to the ground after takeoff. This is very dangerous to the uninformed or inexperienced. On the website of the club association, it states that there is a camping ground nearby, meant for use by glide plane enthusiasts and club members. There is no mention of this Airfield being a popular location for drone enthusiasts or non-flight related activities. However, it should be noted that since motorised planes take off and land from Terlet Airfield, it is not a stretch to imagine larger UAVs using the runway as well. This could be a future expansion possibility, and then the question becomes how safety should be guaranteed.

### USE Stakeholders

As discussed previously, recreational airfields cater to a large variety of user groups, not limited to flight oriented users. As such, the variety of stakeholders in the safety of these airfields is equally large. This Section will identify such stakeholder groups and classify them according to the USE classification.

Stakeholder Definition/Example(s)
Recreational glide flying organisations Organisations using the airfield to offer glide flights to members
Educational flight organisations Organisations offering lessons in flight of motorised airplanes
Other sport organisations Cycling or hiking organisations that are also frequently present at airfields
People wanting to fly recreationally The populace in general with the need for recreational flight
People needing transport The need for people or goods being transported in small planes to land or

take off at places other than the nations largest airports

People wanting to fly drones People living close to the airfields wanting to fly their drone recreationally

in suitable locations

Employees Regular employees of the airfields
Companies Companies wanting to conduct business on the airfield by offering

flights or other services

In the table above, a list of stakeholders in drone flight safety at recreational airfields is introduced. The elements of this list all represent groups that have a stake in safety at recreational airfields, following from the analysis of several recreational airfields in the Netherlands. The interests of these stakeholders play a part in deciding which requirements might be more critical for an anti-drone system at a recreational airfield. Since recreational airfields are usually also prime locations for drone flight, the need to fly drones recreationally is counted as a society stake. The following list sums up the main goals of the various USE categories with regards to recreational airfields.

We see that for the User category mainly safety is of importance. With regards to drone flight, it is essential for the User to be provided with levels of safety not inferior to that in a world or time where no drone flight was present at all. For society, it is of great interest to have these airfields available for flight as recreation, as well as transport to locations not close to major airports. However, it could also be described as a social goal to enable recreational flight with drones by maximising the usable airspace. Enterprise goals are mainly similar to the User goals in that safety is of great importance for companies with a stake in general operations at the airfield, however, maximising the area where drones can be flown safely is also a goal for drone manufacturing companies. In the following section, we further analyse the risk of drone incidents that are specific to this category of an airfield.

### Risk analysis

Different types of airfields with different attributes might be at a higher risk of specific drone-related incidents than others. The most obvious elevated risk level exists because almost all recreational airfields are great places to fly drones. There are a number of non-enforced rules for commercial drone flight, one such being the ability to keep a line of sight to the flying drone at all times during operation. When commercial drone flight gets more popular and more affordable, we expect to see the number of such flights rise sharply in large open spaces away from residential areas especially because these rules are easier to adhere to. This is an important aspect for most commercial drone pilots. Other aspects of recreational airfields that might appeal to pilots of 'over-the-counter' drones might be the ease of access to most recreational airfields and the fact that there are often restaurants or cafes on site, altogether making recreational airfields attractive for a commercial visit for other purposes than to fly planes.

Given this expected increase in the presence of commercial drone flight, it is pivotal that an anti-drone system at recreational airfields is able to maintain safety mainly when faced with smaller drones that are commercially available. One might use the argument that these commercially available drones might soon be equipped with a geofencing system, therefore solving a large part of the problem. However, as drone flight is also technically a form of recreational flight, a recreational airfield might not want to permanently prevent drones from flying through its airspace. As described in the USE stakeholder analysis, society, in general, might be a stakeholder in keeping many suitable areas, including these airfields, open for drone flight. Furthermore, according to the drone classification into C0 up to C4 categories, the smaller drones in the C0 category are not required to be equipped with geo-awareness. As described in the Section on commercial airports, many commercially available drones from this category have an operating range not exceeding 200 meters and a battery-time of about 15 minutes. While this might not be enough range nor usage time to cause serious trouble at a commercial airport, it might certainly be enough to cause an incident on a recreational airfield.

It is also important to note that since recreational airfields are less busy, and the drone category we expect to see most is also that which has the least potential to cause damage due to being relatively small, general risks of drone incidents are reduced compared to major airports. Also, the subjective cost of such an incident might be less than that of an incident at a major airport, since a crash of a smaller plane endangers less human lives, and they are in less danger compared to an incident involving a large jet. Combined with the relatively low number of flights leaving from and arriving at these airfields, this all might suggest a lower risk of drone incidents at recreational airfields and might be seen as a reason to not completely inhibit drone flight here.

### Requirement for the solution

This Section describes a list of requirements for an anti-drone system to serve as a recreational airfield. These were generated after considering airfield specific drone incident risks and the USE stakeholder analysis for these types of airfields. Note that some of these focus on minimising damage to the drone, as drone flight is also regarded as a type of recreational flight. Also, general environmental concerns are taken into account.

# Decision Model investigation

In this section, we will investigate some different approaches to decision models. These decision models were investigated but were chosen not to be the final decision model that we will implement. However, for the sake of completeness of this wiki, we will describe our findings on other decision models in this section.

## Nearest Neighbour Strategy

NearestNeighbour, short NN, is a mathematical decision model. It is a machine learning decision model, in the sense that existing solutions, often denoted as training data, are used for NN to be able to accurately make predictions about new data such as a user which wants a solution for their airport. This decision model can make the choice which solution fits best to the user. Nearest Neighbour is based on the machine learning strategy KNearestNeighbors [83] [84][85].

### Picking variables / attributes

In order for Nearest Neighbour to work, we need to quantify our problem into numerical values. For this, we need to split this up into variables with numerical data. This can be done in the same way as we picked the attributes in section implemented decision model. These are variables that can tell which type of solution will fit best for this case. Examples of these attributes for the solution are, e.g. cost (in €), reliability (in %), range (in m), a hindrance to surroundings (scale from 1 to 10), CO2-emission (in kg CO2 / year), et cetera.

### How does NN work?

So, we now have defined a solution in terms of only numerical variables. Then, for each solution that we have found, we will assign corresponding values to the attributes. An example of how this is done can be found in this part of the solutions section.

Now, how NN then works as follows: it plots the points from the solutions in the n-dimensional plane, where n is the number of variables/attributes that each solution consists of. We have that the first variable will correspond to the first coordinates. Continuing this fashion, the second coordinate corresponds to the second variable or attribute, et cetera. Using these n variables or attributes that we will predetermine, we get a plot of the solutions the n-dimensional plane.

So, now we have that all solutions are quantified in the n-dimensional plane. We now ask the user to fill in these attributes for their desired airport simply. What we mean by this is that the user fills in the desired / optimal cost for the solution, the desired / optimal range for the solution, et cetera. This will again result in a point in the n-dimensional plane.

After that, the decision rule is quite simple; we check the Euclidean distance between this point, which in fact represents the most optimal solution for the user and all the other 'solution points'. We then check for which solution point this distance is minimal. In practice, this solution should correspond to the solution that fits the demands and desires best of all the possible solutions. We decided that, instead of only simply giving the best solution, we would list all solutions, and rank them based on distance.

### Problems and Improvements to NearestNeighbour

There are some problems that come with the development of NearestNeighbour, but fortunately, they can be overcome quite easily. First of all, we need to define what NearestNeighbours should do in the case that two solutions have the same distance. If this is the case, we will simply pick one of the two at random to not unfairly prioritise any solution over the other.

Furthermore, as it stands of now, some variables are more important than others simply due to their scale in distance. For example, one unit difference in cost (euro) contributes equally as one unit difference in reliability (one percent). This would also mean that this decision model would pick a solution that is 10 euros cheaper than a solution that is 11% more reliable. In order to tackle this, we normalise all the attributes. Normalising means that all values will be multiplied such that the values are between zero and one. When an attribute is normalised, the lowest value will be zero, and the highest value will be one. Since we do not focus on the mathematical background, we do not discuss this normalisation in great detail. Further explanation can be found here [86].

Now, another problem is that now all attributes have an equal contribution. However, some attributes might be more important than others. In general, the cost is an attribute that has a higher weight than the attribute CO2 emission. Now, we can counter this by multiplying all normalised attributes by a certain predetermined weight. These weights can be determined with all stakeholders; another option is for the decision model to ask the user which variables he/she finds most important, and then base the weights on the user's preference.

### Strength of Nearest Neighbours

Now, one reason that we chose the Nearest Neighbours is that it is quite easy to grasp. In practice, you simply ask which values he/she would most preferably assign to the attributes of the anti-UAV system, and which attributes are most important. Then, based on that, all that remains is listing the existing solutions based on increasing distance to this point. Furthermore, this makes the decision model quite easy to implement, were we to pick this decision model. Lastly, more solutions can easily be added to this decision model. This can simply be done by adding this point to the n-dimensional plane as described before. Naturally, solutions can then also easily be removed from the decision model by removing this solution point from the aforementioned n-dimensional plane.

A voting advice application (VAA), vote compass, or election compass is not a traditional decision model. A VAA is often displayed as a web application that helps voters find a candidate or a party that stands closest to their preferences. They are a rather new phenomenon in modern election campaigning. The VAAs consider preparation stages and running stages. The preparation stage addresses issues that reflect the most important dimensions of political competition. Furthermore, a database of parties' or candidates' positions on these issues is considered. Then, a formula to calculate the proximity of voters' positions to the positions of the parties or candidates. During the running stage, voters express their views on the aforementioned policy issues. Then, the application provides a personalised voting recommendation for each user. Usually, the output is a ranked list of parties or candidates according to the calculated proximities.

The tendency to vote stimulates the use of VAAs, rather than the reverse, according to Ruusuvirta and Rosema [87]. During the Swiss federal elections in 2007, 16% claimed that VAAs had motivated them to participate in the elections. Another 25% reveal that they have been partially motivated [88]. There exist many popular VAAs as of now. Germany has the `Wahlomat' with 6.700.000 users, whereas Switzerland has the `Smartvote' with 938.403 users. Additionally, the Netherlands has the `Stemwijzer' with 1,5 million users, and the EU has the `EuProfiler' with 919.422 users.

### Stemwijzer Decision Model

The second decision model that we investigated, was the Dutch StemWijzer[89]. This decision model will for a large part be based on the Dutch StemWijzer. The Dutch Stemwijzer is a website that helps people find the political party that fits best with their point of view/opinion. The way this works is as follows, the website presents the user thirty propositions, to which the user can answer to agree, disagree or either be neutral. Then, the user can choose whether there are some subjects that he or she might find more important than the others. StemWijzer[89] scores each political party by counting the number of times the user and the party have the same point of view for a proposition, where the more important propositions give double the score if the point of view aligns. In the end, the score for each political party adds up, and the parties with the highest score fit best with the point of view of the user. This model is explained in greater detail in Implemented Decision Model.

# Decision Model

## Introduction

In this section, we will describe our decision model. First, a description of what a decision model actually is will be given, give a basic understanding of the concept. After that, we will explain what our decision model, in fact, does on a higher level, without all the details inside the decision model. After that, we will explain how the decision model is derived, and how our decision model works on a lower level.

## What is a decision model?

As described before, our decision model gives as output the best solution for anti-UAV systems based on the input of the user. This user can be, for example, an airport seeking to improve on its anti-UAV systems. Due to the enormous growing list of solutions for this, airports may find it difficult to decide for themselves. After our thorough analysis on solutions and types of airports, we have seen that some solutions fit certain airports better than others, and thus we decide to give a systemized model to consult users in this difficult choice.

## Decision Model Investigation

There are a lot of different types of decision models, and so before we implement a decision model, we decided to investigate some of the decision models that were available. Then, when we have done enough investigation, we will decide on the decision model that fits our approach best. The model that fits our description best is described in the sections below. The other models that we have investigated will be discussed in types of decision models.

## How does our decision model work?

The decision model that we will use will for a large part be based on the Dutch StemWijzer[89], which is a website that helps people find the political party that fits best with their point of view/opinion. The way this works is as follows, the website presents the user thirty propositions, to which the user can answer to agree, disagree or either be neutral. Then, the user can choose whether there are some subjects that he or she might find more important than the others. StemWijzer[89] scores each political party by counting the number of times the user and the party have the same point of view for a proposition, where the more critical propositions give double the score if the point of view aligns. In the end, the score for each political party adds up, and the parties with the highest score fit best with the point of view of the user.

The reason for choosing this type of decision model is that it is easy and straightforward for the users since for each question the user only needs to fill out whether he agrees, disagrees or is neutral. Furthermore, the user can decide whether he/she finds certain subjects more important than others. This is really useful in our situation, as airports might find specific attributes of a solution a lot more critical for their airport than others.

In our situation, where we would like to find an anti-UAV system that fits best with a particular airport, the decision model of StemWijzer needs a few minor changes in order to make it work. Instead of asking about the propositions of political parties, we will ask the user questions on the attributes a solution has and are based on the recommendation report. The attributes used for this, are explained in the next section. The questions asked on the attributes of the solutions will be based on the comparison of the attributes between the solutions. However, since a solution of an anti-UAV system can exist of three parts: detection, identification and neutralisation, questions will be asked on all three of these 'sub-solutions'. Users might also indicate if they want a complete solution including detection, identification, and neutralisation, or whether they do not need one or more of these `sub-solutions'. It might be the case that an airport might just only want a UAV detection system, in which case the questions on identification and neutralisation will be skipped, and only a `sub-solution' for detection will be given. Then, the user will be asked to indicate the attributes that are most important to the airport. The scoring of the solutions will work in the same way as StemWijzer calculates the score for political parties, and will be explained in more detail below.

## Goal

Let us take a closer look at the goals of this model. One of the goals of the model is to produce a result that takes the needs and beliefs of an individual as input and propose a solution against unwanted UAVs around airports based on this input. Furthermore, the user can indicate what issues weight more and that thus, are more critical. While this might seem like the goal of the model, it is most definitely not the only goal. With this model and the produced results, we also want to spark a debate when it comes to anti-UAV measures around airports. From our research, we know that many airports (still) do not have appropriate countermeasures against UAVs. These airports would only start looking for measures and solutions after an unwanted UAV threatens the airspace. This model directly goes against the passive behaviour we have seen so much from airports and promotes the discussion of suitable solutions for airports. Furthermore, as this type of model is primarily used for elections, it would be interesting to do further research on this type of model as it is still rather new. Extending this type of model to other fields than elections then could lead to exciting results.

## Attributes

As described above, we will create a decision model that airports can use to decide on which type of anti-UAV system to deploy. For this decision model, we have deconstructed the needs of the airports into particular attributes. These attributes are based on the analysis done on both the solutions and the airport analysis. We distinguished between three different types of airports and identified all the USE-stakeholders for each type. Furthermore, we did a risk analysis for each type of airport and stakeholder analysis. Using this stakeholder analysis, we were able to set up attributes that different airports are interested in. From these interests, we have derived core attributes. We will first summarise a list of these attributes to get a clear overview of what attributes are all taken into account when creating the decision model. It is, of course, hard to note down the internal processes that take place during the design of all of the attributes. This is why there is no clear section that explains this internal process, other than there being an excel sheet that contains all of the attributes against the solutions.

The list of current attributes is as follows:

• Range
• Speed of operation
• Disturbance to the environment
• Effect on different types of drones
• Scalability
• Number of drones it can concurrently handle
• Emission
• Size
• Identification
• Level of autonomy
• Power Outage risks
• Weather
• Uptime
• Portability
• Danger to humans
• Emission
• Destructivity
• Level of training needed

Note that this list is non-exhaustive and that the decision model will be made such extensions in this list can easily be implemented.

## Scoring the solutions

The next step is for the decision model to rank or score these attributes so that the decision model can link the outcome of the attributes to actual solutions. To score these solutions, multiple choice questions were used in the same way as StemWijzer[89] asks their users questions. An example of scoring the attributes based on the questions is as follows:

Q: "The budget for a UAV 'detection' system is 10.000 euro or less."

A:

• Agree
• Neutral
• Disagree

Based on this question, all detection solutions that cost less than 10.000 euro will obtain a point (or two depending on the weight, which will be explained in the next section), if the user answers 'Agree'. On the other hand, all detection solutions that cost more than 10.000 euro will obtain a point (or two) if the user answers 'Disagree'. If the user answers 'Neutral', then none of the solutions will obtain a point for this question. All these questions are justified, and all questions will be explained in greater detail (see section questions), so that each attribute can get a justified and well-calculated score. The main point of this example is to show how we are going to score solutions based on the questions that we ask.

## Weighing the attributes

Now that our decision model has calculated the score of each attribute concerning the preferences of the user, we must also appropriately weigh the attributes. In most cases, the emission does not contribute equally to the choice in solution as the safety of the solution, to give an example. We will weigh these attributes as follows: We ask the user to indicate the attributes that they find more important than others. For these attributes, we will double the score for a solution that aligned with the answer of the user.

## Translating the attributes to advised solutions

After the user finishing stating whether they agree, disagree or feel indifferent towards all propositions, an actual combination of solutions for each of the chosen sub-systems can be proposed. Each of the solution proposed under the section solutions will be grouped in either the `agree', `disagree', or `neutral' category for each proposition. We consider three broad categories for the propositions themselves, namely identification, detection, and neutralisation. Each of these categories considers propositions that relate to the essential attributes coined previously. Scoring the solutions for illegal drone activity will be done based on the answers that the individual using the decision model provides.

So, we now have a way of scoring and weighing the demands of airports based on the propositions given below. We also have a way to score the given solutions based on the attributes that we have deconstructed from the needs of the stakeholders. What now remains is linking the solutions for each category and the outcome of the prepositions together. For each proposition, if the user agrees, all solutions in the `agree' category for that proposition gain 1 point. If the user, however, disagrees, all solutions in the `disagree' category for that proposition gain 1 point. Furthermore, the user can also skip the proposition if they do not care about the attribute coined for that proposition. In the end, the user can also indicate which attributes are more important to them. All these attributes will gain a multiplier of 2. Additionally, the user can deselect solutions that they do not want to be taken into account during the final result presentation.

For example, let us consider a solution `x' for category `y' and the attributes: `cost', `scalability', and `safety'. Let us assume there is only a single proposition for each attribute. Let the user answer on the propositions such that solution `x' is the right solution for the attributes `cost' and `safety', but not on the `scalability'. Furthermore, the user has indicated that cost is more important than the other attributes. The final score the solution `x' then gets is: 2 (cost) + 0 (scalability) + 1 (safety) = 3. By scoring each of the solutions in this manner, we can, in the end, advise the solutions with the highest scores for each of the categories (detection, identification, and neutralisation) to fit best with the demands of the airport at stake. Note that these are not final decisions that the airport should blindly follow. Rather, we intend to provide a recommendation based on the needs of the airport.

Note that this decision model recommends a type of solution rather than an actual solution. That is, if one were to buy the `recommended' solution suggested by the decision model, this solution might not work. Preferably, the type of methods used for the various subcategories is given rather than an actual solution.

## Propositions

In this section, we consider propositions regarding each of the attributes coined earlier. Since we have three categories of solutions (detection, identification, and neutralisation).

The individual stating whether or not they agree with the propositions should be as least restrictive as possible. That is, one should only agree with certain propositions when they really need to place those restrictions upon the solution.

### How are the propositions made?

The procedure is as follows: in a brainstorm meeting with all group members, the most essential attributes regarding anti-UAV mechanisms are discussed. These have been found through exhaustive research of existing anti-UAV mechanisms and illegal UAV activity surrounding airports. Furthermore, exhaustive research of existing solutions and solutions that might be possible in the near future guides the solutions that are considered. Based on all this information, the group members make a list of around fifty statements. The statements are then considered from the point of view from each solution that is to be considered. From this point of view, it is possible to indicate whether they agree or disagree with the statement, or whether they take a neutral position. In particular, propositions about which the solutions clearly disagree on are included in the list of propositions. If there are statements that all solutions agree on, then they are dropped. After all, the individual filling in the list has nothing to choose from when all solutions are on the same side of the fence. Ultimately, a list of a maximum of thirty statements will be created that form the final list. Throughout this whole process, the group members strive to make the statements as clear and objective as possible.

See the image below for a visualisation of this process.

Visualisation of the design process of propositions.

### Scoring

Let us consider a bit more in-depth on how the scoring of each proposition goes. We use the method of `most similarities'. It works as follows: every statement where the user and solution give the same answer (e.g., agree, disagree, or neutral) counts as a point. Double points are given for the statements to which you assign extra weight, which is done at the end. All points are summed up at the end. The user can then observe what solution has the highest score. This solution having the highest score means that this solution had the same position as you the most often.

### Wrong solution?

We cover a maximum of thirty statements and, based on those statements, examine which solution you have the most agreement with. We try to focus on topics that are important and current when it comes to the technological development of UAVs and their countermeasures. It may, however, happen that you end up at one of the solutions that you did not expect nor agree with. When this happens, it means that, apparently, you have many similarities with that solution, apart from the reasons you initially had that caused you to be surprised.

When you arrive at a solution other than what you had expected, it is advisable to look carefully at which points there are similarities. Perhaps this makes you think about choosing a solution. Of course, however, there can also be very good reasons to go for a different solution. We only provide a tool to discover substantive differences between the solutions. It is always encouraged to keep thinking for yourself!

### Propositions

The propositions can be found in the `Categorising the detection solutions' section of the categorising page.

## Unevenness

If one were to perform 100, 1.000, or even 10.000 random walks through the proposed decision model, not all solutions would appear equally as likely. This phenomenon is due to the fundamental construction of the decision model. As the propositions are designed such that they really highlight the differences between all solutions, it is possible for a large percentage of solutions to either agree or disagree. This way, we often get uneven splits. That is, for example, 80% of all solutions `agree' with a proposition, whereas only 20% of all solutions `disagree' with a proposition. Then, we already observe that not all solutions are equally as likely to appear when it comes to filling in the complete decision model. It is even the case that there are quite a few solutions that are always more inclined to appear high on the ranking assuming that the user does assign extra weights to specific attributes. This `bias' is introduced because of a certain solution simply being `better' in most aspects than another solution. Does this then invalidate the other solution? No, it certainly does not. We would argue that it would be worth it to consider any solution that differs from (most) other solutions even if this is only regarding a single attribute. This different solution would then bring more variety to the decision model, which in turns result in more options that can be taken.

Thus, the uneven distribution of solutions into categories and specific solutions that have a `bias' has been done on purpose due to the fundamental construction of both the decision model and its propositions.

## Web App

We pack the decision model in a Web App for the convenience of the user. Information regarding this Web App can be found on the Web App page.

# Decision Model validation

## Introduction

When introducing a decision model, it is important to both validate and verify that decision model. This is especially important when it comes to computational models. When it comes to model verification, we ask ourselves the following question: `Does the model perform as intended?'. This question is asked in order to verify that, for example, the model has been programmed correctly. Furthermore, it verifies if the algorithm has been implemented properly and if the model does not contain errors, oversights, or bugs. We also have model validation. Here, we ask ourselves the following question: `Does the model represent and correctly reproduce the behaviors of the real world system?'. Validation ensures that the model meets its intended requirements in terms of the methods employed and the results obtained. The ultimate goal of model validation is to make the model useful in the sense that the model addresses the right problem, provides accurate information about the system being modeled, and to makes the model actually used[91].

## What now?

Unlike physical systems, for which there are well-established procedures for model validation, no such guidelines exist for social modeling. Unfortunately for the implemented decision model, there is no easy or clear way to validate and verify the model. This is mainly due to the model containing much subjectivity through human decision making. When users of the decision model use it, they have to provide input themselves. These inputs are not just numbers, but they are about whether or not the user agrees or disagrees with a proposition. This makes it hard to both validate and verify the model in a traditional way. In the case of models that contain elements of human decision making, validation becomes a matter of establishing credibility in the model. Verification and validation work together by removing barriers and objections to model use. The task is to establish an argument that the model produces sound insights and sound data based on a wide range of tests and criteria that `stand-in' for comparing model results to data from the real system[91]. This process is akin to developing a legal case in which a preponderance of evidence is compiled about why the model is a valid one for its purported use. In order to still do some verification, we use subject matter experts in order to gain a grasp of the credibility of the model. We implement ways to measure this credibility through evaluation and role-playing.

## Credibility

As coined earlier, we want to somehow make the credibility of the model tangible. We do this through evaluation and role-playing. A group of domain experts will do the evaluation. These domain experts consist of both the group working on this project and higher-ups that go over anti-drone mechanisms at Eindhoven Airport. We asked higher-ups at Eindhoven Airport that go over anti-drone mechanisms to spread the decision model questionnaire and have it be filled in by numerous individuals that all agree on the interests, needs, and characteristics of Eindhoven Airport. Furthermore, we ask for an initial solution that they think is the best from the list of all the solutions we forged. It is then interesting to see if these individuals get the same results for the decision model and if they agree with the decision model. Additionally, it is interesting to compare the initial solution they thought would be best for the recommended solution they got and what they think of the recommended solution. Are they surprised? Are they not surprised at all? Does the recommended solution provide new insights?

As we do not want to depend on a select few individuals from Eindhoven Airport alone, we also propose an example scenario where the user taking the questionnaire becomes a higher-up of a clearly defined airport that has to design a mechanism against unwanted UAVs. This is the role-playing method to establish credibility. This includes the needs, wants, and beliefs of this airport. We, internally, take this questionnaire as well. Afterward, we compare the initial thought of solutions, the recommended solutions, and the opinion of the recommended solution for the proposed airport.

## Methods

Let us consider the two methods coined earlier for testing the credibility of the decision model to a certain degree.

### Evaluation

Testing the credibility of the model through evaluation will be done, as briefly introduced earlier, by domain experts filling in a questionnaire that incorporates the decision model. We have sent a questionnaire to higher-ups at Eindhoven Airport that go over mechanisms to counter illegal drone activity around their airport. Additionally, we fill in this questionnaire ourselves from the perspective of Eindhoven Airport. This questionnaire first asks for the initial thought of the best solution from the list of solutions proposed. Then, the individual uses the decision model to obtain a recommended solution. Afterward, the opinion of the individual will be asked. Does the individual think this solution was to be expected? Does the solution make sense when holding it against the values and beliefs involved? What we are particularly interested in with this way of verification is seeing how much credibility we can give the recommended solutions based on the values and beliefs used for the input. We then collect all the information and analyse it by comparing the results provided to one another. This will then be used for assessing the credibility of the model.

The questionnaire we propose can be observed below.

#### Questionnaire

This file presents a questionnaire that takes into consideration questions that are used in the decision model. The goal of this decision model is to propose a solution for unwanted UAV presence around any type of airport. The primary goal of this questionnaire, that considers propositions, is to get feedback on the questions and the result of the model. This questionnaire is the basis of the decision model that we have implemented in order to recommend solutions against unwanted UAVs for stakeholders such as commercial airports and recreational airfields. Note that throughout this questionnaire, we use the point of view of Eindhoven Airport. That is, all propositions should be answered with the needs, wants, and ideals of Eindhoven Airport in mind. We address a multitude of propositions in the questionnaire, as well as provide context and motivation for these propositions. The motivation and context provided with each proposition are mainly for support and explanation of the proposition.

We have decided to split the questionnaire into propositions that consider the two main types of anti-UAV solutions, namely detection, and neutralisation. On the one hand, the propositions that consider a solution for detection only provides a means to alert the airport of the presence of a UAV. On the other hand, the propositions that consider a solution for neutralisation only provides a means to take down the UAV once detected. Note that this questionnaire only considers the first draft of propositions and that this might change later on.

For each proposition, the individual taking the questionnaire has to indicate to what extent they agree with the proposition. The options presented are `disagree’, `neutral’, and `agree’. The individual can indicate which option they choose by putting an `X’ in the respective cell. This system is used rather than a 5-point scale system as only an indication of what the solution has to offer is needed. Furthermore, it is incredibly complicated to divide solutions into various scales when compared to when considering two main groups.

This questionnaire also has a PDF-format, which can be found here.

General questions

We first consider some general questions in order to process this feedback to improve the current decision model and the questions involved.

• What do you personally think are the best solutions and why when it comes to detecting unwanted UAVs in the airspace?
• What do you personally think are the best solutions and why when it comes to neutralising unwanted UAVs in the airspace?
• How useful do you think a framework is that can give an indication on what kind of solution fits the needs, wants, and ideals of an airport. Note that this is not only meant for commercial airports, but also for recreational, and military ones.

Detection

1. I want to be advised on an anti-UAV detection solution

• Agree
• Neutral
• Disagree

Category: Need for a solution

Explanation: Because of the two different types of anti-UAV solutions, we decided to give the user the possibility only to pick one of either two types. Of course, it is still possible to be recommended for both types of solutions. This is done by agreeing to this proposition and the same proposition in the neutralisation questionnaire.

Motivation: Certain small airports may decide due to budget constraints only to invest in detecting solutions, and merely to wait for the unwanted UAV to go away. Furthermore, certain airports which already have a decent neutralisation solution and do not want to invest in that again may only opt for a detection system.

2. The detection system must be able to detect UAVs within a range of 4000 meters

• Agree
• Neutral
• Disagree

Category: Range

Explanation: The solution must work as described in the area inscribed by a circle with a radius of 4000m, centered at the detecting part of the solution.

Motivation: The range has an enormous influence on the cost of the solution, which the user most likely wants to minimize, while also having a proper solution. For small airports, there is no immediate need to have a solution that covers three times the area of the airport. For larger airports, a solution that only covers half of the area is also not a favourable option.

3. The detection system must detect illegal UAV presence within less than 1 second

• Agree
• Neutral
• Disagree

Category: Speed of Operation

Explanation: The time between the unwanted UAV entering the range of the anti-UAV solution, and the actual detection, must be less than one second.

Motivation: The timing of detecting unwanted UAVs can be crucial at certain airports where security is a top priority, such as military airports. However, for some airports, the timing must be done quickly, but not close to instant.

4. The detection system must not make any loud noises annoying people around the airport

• Agree
• Neutral
• Disagree

Category: Disturbance of the environment

Explanation: Certain solutions can emit a constant sound during operation, which could be an annoyance to people at or around the airport. Furthermore, some neutralisation solutions can also cause quite a loud noise when they are being operated.

Motivation: The annoyance of people can be a less crucial factor in very remote airports with few passengers, such as military bases. However, at large airports with lots of (easily frightened) passengers, one might refrain from solutions which make loud noises.

5. The detections system must be able to detect UAVs from all the categories(C1-C4)

• Agree
• Neutral
• Disagree

Category: Effect on Different Types of UAVs

Explanation: There are different types of commercial UAVs, ranging from C1 being very small UAVs, to C4 being large and heavy UAVs. Some solutions can be very effective on smaller UAVs, but the larger UAVs may require more costly solutions.

Motivation: Smaller recreational airports may decide only to be able to detect or neutralise smaller UAVs, since neutralising larger UAVs can result in more expensive solutions. If an airport concludes from investigations that they will most likely never encounter the larger C4 UAVs, then they can opt for a solution that only takes down the smaller UAVs.

6. The detection system must be able to scale with the growth of the airport in size

• Agree
• Neutral
• Disagree

Category: Scalability

Explanation: When an airport grows in terms of size due to economic prosperity, the solutions must be able to easily expand with the growing airport. Some detection solutions, for example, can be more easily scaled by adding another small subpart, whereas other solutions may require adding a whole new unit as if you have two systems.

Motivation: Some airports have already planned to grow and extend over the coming ten years. However, some airports have already reached their cap, meaning that they know that they will not scale up in the coming decade. For these airports, it is not wise to spend extra on solutions that have invested research into making their solutions more scalable.

7. The detection system must be able to detect multiple UAVs concurrently

• Agree
• Neutral
• Disagree

Category: Number of Drones it Can Handle

Explanation: Some solutions can handle multiple drones concurrently. On the other hand, some solutions (such as an aimed jammer), can only be aimed at one UAV. Then, only one UAV can be detected or neutralised at the same time.

Motivation: There are smaller airports that argue that the probability of two drones causing a disturbance at the same time is highly unlikely. Especially when saving costs, it might be wise to not spend extra money on more expensive solutions that can handle multiples UAVs concurrently.

8. The detection system must not emit any CO2

• Agree
• Neutral
• Disagree

Category: Emission

Explanation: Some solutions can be powered by fossil fuel, meaning that they emit CO2.

Motivation: The transition to green energy can be the main priority for airports, whereas the emission of CO2 can be of much less importance for other airports who care less about these regulations.

9. The detection system must not be larger than 1 m3

• Agree
• Neutral
• Disagree

Category: Size

Explanation: A solution is a physical object, which takes up a particular space. Some solutions are much more compact than other solutions.

Motivation: Some airports may be small and not have enough space to have specific solutions that take up too much space.

10. The detection system must be able to identify the UAV properly

• Agree
• Neutral
• Disagree

Category: Identification

Explanation: Regulated drones also emit an identification signal, from which for example the product code and links to the owner can be enclosed. This proposition states that the solution is able to not only detect but also identify drones that emit these identification signals.

Motivation: Although not all drones emit these signals, some airports may find it worth the cost to be able to identify these drones.

11. The detection system must be able to detect UAVs automatically without needing any human interaction

• Agree
• Neutral
• Disagree

Category: Level of Autonomy

Explanation: For specific solutions, a certain extent of human interaction is needed in order for the detection system to operate. This proposition puts a constraint of the detection system not requiring any form of human interaction.

Motivation: In some instances where 24/7 protection is needed, it might be useful not to need any human interaction when it comes to the services provided by the detection system. This is especially useful since human interaction only requires more effort that could potentially result in errors being introduced.

12. The detection system must be able to operate in the event of a power outage

• Agree
• Neutral
• Disagree

Category: Power Outage

Explanation: This proposition states that the detection system must be able to operate after there has been a power outage. This can be through various ways, such as the detection system making use of a battery.

Motivation: For some airports, it is vital that even after a power outage, the detection system still functions. It is, however, also possible that this is not a significant issue.

13. The detection system must be able to operate under any weather condition

• Agree
• Neutral
• Disagree

Category: Weather

Explanation: This proposition states that the detection system must be able to detect UAVs under any weather condition. This means that UAVs should be detected even when there are hazardous conditions.

Motivation: Some individuals might not want to put this constraint upon the solution as UAVs might not be able to fly under certain hazardous conditions.

14. The detection system must be able to operate 24/7 (assuming no outages, et cetera take place)

• Agree
• Neutral
• Disagree

Category: Time

Explanation: This proposition focuses on the solution providing 24/7 coverage when it comes to the detection of the UAVs in the airspace around the airport within a certain distance.

Motivation: For some airports, it might be essential that there is 24/7 coverage because there are flights 24/7. For other airports, this might not be as important as they do not consider flights 24/7.

15. The detection system must be able to detect UAVs at night

• Agree
• Neutral
• Disagree

Category: Time

Explanation: This proposition focuses on the constraint that UAVs should not merely be detected at daytime, but also at nighttime.

Motivation: Certain airfields (recreational) where only flights are active at certain times during a week with set hours are not as interested in solutions that provide their services 24/7. Then, for these instances, it is attractive to consider solutions that contain fewer constraints due to this relieving the costs of the solution.

16. The detection system must be able to be moved around instead of the solution being a `permanent’ installation

• Agree
• Neutral
• Disagree

Category: Portability

Explanation: An airport can have the preference of a solution being portable. With this, we mean that it is possible for this solution to be `picked up’ and deployed elsewhere. This results in the airport being able to deploy the solution almost anywhere in their area while not having to invest in a solution that covers the whole area by itself.

Motivation: Certain airports might not require a fully automated system that is active 24/7 due to financial constraints. Then, it is possible that they are interested in a less expensive solution that does not need to be active 24/7. Considering a portable solution is then an option. This solution can then be deployed when needed.

Neutralisation

1. The neutralisation system must be able to neutralize UAVs within a range of 1000m from the neutralisation system

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

2. The neutralisation system may neutralise unwanted UAVs within a few minutes rather than instantly

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

3. The neutralisation system must not pose any threat to humans, for example when a UAV falls from the sky after being neutralised

• Agree
• Neutral
• Disagree

Category: Danger to Humans

Explanation: Some solutions, such as lasers, damage a UAV mid-air, meaning that it will most likely fall to the ground. Other solutions, however, do not have this issue.

Motivation: Crowded airports may want to invest money in order to minimize the danger to humans. However, other airports where there are much less passengers, the risk is also lower and hence, airports may decide not to spend too much money on this.

4. The neutralisation system must not emit any CO2

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

5. The neutralisation system must be suitable to use in locations close to residential areas

• Agree
• Neutral
• Disagree

Category: Disturbance to the Environment

Explanation: Some solutions are less conservative than other solutions. For example, some solutions can cause great harm to others when misused, which is especially harmful when the airport is close to any residential areas.

Motivation: Some airports that are located in a crowded area might be looking for solutions that cause less danger to the immediate environment, whereas airports that are located in practically the middle of nowhere do not have to worry about this.

6. The neutralisation system must be able to neutralise non-commercial UAVs, those that might not be regulation conforming

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

7. The neutralisation system must be able to neutralise commercial UAVs

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

8. The neutralisation system must be easy to extend

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

9. The neutralisation system must be able to neutralise swarms of UAVs simultaneously, rather than only being able to deal with a single UAV at a time

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

10. The neutralisation system must be able to neutralise UAVs under any weather circumstance

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

11. The neutralisation system must be able to operate 24/7

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

12. The neutralisation system must be able to neutralise UAVs at night

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

13. The neutralisation system must be able to be moved around instead of the solution being a `permanent’ installation

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

14. The neutralisation system must be able to be used without training of the employees

• Agree
• Neutral
• Disagree

Category: Level of Training

Explanation: Some solutions are much more complex than others, and require a significant extra training course for the employees that operate these solutions. On the other hand, some other solutions are much easier to use.

Motivation: Smaller airports who do not want to invest in the extra training hours may want a solution that does not take a lot of training, especially when it is only one employee who needs to be trained. Furthermore, airports where there are a lot of part-time employees might suffer more from having to train all these people.

15. The neutralisation system must be able to operate in the event of a power outage

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

16. The neutralisation system must be able to neutralise UAVs without human input

• Agree
• Neutral
• Disagree

This proposition has been explained and motivated in the section for detection.

Closing questions

It is important to obtain feedback and to use this appropriately in order to improve the current decision model and its questions.

• What is your opinion on the different categories used for the propositions? Were they diverse enough or not at all? Is a certain category that you expected missing?
• What is your opinion on the propositions proposed? Were they diverse enough or not at all? Is a certain proposition that you expect missing?
• Other remarks

Thank you for filling in this questionnaire.

## Validation by Domain Experts

As described before, we have sent the questionnaire above to the higher-ups at Eindhoven Airport that have the responsibility of the anti-drone systems. In our correspondence, we were assured that if we sent the questionnaire, we would get the feedback only a couple of work days later. Thus, we did send the questionnaire to this group of domain experts. Unfortunately, we did not receive the feedback during the duration of the course, and thus we were not able to analyze the feedback of the domain experts. Although it is unfortunate, we did learn an important lesson that relying on external sources can be unpredictable at times.

## Internal Validation

So, in order to still provide a proper validation, we also did the validation internally, by all of the group members. Over the last eight weeks, we have done an extensive literature research on the matter, and thus we also consider ourselves as people who can validate the model. As described before, we would have also let domain experts at Eindhoven Airport help us with this, but unfortunately their promise was not met.

### Picking an airport to use for validation

So, as described, our approach was as follows. We picked a specific airport and filled in the questionnaire on their behalf. Since we do not know all the needed information for a certain existing airport in the Netherlands, we decided to come up with our own mock-up airport. We know all the important attributes, beliefs and wants of this airport, which allows us to fill in the questionnaire on their behalf. We used the information from our airport analysis to come up with this airport and the reasoning behind what to answer to which proposition. Examples of important attributes we took into account were among other the size of the airport and the amount of daily departures and arrivals.

### Filled in proposition on behalf of this airport

So, when we created our mock-up airport, we came up with these answers to the propositions, with corresponding motivation:

#### Detection

1. Q: I want to be advised on an anti-UAV detection solution.

A: Agree

Motivation: Because we as an airport are looking for a system that is able to detect UAVs.

2. Q: The detection system must be able to detect UAVs within a range of at least 4000m.

A: Disagree.

Motivation: Our airport is not that big; we only need a range of about 1000m, which is much less than that.

3. Q: The detection system must detect illegal UAV presence within 1 second.

A: Agree.

Motivation: One second can be critical when it comes to drone encounters, and since we prioritize safety, we are willing to spend more to get a solution that operates quickly.

4. Q: The detection system must not make any loud noises annoying people around the airport.

A: Agree.

Motivation: We do not want our passengers or people living nearby to get annoyed by our anti-UAV solution; we are willing to invest a little in order to have a less loud solution.

5. Q: Most detection systems make use of Radar techniques to detect unwanted UAVs, however, privately built drones can be made of materials such that they are not picked up by radar systems. Hence, the detection system should not only rely on Radar techniques for detection.

A: Agree.

Motivation: We foresee that when the technology of radar techniques will change, then all solutions that rely on this radar technique will be useless.

6. Q: The detection system must be able to scale with the growth of the airport in size.

A: Agree.

Motivation: We want and are able to expand more due to our location, and we have concrete plans to scale up our airport in the near future. Hence, we deem it necessary to make the solution scalable. We think this is so important that we will make this a critical proposition by checking the box mandatory property.

7. Q: The detection system must be able to detect multiple UAVs concurrently.

A: Disagree.

Motivation: We are an airport that do not see a lot of drone encounters and hence, we estimated that the probability of having multiple drone encounters concurrently is extremely small. Hence, we decide not to invest into this solution.

8. Q: The detection system must not emit any CO2.

A: Agree.

Motivation: With all airports being more environmentally friendly, we decide that we also want our airport to go in this direction.

9. Q: The detection system must fit in an area of 0.5m^3.

A: Neutral.

Motivation: We have plenty of space, so we do not really care about the size of the solution.

10. Q: The detection system must be able to properly identify the UAV.

A: Disagree.

Motivation: Only the commercial drones can be properly identified. We foresee that the ones that seek to do harm will not be able to be identified, so this attribute is not important to us.

11. Q: The detection system must be able to detect UAVs automatically.

A: Agree.

Motivation: We do not want to have someone permanently occupied by detecting UAVs. The whole reasons of such a system was to have this automized.

12: Q: The detection system must be able to operate in the event of a power outage

A: Disagree.

Motivation: In the event of a power outage, we will not fly our airplanes anyways and hence, we do not find it necessary to detect drones when there are no departures and arrivals.

13: Q: The detection system must be able to operate under any weather condition.

A: Agree.

Motivation: Even in bad weather, we might fly our airplanes and then we must most certainly have a detection system, especially when they are not easily visible.

14: Q: The detection system must be able to operate 24/7 (assuming no outages, et cetera take place).

A: Agree.

Motivation: We may have long days at the airport, and then we want the detection system to be working all the time.

15: Q: The detection system must be able to detect UAVs at night.

A: Agree.

Motivation: Although we do not have departing or arriving airplanes in the middle of the night, we certainly have airplanes departing or arriving very early in the morning, or very late in the evening. At those times, it is dark, and also then we want to be able to properly detect drones.

16: Q: The detection system must be able to be moved around instead of the solution being a `permanent’ installation.

A: Disagree.

Motivation: Since we are a small airport, we do not find it necessary to have the solution portable, as long as it has full range at its permanent installation spot.

#### Neutralization

17. Q: I want to be advised on a anti-UAV neutralisation solution.

A: Agree.

Motivation: We do not want to stand idly by when we have detected a drone. We also want to be able to neutralize it when we find it necessary.

18. Q: The neutralisation system must be able to neutralise UAVs within a range of at least 1000m from the neutralisation system.

A: Agree.

Motivation: The range of this is about the range of our airport that we want to be able to actively defend, so this fits our airport well.

19. Q: The neutralisation system may neutralise unwanted UAVs within a few minutes rather than instantly.

A: Disagree.

Motivation: We consider a few minutes too long to wait; if there is something we must be able to intervene quicker than a few minutes.

20. Q: The neutralisation system must not pose any threat to humans, for example when a drone falls from the sky after being neutralised.

A: Agree.

Motivation: We do not want people to get hurt by our anti-UAV solution.

21. Q: The detection system must not emit any CO2.

A: Agree.

Motivation: The same as for the detection system.

22. Q: The neutralisation system must be suitable to use in locations close to residential areas.

A: Disagree.

Motivation: Our airport is not located near residential areas, so we do not have to worry about that.

23. Q: Almost all commercially available UAVs use predictable Radio Frequencies for communication, however, the neutralisation system must also be able to neutralise UAVs (possibly privately built) that use other communication methods.

A: Agree.

Motivation: The same as for the detection system.

24. Q: The system must be able to neutralise a drone threat without destroying the drone in question.

A: Agree.

Motivation: In order to not create chaos, and to better find out who controlled the drone, we find it more convenient to be able to neutralize the drone without e.g. 'shooting it out of the sky'.

25. Q: The neutralisation system must be easy to extend.

A: Agree.

Motivation: As described before, we want to scale up our airport, and hence the neutralization system must be easy to extend.

26. Q: The neutralisation system must be able to neutralize swarms of drones simultaneously, rather than one at a time.

A: Disagree.

Motivation: The same reasoning as to why we do not deem it necessary to detect multiple drones concurrently.

27. Q: The neutralisation system must be able to neutralize drones under any weather circumstance.

A: Agree.

Motivation: The same as why the detection system must be able to operate under any weather circumstances.

28. Q: The neutralisation system must be able to operate 24/7.

A: Agree.

Motivation: The same as why the detection system must be able to operate 24/7.

29. Q: The neutralisation system must be able to neutralise UAVs at night.

A: Agree.

Motivation: The same as why the detection system must be able to detect UAVs at night.

30. Q: The neutralisation system must be able to be moved around instead of a permanent installation.

A: Agree.

Motivation: With highly agile UAVs, we also want to be able to move the neutralization installation, since we think this is more cost-effective.

31. Q: The neutralisation system must be able to be used without training of the employees.

A: Disagree.

Motivation: Although this is more desirable, we do not think it is possible to employ such a neutralization system without training some employees.

32. Q: The neutralisation system must be able to operate in the event of a power outage.

A: Disagree.

Motivation: The same as why the detection system should not necessarily be able to operate in the event of a power outage.

33. Q: The neutralisation system must be able to neutralize drones without human input.

A: Disagree.

Motivation: We are unsure whether it would be ethical and 100% reliable to take out drones without human input; we should at least have somebody responsible over the neutralization.

#### More important propositions

There are some propositions which we find more important than others, so in the last window we checked the following propositions to be more important to us:

Proposition 3: he detection system must detect illegal UAV presence within 1 second.

Motivation: We find it very important for the anti-UAV system to be quickly deployable.

Proposition 11: The detection system must be able to detect UAVs automatically.

Motivation: We think that the automization is very important, because else we would constantly have to place precious personnel on the detection system.

Proposition 13: The detection system must be able to operate under any weather condition.

Motivation: We would have a huge weakness if we could only neutralize UAVs in good weather, then we could not use them for a significant amount of the time.

Proposition 18: The neutralisation system must be able to neutralise UAVs within a range of at least 1000m from the neutralisation system.

Motivation: We must be able to reach all parts of of our airport, because else we would again have a significant weakness.

Proposition 20: The neutralisation system must not pose any threat to humans, for example when a drone falls from the sky after being neutralised.

Motivation: We want to ensure the safety of everybody at the airport, and when there can be UAVs falling down, we would consider this harmful for the passengers and our reputation.

Proposition 25: The neutralisation system must be easy to extend.

Motivation: Analogous to the scalability of the airport; since we want to extend the airport in the near future, the extendability of the neutralisation system is of great importance.

Proposition 27: The neutralisation system must be able to neutralize drones under any weather circumstance.

Motivation: See motivation of proposition 13.

### Outcome of the decision model for this airport

After filling the questionnaire, we filled the results of this questionnaire into our decision model. As expected, the decision model gave as output a list of anti-UAV solutions, together with a percentage score. These were the results:

Figure 1: Picture of the best solutions for this airport from the decision model.

As you can see, we have only listed the best seven solutions. We did this because the other solutions had a score lower than this, and thus we would argue that including and explaining all possible solutions, even those with a low score, would be superfluous for the validation. We can see that we have seven solutions, each with a proper matching percentage, ranging from 76.9% to 66.7%.

If any reader would be interested in reproducing these results, one could go to drones.jortdebokx.nl (assuming it is still online) and fill in the propositions as we described in the section above. Alternatively, one could also try out some propositions, or could do a similiar approach as we did in this validation test.

### Discussion of the results for this airport

Now, we discussed with all the group members whether these outcomes did make sense. It was interesting to see that there are no solutions that have a higher matching percentage than 77% with our mock-up airport. However, after looking through all the solutions that we have gathered, we indeed concluded that these top 7 solutions would be the best fit for our airport. So apparently there is no solution in the list that almost perfectly matches the expectations of the mock-up airport. After close investigation of the highest matched solution attributes against the needs of our mock-up airport, we indeed see that the top solutions mostly cover the airport's needs for a large part. First and foremost were the arguments that all these solutions were easily scalable and all had a neutralization range of 1000m. These were the some of the most important aspects for this airport and all solutions with a matching higher than 60% also agreed to these specifications. We could also cover all other propositions and compare these with the attributes of the solutions but that would simply not be very interesting and also would only just validate the logic behind the code of the web app.

What is also good to see is that some solutions that we included, of which we knew would not be ideal solutions, were not included in the top matches. For example, we also included human sight as a detection system, or eagles as a neutralization system, but all these solutions had a very low match compared to the others. For example, human sight only had a match of 20.5%, which makes sense since this is not a very good solution in the first place. This is also not a solution that any airport would not have come up with themselves. We did, however, include these solutions to give as much freedom for the users, and also to validate that solutions which are less good are not picked that often. As well as giving the users a comparison between such simple solutions and more sophisticated solutions, such that the users can decide for themselves whether these differences are worth investigating in. After all, good solutions having a good match is important, but it is also important that solutions that are not so good also get a lower match rating. After performing our internal validation, we indeed found this to be the case. It was unfortunate that our decision model could not be validated by external users, however, we hope that this validation, whilst far from perfect, gives some impression of the performance/accuracy of the decision model.

### Conclusion of the validation for this airport

We conclude that indeed the decision model passed this validation test. We saw that indeed all solutions that did not match the mandatory property of being easy to extend successully filtered out all solutions that did not match this criteria. Furthermore, we also saw that we indeed got solutions with a proper matching percentage, with some solutions being well over a 70% match. Furthermore, we also saw that it did not happen that a lot of solutions got the same score, which was something we were slightly worried about in the beginning.

Furthermore, considering the solutions with a high match, we also concluded that these would fit the airport very well, as the specifications of those solutions seemed to allign very well with the beliefs and wants of our airport. With respect to the solutions that got a very low score, we saw that all these were indeed solutions that were either not a very good idea for any airport, or solutions that simply were not a good match with our airport in question. From this, we conclude that the decision model passed this validation test.

# Categorising solutions

Now that we have set up the propositions for the decision model, it is also important to link solutions to those propositions. In other words, decide which solutions 'agree' or 'disagree' with the propositions, to be able to give them points in the decision model as elaborated on before.

## Concrete solutions

In this section, we provide a non-exhauste list of solutions. This list of solutions can easily be extended by us or by future groups that want to continue with our work.

Table 1: Concrete solutions
ID Name of solution Link to solution
1 SKYWALL https://openworksengineering.com/skywall-100/
5 NetGun https://www.dronedefence.co.uk/products/netgun-x1/
8 Raytheon https://www.raytheon.com/capabilities/products/lasers
9 Repulse24 https://www.repulsedrones.com/products-repulse-24.php
10 HEL https://www.rheinmetall-defence.com/en/rheinmetall_defence/...
11 SkyPatriot http://www.rinicom.com/products/drone-detection/sky-patriot
12 FalconShield http://www.us.selex-es.com/documents/6283866/64370908/body_Falcon_Shield_LQ_mm08605_.pdf
13 SkySec https://www.skysec.ch
15 Fuyuda Portable Counter Drone Defense System http://fuyuda.en.made-in-china.com/product/GMOQKwpYbNAa/...
16 HiGH+MiGHTY SKYNET http://anti-drones.net/
17 Hikvision Defender Series http://www.hikrobotics.com/en/visioninfo.aspx?k1=1&k2=8&k3=37&k4=83&id=86
18 HP Marketing and Consulting HP47 http://www.hp-jammer.de/c-uav.php
19 IXI Technology Drone Killer http://ixitech.com/products/drone-killer/
20 Jiun An Technology Raysen MD1 http://www.jiunan.com.tw/en/defense/MulticopterDefender.html
21 Kalashnikov REX1 https://www.sofmag.com/russias-kalashnikovs-rex-1-counter-uav-rifle-a-man-portable-drone-jammer/
24 LocMas Stupor http://mil.today/2017/Science9/
25 MC-CLIC anti uav rifle http://www.monacolife.net/more-details-emerge-of-made-in-monaco-anti-drone-rifle/
26 Mitsubishi Drone Detterence System http://edm01.ubmthailand.com/2017/DEF2017/ShowDaily/Day4/ShowDailyADJ_Day4.pdf
29 ctrl+sky http://apsystems.tech/en/
32 İHTAR https://www.aselsan.com.tr/en-us/press-room/Brochures/Air-and-Missile-Defense-Systems/IHTAR_ENG.pdf
33 Gamekeeper 16U http://www.aveillant.com/files/2514/7671/3392/v0.9_Gamekeeper_16U...
34 Battelle DroneDefender https://www.battelle.org/government-offerings/national-security/aerospace-systems/...
35 AUDS Anti-UAV http://www.blighter.com/images/pdfs/fact-sheets/auds-fact-sheet.pdf
36 Boeing MEHEL https://breakingdefense.com/2016/10/stryker-mehel-general-dynamics-at-ausa-video/
37 Bss UAV blocker http://www.bssholland.com/product/counter-wifi-uav-solution-drone-blocker/
38 Eagles Solutions gathered by us from different webpages
39 Human sight Solutions gathered by us from different webpages
40 Geofencing Solutions gathered by us from different webpages
41 Missiles Solutions gathered by us from different webpages
42 Other UAV with a net Solutions gathered by us from different webpages

## Categorising the detection solutions

In this section, we categorise the detection solutions for each proposition regarding detection. That is, we map the solution identification numbers to either `agree' or `disagree' for each proposition. Mapping these solutions to a certain category was done internally based on the comparison charts that compared all attributes to each solution.

Table 2: Detection solutions categorised
Proposition ID of solutions that Agree ID of solutions that Disagree
I want to be advised on a anti-UAV detection solution 2,4,6,11,12,26,28,29,31,32,33,35,37,39,40
The detection system must be able to detect UAVs within a range of at least 4000m 4,6,13,28,32,33,35,37 2,11,12,26,29,31,39,40
The detection system must detect illegal UAV presence within 1 second 2,4,6,12,26,28,29,31,32,33,35,37,40 11,39
The detection system must not make any loud noises annoying people at / around the airport 2,4,6,11,12,26,28,29,31,32,33,35,37,39 40
Most detection systems make use of Radar techniques to detect unwanted UAVs, however, privately built drones can be made of materials such that they are not picked up by radar systems. Hence, the detection system should not only rely on Radar techniques for detection 2,11,12,29,31,32,35,39 4,6,26,28,33,37,40
The detection system must be able to scale with the growth of the airport in size 2,4,6,11,12,26,28,29,31,32,33,35,37,39 40
The detection system must not make any loud noises annoying people at / around the airport 2,4,6,11,12,26,28,29,31,32,33,35,37,39 40
The detection system must be able to detect multiple drones concurrently 2,4,6,11,12,26,28,29,31,32,33,35,37,40 39
The detection system must not emit any CO2 2,4,6,11,12,26,28,29,31,32,33,35,37,39,40
The detection system must fit in an area of 0.5m^3 4,11,37 2,6,12,26,28,29,31,32,33,35,39,40
The detection system must be able to properly identify the UAV 2,4,6,11,12,28,29,31,33,35,37 26,32,39,40
The detection system must be able to detect UAVs automatically 2,4,6,11,12,26,28,29,31,32,33,35,37,40 39
The detection system must be able to operate in the event of a power outage 12,29,31,32,37,39 2,4,6,11,26,28,33,35,40
The detection system must be able to operate under any weather condition 2,4,12,28,29,31,32,33,35,40 6,11,26,37,39
The detection system must be able to operate 24/7 2,4,6,11,12,26,28,29,31,33,35,37,40 39
The detection system must be able to detect UAVs at night 2,4,6,12,26,28,29,31,32,33,35,37,40 11,39
The detection system must be portable 2,6,26,29,31,32,37,39 4,12,28,33,35,40

## Categorising the neutralisation solutions

In this section, we categorise the neutralisation solutions for each proposition regarding detection. That is, we map the solution identification numbers to either `agree' or `disagree' for each proposition. Mapping these solutions to a certain category was done internally based on the comparison charts that compared all attributes to each solution.

Table 3: Neutralisation solutions categorised
Proposition ID of solutions that Agree ID of solutions that Disagree
I want to be advised on a anti-UAV neutralisation solution 1,2,4,5,7,9,10,12,13,15,16,17,

18,19,20,21,22,23,24,25,26,30,31, 34,35,36,37,38,40,41,42,

The neutralisation system must be able to neutralise UAVs within a range of at least 1000m from the neutralisation system 2,4,9,10,13,15,16,20,21,

22,23,25,26,31,36,37

1,5,7,12,17,18,19,24,30,

34,35,38,40,41,42

The neutralisation system may neutralise unwanted UAVs within a few minutes rather than instantly 2,4,7,9,10,12,15,16,17,18,19,20,21,22,23

24,25,26,30,31,34,35,36,37,38,40,41,42

1,5,13
The neutralisation system must not pose any threat to humans, for example when a drone falls from the sky after being neutralised 2,4,7,9,12,13,15,16,17,18,19,20,

21,22,23,24,25,26,31,34,35,37,40

1,5,10,30,36,38,41,42
The detection system must not emit any CO2 1,2,4,5,7,9,10,12,13,15,16,17,18,19,20,21

22,23,24,25,26,30,31,34,36,37,38,40

41
The neutralisation system must be suitable to use in locations close to residential areas 1,2,4,5,7,9,12,13,15,16,17,18,19,

20,21,22,23,24,25,26,31,34,35,38,40,42

10,30,36,37,41
Almost all commercially available UAVs use predictable Radio Frequencies for communication, however, the neutralisation system must also be able to neutralise UAVs (possibly privately built) that use other communication methods 1,5,9,10,13,30,36,37,38,41,42 2,4,7,12,15,16,17,18,19,20,21,

22,23,24,25,26,31,34,40

The system must be able to neutralise a drone threat without destroying the drone in question. 1,2,4,5,7,9,12,13,15,16,17,18,19,20,21,22,

23,24,25,26,31,34,37,38

10,30,36,40,41,42
The neutralisation system must be easy to extend 1,2,4,5,7,9,10,12,13,15,16,17,18,19,20,21,

22,23,24,25,26,30,31,34,35,37,41,42

36,38,40
The neutralisation system must be able to neutralize swarms of drones simultaneously, rather than one at a time 2,4,6,9,11,12,13,15,23,26,31,35,37,40 1,5,7,10,16,17,18,19,20,21,22,24,

25,30,34,36,38,41,42

The neutralisation system must be able to neutralize drones under any weather circumstance 2,4,9,10,12,15,23,26,30,31,35,40 1,5,7,13,16,17,18,19,20,21,22,24,

25,34,36,37,38,41,42

The neutralisation system must be able to operate 24/7 2,4,7,12,15,17,23,26,30,31,

35,37,40,41,42

1,5,9,10,13,16,18,19,20,

21,22,24,25,34,36,38

The neutralisation system must be able to neutralise UAVs at night 1,2,4,5,7,9,10,12,15,16,17,18,19,20,21

22,23,24,25,30,31,34,35,36,37,38,40,41,42

13
The neutralisation system must be able to be moved around instead of a permanent installation 1,2,4,5,7,9,10,15,16,17,18,19,20,21,

22,24,25,26,32,34,36,37,38,42

12,13,23,30,31,35,40,41
The neutralisation system must be able to be used without training of the employees 2,4,7,9,12,13,31,40 1,5,10,15,16,17,18,19,20,21,22,23,24,25

26,30,31,34,35,36,37,38,41,42

The neutralisation system must be able to operate in the event of a power outage. 1,5,7,9,10,12,13,15,16,17,18,19,20,21,

22,24,25,26,30,31,34,36,38,42

2,4,23,35,37,40,41
The neutralisation system must be able to neutralize drones without human input 2,4,9,10,12,13,15,23,26,31,37,40 1,5,7,16,17,18,19,20,21,22,24,25,

30,34,35,36,38,41,42

# Web Application

Now that we have a decision model in place, it is useful to be able to display/represent it in some way. By doing so, we will be able to validate, test and actually make use of the decision model. This also gives a clear overview of how the decision model is supposed to work. The first step is to find a suitable method to represent the decision model.

## How to implement?

There are various methods of implementing a decision model. The decision model could be implemented using an existing survey/form website, i.e. using google forms[92], SurveyMonkey[93] or another similar website. This method would probably be easiest and the least time consuming, however, those websites are mostly not very flexible and do not adhere to our needs. Such form website mostly offers users to fill in answers to questions, but do not follow up with a calculation of a score and be able to give advice for solutions. Furthermore, some of these websites even restrict 'free' users to a limited number of questions to be asked.

Another option would be to create a Web application, that we can modify to our exact needs. We decided that a client-side application built using Vue.js and some bootstrap would allow for fast development, scalability, portability and can easily be tailored exactly to our needs. The development was done on GitHub, allowing us to host our result on GitHub pages. For the design of our app we took inspiration from the Dutch "Stemwijzer"[94], and an open source voting site called electioncalculator.org[95].

The site displays a series of questions, to which the user can agree, disagree or remain neutral. In addition, the user can also select some questions as being "mandatory". When a question is selected as mandatory, only solutions that follow the users preference for that question will be taken into account when computing the final solution. After having answered all questions, we ask the user to identify solutions that are extra important to him. To these solutions, the app adds an increased weight. After all questions have been answered, the scores and relative % match are computed. The results page shows a sorted list of solutions, together with a description of the solution.

## What it looks like

The voting app is hosted on https://drones.jortdebokx.nl/. Here are some screenshots:

Figure 1: Homepage of the web app.
Figure 2: Questions of the web app.
Figure 3: Extra-important subjects that the user can select of the web app.
Figure 4: Results of the web app.

# Future

## Introduction

The estimated investment in drone hardware when considering the government, the consumer, and enterprise (U.S.A).

It is essential to consider the future when it comes to anti-drone measures around airports in this ever-changing world. The investment in drone hardware is expected to rise even more as can be seen in the figure above. This does include not only the investment in military drones but also commercial Drones. Now that the technology is growing exponentially, it is hard to say what the future of drones will look like.

Altair Aerial creates easy-to-fly drones in the hopes of getting more people into the market with low-cost beginner-friendly UAV technology. Another company, EHANG, wants to turn drones into a taxi service. They have built a quadcopter capable of carrying passengers. It is clear that the technology when it comes to drones will keep increasing exponentially until we hit a certain point, where it is still unknown when this certain point will be hit. The only thing that is, however, clear, is that the non-existing to little countermeasures that exist around airports against drones is not sufficient for the current situation nor will it be for the future if we do not take a critical look at the underlying issue and tackle it.

## Laws

When we consider the laws regarding drones usage in The Netherlands, we observe that we can start considering it from an EU perspective as new laws and regulations are being made. We argue that it is a good thing that these laws are made on higher levels rather than on the level of individual countries as we think there should be some standard when it comes to drones. We think this as the drones pose the same kind of risk when it comes to their usage around airports. The country this happens it does matter little when we consider it from a more general perspective.

## Decision Model

When it comes to the decision model we used, we think that there are still many unexplored areas. Primarily as the decision model used (VAA) is still relatively new. We think it is possible also to consider other types of decision models. We could argue that using different decision models becomes more complex as then, in general, one proposed solution is the final result. It is extremely tough to differentiate between all possible solutions based on their attributes adequately. Thus, a ranking is more straightforward and, in our opinion, also more useful.

## Autonomous UAVs

Many of the current solutions against unwanted UAVs rely on jamming their radio signal. The radio signal is used by a human to control the UAV from a distance. This means that hijacking the radio signal results in taking over the control of the UAV. However, what if the UAV does not use radio signals? Then many of the solutions are rendered useless, and the UAV can fly without any interference from the perspective of the airport. In the near future, we expect that the ability for UAVs to fly without human control will rise. This can be for example by preprogramming a path of the UAV. Another example comes with the rise of Artificial Intelligence: drones flying autonomously. With the use of sensors, a drone could be able to fly all by itself. In this case, again, radio signals are unnecessary, and thus all solutions that exploit the UAV's use of radio signals will be useless. In fact, there are already multiple drones on the market that utilise autonomous flying[96]. This is a shift that we expect to happen in the near future and can be abused by people with malicious intentions.

## Other groups from the Robots Everywhere course

We thought it would be interesting to consider the work of previous groups and current groups when it comes to drone interception. Unfortunately, we were only able to find a single group that tackled this issue. The group found is group 9 of the current semester and year (PRE2018 3).

The overall goal of the project proposed by group 9 is to deliver a prototype and model on how an interceptive drone can be implemented. They started by researching their chosen project. This was followed by them analysing the USE aspects and determining the requirements of their system. Then, they considered multiple design strategy and chose the hardware and created a UML diagram. This was followed by working on a prototype (3D model and mobile application). Then, they evaluated the prototype.

Group 9 decided to design a drone that could take down other drones. They look at how to detect the intruder, how to target it, and how to assure that it has been captured. Furthermore, they provide a dashboard app, which displays the real-time system info and provides critical controls.

The first thing we notice based on the project proposed by group 9 is that there are no precise requirements as to what their solution should offer. It is mentioned that drones should be neutralised, but it is not mentioned if this should be done in a way such that no damage can be done to third parties or anything related. Furthermore, the dashboard app (only offered in a mobile form as of writing this (5th of March, 2019)) makes it possible to send key commands to the drone fleet. This raises some alarms from our side when we consider this drone actually being used in real life as jamming or hijacking the communication between the drone and the app does not prove to be the most difficult task. They argue that their application will also communicate with the interceptor drone (or drones if multiple are deployed) in real time, thus making the whole operation of intercepting and catching an intruder drone much faster. While the idea of amazing, we think this is exceptionally insecure as no security measures have been proposed by the group. We are afraid that the communication between the app and the fleet of drones can be easily taken down, or in the worst case, be hijacked and used for malicious purposes.

# Conclusion

## Introduction

In the conclusion, we want to look back at important aspects of the project and discuss our most exciting findings and results. Where we look at both the literature research we did on unwanted UAV presence at and around airports, as well as the decision model we have created. We try to give a short summary and report on all the things learned while working on this project.

## Literature Research

Looking back at the problem where last few years multiple airports were forced to cancel hundreds or thousands of flights affecting many travellers due to sightings of UAVs at airports. Not only does this negatively affect the passengers but this also costs the airports itself enormous amounts of money. However, from our literature research and interviews with multiple (Dutch) airports, we have learned that many airports have nothing in place to quickly deal with these UAV sightings, let alone have something in place against potential attacks with (weaponised) UAVs. Since we see the increase in the number of consumer UAVs growing each year, this problem seems to become more and more critical. Furthermore, from the interviews, we know that many of these airports without any defences would like to invest in a solution against unwanted UAV presence.

There are many different potential types of solutions against UAVs at airports that can detect, identify/classify, track or neutralise unwanted UAVs. There already even exist multiple concrete systems created by many different companies using a variety of technologies that could be invested in and deployed at these airports. However, the problem is that there are multiple differences between each of the solutions, and finding the best solutions for your airport is extremely difficult. This is where our research and decision model comes into play. We have thoroughly researched many of these solutions and systems as well as many different airports and types of airports in the Netherlands. After researching the differences between all of these systems, the differences between all of the airports and the wants of the stakeholders of the airports, we have come up with a list of attributes of the solutions which are to be to create a decision model.

## Decision Model

We have brainstormed and researched many different types of decision models, and found a voting advice application (VAA) such as the Dutch Stemwijzer, to most closely represent the model we would like to create. Where, instead of advising on political parties by using statements based on issues, we give advice on the best anti-UAV system for your airport based on propositions based on attributes of anti-UAV systems and the airport itself. For setting up these propositions, we used the attributes found in the literature research, to create an enormous table of solutions against the attributes. Then, we formed the propositions for the decision model based on the differences between the attributes of the solutions. In the decision model, we wanted to give the users of the model (airports) some freedom to indicate on attributes that they might find more interesting, or indicate attributes that the solution must necessarily have. Hence we also give the users both of these options. Furthermore, we give the users the option to decide on whether to be advice on detection only solutions, neutralisation only solutions or both. We feel like, with all of these features combined with the exhaustive research and validation of the model, the decision model is a useful anti-UAV decision tool for airports.

To make the decision model more tangible, more accessible to test, we decided to create a web-based application to implement the decision model. This decision model web-app asks the user all of the propositions, where more important attributes and must-haves can be indicated and calculates a score for each of the solutions based on the answers of the user. These scores then indicate which solution fits best with the airport and for the convenience of the user also the percentage of how well a solution matches with the airport is indicated. Lastly, the user can also find descriptions of the advanced solutions on the web-app. For the next couple of weeks, this web-app will be hosted online on the website: https://drones.jortdebokx.nl/.

# Discussion

## Introduction

In the discussion, we want to touch upon some important aspects of the project. We take a step back from the content of the project itself and take a closer look at how the process went and what we can learn from these developments. It is not only the development of the process that can provide useful insights, but also the stumbling blocks encountered on the road towards the final result can provide useful insights.

## Process and limitations

The very first thing of the project, namely picking a topic and what to exactly do with that topic proved to be the most difficult part. In the current day and age, where there are millions of exciting topics when it comes to robotics, it is tough to limit oneself to a single topic. While deciding on a rough draft of what to research was still quite doable and only took a few days, explicitly formulating the deliverables was much harder. The first week consisted of doing a literature study regarding drone interceptions, documented in the State of the Art. While meeting with the professors of 0LAUK0 during the first weekly meeting, it quickly became clear that researching `drone interceptions' was not explicit enough and way too broad. This issue resulted in us focusing more on drone interceptions and illegal drone activity surrounding airports in The Netherlands. Choosing all types of airports (commercial airports, airlines, et cetera) was done on purpose, and it was thought if this would be too broad, but everyone thought this would still be feasible.

Now that the research space was limited to illegal drone activity around airports in The Netherlands, the next steps could be taken. What would the deliverables look like? At first, a literature study seemed like the preferable option. We, however, thought this would be too short for the time we had for the course and thus decided to also design a decision model that airports in The Netherlands could use to decide what their `best' solution against illegal drone activity was. This overabundance of time quickly resulted in the group wanting to produce a recommendation report for airports located in The Netherlands.

Researching what types of decision models exist and which one we could use also proved to be quite tricky. There exist numerous types of decision models available, and therefore, it was rather hard to pick one based on the things we wanted to achieve. We thought we found the right type of decision model to use, but after working out the questions, we found that this did not work as well as we thought. Thus, we had to change the type of decision model we used. This change leads us to a different model, which worked significantly better for the idea we had in mind. It was still tough to design propositions based on the attributes of various solutions as most solutions had the same attributes. In parallel, we started working on a Web App that implemented this decision model. When it comes to communication during this process, it was rather rough. It being rough was partly due to planning not stating detailed deliverables for each week. We did try to incorporate the `delivering deliverables each week', but the deliverables were not explicit enough until we realised this issue. After this issue was brought up, it was tackled appropriately. From there on out, things went better and better. No real issues were encountered afterwards.

## Main takeaways

The main takeaways from this project that should definitely be kept in mind in future projects are as follows:

• Make the planning extremely detailed. Note down who should do what and when it should be finished. Make sure that the items in this planning are formulated as detailed deliverables rather than letting them be abstract ideas.
• Verify the work of others against the deliverables described in the planning. Furthermore, talk about these things if they are not as they should be.
• Meeting up together to work on certain aspects can be useful, but this should be minimised as much as possible. Using a planning with detailed deliverables and not more than two people on a certain item in the planning is preferable.

## Objectives

Let us now consider all of the initially defined objectives in the project setup and if/how we met these.

• Gaining insight into accidents and incidents involving various forms of drones.

We most definitely obtained insights into accidents and incidents involving various forms of drones in the `State of the Art' section. In short, we found that this number was raising and raising together with the number of customers of drones. Therefore, we can argue that we meet this objective.

• Identify and specify the currently existing countermeasures and counter mechanisms against drones and UAVs in general.

We researched existing countermeasures against UAVs through literature research and simply looking at the market. We e-mailed various airports and obtained some responses. From the responses and the literature research, we observed that there were still little to no countermeasures against unwanted UAVs in place. Therefore, we can state that we meet this objective.

• Identify and specify the USE stakeholders of the problem space and their interests regarding possible solutions.

The USE stakeholders of the problem space were analysed in-depth. That is, we considered multiple types of airports, namely commercial airports, recreational airfield, and military airbases. We provided an in-depth analysis of all of these regarding the issues they are specifically facing, a risk analysis, what a solution should provide for them, and what their characteristics are. Therefore, we can argue that we meet this objective.

• Propose multiple possible solutions to the problem.

We did extensive research on solutions in two (or three) categories when it comes to solutions against unwanted UAVs. These categories are detection (and identification) and neutralisation. We offer many different types of solutions and also many specific solutions from companies that would solve part of the problem. Then, a combination of these solutions can be used as a `final' solution. Therefore, we can state that we meet this objective.

• Identify the advantages and the disadvantages centred around user interests for each provided solution.

After collecting all solutions, we listed their advantages and disadvantages through filling in all of their attributes and observing how they performed against one another. Therefore, we can argue that we meet this objective.

• Validate and verify that our proposed solutions solve the discussed problems with respect to the USE stakeholders and their interests.

When noting down each solution, it was heavily thought about if they would actually solve either the detection or neutralisation issue. This has also been described when it comes to all of the solutions. Therefore, we can state that we meet this objective.

• Design a basic decision model around providing solutions for airports against UAVs.

A basic decision model was designed. The type of this design model was a VAA, and it was heavily inspired by the Stemwijzer while making some improvements such as implementing the MoSCoW model in order to make the model fit our situation. Therefore, we can argue that we meet this objective.

Now, as we have met all objectives, we can conclude that we achieved all of the initially defined objectives successfully.

## Deliverables

Let us now consider if the proposed deliverables as defined in the project setup have been finished and delivered.

• A presentation regarding the problem and possible solutions

We have made a presentation regarding the problem description, our literature research, and the decision model we have made that incorporates all the solutions we found. Therefore, we can state that we finished this deliverable.

• A literature study in the form of coherent Wiki pages in a hierarchical manner

The Wiki page of our group functions as the literature study. This literature study is available in two manners, namely a hierarchical structure of Wiki pages and a single page that incorporates the whole study. This latter was done as an extra based on the behaviour of many other groups. Therefore, we can argue that we finished this deliverable.

• A Web App that implements a decision model

A Web App that implements the decision model has been finished. This Web App can be observed in the `Web App' section. Therefore, we can argue that we finished this deliverable.

## Future development

Most of the areas for future development have been coined on the Future page.

# Notes

We present notes that were taken during the meeting with the professors.

## Week 2

• We should focus on a certain context rather than considering the problem in general.
• Consider when something is allowed and when it is not allowed with respect to drones around airports.
• Look at a debate Royakkers attended regarding rules and the regulation of drones.
• Look at the guidelines that should be set in stone on the first of March in the Netherlands.

## Week 3

• For the assignment, we have to have a clear idea of the product we want to deliver, in our case for airports. We have decided to create a decision model for airports to be able to choose a possible solution based on certain characteristics of the airports and dependent on the budget for investment in the technologies.
• Look at/investigate the information/decisions that need to be used in the decision model. A possible way of doing this is to look at the difference between the airports of the Netherlands and look at how this difference could affect the chosen solution.
• Get requirements of airports to help get a clear vision of the needs of airports for the solution, so contact Dutch airports.
• Do a cost analysis, on basis of Gatwick airport attack, to check how much airports want to invest on the solutions.
• For the report, do the following:
• Research literature on how to create a decision model.
• A more concise and clear list of existing solutions.
• Expand the list of advantages and disadvantages of the solutions on the basis of demands of airports, such that the solution of use of eagles and geofencing does not seem like the best solution anymore.

## Week 4

• We should be very explicit about everything we write down such that no confusion can be created
• Further dive into the literature of decision models
• Give more body to the problem description

## Week 5

• For the assignment, we talked about our recommendation report. We were told that we needed to have a much clearer and concrete idea of what kind of deliverable we have at the end of the project.
• As of now, we have that we want to make a decision model that recommends an ideal type of anti-UAV communication system for a given user of the decision model. He / She fills in her preferences in the decision model and based on that and the underlying decision rule in the decision model, we can give a (list of) recommendation(s) to the user.
• We need to look in getting a more concrete picture of this decision model, and also need to investigate into different types of decision models.

## Week 6

• The professors were happy with the surprising news regarding the contact with a higher-up at Eindhoven Airport that was interested in the project.
• We should use this contact in order for model validation.
• A design of a solution for Eindhoven Airport was out of the question as it would be too much extra effort.
• The idea of the decision model and its workings seemed good, but we had to be careful and validate it as it is possible to turn some knobs and let certain things weigh more and others weigh less. It is now important to get the right configuration and prove that this configuration is right.

## Week 7

• We discussed how the decision model had been completed; integration with a web app is all that remains to be functional.
• Gave a quick demo of how the web app now looks: only adding of questions needed to be added.
• Professors suggested adding a 'critical' checkbox since some criteria must be met by some users. This would then integrate parts of the MoSCoW model in order to improve the current model.
• Presentation should be around 10 minutes + 5 minutes demo + 5 minutes questions from other groups.
• Emotional farewell considering it was the last meeting.

Back to the root page.

# References

1. 1.0 1.1 "Gatwick Airport: Drones ground flights", 20 December 2018. Retrieved on 2019-02-06.
2. 2.0 2.1 2.2 2.3 2.4 2.5 2.6 Yin, Tung. "Game of drones: defending against drone terrorism", Tex. A&M L, 2015. Retrieved on 2019-02-06.
3. 3.0 3.1 3.2 Nguyen, P., Ravindranatha, M., Nguyen, A., Han, R., & Vu, T. "Investigating Cost-effective RF-based Detection of Drones", ACM, June 2016. Retrieved on 2019-02-06.
4. Revell, T. "Clash of the Drones", NewScientist, February 2018. Retrieved on 2019-02-07.
5. UK Department for Transport, "Small Remotely Piloted Aircraft Systems (drones) Mid-Air Collision Study", July 2017, Retrieved on 2019-02-07.
6. Civil Aviation Authority, "Drone Safety Risk: An assessment", January 2018,. Retrieved on 2019-02-07.
7. 7.0 7.1 Liberatore, S., "How do you catch a drone? With an even BIGGER drone and a giant net: Tokyo police reveal bizarre 'UAV catcher'", DailyMail, December 2015, Retrieved on 2019-02-07.
8. 8.0 8.1 Burns, M., https://techcrunch.com/2016/03/04/the-skywall-100-bazooka-captures-drones-with-a-giant-net/?guccounter=1 "The SkyWall 100 bazooka captures drones with a giant net"], TechCrunch, 2016, Retrieved on 2019-02-07.
9. Royakkers, L. M. M., & Est, van, Q. C. (2015). A literature review on new robotics: automation from love to war.International Journal of Social Robotics, 7(5), 549-570. DOI: 10.1007/s12369-015-0295-x
10. Royakkers, L. M. M., & Steen, M. (2017). Developing tools to counteract and prevent suicide bomber incidents:A case study in Value Sensitive Design. Science and Engineering Ethics, 23(4), 1041-1058. DOI:10.1007/s11948-016-9832-8
11. 11.0 11.1 Jahangir, M., & Baker, C. "Persistence Surveillance of Difficult to Detect microdrones with L-band 3-D Holographic RadarTM", Sensor Signal Processing for Defence (SSPD), September 2016. Retrieved on 2019-02-07.
12. 12.0 12.1 Moses, A., Rutherford, M. J., & Valavanis, K. P "Radar-Based Detection and Identification for Miniature Air Vehicles", Control Applications (CCA), September 2011. Retrieved on 2019-02-07.
13. Finn, R. L., & Wright, D. (2016). Privacy, data protection and ethics for civilian drone practice: A survey of industry, regulators and civil society organisations. Computer Law & Security Review, 32(4), 577-586.
14. Lin, P., Abney, K., & Bekey, G. (2011). Robot ethics: Mapping the issues for a mechanised world. Artificial Intelligence, 175(5-6), 942-949.
15. Joh, E. E. (2016). Policing police robots. UCLA L. Rev. Discourse, 64, 516.
16. Cavoukian, A."Privacy and drones: Unmanned aerial vehicles", Ontario: Information and Privacy Commissioner of Ontario, Canada, 2012. Retrieved on 2019-02-07.
17. 17.0 17.1 Etak Systems LLC."Anti-drone flight protection systems and methods", Google patents, 2016. Retrieved on 2019-02-07.
18. Wild, G., Murray, J. and Baxter, G."Exploring Civil Drone Accidents and Incidents to Help Prevent Potential Air Disasters", Aerospace, 2016. Retrieved on 2019-02-08.
19. Clothier, R., Walker, R."Determination and Evaluation of UAV Safety Objectives", Queensland University of Technology, 2006. Retrieved on 2019-02-08.
20. Slavimir S. Nikolić "An innovative response to commercial UAV menace - Anti-UAV falconry", Educons University, 2017. Retrieved on 2019-02-09.
21. Secretary of State for Transport, UK "Taking Flight: The Future of Drones in the UK", UK Secretary of State for Transport, 2019. Retrieved on 2019-02-09.
22. "Battelle defense against drones", Battelle Memorial Institute, 2019. Retrieved on 2019-02-09.
23. Lockheed Martin Corp."Counter-unmanned aerial vehicle system and method", Google patents, 2016. Retrieved on 2019-02-10.
24. Parker, D., Stern, D., Pierce, L."Deterent for unmanned aerial systems", Google patents, 2016. Retrieved on 2019-02-10.
25. [1], EASA, 2018. Retrieved on 2019-02-10.
26. "Gatwick spends £5m on airport anti-drone measures", The Financial Times, 2019. Retrieved on 2019-02-10.
27. Hallie Detrick, Gatwick's December Drone Closure Cost Airlines \$64.5 million http://fortune.com/2019/01/22/gatwick-drone-closure-cost/
28. Shutting down Dubai International Airport due to a drone costs \$100,000 a minute https://www.arabianbusiness.com/content/375851-drone-costs-100000-minute-loss-to-uae-airports
29. Traffic and transport figures https://www.schiphol.nl/en/schiphol-group/page/transport-and-traffic-statistics/
30. "International Aviation Transport Industry: Fact Sheet", December 2018. Retrieved on 11-02-2019.
31. "CNN: How can a drone bring an airport to a standstill?", December 2018. Retrieved on 11-02-2019.
32. "The Independent: Gatwick drone disruption cost over £50 million", January 2019. Retrieved on 11-02-2019.
33. 33.0 33.1 "European Aviation Laws", November 2018, Retrieved on 11-02-2019
34. "Huffpost: Why flight attendants hate delays more than you", July 2016, Retrieved on 11-02-2019
35. "Loyalty Travels: Why Do Flights Get Delayed – 15 Reasons Why Your Next Flight May Be Delayed", 2018, Retrieved on 12-02-2019
36. 36.0 36.1 Drs. C. van Nieuwenhuizen Wijbenga. "Beantwoording vragen van het lid Remco Dijkstra (VVD) over drones bij Londen-Gatwick", Ministerie van Infrastructuur en Waterstaat, 15 January 2019, Retrieved on 14-02-2019
37. 37.0 37.1 [https://www.theguardian.com/world/2019/jan/03/heathrow-and-gatwick-millions-anti-drone-technology "The Guardian: Heathrow and Gatwick invest millions in anti-drone technology", January 2019, Retrieved on 13-02-2019
38. 38.0 38.1 Thuy Ong. "Dutch police will stop using drone-hunting eagles since they weren't doing what they're told", 12 December 2017, Retrieved on 14-02-2019
39. 39.0 39.1 39.2 Adam Bannister. "With anti-drone tech on the market, why was Gatwick Airport so unprepared?", December 21 2018, Retrieved on 14-02-2019
40. "MIT Technology Review: Laws and Ethics Can’t Keep Pace with Technology", Written by V. Wadhwa, April 2014, Retrieved on 12-02-2019
41. "Does Technology Require New Law?", Written by D. Friedman, January 2001, Retrieved on 12-02-2019
42. [https://www.mediamarkt.nl/nl/product/_dji-ryze-tello-powered-by-dji-1556528.html "MediaMarkt Drone: DJI Ryze Tello Powered by DJI", Retrieved on 12-02-2019.
43. "Bright: Nieuwe regels voor drones gaan medio 2019 in" November 2018, Retrieved on 12-02-2019.
44. "Tweede Kamer der Staten-Generaal: Rondetafelgesprek over Drones en killer robots", January 2019, Retrieved on 13-02-2019
45. U.S.A. Government https://www.faa.gov/news/fact_sheets/news_story.cfm?newsId=20516
46. Jim Fisher, Drone Regulations: What You Need to Know, Aug. 23 (2018) https://www.pcmag.com/article2/0,2817,2491507,00.asp
47. Regeling modelvliegen https://wetten.overheid.nl/BWBR0019147/2015-11-07
48. Dutch Government, Rules for recreational use of drones https://www.government.nl/topics/drone/rules-pertaining-to-recreational-use-of-drones
49. Veilig vliegen met drones https://www.rijksoverheid.nl/binaries/content/gallery/rijksoverheid/content-afbeeldingen/onderwerpen/drone/drone-2018.jpg
50. Welke vergunning heb ik nodig voor mijn drone? https://www.rijksoverheid.nl/onderwerpen/drone/vraag-en-antwoord/vergunning-drone
51. Regels voor drones: verschillen tussen recreatief en beroepsmatig gebruik https://www.rijksoverheid.nl/onderwerpen/drone/documenten/brochures/2016/07/06/regels-voor-drones-verschillen-tussen-recreatief-en-beroepsmatig-gebruik
52. 52.0 52.1 Beantwoording vragen Schriftelijk Overlag drones, 1 November 2018, Ministerie van Infrastructuur en Waterstaat https://www.rijksoverheid.nl/onderwerpen/drone/documenten/kamerstukken/2018/11/01/beantwoording-vragen-schriftelijk-overleg-drones
53. 53.0 53.1 Beantwoording vragen van het lid Remco Dijkstra (VVD) over drones bij Londen-Gatwick , 15 Jan. 2019 https://www.rijksoverheid.nl/onderwerpen/drone/documenten/kamerstukken/2019/01/15/beantwoording-vragen-van-het-lid-remco-dijkstra-vvd-over-drones-bij-londen-gatwick
54. 54.0 54.1 54.2 Nieuwe Europese regels https://www.ilent.nl/onderwerpen/drones/nieuwe-europese-regels
55. 55.0 55.1 Opinion No 01/2018, Introduction of a regulatory framework for the operation of unmanned aircraft systems in the open and specific categories https://www.easa.europa.eu/sites/default/files/dfu/Opinion%20No%2001-2018.pdf
56. Gettinger, D., & Michel, A. H. " "Drone sightings and close encounters: An analysis", Center for the Study of the Drone, Bard College, 2015. Retrieved on 2019-02-14.
57. Snapshot: DHS Silicon Valley Innovation Program Successfully Transitions Three Technologies to CBP https://www.dhs.gov/science-and-technology/news/2019/02/05/snapshot-svip-successfully-transitions-three-technologies-cbp
58. Super Bowl: experimental radar aims to stop drone drama at game https://www.theguardian.com/technology/2019/jan/28/super-bowl-drones-radar-start-up-experiment
59. The U.S. government showed just how easy it is to hack drones made by Parrot, DBPower and Cheerson. (2017). Recode. Retrieved 15 February 2019, from [2]
60. 60.0 60.1 Haye Kesteloo, Dutch police halts use of eagles to intercept drones (2017) [3]
61. 61.0 61.1 61.2 61.3 61.4 61.5 61.6 CBS opendata "Luchtvaart; maandcijfers Nederlandse luchthavens van nationaal belang" Retrieved on 2019-02-20.
62. 62.0 62.1 Vliegveldinfo"Grootste vliegveld Europa" Retrieved on 2019-02-20.
63. 63.0 63.1 63.2 Royal schiphol group"Amsterdam airport schiphol airport facts" Retrieved on 2019-02-20.
64. Royal schiphol group "Schiphol verkeer en vervoer cijfers" Retrieved on 2019-02-20.
65. "Wikipedia: Eindhoven Airport", Retrieved on 20-02-2019
66. "Koninklijke Luchtmacht: Vliegbasis Eindhoven", Retrieved on 20-02-2019
68. "Wikpedia: Hurricane Irma", Retrieved on 20-02-2019.
69. NACO,EA Infra werkgroep [https://www.google.com/url?sa=i&source=images&cd=&ved=2ahUKEwig9MLQ2szgAhXFblAKHejfBZ0Q5TV6BAgBEAs&url=http%3A%2F%2Fsamenopdehoogte.nl%2Fover-ons%2Fnieuws%2Fairport-infrastructuur&psig=AOvVaw1LoO6la2qC5v9x5hVz6RDh&ust=1550834409875173 "Eindhoven Airport Master Plan Airport Infrastructuur"] May 2018, Retrieved on 21-02-2019.
70. 70.0 70.1 ""Routes Online: Eindhoven Airport, Retrieved on 20-02-2019
71. "Routes Online: Eindhoven Airport, Milestone Year 2018", Retrieved on 20-02-2019
72. Rotterdam The Hague Airport "Rotterdam The Hague airport, overzicht verkeer en vervoer per kalenderjaar" Retrieved on 2019-02-20.
73. "Wikipedia: Rotterdam The Hague Airport", Retrieved on 20-02-2019.
74. 74.0 74.1 "Rotterdam The Hague Airport: Geschiedenis", Retrieved on 20-02-2019.
75. 75.0 75.1 75.2 75.3 75.4 75.5 75.6 75.7 Google "Google Maps" Retrieved on 21-02-2019
76. "Rotterdam The Hague Airport: Feiten en Cijfers 2017", Retrieved on 20-02-2019.
77. "Rotterdam The Hague Airport: Overzicht verkeer en vervoer 2018", Retrieved on 20-02-2019.
78. "https://www.maa.nl/bestemmingen/"
79. 79.0 79.1 DaftLogic, "Google Maps Area Calculator Tool", retrieved on 10-03-2019
80. "https://www.groningenairport.nl/bestemmingen-overzicht"
81. 81.0 81.1 David Schaar, "Analysis of Airport Stakeholders", The Volgenau School of Information Technology and Engineering, Retrieved on 05-03-2019
82. Abbie, Outstandingdrone, "https://www.outstandingdrone.com/drones-under-250-grams/", Retrieved on 05-03-2019
83. "Brilliant.org: K-nearest Neighbors", Retrieved 12 March 2019
84. Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. An Introduction to Statistical Learning with Applications in R. Springer, first edition, 2017.
85. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data mining, Interface, and Prediction. Springer, second edition, 2017.
86. "Wikipedia: Feature Scaling", Retrieved on 12 March 2019
87. Ruusuvirta, O., & Rosema, M. (2009, September). Do online vote selectors influence electoral participation and the direction of the vote? In ECPR general conference (pp. 13-12).
88. Ladner, A., & Pianzola, J. (2010, August). Do voting advice applications affect electoral participation and voter turnout? Evidence from the 2007 Swiss Federal Elections. In International Conference on Electronic Participation (pp. 211-224). Springer, Berlin, Heidelberg.
89. 89.0 89.1 89.2 89.3 89.4 "StemWijzer", Retrieved on 14-03-2019
90. What is the Decision Model? http://www.ittoday.info/ITPerformanceImprovement/Articles/2011-03VonHalleGoldberg.html
91. 91.0 91.1 Model Verification and Validation, Charles M. Macal http://jtac.uchicago.edu/conferences/05/resources/V&V_macal_pres.pdf