Web Application - Group 4 - 2018/2019, Semester B, Quartile 3: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
No edit summary
 
(22 intermediate revisions by 4 users not shown)
Line 1: Line 1:
link rel="shortcut icon" href="https://www.tue.nl/favicon-64.ico" type="image/x-icon">
<div style="font-family: 'Georgia'; font-size: 15px; line-height: 1.5; max-width: 800px; word-wrap: break-word; color: #333; font-weight: 400; box-shadow: 0px 25px 35px -5px rgba(0,0,0,0.75); margin-left: auto; margin-right: auto; padding: 70px; background-color: white; padding-top: 30px;">
<link rel=http://cstwiki.wtb.tue.nl/index.php?title=PRE2018_3_Group4&action=edit"stylesheet" type="text/css" href="theme.css">
<div style="width:calc(100vw - 50px);background-color:#EFEFEF;padding-bottom:35px;position:absolute;top:0;left:0;">
<div style="font-family: 'q_serif', Georgia, Times, 'Times New Roman', serif; font-size: 14px; line-height: 1.5; max-width: 750px; word-wrap: break-word; color: #333; font-weight: 400; box-shadow: 0px 25px 35px -5px rgba(0,0,0,0.75); margin-left: auto; margin-right: auto; padding: 70px; background-color: white; padding-top: 25px;">


<div style="display: block; position: absolute; right: 0%;">
<div style="display: block; position: absolute; right: 6%;">
; Page navigation
; Page navigation
# [[PRE2018_3_Group4 | Root]]
# [[PRE2018_3_Group4 | Root]]
# [[Notes - Group 4 - 2018/2019, Semester B, Quartile 3|Notes]]
# [[Notes - Group 4 - 2018/2019, Semester B, Quartile 3|Notes from meeting]]
# [[Initial ideas - Group 4 - 2018/2019, Semester B, Quartile 3|Initial ideas]]
# [[Initial ideas - Group 4 - 2018/2019, Semester B, Quartile 3|Initial ideas]]
# [[Project setup - Group 4 - 2018/2019, Semester B, Quartile 3|Project setup]]
# [[Project setup - Group 4 - 2018/2019, Semester B, Quartile 3|Project setup]]
# [[General problem - Group 4 - 2018/2019, Semester B, Quartile 3|General problem]]  
# [[General problem - Group 4 - 2018/2019, Semester B, Quartile 3|General problem description]]  
# [[State of the Art - Group 4 - 2018/2019, Semester B, Quartile 3|State of the Art]]
# [[State of the Art - Group 4 - 2018/2019, Semester B, Quartile 3|State of the Art]]
# [[Specific problem - Group 4 - 2018/2019, Semester B, Quartile 3|Specific problem]]
# [[Specific problem - Group 4 - 2018/2019, Semester B, Quartile 3|Specific problem description]]
# [[Present situation - Group 4 - 2018/2019, Semester B, Quartile 3|Present situation]]
# [[Present situation - Group 4 - 2018/2019, Semester B, Quartile 3|Present situation]]
# [[Drones - Group 4 - 2018/2019, Semester B, Quartile 3|Drones]]
# [[Drones - Group 4 - 2018/2019, Semester B, Quartile 3|Drone analysis]]
# [[Solutions - Group 4 - 2018/2019, Semester B, Quartile 3|Solutions]]
# [[Solutions - Group 4 - 2018/2019, Semester B, Quartile 3|Solution analysis]]
# [[Airports under a microscope - Group 4 - 2018/2019, Semester B, Quartile 3|Airports under a microscope]]
# [[Airports under a microscope - Group 4 - 2018/2019, Semester B, Quartile 3|Airport analysis]]
# [[Recommendation report - Group 4 - 2018/2019, Semester B, Quartile 3|Recommendation report]]
# [[Types of Decision Models - Group 4 - 2018/2019, Semester B, Quartile 3 | Decision Model investigation]]
# [[Decision Model - Group 4 - 2018/2019, Semester B, Quartile 3 | Decision Model implementation]]
# [[Decision Model validation - Group 4 - 2018/2019, Semester B, Quartile 3|Decision Model validation]]
# [[Categorizing solutions - Group 4 - 2018/2019, Semester B, Quartile 3|Categorising solutions]]
# [[Web_Application_-_Group_4_-_2018/2019,_Semester_B,_Quartile_3 | Web Application]]
# [[Future - Group 4 - 2018/2019, Semester B, Quartile 3|Future]]
# [[Future - Group 4 - 2018/2019, Semester B, Quartile 3|Future]]
# [[Types of Decision Models - Group 4 - 2018/2019, Semester B, Quartile 3 | Decision Model Investigation]]
# [[Decision Model - Group 4 - 2018/2019, Semester B, Quartile 3 | Implemented Decision Model]]
# [[Conclusion - Group 4 - 2018/2019, Semester B, Quartile 3|Conclusion]]
# [[Conclusion - Group 4 - 2018/2019, Semester B, Quartile 3|Conclusion]]
# [[Discussion - Group 4 - 2018/2019, Semester B, Quartile 3|Discussion]]
# [[Discussion - Group 4 - 2018/2019, Semester B, Quartile 3|Discussion]]
</div>
</div>


= Introduction =
= Web Application =
When introducing a decision model, it is important to both validate and verify that decision model. This is especially important when it comes to computational models. When it comes to model verification, we ask ourselves the following question: `Does the model perform as intended?'. This question is asked in order to verify that, for example, the model has been programmed correctly. Furthermore, it verifies if the algorithm has been implemented properly and if the model does not contain errors, oversights, or bugs. We also have model validation. Here, we ask ourselves the following question: `Does the model represent and correctly reproduce the behaviors of the real world system?'. Validation ensures that the model meets its intended requirements in terms of the methods employed and the results obtained. The ultimate goal of model validation is to make the model useful in the sense that the model addresses the right problem, provides accurate information about the system being modeled, and to makes the model actually used<ref name="v&v">Model Verification and Validation, Charles M. Macal http://jtac.uchicago.edu/conferences/05/resources/V&V_macal_pres.pdf</ref>.
Now that we have a decision model in place, it is useful to be able to display/represent it in some way. By doing so, we will be able to validate, test and actually make use of the decision model. This also gives a clear overview of how the decision model is supposed to work. The first step is to find a suitable method to represent the decision model.


= What now? =  
== How to implement? ==
Unlike physical systems, for which there are well-established procedures for model validation, no such guidelines exist for social modeling. Unfortunately for the implemented decision model, there is no easy or clear way to validate and verify the model. This is mainly due to the model containing much subjectivity through human decision making. When users of the decision model use it, they have to provide input themselves. These inputs are not just numbers, but they are about whether or not the user agrees or disagrees with a proposition. This makes it hard to both validate and verify the model in a traditional way. In the case of models that contain elements of human decision making, validation becomes a matter of establishing credibility in the model. Verification and validation work together by removing barriers and objections to model use. The task is to establish an argument that the model produces sound insights and sound data based on a wide range of tests and criteria that `stand-in' for comparing model results to data from the real system<ref name="v&v">Model Verification and Validation, Charles M. Macal http://jtac.uchicago.edu/conferences/05/resources/V&V_macal_pres.pdf</ref>. This process is akin to developing a legal case in which a preponderance of evidence is compiled about why the model is a valid one for its purported use. In order to still do some verification, we use subject matter experts in order to gain a grasp of the credibility of the model. We implement ways to measure this credibility through evaluation and role-playing.
There are various methods of implementing a decision model. The decision model could be implemented using an existing survey/form website, i.e. using google forms<ref name="forms">Google [https://forms.google.com "Google Forms"] Retrieved on 19-03-2019</ref>, SurveyMonkey<ref name="surveymonkey"> [https://www.surveymonkey.com/ "SurveyMonkey"] Retrieved on 19-03-2019</ref> or another similar website. This method would probably be easiest and the least time consuming, however, those websites are mostly not very flexible and do not adhere to our needs. Such form website mostly offers users to fill in answers to questions, but do not follow up with a calculation of a score and be able to give advice for solutions. Furthermore, some of these websites even restrict 'free' users to a limited number of questions to be asked.


= Credibility =
Another option would be to create a Web application, that we can modify to our exact needs. We decided that a client-side application built using Vue.js and some bootstrap would allow for fast development, scalability, portability and can easily be tailored exactly to our needs. The development was done on GitHub, allowing us to host our result on GitHub pages. For the design of our app we took inspiration from the Dutch "Stemwijzer"<ref name="Stemwijzer">https://www.stemwijzer.nl</ref>, and an open source voting site called electioncalculator.org<ref name="electioncalculator">Jaroslav Semančík, Michal Škop,[https://electioncalculator.org/ "electioncalculator"], KohoVolit.eu, Retrieved on 19-03-2019</ref>.  
As coined earlier, we want to somehow make the credibility of the model tangible. We do this through evaluation and role-playing. A group of domain experts will do the evaluation. These domain experts consist of both the group working on this project and higher-ups that go over anti-drone mechanisms at Eindhoven Airport. We asked higher-ups at Eindhoven Airport that go over anti-drone mechanisms to spread the decision model questionnaire and have it be filled in by numerous individuals that all agree on the interests, needs, and characteristics of Eindhoven Airport. Furthermore, we ask for an initial solution that they think is the best from the list of all the solutions we forged. It is then interesting to see if these individuals get the same results for the decision model and if they agree with the decision model. Additionally, it is interesting to compare the initial solution they thought would be best for the recommended solution they got and what they think of the recommended solution. Are they surprised? Are they not surprised at all? Does the recommended solution provide new insights?


As we do not want to depend on a select few individuals from Eindhoven Airport alone, we also propose an example scenario where the user taking the questionnaire becomes a higher-up of a clearly defined airport that has to design a mechanism against unwanted UAVs. This is the role-playing method to establish credibility. This includes the needs, wants, and beliefs of this airport. We, internally, take this questionnaire as well. Afterward, we compare the initial thought of solutions, the recommended solutions, and the opinion of the recommended solution for the proposed airport.  
The site displays a series of questions, to which the user can agree, disagree or remain neutral. In addition, the user can also select some questions as being "mandatory". When a question is selected as mandatory, only solutions that follow the users preference for that question will be taken into account when computing the final solution. After having answered all questions, we ask the user to identify solutions that are extra important to him. To these solutions, the app adds an increased weight. After all questions have been answered, the scores and relative % match are computed. The results page shows a sorted list of solutions, together with a description of the solution.


= Methods =  
== What it looks like ==
Let us consider the two methods coined earlier for testing the credibility of the decision model to a certain degree.
The voting app is hosted on https://drones.jortdebokx.nl/. Here are some screenshots:


== Evaluation ==
[[File:homepage.png| 700 px |thumb|upright=4|center|alt=Missing image|Figure 1: Homepage of the web app.]]
Testing the credibility of the model through evaluation will be done, as briefly introduced earlier, by sending a questionnaire to higher-ups at Eindhoven Airport that go over mechanisms to counter illegal drone activity around their airport. What we are particularly interested in with this way of verification if what the recommendation proposed by the decision model based on the inputs of the individuals fits the airport. With this, we want to test if the decision model proposes solutions that would work for the airport.
[[File:question-example.png| 700 px |thumb|upright=4|center|alt=Missing image|Figure 2: Questions of the web app.]]
[[File:selection.png| 700 px |thumb|upright=4|center|alt=Missing image|Figure 3: Extra-important subjects that the user can select of the web app.]]
[[File:results.png| 700 px |thumb|upright=4|center|alt=Missing image|Figure 4: Results of the web app.]]


== Role-playing ==
----
Testing the credibility of the model through role-playing will be done by proposing an example scenario where the individual is a higher-up at an airport company. Here, this individual will decide on what mechanisms to consider when it comes to illegal drone activity. This will be done based on certain information given to the individual.
Back to the [[PRE2018_3_Group4 | root page]].


= References =
= References =
<references/>
<references/>

Latest revision as of 15:00, 1 April 2019

Web Application

Now that we have a decision model in place, it is useful to be able to display/represent it in some way. By doing so, we will be able to validate, test and actually make use of the decision model. This also gives a clear overview of how the decision model is supposed to work. The first step is to find a suitable method to represent the decision model.

How to implement?

There are various methods of implementing a decision model. The decision model could be implemented using an existing survey/form website, i.e. using google forms[1], SurveyMonkey[2] or another similar website. This method would probably be easiest and the least time consuming, however, those websites are mostly not very flexible and do not adhere to our needs. Such form website mostly offers users to fill in answers to questions, but do not follow up with a calculation of a score and be able to give advice for solutions. Furthermore, some of these websites even restrict 'free' users to a limited number of questions to be asked.

Another option would be to create a Web application, that we can modify to our exact needs. We decided that a client-side application built using Vue.js and some bootstrap would allow for fast development, scalability, portability and can easily be tailored exactly to our needs. The development was done on GitHub, allowing us to host our result on GitHub pages. For the design of our app we took inspiration from the Dutch "Stemwijzer"[3], and an open source voting site called electioncalculator.org[4].

The site displays a series of questions, to which the user can agree, disagree or remain neutral. In addition, the user can also select some questions as being "mandatory". When a question is selected as mandatory, only solutions that follow the users preference for that question will be taken into account when computing the final solution. After having answered all questions, we ask the user to identify solutions that are extra important to him. To these solutions, the app adds an increased weight. After all questions have been answered, the scores and relative % match are computed. The results page shows a sorted list of solutions, together with a description of the solution.

What it looks like

The voting app is hosted on https://drones.jortdebokx.nl/. Here are some screenshots:

Missing image
Figure 1: Homepage of the web app.
Missing image
Figure 2: Questions of the web app.
Missing image
Figure 3: Extra-important subjects that the user can select of the web app.
Missing image
Figure 4: Results of the web app.

Back to the root page.

References

  1. ↑ Google "Google Forms" Retrieved on 19-03-2019
  2. ↑ "SurveyMonkey" Retrieved on 19-03-2019
  3. ↑ https://www.stemwijzer.nl
  4. ↑ Jaroslav Semančík, Michal Škop,"electioncalculator", KohoVolit.eu, Retrieved on 19-03-2019
Retrieved from "https://cstwiki.wtb.tue.nl/index.php?title=Web_Application_-_Group_4_-_2018/2019,_Semester_B,_Quartile_3&oldid=71067"