PRE2019 3 Group7

Group Members

Name	Study	Student ID
Daan Schalk		0962457
Job Willems		1003011
Jasper Dellaert		1252454
Sanne van Wijk		1018078
Wietske Blijjenberg		1025111

Subject

Simulating populations using AI.

Objectives

• Construct an AI simulation with which we can evaluate the likelihood of adverse ecological effects on populations occurring as a result of exposure to physical or chemical stressors
• Analysing the reliability of such a simulation by comparing the results to scenarios which have happened in the past.
• Analysing the possibilities and shortcomings of such a simulation.

Users

Simulating populations can be useful in a lot of fields. A good-working simulation will therefore have a lot of users. Firstly, there are of course the biologists and behavioral analysts, who can perform experiments to try and understand populations better. A simulation like this can also be used in a population viability analysis, which is a species-specific method of risk assessment frequently used in conservation biology. Then we have the ecologists and government organisations who can use this tool for ecological risk assessment when they are making wildlife plans. Lastly, such a simulation can be useful in education to give students a better understanding of evolution and animal behaviour. (Avida-ED)

State-of-the-art

• Multi-agent Based Simulation: Where Are the Agents?

https://link.springer.com/chapter/10.1007/3-540-36483-8_1

• VORTEX: a computer simulation model for population viability analysis

https://www.publish.csiro.au/WR/WR9930045

• A Generalized Computer Simulation Model for Fish Population Studies

https://afspubs.onlinelibrary.wiley.com/doi/abs/10.1577/1548-8659(1969)98[505:AGCSMF]2.0.CO;2

• Artificial Intelligence techniques: An introduction to their use for modelling environmental systems

https://www.sciencedirect.com/science/article/abs/pii/S0378475408000505

• An artificial intelligence modelling approach to simulating animal habitat interactions

https://www.researchgate.net/profile/Jane_Packard2/publication/237332438_An_artificial_intelligence_modelling_approach_to_simulating_animalhabitat_interactions/links/5b882577a6fdcc5f8b72005e/An-artificial-intelligence-modelling-approach-to-simulating-animal-habitat-interactions.pdf

• Application of Multi-agent Simulation in Animal Epidemic Emergency Management: Take an Example of AFS (Africa Fever Swine) Policy

http://www.dpi-proceedings.com/index.php/dtetr/article/view/31843

• A Study of AI Population Dynamics with Million-agent Reinforcement Learning (2018)

https://dl.acm.org/doi/10.5555/3237383.3238096

• Simulating exposure-related behaviors using agent-based models embedded with needs-based artificial intelligence (2018)

https://www.nature.com/articles/s41370-018-0052-y/ Context: HUMANS exposure to a chemical. Because descriptions of where and how individuals spend their time are important for characterizing exposures to chemicals in consumer products and in indoor environments, and the existing method is difficult and labor-intensive, a simulation of longitudinal patterns in human behaviour was created. This is an agent-based model with a needs-based AI. Needs-based because humans make their decisions to take actions in order to fulfil needs. The paper describes how it is implemented. Meets critical need in field of exposure assessment. Only addresses a few needs, and not the complex ones.

• Next-generation ecological risk assessment: Predicting risk from molecular initiation to ecosystem service delivery

https://www.sciencedirect.com/science/article/pii/S0160412016300824 There have been exciting developments in in vitro testing and high-throughput systems that measure responses to chemicals at molecular and biochemical levels of organization, but the linkage between such responses and impacts of regulatory significance – whole organisms, populations, communities, and ecosystems – are not easily predictable. This article describes some recent developments that are directed at bridging this gap and providing more predictive models that can make robust links between what we typically measure in risk assessments and what we aim to protect.

• Population based training of neural networks. Population based training discovers a schedule of hyperparameter settings rather than following the generally sub-optimal strategy of trying to find a single fixed set to use for the whole course of training.

https://arxiv.org/abs/1711.09846 (the paper)

https://deepmind.com/blog/article/population-based-training-neural-networks (a blogpost about the paper)

https://www.youtube.com/watch?v=l-Ga0E9vldg (a talk about the paper)

• A talk by Jeff Clune (http://jeffclune.com/) about recent (2019) avancements in population-based search. Focusing on explicitly searching for behavioral diversity, open-ended search and indirect encoding.

https://www.youtube.com/watch?v=g6HiuEnbwJE

• this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress.

https://ieeexplore.ieee.org/abstract/document/4141061

• An overview of a simulation of an ecosystem housing predators and prey. The simulation has much hard coded behavior, allowing the simulation to get more realistic.

https://www.youtube.com/watch?v=r_It_X7v-1E

• An overview of a simulation of an ecosystem housing creatures based on neural networks. With robust neural networks and no hard coded behaviours, this simulation allows for more emergent behaviour and potential realism at the cost of current realism.

https://www.youtube.com/watch?v=myJ7YOZGkv0

• An overview of a simulation of an ecosystem with a complex environment. This AI has hard coded features for interacting with the environment, however it can still evolve a neural network, striking a balance between the previous two simulations.

https://www.youtube.com/watch?v=E-zcUzK8k7U

• This paper describes a novel system for creating virtual creatures that move and behave in simulated three-dimensional physical worlds. A genetic language is presented that uses nodes and connections as its primitive elements to represent directed graphs, which are used to describe both the morphology and the neural circuitry of these creatures.

http://www.karlsims.com/papers/siggraph94.pdf

• This paper explores selecting for evolvability in neural networks. Evolvability Search enables generating evolvability more easily and directly, facilitating its study and understanding, and may inspire future practical algorithms that increase evolvability without significant computational overhead.

http://www.evolvingai.org/mengistu-lehman-clune-2016-evolvability-search-directly

• In this paper digital organisms were used to investigate the ability of natural selection to adjust and optimize mutation rates.

http://www.evolvingai.org/clune-misevic-ofria-lenski-2008-natural-selection-fails

• This paper explores novelty search, a new type of Evolutionary Algorithm, has shown much promise in the last few years. A common criticism of Novelty Search is that it is effectively random or exhaustive search because it tries solutions in an unordered manner until a correct one is found. Its creators respond that over time Novelty Search accumulates information about the environment in the form of skills relevant to reaching uncharted territory, but to date no evidence for that hypothesis has been presented.

http://www.evolvingai.org/velez-clune-2014-novelty-search-creates-robots

• Exploring the Relationship between Experiences with Digital Evolution and Students' Scientific Understanding and Acceptance of Evolution (2018)

https://avida-ed.msu.edu/files/curricula/ABT_Exploring_Relationship__Understanding_Acceptance_Evo.pdf Uses a research-based platform for digital evolution in the classroom, found that engagement in lessons with Avida-ED both supported studentlearning of fundamental evolution concepts and was associated with an increase in student acceptance of evolution as evidence-based science. Also found a significant, positive association between increased understanding and acceptance. --> arguments for education as one of the users

• MABE (Modular Agent Based Evolver): A framework for digital evolution research (2017)

https://www.mitpressjournals.org/doi/pdf/10.1162/isal_a_016 MABE is a modular and reconfigurable digital evolution research tool designed to minimize the time from hypotheses generation to hypotheses testing. MABE provides an accessible framework which seeks to increase collaborations and to facilitate reuse by implementing only features that are common to most experiments, while leaving experimentally dependent details up to the user. "One difficulty in Digital Evolution research stems from the need to develop the software used to conduct the re-search"

• Artificial Intelligence Techniques to Enhance Actors’ Decision Strategies in Socio-­ecological Agent-­ Based Models (2016)

https://scholarsarchive.byu.edu/iemssconference/2016/Stream-D/19/ Title is pretty self-explanatory. Provides an analysis of the types of AI learning algorithms employed in various application domains which use Agent-Based Models, their specific operationalization in an agent’s decision-­making for various tasks, treatment of spatial and social environment in the design of AI learning algorithms, and the level of empirical information used in ABM. Also highlights the trends in the current practice of AI learning algorithms used to enhance ABMs.

• Agent-based model calibration using machine learning surrogates (2018)

https://www.sciencedirect.com/science/article/pii/S0165188918301088 Tackles parameter space exploration and calibration of agent based models by combining machine-learning and intelligent iterative sampling. Results domanstrate that machine learning surrogates obtained using the proposed iterative learning procedure provide a quite accurate proxy of the true model and dramatically reduce the computation time necessary for large scale parameter space exploration and calibration.

• Effects of mass extinction on community stability and emergence of coordinated stasis with digital evolution (2018)

http://en.cnki.com.cn/Article_en/CJFDTotal-NJNY201801012.htm Research based on digital evolution, can be used as application example.

• Representing the acquisition and use of energy by individuals in agent‐based models of animal populations (2012)

https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210x.12002# Exactly as the title suggests. Suggestion and evaluation of how to model animal energy needs in agent-based models.

• The role of agent-based models in wildlife ecology and management (2011)

https://www.sciencedirect.com/science/article/pii/S0304380011000524 Wildlife-management ABMs disentangle habitat use and quality, and represent dynamic environments. Using adaptive movement ecology in changing landscapes permits scenario planning of future habitats. ABMs are excellent tools encompassing multiple disciplines and stakeholder interests. Can be used to substantiate arguments about why evolution simulations are useful in wildlife ecology and management.

• Using stylized agent-based models for population–environment research: a case study from the Galápagos Islands (2010)

https://link.springer.com/article/10.1007/s11111-010-0110-4 More about the utility of ABM's : here they are named useful for sharpening conceptualizations of population–environment systems, testing alternative scenarios, and uncovering critical data gaps. (Also about trade-offs between model complexity and abstraction.)

• Evolutionary computing

https://www.cs.vu.nl/~gusz/papers/ec-intro-Eiben-Schoenauer.pdf This paper gives a general overview into evolutionary computing

Approach

1 Create a simulated environment where AI based on neural networks can roam.

2 Edit factors such as amount of food and maximum speed of the AI to see how this influences the AI.

3 Analyze the results of step 2, iterate and try to find interesting stuff.

4 Document everything interesting

Planning

• Week 1: research state-of-the-art, finalise plan
• Week 2: More research, state requirements for simulation
• Week 3: Implemented first version of simulation with only basic features
• Week 4: Implemented second version simulation with requirements implemented
• Week 5: Performing simulation, documenting results
• Week 6: Performing simulation, documenting results
• Week 7: Compare results to real life, create conclusion
• Week 8: Finalise wiki

Milestones

Understanding of state of the art

A list of features we might want to implement into our simulation

A minimal viable product: a simulation that can house AI based on neural networks

The simulation but with some of the features of the list above implemented

Having research results by watching the simulations

Having research results by changing some factors of the simulation (such as amount of food)

Documenting our results and comparing them to real life to create a conclusion

Deliverables

• A simulated environment where AI based on neural networks can roam.
• This wiki page, which contains our process, research and the results of our analysis.
• A presentation

Who is doing what

Week 1

Name	Total	Break-down
All	2h	Discussing the subject
Daan
Job
Sanne	3h	Making a draft for the wiki (1/2 h), Gathering links for the State of the Art (2.5h)
Jasper
Wietske	4h	Working on the wiki (0.5 h), Gathering articles for state-of-the-art (3.5 h)