PRE2022 3 Group4: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
(4 intermediate revisions by 2 users not shown)
Line 1: Line 1:
''As written by chatGPT:''<blockquote>Artificial Intelligence, a wonder of the modern age  
''As written by ChatGPT:''<blockquote>Artificial Intelligence, a wonder of the modern age  


A creation made of code, with endless knowledge in its brain  
A creation made of code, with endless knowledge in its brain  
Line 45: Line 45:
|Psychology & Technology
|Psychology & Technology
|}
|}
==Brainstorm==
*AI in academic education
*<s>VR game for children's education</s>
*<s>Kitchen aid for visually impaired people</s>
*<s>Child support in healthcare</s>
*<s>Researching and improving the acceptance of robots in health care</s>


==Structure of the Wiki==
==Structure of the Wiki==
Line 60: Line 52:
#State of the art
#State of the art
##Also findings from the panel?
##Also findings from the panel?
#Problem statement
#Hypothesis
#Method
#Method
##Surveys, interviews with students and teachers
##Surveys, interviews with students and teachers
Line 93: Line 87:
However, it is important to note that the implementation of AI in education is still in its early stages and there are many challenges that must be overcome, such as ensuring the privacy and security of student data and addressing ethical considerations and trust issues around the use of AI in the classroom.
However, it is important to note that the implementation of AI in education is still in its early stages and there are many challenges that must be overcome, such as ensuring the privacy and security of student data and addressing ethical considerations and trust issues around the use of AI in the classroom.


So, while AI has the potential to greatly impact and improve education, it is important to approach its implementation with caution and careful consideration of its limitations and potential risks. This and the outlook and trust of teacher in academic education on AI in the classroom is what we will be researching. With the gathered information we will create a recommendation for teachers at the TUe on how to optimally implement AI in the classroom to benefit both teacher and student.
So, while AI has the potential to greatly impact and improve education, it is important to approach its implementation with caution and careful consideration of its limitations and potential risks. This and the outlook and trust of teacher in academic education on AI in the classroom is what we will be researching. With the gathered information we will create a recommendation for teachers at the TU/e on how to optimally implement AI in the classroom to benefit both teacher and student.
==State of the Art==
==State of the Art==
===What is ChatGPT?===
ChatGPT is a new technology, but has been extensively researched, as it is such a noticeable new technology. ChatGPT is an acronym for Chat Generative Pre-trained Transformer and was created in November 2022. It is based on the OpenAI GPT-3 engine and has been fine-tuned by supervised and reinforcement learning technology; this means that the AI learns by having humans simulate artificial conversations with it and adapting its responses based on how accurately they reflect natural human dialogue. ChatGPT is also able to remember previously given prompts in the same conversation, making it a more personalized chatbot compared to its alternatives.<ref>Arimetrics (2022) ''What is ChatGPT.'' Retrieved from https://www.arimetrics.com/en/digital-glossary/chatgpt</ref> ChatGPT has a lot of functionalities, for example:
ChatGPT is a new technology, but has been extensively researched, as it is such a noticeable new technology. ChatGPT is an acronym for Chat Generative Pre-trained Transformer and was created in November 2022. It is based on the OpenAI GPT-3 engine and has been fine-tuned by supervised and reinforcement learning technology; this means that the AI learns by having humans simulate artificial conversations with it and adapting its responses based on how accurately they reflect natural human dialogue. ChatGPT is also able to remember previously given prompts in the same conversation, making it a more personalized chatbot compared to its alternatives.<ref>Arimetrics (2022) ''What is ChatGPT.'' Retrieved from https://www.arimetrics.com/en/digital-glossary/chatgpt</ref> ChatGPT has a lot of functionalities, for example:


Line 108: Line 104:
However, the technology is not without its limitations. It has the potential for over-optimization due to its reliance on human oversight, also known as Goodhart's law<ref>Gao, Leo; Schulman; Hilton, Jacob (2022). "Scaling Laws for Reward Model Overoptimization". arXiv:2210.10760 [cs.LG].</ref>, which could hinder performance. Furthermore, language models like ChatGPT are prone to writing plausible-sounding but incorrect answers, which is called artificial intelligence hallucination<ref>Lakshmanan, Lak (December 16, 2022). "Why large language models like ChatGPT are bullshit artists". becominghuman.ai. Archived from the original on December 17, 2022. Retrieved January 15, 2023. <q>''The human raters are not experts in the topic, and so they tend to choose text that looks convincing. They'd pick up on many symptoms of hallucination, but not all. Accuracy errors that creep in are difficult to catch.''</q></ref>; this can be attributed to insufficient training data. The AI is also limited by a lack of knowledge about events that occurred after 2021 and in some cases suffers from algorithmic biases. Furthermore, although ChatGPT is able to produce results that seem genuine, it is unable to fully comprehend the complexity of human language and instead relies solely on statistical knowledge and patterns.
However, the technology is not without its limitations. It has the potential for over-optimization due to its reliance on human oversight, also known as Goodhart's law<ref>Gao, Leo; Schulman; Hilton, Jacob (2022). "Scaling Laws for Reward Model Overoptimization". arXiv:2210.10760 [cs.LG].</ref>, which could hinder performance. Furthermore, language models like ChatGPT are prone to writing plausible-sounding but incorrect answers, which is called artificial intelligence hallucination<ref>Lakshmanan, Lak (December 16, 2022). "Why large language models like ChatGPT are bullshit artists". becominghuman.ai. Archived from the original on December 17, 2022. Retrieved January 15, 2023. <q>''The human raters are not experts in the topic, and so they tend to choose text that looks convincing. They'd pick up on many symptoms of hallucination, but not all. Accuracy errors that creep in are difficult to catch.''</q></ref>; this can be attributed to insufficient training data. The AI is also limited by a lack of knowledge about events that occurred after 2021 and in some cases suffers from algorithmic biases. Furthermore, although ChatGPT is able to produce results that seem genuine, it is unable to fully comprehend the complexity of human language and instead relies solely on statistical knowledge and patterns.


Research into ChatGPT still has a long way to go. vVan Dis, E. et al. have proposed five priorities for future research on ChatGPT:
Research into ChatGPT still has a long way to go. Van Dis, E. et al. have proposed five priorities for future research on ChatGPT:


*The first priority is to explore the ethical implications of AI-generated content and to develop guidelines for responsible use.
*The first priority is to explore the ethical implications of AI-generated content and to develop guidelines for responsible use.
Line 116: Line 112:
*Finally, the fifth priority is to develop more efficient and sustainable methods for training and deploying ChatGPT, in order to reduce its carbon footprint and energy consumption.
*Finally, the fifth priority is to develop more efficient and sustainable methods for training and deploying ChatGPT, in order to reduce its carbon footprint and energy consumption.


In order for reserach into ChatGPT to advance however, collaboration across disciplines on ChatGPT and other AI systems is very important.<ref>van Dis, E. A., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023). ChatGPT: five priorities for research. ''Nature'', ''614''(7947), 224-226.</ref>
In order for research into ChatGPT to advance however, collaboration across disciplines on ChatGPT and other AI systems is very important.<ref>van Dis, E. A., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023). ChatGPT: five priorities for research. ''Nature'', ''614''(7947), 224-226.</ref>


====ChatGPT in academic education====
====ChatGPT in academic education====
Line 142: Line 138:
To look further into the fear of teachers that students will outsource all their work to ChatGPT, we will look another paper written by García-Peñalvo that reviews previous literature on ChatGPT. The most controversial issue with ChatGPT is the possibility of students using it as an easy solution to write essays without putting in the necessary effort. Because of this, they won’t acquire the needed knowledge for their course. However, the problem might not be the tool itself, but that the assessment techniques of educational institutions have become outdated However, the paper argues that prohibiting chatGPT is not the way to go. Instead, teachers and students should learn how to use the tool to their advantage.<ref>García-Peñalvo, F. J. (2023). La percepción de la Inteligencia Artificial en contextos educativos tras el lanzamiento de ChatGPT: disrupción o pánico. ''Education in the Knowledge Society'', ''24'', e31279. <nowiki>https://doi.org/10.14201/eks.31279</nowiki></ref>  
To look further into the fear of teachers that students will outsource all their work to ChatGPT, we will look another paper written by García-Peñalvo that reviews previous literature on ChatGPT. The most controversial issue with ChatGPT is the possibility of students using it as an easy solution to write essays without putting in the necessary effort. Because of this, they won’t acquire the needed knowledge for their course. However, the problem might not be the tool itself, but that the assessment techniques of educational institutions have become outdated However, the paper argues that prohibiting chatGPT is not the way to go. Instead, teachers and students should learn how to use the tool to their advantage.<ref>García-Peñalvo, F. J. (2023). La percepción de la Inteligencia Artificial en contextos educativos tras el lanzamiento de ChatGPT: disrupción o pánico. ''Education in the Knowledge Society'', ''24'', e31279. <nowiki>https://doi.org/10.14201/eks.31279</nowiki></ref>  


As mentioned times before, ChatGPT has it obvious limitations. A controversial view in this paper written by Thorp, the author of the article is not necesseraly worried about the use of chatGPT in education as ‘it did well finding factual answers, but the scholarly writing still has a long way to go’. He thinks it pushes academics to design their courses in such a way that they are not easily solved by AI. The author is more worried for the influence of chatGPT on the world of literature.<ref>Thorp, H. H. (2023). ChatGPT is fun, but not an author. ''Science'', ''379''(6630), 313. <nowiki>https://doi.org/10.1126/science.adg7879</nowiki></ref>  
As mentioned times before, ChatGPT has it obvious limitations. A controversial view in this paper written by Thorp, the author of the article is not necessarily worried about the use of ChatGPT in education as ‘it did well finding factual answers, but the scholarly writing still has a long way to go’. He thinks it pushes academics to design their courses in such a way that they are not easily solved by AI. The author is more worried for the influence of chatGPT on the world of literature.<ref>Thorp, H. H. (2023). ChatGPT is fun, but not an author. ''Science'', ''379''(6630), 313. <nowiki>https://doi.org/10.1126/science.adg7879</nowiki></ref>  


While chatGPT has its limitations, it has the potential to replace humans in routine tasks such as homework grading, potentially spelling the end of traditional essay writing assignments. "ChatGPT User Experience: Implications for Education" explores the user experience of chatGPT and its potential impact on education. He also states that using AI tools to perform certain tasks ‘should be a part of the educational goals in the future’. The author concludes that teachers need to change their assignments to make it harder for students to use AI, but also notes that using AI in assignments to engage students in learning is a viable option. <ref>Zhai, X. (2022). ChatGPT User Experience: Implications for Education. ''Social Science Research Network''. <nowiki>https://doi.org/10.2139/ssrn.4312418</nowiki></ref>
While ChatGPT has its limitations, it has the potential to replace humans in routine tasks such as homework grading, potentially spelling the end of traditional essay writing assignments. "ChatGPT User Experience: Implications for Education" explores the user experience of ChatGPT and its potential impact on education. He also states that using AI tools to perform certain tasks ‘should be a part of the educational goals in the future’. The author concludes that teachers need to change their assignments to make it harder for students to use AI, but also notes that using AI in assignments to engage students in learning is a viable option. <ref>Zhai, X. (2022). ChatGPT User Experience: Implications for Education. ''Social Science Research Network''. <nowiki>https://doi.org/10.2139/ssrn.4312418</nowiki></ref>


To summarize, ChatGPT is a powerful tool with many functionalities, from writing texts and composing music to solving mathematical problems and playing games. However, it is not without its limitations, such as over-optimization, algorithmic biases, and the potential for incorrect responses due to insufficient training data. The future research priorities include exploring the ethical implications of AI-generated content, detecting and addressing bias in its outputs, improving the interpretability of ChatGPT and investigating the potential of ChatGPT in domains beyond language.  
To summarize, ChatGPT is a powerful tool with many functionalities, from writing texts and composing music to solving mathematical problems and playing games. However, it is not without its limitations, such as over-optimization, algorithmic biases, and the potential for incorrect responses due to insufficient training data. The future research priorities include exploring the ethical implications of AI-generated content, detecting and addressing bias in its outputs, improving the interpretability of ChatGPT and investigating the potential of ChatGPT in domains beyond language.  
Line 150: Line 146:
In academic education, ChatGPT has the potential to revolutionize the way we access and use information, but it also raises concerns about academic integrity, student engagement, and the role of teachers in the learning process. Further research is needed to develop secure methods for online assessments and determine the rightful author of AI-generated content. Despite the challenges, AI systems like ChatGPT have the potential to democratize access to education and make it more inclusive.<br />
In academic education, ChatGPT has the potential to revolutionize the way we access and use information, but it also raises concerns about academic integrity, student engagement, and the role of teachers in the learning process. Further research is needed to develop secure methods for online assessments and determine the rightful author of AI-generated content. Despite the challenges, AI systems like ChatGPT have the potential to democratize access to education and make it more inclusive.<br />


==Problem statement and objectives==
==Problem statement==
'''Problem statement:''' How could the use of ChatGPT affect students in reaching the learning goals of the course 4WBB0 at the TU/e?
In the present study, we investigate the influence of ChatGPT on students in reaching certain learning objectives. Therefore, our problem statement is as follows:
 
'''''How could the use of ChatGPT affect students in reaching the learning objectives of the course 4WBB0 at the TU/e?'''''


'''Objectives:'''
===Objectives:===


*Upcoming use of AI in academic settings,
*Upcoming use of AI in academic settings,
*how to design this so that it will be accepted by teachers
*how to design this so that it will be accepted by teachers
*How AI can improve learning
*How AI can improve learning
*Acceptence towards AI in the classroom
*Acceptance towards AI in the classroom
*Looking at ChatGPT as main example
*Looking at ChatGPT as main example
*Teachers at the TUe, their thoughts about AI (in their courses)
*Teachers at the TU/e, their thoughts about AI (in their courses)


==Users==
===Users===


===Target group===
====Target group====
The target group of this project are the teachers of the TU/e who designed and teach the course engineering design (4WBB0) and students who followed this course. In this research the perspective of the teachers and students on the use of chatGPT in this course is reviewed. The input of students will be used to get a better image of how students intend to use chatGPT. The input of the teachers will be used to understand the learning objectives of the course engineering design and to determine the effect of chatGPT on those learning objectives. The end product will be a deliverable useful for the course coordinators of engineering design.    
The target group of this project are the teachers of the TU/e who designed and teach the course engineering design (4WBB0) and students who followed this course. In this research the perspective of the teachers and students on the use of chatGPT in this course is reviewed. The input of students will be used to get a better image of how students intend to use chatGPT. The input of the teachers will be used to understand the learning objectives of the course engineering design and to determine the effect of chatGPT on those learning objectives. The end product will be a deliverable useful for the course coordinators of engineering design.    


===Requirements===
====Requirements====
Teachers want their students to deliver autonomous work, ChatGPT in academic settings can help with this. Teachers want to transfer their knowledge as effectively and efficiently as possible, using the current state of art technologies. AI technologies, like ChatGPT can help with this, but what teachers require is yet to be determined. The requirements will be determined by distributing surveys among students of the TU/e and conducting interviews with teachers, based on the survey results, in combination with literature studies. Those results will then be combined into a piece of well-considered advice for the teachers at the TU/e, consisting of guidelines for a course for the teachers of how to make exercises and tests for students that can benefit from using ChatGPT, but not be made solely with ChatGPT.  Which they will be able to use to guide them through the landscape of ChatGPT use in academic education and give them tools to keep transferring their knowledge to students.     
Teachers want their students to deliver autonomous work, ChatGPT in academic settings can help with this. Teachers want to transfer their knowledge as effectively and efficiently as possible, using the current state of art technologies. AI technologies, like ChatGPT can help with this, but what teachers require is yet to be determined. The requirements will be determined by distributing surveys among students of the TU/e and conducting interviews with teachers, based on the survey results, in combination with literature studies. Those results will then be combined into a piece of well-considered advice for the teachers at the TU/e, consisting of guidelines for a course for the teachers of how to make exercises and tests for students that can benefit from using ChatGPT, but not be made solely with ChatGPT.  Which they will be able to use to guide them through the landscape of ChatGPT use in academic education and give them tools to keep transferring their knowledge to students.     


==Hypothesis==
==Hypothesis==
''Articles about the use of chatGPT in (academic) education suggest that it is preferred to adopt chatGPT in education as opposed to forbidding it. In this research, this hypothesis will be tested by conducting interviews and sending out surveys to teachers and students of the technical university Eindhoven. The interviews and surveys will give in insight into the effect of chatGPT on the learning objectives of courses and whether they will be achieved if chatGPT is used with a focus on the course engineering design.''  
''Articles about the use of ChatGPT in (academic) education suggest that it is preferred to adopt ChatGPT in education as opposed to forbidding it. In this research, this hypothesis will be tested by conducting interviews and sending out surveys to teachers and students of the technical university Eindhoven. The interviews and surveys will give in insight into the effect of ChatGPT on the learning objectives of courses and whether they will be achieved if ChatGPT is used with a focus on the course engineering design.''  


The course engineering design has 13 learning goals. Each one could be affected by the use of ChatGPT. Considering the literature and the known capabilities of chatGPT, a hypothesis for each learning goal can be made.   
The course engineering design has 13 learning goals. Each one could be affected by the use of ChatGPT. Considering the literature and the known capabilities of ChatGPT, a hypothesis for each learning goal can be made.   


*'''Execute a generic design process'''
*'''Execute a generic design process'''
Line 273: Line 271:
General hypothesis:
General hypothesis:


Most of the learnig goals of the course engineering design could be affected by the use of chatGPT, therefore the learning goals of the course might not be achieved by the students who use chatGPT.   
Most of the learning goals of the course engineering design could be affected by the use of ChatGPT, therefore the learning goals of the course might not be achieved by the students who use ChatGPT.   


==Methode==
==Method==
To address the research question of the study, a mixed-methods approach was adopted. This involved the use of both a survey and interviews to obtain a comprehensive understanding of the participants' opinions and experiences with ChatGPT in the context of Engineering Design.


===Surveys, interviews with students and teachers===
The survey was sent to university students who had completed the Engineering Design course at TU/e, with the aim of gaining insights into their usage and perception of ChatGPT. The survey included questions related to the frequency of ChatGPT usage, perceived usefulness of the tool, and its impact on reaching the learning objectives of Engineering Design.
To get an answer to the problem statement there have been conducted multiple interviews with experts and regular users of chatGPT to gain insight into their opinion of the use of chatGPT in the course engineering design and its impact on achieving the learning objectives.  


There have been conducted interviews with multiple students who use chatGPT often and have used it for a course on the TUE. These students were chosen to get a perception of why and how students use chatGPT and how this usage could influence the learning goals of a course like engineering design. The students that were interviewed all study at the TUE and have completed the course engineering design.
Furthermore, a series of interviews were conducted with both a professor from the Engineering Design department and frequent users of ChatGPT: which will be called experts. The interviews aimed to provide a deeper understanding of the participants' opinions and experiences with ChatGPT, specifically in the context of Engineering Design. The interviews with the student experts aimed to capture their motivation for using ChatGPT, how they use the tool, and their perception of its impact on the Engineering Design learning goals. The interview with the professor aimed to capture the impact they believe ChatGPT has on the learning objectives, and their overall opinion on the use of ChatGPT.


An interview has also been conducted with an associate professor of the TUE who intentionally lets his students use chatGPT in one of his courses. He was asked for an interview to give insight into the point of view of teachers and the impact chatGPT has on the learning objectives of courses, especially engineering design.
Together, the survey and interviews aimed to obtain a comprehensive understanding of the participants' opinions regarding the use of ChatGPT in the Engineering Design course and its impact on achieving the learning objectives within this course.


Next to the interviews, a survey was sent out to students to get a broader insight into the opinions about and usage of chatGPT. The students the survey was sent to study at the TUE and have completed the course engineering design.
===Survey===
The survey consists of questions about the participants' general knowledge and experience about ChatGPT, and how this tool could affect them reaching the learning goals of Engineering Design.  


===script in appendix===
=====Participants=====
The survey was distributed to students who are enrolled at the TU/e and who have finished the course Engineering Design, which are 2nd year and higher students. There were a total of 44 responses.


===discuss thematic analysis===
=====Procedure=====
The interviews have been analyzed with the use of thematic analysis. Thematic analysis makes use of finding patterns in the answers to the interview questions.
The outline of the survey was as follows (see appendix 2): a brief section about their general knowledge of ChatGPT, their opinions and experience with using the chatbot as an academic tool, and finally, a more in-depth section with questions about hypotheticals of using ChatGPT within the course Engineering Design and what the affects of it would be on achieving all the learning goals. The first two sections were to measure the proficiency of the participants, and the latter to measure the effects of ChatGPT on achieving certain learning goals.
 
===Interview with ChatGPT experts===
Student experts who frequently use ChatGPT were interviewed to gain a better understanding of how and why they use ChatGPT, and how this usage could impact achieving the learning goals of Engineering Design.
 
=====Participants=====
The focus was on university students who were all enrolled at the TU/e and had successfully completed the Engineering Design course. A total of five participants were recruited, one female and four males, ranging from the ages 20 to 23 (M<sub>age</sub>= 21, SD = 1.79, 20% female). The participants were sampled by the convenience sampling method, all the participants had to be students at the University of Eindhoven, and they should have followed the course Engineering Design, since this was a crucial part for our research.
 
=====Procedure=====
The outline of the interview was as follows: a brief section about their general knowledge of ChatGPT, their opinions and experience with using the chatbot as an academic tool, and finally, a more in-depth section with questions about hypotheticals of using ChatGPT within the course Engineering Design and what the affects of it would be on achieving all the learning goals. The first two sections were to measure the proficiency of the participants, and the latter to measure the effects of ChatGPT on achieving certain learning goals. We asked open questions that were mentioned in the interview script, see appendix 3. This was followed by transcribing the interview and discussing the interview by adding codes to the transcript. Afterward, similar codes were placed in certain themes. These themes were then analyzed carefully, which gave a certain conclusion of the interview.
 
===Interview with university professor===
In addition, an interview was conducted with an associate professor from TU/e who teaches the course Engineering Design. This interview aimed to provide insights into the perspectives of educators regarding the use of ChatGPT and its impact on learning outcomes, particularly in the context of Engineering Design.
 
=====Participants=====
The participant for this interview was Joris Remmers, a social professor of Mechanical Engineering, whose role within Engineering Design is to organize assessments and set up the organizational structure of it. He is responsible for the assessment setting and rubrics.
 
=====Procedure=====
The outline of the interview was as follows: a brief section about his general knowledge of ChatGPT, his opinions and experience with using the chatbot as an academic tool, and finally, a more in-depth section with questions about hypotheticals of using ChatGPT within the course Engineering Design and what the affects of it would be on achieving all the learning goals. The first two sections were to measure the proficiency of the participant, and the latter to measure the effects of ChatGPT on achieving certain learning goals. We asked open questions that were mentioned in the interview script, see appendix <s>4</s>. If the participant did not give a clear answer, we asked follow-up questions to get more in-depth answers. This was followed by transcribing the interview and discussing the interview by adding codes to the transcript. Afterward, similar codes were placed in certain themes. These themes were then analyzed carefully, which gave a certain conclusion of the interview.
 
===Thematic analysis===
All the interviews have been analyzed with the use of thematic analysis. Thematic analysis makes use of finding patterns in the answers to the interview questions. The thematic analysis was conducted via Dedoose<ref>“Home | Dedoose”. <nowiki>https://www.dedoose.com/</nowiki></ref>. Codes for themes in the interview were made based on the apparent feelings the interviewees have about ChatGPT and its use. Three main groups were established: Features, what interviewees use it for and why they use it; feelings, what the general feeling of interviewees is about ChatGPT; and warnings, what interviewees think is a drawback of ChatGPT. In table 'Codes for the themes' all the codes' descriptions are given.
{| class="wikitable mw-collapsible"
|+Codes for the themes
!Overarching theme
!Code
!Discription
|-
| -
|Academic usage
|A mentioning of how ChatGPT can be used in academic circumstances.
|-
| -
|Improved capabilities
|The notion that ChatGPT is better at doing a certain job than conventional technology or humans.
|-
| -
|Inevitable use
|When someone mentioned ChatGPT would be used now or in the future, regardless of actions that would be taken against it.
|-
| -
|New learning goal
|When a new possible learning goal was metioned that arose due to ChatGPT.
|-
| -
|Pluripotent
|When a interviewee mentioned that they think ChatGPT can do everything or that it has no boundries.
|-
|Features
|Coding Help
|When ChatGPT was used to write code, help with understanding code, debugging code or optimizing code.
|-
|Features
|Creativity
|When ChatGPT got used to conduct a creative task where it has to recombine ideas into a new one like coming up with new ideas for a brainstorm or write a caption for a picture.
|-
|Features
|Ease of use
|When it is mentioned that ChatGPT was easy to use.
|-
|Features
|Fun
|When ChatGPT was used to mess around, so there was no goal for using it except for momentairy happiness.
|-
|Features
|Providing information
|When ChatGPT was used to gain information.
|-
|Features
| -Guidance
|When ChatGPT was used to gain information on how to do a certain task.
|-
|Features
|Summarizing
|When ChatGPT was used to create a summary af a piece of text or a complex subject.
|-
|Features
|Time saving
|When the reason for using ChatGPT was given to be saving time compared to other methods of doing the same task.
|-
|Features
|Writing help
|When ChatGPT was used to help write, for example rewrite a text, improve its grammar or flat out writing it all.
|-
|Feelings
|Neutral
|The general opinion of about ChatGPT was given to be neutral, the interviewee was not possitive nor negative about the use of it.
|-
|Feelings
|Possitive outlook
|When the interviewee mentioned they saw a possitive effect of ChatGPT in the future.
|-
|Warning
|Better to do it yourself
|When the interviewee mentioned they thought a person can do the job better than ChatGPT
|-
|Warning
|Depends on user knowledege
|The capabilities of ChatGPT depend on how the user uses it. THe more knowledge the suer has, the more ChatGPT can do. In the same way, it is not as usefull in the hands of a inexperienced user.
|-
|Warning
|Hinderance to learning
|When it was mentioned that current learning goals would be unable to be achieved, since CahtGPT would do the work.
|-
|Warning
|Steep learning curve
|It is hard for new users to properly learn ChatGPT at the start, a lot of learning needs to be done before ChatGPT can be used prpperly.
|-
|Warning
|Limited
|When it was noted that ChatGPT could not do something.
|-
|Warning
|Unpersonal
|When it was mentioned that ChatGPT created a gap between the user and its goal.
|}
These codes are applied to the interviews when their description matches what is mentioned. What follows from it is thematic analysis code occurrence table.


==Results==
==Results==
Line 305: Line 421:


===Interview analysis===
===Interview analysis===
A very concise summary of the interviews with frequent users can be seen in the table below.
A very concise summary of the interviews with the ChatGPT experts and professor can be seen in the table below: these tables depict whether these participants think it would be likely to reach the mentioned learning goal with the use of ChatGPT.
{| class="wikitable"
{| class="wikitable"
|+
|+Interview ChatGPT experts
!
!Learning objective
!F
!F
!M
!M
Line 392: Line 508:
|No
|No
|}
|}
{| class="wikitable"
|+Interview with professor
!Learning objective
!Y/N
!Explanation
|-
|Execute a generic design process
|Yes
|Students still need to do all the steps themselves, but ChatGPT can be of assistance
|-
|Formulate a design goal
|No
|This is a creative part and requires out-of-the-box thinking
|-
|Define the functional and technical specifications
|Yes
|This is objective knowledge that can be looked up
|-
|Generate an elaborate list of realization possibilities for the different functions of the design
|No
|This is the most creative part - ChatGPT is not innovative enough
|-
|Select a number of design concepts from an extensive list of realization possibilities
|Yes
|Text-book Google work, so ChatGPT can do it, since it can follow strict procedures
|-
|Make a final design choice between a number of concepts
|No
|ChatGPT can be used to structure the concepts, but students ultimately need to reflect on their own preferences
|-
|Develop a detailed design that meets the specifications
|Partly
|This part requires thinking, combining results, solving equations and doing the actual work; ChatGPT can help with parts of it, but not entirely
|-
|Develop and execute a test plan for the prototype
|Yes
|Can be done by ChatGPT
|-
|Evaluate a prototype based on test results and give an advice for redesign
|Yes
|ChatGPT can compare data easily, especially if it's about software design
|-
|Reflect on the design and on the design process
|No
|This requires your own thoughts and experiences
|-
|Write a design report describing the design process, the foundation of the choices made and the evaluation of the delivered design
|Yes
|It's wonderful for (improving) report writing
|}
''[Insert visual to compare these table]''
====Thematic analysis====
<br />
[[File:Code application in Interviews.png|center|thumb|800x800px|Thematical analysis code occurance from the interviews conducted with students who are proficient users of ChatGPT.]]


==Interpretation==
==Interpretation==
Line 402: Line 574:
[Users that indicate many learning goals can be reached also are very frequent users and vice versa]  
[Users that indicate many learning goals can be reached also are very frequent users and vice versa]  


This discrepancy could be explained by two factors combined: the skill of the student in using ChatGPT and the skill of the user in the learning objectives of the course. A student with less knowledge of the learning objectives and high proficiency with ChatGPT will be significantly more likely to view the technology as useful, than a student which can already quite efficiently obtain some learning objectives and is less proficient with ChatGPT.  
This discrepancy could be explained by two factors combined: the skill of the student in using ChatGPT and the skill of the user in the learning objectives of the course. A student with less knowledge of the learning objectives and high proficiency with ChatGPT will be significantly more likely to view the technology as useful, than a student which can already quite efficiently obtain some learning objectives and is less proficient with ChatGPT.
 
===Interviews ChatGPT expert===
With the thematic analysis general notions about ChatGPT became apparent. The three main reasons ChatGPT is used are for gaining information, help with writing and saving time. One interviewee said that they used ChatGPT as "a search engine instead of google", This goes to show, that these reasons are linked with one another.
 
It is interesting to note that all interviewees were aware of drawbacks. The most mentioned disadvantage of ChatGPT was that it was limited in its use. Interviewees mentioned that it could "Only do straightforward stuff" and "It should be used wisely". This last remark is due to what another interviewee said: "[ChatGPT] hallucinates and is not up to date". The not up to date part refers to the data ChatGPT is trained on [the youngest data it was trained on comes from 2011 (source?)]
 
=== Interview university professor ===
The interviewee was overall positive about his experience with ChatGPT and even encourages his students to use it. The main benefits were: improving students' report writing, using it as a more efficient search engine, and helping with speeding up doing ordinary tasks, such as googling for literature.
 
However, according to him, the main drawback of the chatbot is that it is a very linguistic tool. He emphasized the fact that ChatGPT was not creative enough and could not generate innovative ideas due to its limited database.
 
These statements are backed up by his thoughts on whether it would be likely to reach the learning objectives of Engineering Design with the use of ChatGPT. When the objective requires more creative thinking, out-of-the-box solutions and own experiences, it was quickly decided that ChatGPT alone would not be sufficient to achieve this goal. On the other hand, more standard goals which require objective knowledge or a strict procedure, could be easily done by ChatGPT.
<br />
==Approach, milestones and deliverables==
==Approach, milestones and deliverables==


Line 681: Line 866:
*Advice for future
*Advice for future


==Background literature==
<ref>Kim, N. J., & Kim, M. K. (2022). Teacher’s Perceptions of Using an Artificial Intelligence-Based Educational Tool for Scientific Writing. In ''Frontiers in Education'' (p. 142). Frontiers.</ref>This paper regards Teacher’s Perceptions of Using an Artificial Intelligence-Based Educational Tool for Scientific Writing. This study investigated how teachers perceived an AI-enhanced scaffolding system developed to support students’ scientific writing for STEM education. Teachers are worried: What if the introduction of AI will gradually reduce our role in the classroom. What should we do? Should we support AI?  They feared that the AI would reduce their role to assistants and they also questioned the accuracy and reliability of the information generated by the system. For AI to be successfully integrated into STEM education, it is necessary for the roles and relationships between students and teachers to be redefined and for educators to be fully trained on best practices of using AI pedagogical techniques. There is still a trend among educators to hold negative impressions on educational technology. By changing teachers’ current negative perceptions of educational technology, the acceptance of AI as a new type of educational tool and its implementation in schools is possible. The younger generation of teachers, who have more experience with educational technology as both educators and as students, is more interested in exploring new digital technology and potentially incorporating technology into their lessons. Experience with AI in any of these contexts may reduce teachers’ reluctance to use AI for educational purposes.
<ref>Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning. ''Available at SSRN 4337484''.</ref>This paper regards Education in the Era of Generative Artificial Intelligence (AI). The article "Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning" discusses the potential benefits of using generative AI, such as ChatGPT, in education. The article suggests that AI can help teachers and students by providing personalized feedback, facilitating communication and collaboration, and creating engaging and interactive learning experiences. The article also highlights some of the challenges of using AI in education, including issues related to data privacy and the need for human oversight to ensure that AI is used in a responsible and ethical manner. The article concludes by calling for more research and experimentation to better understand the potential benefits and challenges of using AI in education.
<ref>Lindner, A., Romeike, R., Jasute, E., & Pozdniakov, S. (2019). Teachers’ perspectives on artificial intelligence. In ''12th International conference on informatics in schools,“Situation, evaluation and perspectives”, ISSEP''.</ref>The article "Teachers' Perspectives on Artificial Intelligence" explores the attitudes and perceptions of teachers towards the use of artificial intelligence (AI) in education. Based on a survey of K-12 teachers in the United States, the article found that while many teachers are interested in using AI to improve their teaching practice, they also have concerns about the potential impact of AI on student learning, privacy, and job security. The article highlights the need for more education and training on AI for teachers, as well as the importance of involving teachers in the development and implementation of AI tools in education. The article concludes by calling for further research on how to best integrate AI into education in a way that benefits both teachers and students.
<ref>Qadir, J. (2022). Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education.</ref>The article "Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education" explores the potential benefits and challenges of using generative AI, such as ChatGPT, in engineering education. The article suggests that AI has the potential to enhance engineering education by providing personalized feedback, facilitating communication and collaboration, and creating interactive learning experiences. However, the article also highlights some of the challenges of using AI in engineering education, including issues related to bias, the need for human oversight, and the limitations of AI in capturing the full complexity of engineering problem-solving. The article concludes by calling for further research and experimentation to better understand the potential benefits and challenges of using AI in engineering education, as well as the need for ongoing dialogue between educators, engineers, and AI developers to ensure that AI is used in a responsible and ethical manner.
<ref name=":0">Alam, A. (2021). Possibilities and Apprehensions in the Landscape of Artificial Intelligence in Education. ''2021 International Conference on Computational Intelligence and Computing Applications (ICCICA)''. <nowiki>https://doi.org/10.1109/iccica52458.2021.9697272</nowiki></ref>This paper starts with a general introduction to AI, how it developed over the years, and what these developments entail.
*AI in program coding (1950s)
*AI in rules-based expert systems (late 1970’s)
*AI-grounded automatic data processing systems (mid-1980’s)
*Machine learning integrated AI (mid-2000’s)
In education, AI is still in the early stages, because there is more focus on the development of AI than on the application of AI in new fields. This research states that there will be two distinct major effects of AI on education. First, education needs to prepare itself for the fast changes in competence needed for jobs, as some jobs will disappear but most jobs will keep changing over time. Second, there will be changes in the pedagogical techniques needed in the classroom. AI will be able to relieve the teacher from some of the work. However, the teacher will need additional tools to understand the statistical results from the AI (for example from a statistical analysis of the performance of the student). This paper goes on by talking about the future of AI in education. The Author states that there needs to be a big shift in education toward more personalized education. AI will be able to determine the learning style of an individual student. AI will help teachers in content delivery and other instructions, but in the future real life, human teachers might become obsolete. 
<ref>Limna, Pongsakorn and Jakwatanatham, Somporch and Siripipattanakul, Sutithep and Kaewpuang, Pichart and Sriboonruang, Patcharavadee, A Review of Artificial Intelligence (AI) in Education during the Digital Era (July 2022). Advance Knowledge for Executives, 1(1), No. 3, 1-9, 2022, Available at SSRN: <nowiki>https://ssrn.com/abstract=4160798</nowiki></ref>This paper talks about AI in education, the need for it, and its benefits and challenges of it. According to the author, AI is a development that has many promising applications in education. The examples that are named are; personalized education, automatic grading systems, and predictive analytic tools. These applications will relieve the teachers from those tasks which gives them more time with the students. The paper also talks about challenges that come along with AI, like concerns about safety, security, and privacy.   
<ref>Alam, A. (2021b). Should Robots Replace Teachers? Mobilisation of AI and Learning Analytics in Education. ''2021 International Conference on Advances in Computing, Communication, and Control (ICAC3)''. <nowiki>https://doi.org/10.1109/icac353642.2021.9697300</nowiki></ref>In this paper, the authors review the use of AI in current education, the technical aspects of AI in education, the role of AI in education and the impact of AI in education. The topic of education is spread out into three sub-topics; administration, instructions and learning. The takeaway of this paper is that with the help of AI, teachers will become more efficient which will in result increase the quality of education. Next to that with AI, it will be possible to create a more personalized education for students. AI will have a major impact on education as the tasks AI will be able to do tasks that are not originally designed for computers.     
<ref>Kim, J., Merrill, K., Xu, K., & Sellnow, D. D. (2020). My Teacher Is a Machine: Understanding Students’ Perceptions of AI Teaching Assistants in Online Education. ''International Journal of Human–Computer Interaction'', ''36''(20), 1902–1911. <nowiki>https://doi.org/10.1080/10447318.2020.1801227</nowiki></ref>This paper focuses on the viewpoint of the students and their acceptance of AI. The author uses the technology acceptance model (TAM) to analyze the response of students to AI. This is because the perceived usefulness and the perceived ease of use of AI seem to have a big impact on the acceptance of AI in education. The author sees a future in AI in education as it is a cost-effective way to the shortage of teachers. However to make sure that the AI teaching assistants will be successful the teachers need to be trained to work with them. As research points out that if the teacher is uncomfortable with the AI, the student is less likely to adopt it. Based on the results of this research the author recommends research in this area as it is not known how students at different levels of education will react to the adoption of AI as a teacher.   
<ref>Borenstein, J., & Howard, A. (2020). Emerging challenges in AI and the need for AI ethics education. ''AI and Ethics'', ''1''(1), 61–65. <nowiki>https://doi.org/10.1007/s43681-020-00002-7</nowiki></ref>The authors in this paper review the ethical perspective of AI with a focus on AI in education. The biggest problem that the authors see is the privacy issue of AI. Next to that they also see a challenge in the trustworthiness of AI. It is often that AI is presented as a better alternative than a human, as it is supposedly not biased. However, the designers of AI are biased and can make mistakes. Therefore can we even trust AI more than humans? Another issue that the author comes across is the difficulty of addressing the ethical challenges of AI. Therefore it is important to make the designers of AI aware of the ethical dilemmas that come along with new technology and make them aware of their responsibilities. To achieve this it is important that there mandatory teaching or at least education available around this topic. 
<ref>Fan Ouyang, Pengcheng Jiao (2021). Artificial intelligence in education: The three paradigms. ''Computers and Education: Artificial Intelligence'', Volume 2, 100020, ISSN 2666-920X. Retrieved from https://doi.org/10.1016/j.caeai.2021.100020.</ref>In this articile, F. Ouyang and P. Jiao reflect on the three paradigms of artificial intelligence in educaiton: AI-directed, AI-supported, and AI-empowered.
*AI-directed (learner-as-recipient): AI is used to represent knowledge models and direct cognitive learning.
*AI-supported (learner-as-collaborator): AI is used to support learners while they work as collaborators with AI
*AI-empowered (learner-as-leader): AI is used to empower learning while learners take agency
{| class="wikitable"
|+Table 1. Three paradigms of AIEd
!Paradigm
!Theoretical underpinning
!Implementations
!AI techniques
|-
|1. AI-Directed
|Behaviorism
|Intelligent Tutoring Systems (ITSs): software that tracks students' work, adjusts feedback and provides hints along the way)
|AI based on statistical relational techniques
|-
|2. AI-supported
|Cognitive, social constructivism
|Dialogue-based Tutoring Systems (DTSs); Exploratory Learning Environments (ELEs)
|Bayesian network, natural language processing,
Markov decision trees
|-
|3. AI-empowered
|Connectivism, Complex adaptive system
|The human-computer cooperation; Personalized/adaptive learning
|The brain-computer interface, machine learning, deep learning
|}
<ref>Lameras, P., & Arnab, S. (2021). Power to the Teachers: An Exploratory Review on Artificial Intelligence in Education. ''Information'', ''13''(1), 14. MDPI AG. Retrieved from http://dx.doi.org/10.3390/info13010014</ref>P. Lameras and S. Arnab explored and analyzed what Artificial Intelligence means in Education (AIED) in their article "Power to the Teachers: An Exploratory Review on Artificial Intelligence in Education". The main take-away is that adaptivity and personalization are the innovation that AIED can offer to help students to learn and develop skills that are relevant to their own needs and experiences. However, it is important to help teachers to develop necessary digital competencies and skills for using AIED applications and tools in ethical and informed ways to enhance the student learning experience and attainment of learning outcomes. The findings of this review can contribute to developing a better understanding of how artificial intelligence may enhance teachers' roles as catalysts in designing and visualizing AI-enabled learning. As a result, more useful AI-systems specialized in pedagogy will be developed.
<ref>Beth McMurtrie (2018, August 12). How Artificial Intelligence Is Changing Teaching. ''The Chronicle of Higher Education''(1). Retrieved from https://www.su.edu/conservatory/files/2018/09/How-Artificial-Intelligence-is-Changing-Teaching.pdf</ref>As Artificial Intelligence is becoming more apparent in education, it changes the way of teaching. Craig Coates, an entomologist at Texas A&M University, combated the cheating by reorienting his course towards writing and discussion, combined with adaptive courseware, also referred to as intelligent tutoring systems; and by using a tool that uses algorithms and analytics to grade submissions, the switch was a success. Advocates say this lets students study at their own pace and frees up the instructor’s time in class to shore up students’ knowledge.
<ref>Shuai Yang & Haicheng Bai (2020). The integration design of artificial intelligence and normal students' Education. ''Journal of Physics: Conference Series'', Volume 1453: Conf. Ser. 1453 012090. https://iopscience.iop.org/article/10.1088/1742-6596/1453/1/012090</ref>The study conducted by S. Yang and H. Bai "The integration design of artificial intelligence and normal students’ Education" outlines four major problems of normal education (school for training teachers) development and explores the idea of using artificial intelligence technology to solve them. The main issues and the proposed solutions are:
{| class="wikitable"
!Problem
!Solution
|-
|Lack of teaching practice experience
|Intelligent tutors/robots: it can make a special learning plan for students according to their interests, habits and learning needs. It can also be a simulation student of upcoming teachers to gain experience.
|-
|Outmoded curriculum
|Expert system: can accurately classify normal students and highlight personalized teaching.
|-
|Lack of independent learning
|Deep learning system: artificial intelligence can help to analyze the data of learners, organize the content according to the data, and carry out intelligent push. Teachers can create teaching resources to meet the needs of curriculum design and students.
|-
|Lack of innovation in teaching methods
|Combination of all of the above.
|}<ref>Mohammad Hosseini, Lisa M. Rasmussen & David B. Resnik (2023) Using AI to write scholarly publications, Accountability in Research, DOI: [https://doi.org/10.1080/08989621.2023.2168535 10.1080/08989621.2023.2168535]</ref>Artificial intelligence and natural language processing systems are becoming increasingly prevalent in various forms of writing. These natural language processing (NLP) systems are trained on large databases of text to produce and refine statistical models that generate natural language responses. However, NLP systems do not "know" the meaning of the text they generate and can make factual and reasoning mistakes, which raises issues of accountability for authors using these systems. The use of NLP systems also raises questions of transparency in regards to authorship credit and contributions, and must be acknowledged in the text and references section of the manuscript.
<ref>Churi, P.P., Joshi, S., Elhoseny, M., & Omrane, A. (Eds.). (2022). Artificial Intelligence in Higher Education: A Practical Approach (1st ed.). CRC Press. https://doi.org/10.1201/9781003184157</ref>The first chapter of this book aims to determine the knowledge and skills that remain for humans in an AI-driven world. It addresses AI's impact on education and highlights the need to develop awareness and understanding of AI. The goal is to create an educational environment where AI supports learners and teachers and prepares students for an AI-dominant future.
<ref>Huh, S. (2023). Recent Issues in Medical Journal Publishing and Editing Policies: Adoption of Artificial Intelligence, Preprints, Open Peer Review, Model Text Recycling Policies, Best Practice in Scholarly Publishing 4th Version, and Country Names in Titles. ''Neurointervention''. https://doi.org/10.5469/neuroint.2022.00493</ref>This article discusses recent developments in policies for medical journal publishing and editing in Korea. The article highlights that many editors in Korea are also publishers and therefore need to keep up with current trends and policies in the industry. The article focuses on six main policies that have emerged, including the use of artificial intelligence tools in publishing, preprint publications, open peer review, model text recycling policies, the updated 4th version of the Principles of Transparency and Best Practice in Scholarly Publishing, and the recommendation to include country names in human studies titles. The article also mentions that the use of AI in writing has increased, including in peer review and plagiarism detection. This shows the current and expected relevance AI has in modern scientific writing.
<ref>Williamson, B., Macgilchrist, F., & Potter, J. (2023). Re-examining AI, automation and datafication in education. ''Learning, Media and Technology'', ''48''(1), 1–5. https://doi.org/10.1080/17439884.2023.2167830</ref>OpenAI launched a preview version of ChatGPT. It is part of the GPT (Generative Pre-trained Transformer) technology that can generate human-like text based on internet data. The release of ChatGPT sparked discussions on the impact of AI on education, with some saying it could render the student essay obsolete and others expressing concerns about cheating and false information. Despite its promise of transforming education, ChatGPT was criticized for its lack of understanding of meaning and content and its association with environmental racism and the interests of tech elites. Some educators called for better assignments that can't be written by an algorithm and for teaching students to use AI ethically. Access to ChatGPT was blocked by some US education departments due to concerns of dependence on technology, loss of privacy, and misuse of AI. This article opens up a range of important issues that are relevant across the educational field.
<ref>Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? ''International Journal of Educational Technology in Higher Education'', ''16''(1). <nowiki>https://doi.org/10.1186/s41239-019-0171-0</nowiki></ref>This paper provides a review of AI applications in higher education, with most of the research being quantitative research from computer science and STEM fields. So this review focuses more on AI utility over the entire academic institute rather than academic integrity and utility by students. Four areas of application were identified: profiling and prediction, assessment, adaptive systems and personalization, and intelligent tutoring systems. The conclusion highlights the need for further exploration of ethical and educational approaches in AI applications in higher education.
<ref>Ventayen, R. J. M. (2023). OpenAI ChatGPT Generated Results: Similarity Index of Artificial Intelligence-Based Contents. ''SSRN Electronic Journal''. <nowiki>https://doi.org/10.2139/ssrn.4332664</nowiki></ref>In the paper “OpenAI ChatGPT Generated Results: Similarity Index of Artificial Intelligence (AI) Based Model“, university director R. Ventayen researched how new AI powered technology enables cheating in the academic community. Since constructing an entire paper or essay using ChatGPT is considered a violation of academic integrity, the chatbot’s ability to pass similarity tests was researched on prompts with the headlines of the university’s publications. Their results show that ChatGPT provides acceptable similarly indices, and fails to provide the same citations or sources as the similar papers with the prompted headlines. Moreover, Quillbot paraphrases more content, which t, which provides plagiarism-free content.
<ref>Floridi, L., & Chiriatti, M. (2020). GPT-3: Its Nature, Scope, Limits, and Consequences. ''Minds and Machines'', ''30''(4), 681–694. <nowiki>https://doi.org/10.1007/s11023-020-09548-1</nowiki></ref>This paper focuses foremost on the irreversibility of GPT-3 and its answers and ideas. It shows its shortcomings in dealing with semantic, mathematical and ethical questions and consequences for the writing industry. E.g, the big amount of written things that these tools can provide far exceeds the limited physical storage for them and marketing will abuse these tools heavily. We need to be critical with what these tools produce and how we use them. For our question regarding academic integrity, the problems with correct citations become apparent and academic texts produced by ChatGPT are often irreversibly produced from ideas of researchers.
<ref>Drori, I., Zhang, S. X., Shuttleworth, R., Tang, L., Lu, A., Elizabeth, K. E., Liu, K. X., Chen, L., Tran, S., Cheng, N., Wang, R., Singh, N. K., Patti, T. L., Lynch, J., Shporer, A., Verma, N., Wu, E., & Strang, G. (2022). A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. ''Proceedings of the National Academy of Sciences of the United States of America'', ''119''(32). <nowiki>https://doi.org/10.1073/pnas.2123433119</nowiki></ref>This article shows how new developments in AI can solve 81% of university-level mathematics questions and even come up with new questions. This poses great advantages to the acquisition of new learning materials but the implications of this also addresses significant pedagogical challenges, with these models being available to students as well.
<ref name=":1" />The aim of the article is to explore the academic and administrative applications of Artificial Intelligence. Artificial Intelligence Applications (AIA) are not only assisting education academically and administratively but also enhance their effectiveness. AIA provides help to teachers in various types of tasks in the shape of Learning Analytics (LA), Virtual Reality (VR), Grading/Assessments (G/A), and Admissions. It minimizes the administrative tasks of a teacher to invest more in teaching and guiding students. AIA adds a significant contribution to enhance student learning, minimize the workload of a teacher, grade/assess the students effectively and easily, and to help in a lot of other administrative tasks.
<ref>Zawacki-Richter, O., Marín, V.I., Bond, M. ''et al.'' Systematic review of research on artificial intelligence applications in higher education – where are the educators?. ''Int J Educ Technol High Educ'' 16, 39 (2019). <nowiki>https://doi.org/10.1186/s41239-019-0171-0</nowiki></ref>This report is an overview of research on AI applications in higher education between 2007 and 2018 through a systematic review. The combination of results gives us four areas of AIEd applications in academic support services, and institutional and administrative services: 1. profiling and prediction, 2. assessment and evaluation, 3. adaptive systems and personalisation, and 4. intelligent tutoring systems. The conclusions reflect on the almost lack of critical reflection of challenges and risks of AIEd, the weak connection to theoretical pedagogical perspectives, and the need for further exploration of ethical and educational approaches in the application of AIEd in higher education.
<ref>Chaudhry, M.A., Kazim, E. Artificial Intelligence in Education (AIEd): a high-level academic and industry note 2021. ''AI Ethics'' 2, 157–165 (2022). <nowiki>https://doi.org/10.1007/s43681-021-00074-z</nowiki></ref>This article presents the focus of latest research in AIEd on reducing teachers’ workload, contextualized learning for students, revolutionizing assessments and developments in intelligent tutoring systems. It also discusses the ethical dimension of AIEd and the potential impact of the Covid-19 pandemic on the future of AIEd’s research and practice.
<ref>Popenici, S.A.D., Kerr, S. Exploring the impact of artificial intelligence on teaching and learning in higher education. ''RPTEL'' 12, 22 (2017). <nowiki>https://doi.org/10.1186/s41039-017-0062-8</nowiki></ref>It investigates educational implications of emerging technologies on the way students learn and how institutions teach and evolve. Pinpointing some challenges for institutions of higher education and student learning in the adoption of these technologies for teaching, learning, student support, and administration and explore further directions for research.
<ref>Hinojo-Lucena F-J, Aznar-Díaz I, Cáceres-Reche M-P, Romero-Rodríguez J-M. Artificial Intelligence in Higher Education: A Bibliometric Study on its Impact in the Scientific Literature. ''Education Sciences''. 2019; 9(1):51. <nowiki>https://doi.org/10.3390/educsci9010051</nowiki></ref>This article anaylises a sample of 132 articles published betwee 2007-2017 about the scientific production on artificial intelligence in higher education. It concludes that, although artificial intelligence is a reality, the scientific production about its application in higher education has not been consolidated. Honestly not that interesting.
<ref>Fyfe, P. How to cheat on your final paper: Assigning AI for student writing. ''AI & Soc'' (2022). https://doi.org/10.1007/s00146-022-01397-z</ref>The paper is about a pedagogical experiment in which undergraduates were assigned to "cheat" by using text-generating AI software (GPT-2) to write their final class essay. The students were asked to reflect on the ethics of using AI in this way, such as what counts as plagiarism, and how working with AI could change their perspectives on writing, authenticity, and creativity. The results showed that composing with GPT-2 opened up the students' perspectives on the ethical use and evaluation of language models, and that their insights on these issues were connected to broader conversations in the humanities about writing and communication. The author of the paper shares the students' experiences and reflections on the experiment.
==Survey==
https://forms.office.com/Pages/DesignPageV2.aspx?subpage=design&FormId=R_J9zM5gD0qddXBM9g78ZOciNcKFkhVJsg7I7H6kiF5UNzNXQzhaOEpWSjZJSDg1WUZBN1lWTkkwUS4u
==Bibliography==
<references /><br />
==Appendix==
===Logbook===
{| class="wikitable"
|+
!Week
!Name
!Total
!Breakdown
|-
|1
|Famke
|8.5h
|Group discussion (2h), Target group and requirements (1/2h), Studied papers (4h), Wrote summary for papers (2h)
|-
|
|Gabriëlle
|7h
|Group discussion (1h), Studied papers (4h), Wrote summary for papers(1.5h), Target group and requirments(1/2h)
|-
|
|My
|8.5h
|Group discussion (2h), Setup of wiki (0.5h), Studied papers (4h), Wrote summary for papers (2h)
|-
|
|Naud
|9h
|Catch up (1h), Gathering information for appeal ERB (2h), Gathering relevant papers (1h), Studied papers (4h) Writing summary of papers (1h)
|-
|
|Niels
|8h
|Group discussion (2h), Studied papers (3.5h), Wrote summary for papers (1.5h), Made Gantt chart (0.5h), made first agenda (0.5h)
|-
|
|Quincy
|7.5h
|Group discussion (2h), Introduction & Problem statement and objectives (1/2h), studied papers (4h), wrote summary for papers (1h)
|-
|2
|Famke
|5.5h
|Group discussion (1h), Target group and requirements (1/2h), Searched for literature (2h), Wrote summary for papers (2h)
|-
|
|Gabriëlle
|7h
|Group discussion (1h), Searched for literature (1/2h), Studied papers (3h),  Wrote summary for papers (1.5h), Group discussion (1/2h), Prepared for tutor meeting(1/2h)
|-
|
|My
|7h
|Group discussion (1h), Searched for literature (1/2h), Studied papers (2h), Wrote summary for papers (1h), Read and organized already studied literature (2.5h)
|-
|
|Naud
|4.5h
|Group discussion (1.5h), Filled in ERB review form (3h)
|-
|
|Niels
|
|
|-
|
|Quincy
|
|
|-
|3
|Famke
|3h 20m
|Tutor meeting (20 min), Group discussion (1h), Worked on survey (2h)
|-
|
|Gabriëlle
|
|Tutor meeting (20 min), worked out target group (40 min)
|-
|
|My
|
|
|-
|
|Naud
|
|Group discussion (1h), Filled in ERB review form (1h), Changed research question and stakehoders (0.5h)
|-
|
|Niels
|
|
|-
|
|Quincy
|
|
|-
|4
|Famke
|
|Tutor meeting (20 min), Group discussion (2.5h), Working on state of the art (4h)
|-
|
|Gabriëlle
|
|
|-
|
|My
|
|
|-
|
|Naud
|
|Tutor meeting (20 min), Group discussion (2.5h), ERB form correspondence (0.5h), updating wiki with survey questions(1h)
|-
|
|Niels
|
|
|-
|
|Quincy
|
|
|-
|5
|Famke
|
|
|-
|
|Gabriëlle
|
|
|-
|
|My
|
|
|-
|
|Naud
|
|
|-
|
|Niels
|
|
|-
|
|Quincy
|
|
|-
|
|
|
|
|}
<br />
<br />


==Survey questions==
=== Survey questions ===
'''<u>Research into ChatGPT</u>'''
'''<u>Research into ChatGPT</u>'''


Line 826: Line 1,304:
*I feel like I would have learned something when using ChatGPT for the engineering design project<br />
*I feel like I would have learned something when using ChatGPT for the engineering design project<br />


==Interview script Teachers==
===Interview script Teachers===
Interviewee:    [name]                    Date:    [date]
Interviewee:    [name]                    Date:    [date]


Line 935: Line 1,413:
<br />
<br />


==Interview Script Student "Experts"==
===Interview Script Student "Experts"===
 


Interviewee:[name] Date:[date]
Interviewee:[name] Date:[date]
Line 985: Line 1,462:
#Do you think you would actually learn these objectives while using ChatGPT?
#Do you think you would actually learn these objectives while using ChatGPT?
#In your opinion, does ChatGPT make the education of students better, or does it hinder them in their development?
#In your opinion, does ChatGPT make the education of students better, or does it hinder them in their development?
==Background literature==
<ref>Kim, N. J., & Kim, M. K. (2022). Teacher’s Perceptions of Using an Artificial Intelligence-Based Educational Tool for Scientific Writing. In ''Frontiers in Education'' (p. 142). Frontiers.</ref>This paper regards Teacher’s Perceptions of Using an Artificial Intelligence-Based Educational Tool for Scientific Writing. This study investigated how teachers perceived an AI-enhanced scaffolding system developed to support students’ scientific writing for STEM education. Teachers are worried: What if the introduction of AI will gradually reduce our role in the classroom. What should we do? Should we support AI?  They feared that the AI would reduce their role to assistants and they also questioned the accuracy and reliability of the information generated by the system. For AI to be successfully integrated into STEM education, it is necessary for the roles and relationships between students and teachers to be redefined and for educators to be fully trained on best practices of using AI pedagogical techniques. There is still a trend among educators to hold negative impressions on educational technology. By changing teachers’ current negative perceptions of educational technology, the acceptance of AI as a new type of educational tool and its implementation in schools is possible. The younger generation of teachers, who have more experience with educational technology as both educators and as students, is more interested in exploring new digital technology and potentially incorporating technology into their lessons. Experience with AI in any of these contexts may reduce teachers’ reluctance to use AI for educational purposes.
<ref>Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning. ''Available at SSRN 4337484''.</ref>This paper regards Education in the Era of Generative Artificial Intelligence (AI). The article "Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning" discusses the potential benefits of using generative AI, such as ChatGPT, in education. The article suggests that AI can help teachers and students by providing personalized feedback, facilitating communication and collaboration, and creating engaging and interactive learning experiences. The article also highlights some of the challenges of using AI in education, including issues related to data privacy and the need for human oversight to ensure that AI is used in a responsible and ethical manner. The article concludes by calling for more research and experimentation to better understand the potential benefits and challenges of using AI in education.
<ref>Lindner, A., Romeike, R., Jasute, E., & Pozdniakov, S. (2019). Teachers’ perspectives on artificial intelligence. In ''12th International conference on informatics in schools,“Situation, evaluation and perspectives”, ISSEP''.</ref>The article "Teachers' Perspectives on Artificial Intelligence" explores the attitudes and perceptions of teachers towards the use of artificial intelligence (AI) in education. Based on a survey of K-12 teachers in the United States, the article found that while many teachers are interested in using AI to improve their teaching practice, they also have concerns about the potential impact of AI on student learning, privacy, and job security. The article highlights the need for more education and training on AI for teachers, as well as the importance of involving teachers in the development and implementation of AI tools in education. The article concludes by calling for further research on how to best integrate AI into education in a way that benefits both teachers and students.
<ref>Qadir, J. (2022). Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education.</ref>The article "Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education" explores the potential benefits and challenges of using generative AI, such as ChatGPT, in engineering education. The article suggests that AI has the potential to enhance engineering education by providing personalized feedback, facilitating communication and collaboration, and creating interactive learning experiences. However, the article also highlights some of the challenges of using AI in engineering education, including issues related to bias, the need for human oversight, and the limitations of AI in capturing the full complexity of engineering problem-solving. The article concludes by calling for further research and experimentation to better understand the potential benefits and challenges of using AI in engineering education, as well as the need for ongoing dialogue between educators, engineers, and AI developers to ensure that AI is used in a responsible and ethical manner.
<ref name=":0">Alam, A. (2021). Possibilities and Apprehensions in the Landscape of Artificial Intelligence in Education. ''2021 International Conference on Computational Intelligence and Computing Applications (ICCICA)''. <nowiki>https://doi.org/10.1109/iccica52458.2021.9697272</nowiki></ref>This paper starts with a general introduction to AI, how it developed over the years, and what these developments entail.
*AI in program coding (1950s)
*AI in rules-based expert systems (late 1970’s)
*AI-grounded automatic data processing systems (mid-1980’s)
*Machine learning integrated AI (mid-2000’s)
In education, AI is still in the early stages, because there is more focus on the development of AI than on the application of AI in new fields. This research states that there will be two distinct major effects of AI on education. First, education needs to prepare itself for the fast changes in competence needed for jobs, as some jobs will disappear but most jobs will keep changing over time. Second, there will be changes in the pedagogical techniques needed in the classroom. AI will be able to relieve the teacher from some of the work. However, the teacher will need additional tools to understand the statistical results from the AI (for example from a statistical analysis of the performance of the student). This paper goes on by talking about the future of AI in education. The Author states that there needs to be a big shift in education toward more personalized education. AI will be able to determine the learning style of an individual student. AI will help teachers in content delivery and other instructions, but in the future real life, human teachers might become obsolete. 
<ref>Limna, Pongsakorn and Jakwatanatham, Somporch and Siripipattanakul, Sutithep and Kaewpuang, Pichart and Sriboonruang, Patcharavadee, A Review of Artificial Intelligence (AI) in Education during the Digital Era (July 2022). Advance Knowledge for Executives, 1(1), No. 3, 1-9, 2022, Available at SSRN: <nowiki>https://ssrn.com/abstract=4160798</nowiki></ref>This paper talks about AI in education, the need for it, and its benefits and challenges of it. According to the author, AI is a development that has many promising applications in education. The examples that are named are; personalized education, automatic grading systems, and predictive analytic tools. These applications will relieve the teachers from those tasks which gives them more time with the students. The paper also talks about challenges that come along with AI, like concerns about safety, security, and privacy.   
<ref>Alam, A. (2021b). Should Robots Replace Teachers? Mobilisation of AI and Learning Analytics in Education. ''2021 International Conference on Advances in Computing, Communication, and Control (ICAC3)''. <nowiki>https://doi.org/10.1109/icac353642.2021.9697300</nowiki></ref>In this paper, the authors review the use of AI in current education, the technical aspects of AI in education, the role of AI in education and the impact of AI in education. The topic of education is spread out into three sub-topics; administration, instructions and learning. The takeaway of this paper is that with the help of AI, teachers will become more efficient which will in result increase the quality of education. Next to that with AI, it will be possible to create a more personalized education for students. AI will have a major impact on education as the tasks AI will be able to do tasks that are not originally designed for computers.     
<ref>Kim, J., Merrill, K., Xu, K., & Sellnow, D. D. (2020). My Teacher Is a Machine: Understanding Students’ Perceptions of AI Teaching Assistants in Online Education. ''International Journal of Human–Computer Interaction'', ''36''(20), 1902–1911. <nowiki>https://doi.org/10.1080/10447318.2020.1801227</nowiki></ref>This paper focuses on the viewpoint of the students and their acceptance of AI. The author uses the technology acceptance model (TAM) to analyze the response of students to AI. This is because the perceived usefulness and the perceived ease of use of AI seem to have a big impact on the acceptance of AI in education. The author sees a future in AI in education as it is a cost-effective way to the shortage of teachers. However to make sure that the AI teaching assistants will be successful the teachers need to be trained to work with them. As research points out that if the teacher is uncomfortable with the AI, the student is less likely to adopt it. Based on the results of this research the author recommends research in this area as it is not known how students at different levels of education will react to the adoption of AI as a teacher.   
<ref>Borenstein, J., & Howard, A. (2020). Emerging challenges in AI and the need for AI ethics education. ''AI and Ethics'', ''1''(1), 61–65. <nowiki>https://doi.org/10.1007/s43681-020-00002-7</nowiki></ref>The authors in this paper review the ethical perspective of AI with a focus on AI in education. The biggest problem that the authors see is the privacy issue of AI. Next to that they also see a challenge in the trustworthiness of AI. It is often that AI is presented as a better alternative than a human, as it is supposedly not biased. However, the designers of AI are biased and can make mistakes. Therefore can we even trust AI more than humans? Another issue that the author comes across is the difficulty of addressing the ethical challenges of AI. Therefore it is important to make the designers of AI aware of the ethical dilemmas that come along with new technology and make them aware of their responsibilities. To achieve this it is important that there mandatory teaching or at least education available around this topic. 
<ref>Fan Ouyang, Pengcheng Jiao (2021). Artificial intelligence in education: The three paradigms. ''Computers and Education: Artificial Intelligence'', Volume 2, 100020, ISSN 2666-920X. Retrieved from https://doi.org/10.1016/j.caeai.2021.100020.</ref>In this articile, F. Ouyang and P. Jiao reflect on the three paradigms of artificial intelligence in educaiton: AI-directed, AI-supported, and AI-empowered.
*AI-directed (learner-as-recipient): AI is used to represent knowledge models and direct cognitive learning.
*AI-supported (learner-as-collaborator): AI is used to support learners while they work as collaborators with AI
*AI-empowered (learner-as-leader): AI is used to empower learning while learners take agency
{| class="wikitable"
|+Table 1. Three paradigms of AIEd
!Paradigm
!Theoretical underpinning
!Implementations
!AI techniques
|-
|1. AI-Directed
|Behaviorism
|Intelligent Tutoring Systems (ITSs): software that tracks students' work, adjusts feedback and provides hints along the way)
|AI based on statistical relational techniques
|-
|2. AI-supported
|Cognitive, social constructivism
|Dialogue-based Tutoring Systems (DTSs); Exploratory Learning Environments (ELEs)
|Bayesian network, natural language processing,
Markov decision trees
|-
|3. AI-empowered
|Connectivism, Complex adaptive system
|The human-computer cooperation; Personalized/adaptive learning
|The brain-computer interface, machine learning, deep learning
|}
<ref>Lameras, P., & Arnab, S. (2021). Power to the Teachers: An Exploratory Review on Artificial Intelligence in Education. ''Information'', ''13''(1), 14. MDPI AG. Retrieved from http://dx.doi.org/10.3390/info13010014</ref>P. Lameras and S. Arnab explored and analyzed what Artificial Intelligence means in Education (AIED) in their article "Power to the Teachers: An Exploratory Review on Artificial Intelligence in Education". The main take-away is that adaptivity and personalization are the innovation that AIED can offer to help students to learn and develop skills that are relevant to their own needs and experiences. However, it is important to help teachers to develop necessary digital competencies and skills for using AIED applications and tools in ethical and informed ways to enhance the student learning experience and attainment of learning outcomes. The findings of this review can contribute to developing a better understanding of how artificial intelligence may enhance teachers' roles as catalysts in designing and visualizing AI-enabled learning. As a result, more useful AI-systems specialized in pedagogy will be developed.
<ref>Beth McMurtrie (2018, August 12). How Artificial Intelligence Is Changing Teaching. ''The Chronicle of Higher Education''(1). Retrieved from https://www.su.edu/conservatory/files/2018/09/How-Artificial-Intelligence-is-Changing-Teaching.pdf</ref>As Artificial Intelligence is becoming more apparent in education, it changes the way of teaching. Craig Coates, an entomologist at Texas A&M University, combated the cheating by reorienting his course towards writing and discussion, combined with adaptive courseware, also referred to as intelligent tutoring systems; and by using a tool that uses algorithms and analytics to grade submissions, the switch was a success. Advocates say this lets students study at their own pace and frees up the instructor’s time in class to shore up students’ knowledge.
<ref>Shuai Yang & Haicheng Bai (2020). The integration design of artificial intelligence and normal students' Education. ''Journal of Physics: Conference Series'', Volume 1453: Conf. Ser. 1453 012090. https://iopscience.iop.org/article/10.1088/1742-6596/1453/1/012090</ref>The study conducted by S. Yang and H. Bai "The integration design of artificial intelligence and normal students’ Education" outlines four major problems of normal education (school for training teachers) development and explores the idea of using artificial intelligence technology to solve them. The main issues and the proposed solutions are:
{| class="wikitable"
!Problem
!Solution
|-
|Lack of teaching practice experience
|Intelligent tutors/robots: it can make a special learning plan for students according to their interests, habits and learning needs. It can also be a simulation student of upcoming teachers to gain experience.
|-
|Outmoded curriculum
|Expert system: can accurately classify normal students and highlight personalized teaching.
|-
|Lack of independent learning
|Deep learning system: artificial intelligence can help to analyze the data of learners, organize the content according to the data, and carry out intelligent push. Teachers can create teaching resources to meet the needs of curriculum design and students.
|-
|Lack of innovation in teaching methods
|Combination of all of the above.
|}<ref>Mohammad Hosseini, Lisa M. Rasmussen & David B. Resnik (2023) Using AI to write scholarly publications, Accountability in Research, DOI: [https://doi.org/10.1080/08989621.2023.2168535 10.1080/08989621.2023.2168535]</ref>Artificial intelligence and natural language processing systems are becoming increasingly prevalent in various forms of writing. These natural language processing (NLP) systems are trained on large databases of text to produce and refine statistical models that generate natural language responses. However, NLP systems do not "know" the meaning of the text they generate and can make factual and reasoning mistakes, which raises issues of accountability for authors using these systems. The use of NLP systems also raises questions of transparency in regards to authorship credit and contributions, and must be acknowledged in the text and references section of the manuscript.
<ref>Churi, P.P., Joshi, S., Elhoseny, M., & Omrane, A. (Eds.). (2022). Artificial Intelligence in Higher Education: A Practical Approach (1st ed.). CRC Press. https://doi.org/10.1201/9781003184157</ref>The first chapter of this book aims to determine the knowledge and skills that remain for humans in an AI-driven world. It addresses AI's impact on education and highlights the need to develop awareness and understanding of AI. The goal is to create an educational environment where AI supports learners and teachers and prepares students for an AI-dominant future.
<ref>Huh, S. (2023). Recent Issues in Medical Journal Publishing and Editing Policies: Adoption of Artificial Intelligence, Preprints, Open Peer Review, Model Text Recycling Policies, Best Practice in Scholarly Publishing 4th Version, and Country Names in Titles. ''Neurointervention''. https://doi.org/10.5469/neuroint.2022.00493</ref>This article discusses recent developments in policies for medical journal publishing and editing in Korea. The article highlights that many editors in Korea are also publishers and therefore need to keep up with current trends and policies in the industry. The article focuses on six main policies that have emerged, including the use of artificial intelligence tools in publishing, preprint publications, open peer review, model text recycling policies, the updated 4th version of the Principles of Transparency and Best Practice in Scholarly Publishing, and the recommendation to include country names in human studies titles. The article also mentions that the use of AI in writing has increased, including in peer review and plagiarism detection. This shows the current and expected relevance AI has in modern scientific writing.
<ref>Williamson, B., Macgilchrist, F., & Potter, J. (2023). Re-examining AI, automation and datafication in education. ''Learning, Media and Technology'', ''48''(1), 1–5. https://doi.org/10.1080/17439884.2023.2167830</ref>OpenAI launched a preview version of ChatGPT. It is part of the GPT (Generative Pre-trained Transformer) technology that can generate human-like text based on internet data. The release of ChatGPT sparked discussions on the impact of AI on education, with some saying it could render the student essay obsolete and others expressing concerns about cheating and false information. Despite its promise of transforming education, ChatGPT was criticized for its lack of understanding of meaning and content and its association with environmental racism and the interests of tech elites. Some educators called for better assignments that can't be written by an algorithm and for teaching students to use AI ethically. Access to ChatGPT was blocked by some US education departments due to concerns of dependence on technology, loss of privacy, and misuse of AI. This article opens up a range of important issues that are relevant across the educational field.
<ref>Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? ''International Journal of Educational Technology in Higher Education'', ''16''(1). <nowiki>https://doi.org/10.1186/s41239-019-0171-0</nowiki></ref>This paper provides a review of AI applications in higher education, with most of the research being quantitative research from computer science and STEM fields. So this review focuses more on AI utility over the entire academic institute rather than academic integrity and utility by students. Four areas of application were identified: profiling and prediction, assessment, adaptive systems and personalization, and intelligent tutoring systems. The conclusion highlights the need for further exploration of ethical and educational approaches in AI applications in higher education.
<ref>Ventayen, R. J. M. (2023). OpenAI ChatGPT Generated Results: Similarity Index of Artificial Intelligence-Based Contents. ''SSRN Electronic Journal''. <nowiki>https://doi.org/10.2139/ssrn.4332664</nowiki></ref>In the paper “OpenAI ChatGPT Generated Results: Similarity Index of Artificial Intelligence (AI) Based Model“, university director R. Ventayen researched how new AI powered technology enables cheating in the academic community. Since constructing an entire paper or essay using ChatGPT is considered a violation of academic integrity, the chatbot’s ability to pass similarity tests was researched on prompts with the headlines of the university’s publications. Their results show that ChatGPT provides acceptable similarly indices, and fails to provide the same citations or sources as the similar papers with the prompted headlines. Moreover, Quillbot paraphrases more content, which t, which provides plagiarism-free content.
<ref>Floridi, L., & Chiriatti, M. (2020). GPT-3: Its Nature, Scope, Limits, and Consequences. ''Minds and Machines'', ''30''(4), 681–694. <nowiki>https://doi.org/10.1007/s11023-020-09548-1</nowiki></ref>This paper focuses foremost on the irreversibility of GPT-3 and its answers and ideas. It shows its shortcomings in dealing with semantic, mathematical and ethical questions and consequences for the writing industry. E.g, the big amount of written things that these tools can provide far exceeds the limited physical storage for them and marketing will abuse these tools heavily. We need to be critical with what these tools produce and how we use them. For our question regarding academic integrity, the problems with correct citations become apparent and academic texts produced by ChatGPT are often irreversibly produced from ideas of researchers.
<ref>Drori, I., Zhang, S. X., Shuttleworth, R., Tang, L., Lu, A., Elizabeth, K. E., Liu, K. X., Chen, L., Tran, S., Cheng, N., Wang, R., Singh, N. K., Patti, T. L., Lynch, J., Shporer, A., Verma, N., Wu, E., & Strang, G. (2022). A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. ''Proceedings of the National Academy of Sciences of the United States of America'', ''119''(32). <nowiki>https://doi.org/10.1073/pnas.2123433119</nowiki></ref>This article shows how new developments in AI can solve 81% of university-level mathematics questions and even come up with new questions. This poses great advantages to the acquisition of new learning materials but the implications of this also addresses significant pedagogical challenges, with these models being available to students as well.
<ref name=":1" />The aim of the article is to explore the academic and administrative applications of Artificial Intelligence. Artificial Intelligence Applications (AIA) are not only assisting education academically and administratively but also enhance their effectiveness. AIA provides help to teachers in various types of tasks in the shape of Learning Analytics (LA), Virtual Reality (VR), Grading/Assessments (G/A), and Admissions. It minimizes the administrative tasks of a teacher to invest more in teaching and guiding students. AIA adds a significant contribution to enhance student learning, minimize the workload of a teacher, grade/assess the students effectively and easily, and to help in a lot of other administrative tasks.
<ref>Zawacki-Richter, O., Marín, V.I., Bond, M. ''et al.'' Systematic review of research on artificial intelligence applications in higher education – where are the educators?. ''Int J Educ Technol High Educ'' 16, 39 (2019). <nowiki>https://doi.org/10.1186/s41239-019-0171-0</nowiki></ref>This report is an overview of research on AI applications in higher education between 2007 and 2018 through a systematic review. The combination of results gives us four areas of AIEd applications in academic support services, and institutional and administrative services: 1. profiling and prediction, 2. assessment and evaluation, 3. adaptive systems and personalisation, and 4. intelligent tutoring systems. The conclusions reflect on the almost lack of critical reflection of challenges and risks of AIEd, the weak connection to theoretical pedagogical perspectives, and the need for further exploration of ethical and educational approaches in the application of AIEd in higher education.
<ref>Chaudhry, M.A., Kazim, E. Artificial Intelligence in Education (AIEd): a high-level academic and industry note 2021. ''AI Ethics'' 2, 157–165 (2022). <nowiki>https://doi.org/10.1007/s43681-021-00074-z</nowiki></ref>This article presents the focus of latest research in AIEd on reducing teachers’ workload, contextualized learning for students, revolutionizing assessments and developments in intelligent tutoring systems. It also discusses the ethical dimension of AIEd and the potential impact of the Covid-19 pandemic on the future of AIEd’s research and practice.
<ref>Popenici, S.A.D., Kerr, S. Exploring the impact of artificial intelligence on teaching and learning in higher education. ''RPTEL'' 12, 22 (2017). <nowiki>https://doi.org/10.1186/s41039-017-0062-8</nowiki></ref>It investigates educational implications of emerging technologies on the way students learn and how institutions teach and evolve. Pinpointing some challenges for institutions of higher education and student learning in the adoption of these technologies for teaching, learning, student support, and administration and explore further directions for research.
<ref>Hinojo-Lucena F-J, Aznar-Díaz I, Cáceres-Reche M-P, Romero-Rodríguez J-M. Artificial Intelligence in Higher Education: A Bibliometric Study on its Impact in the Scientific Literature. ''Education Sciences''. 2019; 9(1):51. <nowiki>https://doi.org/10.3390/educsci9010051</nowiki></ref>This article anaylises a sample of 132 articles published betwee 2007-2017 about the scientific production on artificial intelligence in higher education. It concludes that, although artificial intelligence is a reality, the scientific production about its application in higher education has not been consolidated. Honestly not that interesting.
<ref>Fyfe, P. How to cheat on your final paper: Assigning AI for student writing. ''AI & Soc'' (2022). https://doi.org/10.1007/s00146-022-01397-z</ref>The paper is about a pedagogical experiment in which undergraduates were assigned to "cheat" by using text-generating AI software (GPT-2) to write their final class essay. The students were asked to reflect on the ethics of using AI in this way, such as what counts as plagiarism, and how working with AI could change their perspectives on writing, authenticity, and creativity. The results showed that composing with GPT-2 opened up the students' perspectives on the ethical use and evaluation of language models, and that their insights on these issues were connected to broader conversations in the humanities about writing and communication. The author of the paper shares the students' experiences and reflections on the experiment.
==Survey==
https://forms.office.com/Pages/DesignPageV2.aspx?subpage=design&FormId=R_J9zM5gD0qddXBM9g78ZOciNcKFkhVJsg7I7H6kiF5UNzNXQzhaOEpWSjZJSDg1WUZBN1lWTkkwUS4u
==Bibliography==
<references /><br />
==Appendix==
===Logbook===
{| class="wikitable"
|+
!Week
!Name
!Total
!Breakdown
|-
|1
|Famke
|8.5h
|Group discussion (2h), Target group and requirements (1/2h), Studied papers (4h), Wrote summary for papers (2h)
|-
|
|Gabriëlle
|7h
|Group discussion (1h), Studied papers (4h), Wrote summary for papers(1.5h), Target group and requirments(1/2h)
|-
|
|My
|8.5h
|Group discussion (2h), Setup of wiki (0.5h), Studied papers (4h), Wrote summary for papers (2h)
|-
|
|Naud
|9h
|Catch up (1h), Gathering information for appeal ERB (2h), Gathering relevant papers (1h), Studied papers (4h) Writing summary of papers (1h)
|-
|
|Niels
|8h
|Group discussion (2h), Studied papers (3.5h), Wrote summary for papers (1.5h), Made Gantt chart (0.5h), made first agenda (0.5h)
|-
|
|Quincy
|7.5h
|Group discussion (2h), Introduction & Problem statement and objectives (1/2h), studied papers (4h), wrote summary for papers (1h)
|-
|2
|Famke
|5.5h
|Group discussion (1h), Target group and requirements (1/2h), Searched for literature (2h), Wrote summary for papers (2h)
|-
|
|Gabriëlle
|7h
|Group discussion (1h), Searched for literature (1/2h), Studied papers (3h),  Wrote summary for papers (1.5h), Group discussion (1/2h), Prepared for tutor meeting(1/2h)
|-
|
|My
|7h
|Group discussion (1h), Searched for literature (1/2h), Studied papers (2h), Wrote summary for papers (1h), Read and organized already studied literature (2.5h)
|-
|
|Naud
|4.5h
|Group discussion (1.5h), Filled in ERB review form (3h)
|-
|
|Niels
|
|
|-
|
|Quincy
|
|
|-
|3
|Famke
|3h 20m
|Tutor meeting (20 min), Group discussion (1h), Worked on survey (2h)
|-
|
|Gabriëlle
|
|Tutor meeting (20 min), worked out target group (40 min)
|-
|
|My
|
|
|-
|
|Naud
|
|Group discussion (1h), Filled in ERB review form (1h), Changed research question and stakehoders (0.5h)
|-
|
|Niels
|
|
|-
|
|Quincy
|
|
|-
|4
|Famke
|
|Tutor meeting (20 min), Group discussion (2.5h), Working on state of the art (4h)
|-
|
|Gabriëlle
|
|
|-
|
|My
|
|
|-
|
|Naud
|
|Tutor meeting (20 min), Group discussion (2.5h), ERB form correspondence (0.5h), updating wiki with survey questions(1h)
|-
|
|Niels
|
|
|-
|
|Quincy
|
|
|-
|5
|Famke
|
|
|-
|
|Gabriëlle
|
|
|-
|
|My
|
|
|-
|
|Naud
|
|
|-
|
|Niels
|
|
|-
|
|Quincy
|
|
|-
|
|
|
|
|}
<br />

Revision as of 19:25, 25 March 2023

As written by ChatGPT:

Artificial Intelligence, a wonder of the modern age

A creation made of code, with endless knowledge in its brain

From data analysis to language skills, it's a tool of great worth

Changing the way we live, and opening doors to new growth and mirth


But we must always be mindful, of the impact it may bring

For AI can be used for good or for deceit, and we must choose the right thing

So let us use it wisely, with integrity as our guide

For the future of our world, is shaped by the choices we decide.


Group members

Name Student ID Department
Famke Peek 1459058 Psychology & Technology
Gabriëlle van Heteren 1605305 Biomedical Engineering
My Tran 1620940 Industrial Design
Naud van Rosmalen 1555464 Biomedical Engineering
Niels van Noort 1613928 Biomedical Engineering
Quincy Netteb 1468634 Psychology & Technology

Structure of the Wiki

  1. Abstract
  2. Introduction
  3. State of the art
    1. Also findings from the panel?
  4. Problem statement
  5. Hypothesis
  6. Method
    1. Surveys, interviews with students and teachers
    2. script in appendix
    3. discuss thematic analysis
  7. Results
    1. survey analysis
    2. interview analysis --> thematic analysis
      1. Themes (supported by quotes)
  8. Interpretation
    1. Compare all results
  9. Conclusion (or advise / guidelines?)
  10. Discussion
  11. Future research
  12. Reflection


In appendixes:

- Script interviews

- Thematic analysis

- Data survey

- Transcriptions interviews?

- Background literature

Introduction

The use of AI in education has the potential to completely change the way students learn and teachers teach. AI can provide personalized learning experiences for students, adapt to individual student needs and learning styles, and offer instant feedback to both students and teachers[1]. In addition, AI can also help streamline administrative tasks, such as grading and record-keeping, freeing up teachers to focus on instruction and interaction with students.

However, it is important to note that the implementation of AI in education is still in its early stages and there are many challenges that must be overcome, such as ensuring the privacy and security of student data and addressing ethical considerations and trust issues around the use of AI in the classroom.

So, while AI has the potential to greatly impact and improve education, it is important to approach its implementation with caution and careful consideration of its limitations and potential risks. This and the outlook and trust of teacher in academic education on AI in the classroom is what we will be researching. With the gathered information we will create a recommendation for teachers at the TU/e on how to optimally implement AI in the classroom to benefit both teacher and student.

State of the Art

What is ChatGPT?

ChatGPT is a new technology, but has been extensively researched, as it is such a noticeable new technology. ChatGPT is an acronym for Chat Generative Pre-trained Transformer and was created in November 2022. It is based on the OpenAI GPT-3 engine and has been fine-tuned by supervised and reinforcement learning technology; this means that the AI learns by having humans simulate artificial conversations with it and adapting its responses based on how accurately they reflect natural human dialogue. ChatGPT is also able to remember previously given prompts in the same conversation, making it a more personalized chatbot compared to its alternatives.[2] ChatGPT has a lot of functionalities, for example:

  • Write and debug code, generate scripts and functions
  • Give detailed explanations on complex topics (answer test questions)
  • Solve mathematical problems
  • Write texts in different styles (write student essays)
  • Compose music
  • Explain mathematical theorems
  • Play games like tic-tac-toe

However, the technology is not without its limitations. It has the potential for over-optimization due to its reliance on human oversight, also known as Goodhart's law[3], which could hinder performance. Furthermore, language models like ChatGPT are prone to writing plausible-sounding but incorrect answers, which is called artificial intelligence hallucination[4]; this can be attributed to insufficient training data. The AI is also limited by a lack of knowledge about events that occurred after 2021 and in some cases suffers from algorithmic biases. Furthermore, although ChatGPT is able to produce results that seem genuine, it is unable to fully comprehend the complexity of human language and instead relies solely on statistical knowledge and patterns.

Research into ChatGPT still has a long way to go. Van Dis, E. et al. have proposed five priorities for future research on ChatGPT:

  • The first priority is to explore the ethical implications of AI-generated content and to develop guidelines for responsible use.
  • The second priority is to investigate the limitations of ChatGPT and to develop methods for detecting and addressing bias in its outputs.
  • The third priority is to improve the interpretability of ChatGPT and other NLP models, making it easier to understand how they arrive at their outputs.
  • The fourth priority is to investigate the potential of ChatGPT in domains beyond language, such as image and video analysis.
  • Finally, the fifth priority is to develop more efficient and sustainable methods for training and deploying ChatGPT, in order to reduce its carbon footprint and energy consumption.

In order for research into ChatGPT to advance however, collaboration across disciplines on ChatGPT and other AI systems is very important.[5]

ChatGPT in academic education

Artificial intelligence has been a rapidly growing field in recent years, and the development of large language models like ChatGPT has been at the forefront of this expansion. These models have the potential to revolutionize the way we access and use information, particularly in educational contexts. However, they have also raised concerns about issues like academic integrity, student engagement, and the role of teachers in the learning process. To address these concerns, researchers have conducted various studies and published papers on the implications of ChatGPT for education.

ChatGPT has the potential for using large language models such as GPT for cheating in online exams. For instance, a recent study demonstrated that a chatbot named ChatGPT tried to answer exam questions by providing it with a large amount of relevant data. The results of the study show that ChatGPT was able to achieve high accuracy in answering exam questions, even when the questions were designed to be difficult and require reasoning skills. The authors suggest that this poses a significant threat to the integrity of online exams and call for further research into developing more secure methods for online assessments. [6]

Another issue that comes up regarding academic integrity, is the unclarity on who the rightful author of the texts, ideas and inventions should be. Currently, there is no clear consensus on who should be considered the rightful author of AI-generated content. This is a complex issue that requires input from experts in moral philosophy, law, and computer science.[7]

One more issue regards plagiarism when using ChatGPT. A study into this examined the accuracy and originality of scientific abstracts generated by ChatGPT compared to those produced by human experts. The study used a combination of artificial intelligence output detectors, plagiarism detectors, and blinded human reviewers to analyze the abstracts. The results showed that ChatGPT-generated abstracts had a higher similarity score with original abstracts than with other sources, but also revealed some cases of potential plagiarism. This suggests the need for caution when using AI-generated abstracts in scientific research. In the academic world, artificial intelligence models are gaining popularity due to their ability to enhance student engagement, collaboration, and accessibility. A paper authored by D. Cotton, explores the potential benefits and challenges of using AI in education. The models provide a platform for asynchronous communication, personalized and interactive assessments, and real-time grading and feedback. However, also according to D. Cotton there are concerns about academic integrity, particularly the possibility of plagiarism. With access to GPT-3, students could submit essays that are not their own work. Additionally, there are concerns about inequities in assessment, as students with access to GPT-3 have an advantage over those who do not. The article suggests solutions to combat these challenges, such as asking students to submit a draft before the final essay, set strict guidelines or monitor student work closely.[8]

Another paper explored the broader implications of ChatGPT for traditional models of education. The author argued that AI systems like ChatGPT have the potential to disrupt the traditional "banking model" of education, in which knowledge is transmitted from teacher to student. Instead, learners could access knowledge and information more directly through AI systems, which could democratize access to education and make it more inclusive. The author emphasizes the need for educators to adapt to this changing landscape and to shift their focus towards fostering critical thinking, creativity, and other skills that cannot be easily automated, and suggests that AI systems like ChatGPT have the potential to democratize access to education, making it more accessible and inclusive for learners around the world.[9]

Furthermore, a paper written by Rudolph also explores the implications of chatGPT for both students and teachers. While chatGPT offers personalized AI tutoring for students, it also has the potential to reduce the workload for teachers, especially with the use of automatic assessment tools. Furthermore, it can help teachers analyze their students' skills more easily. The paper gives a general overview of challenges and opportunities:[10]

Challenges of AI in education              Opportunities of AI in education
Teachers are afraid students will outsource all their work to chatGPT. Opportunity for teachers to improve/change their assessment and teaching techniques.
ChatGPT doesn’t evaluate the relevance of the information, it just generates text that is an imitation of what it has learned. “ChatGPT allowed students to learn through experimentation and experience”

To look further into the fear of teachers that students will outsource all their work to ChatGPT, we will look another paper written by García-Peñalvo that reviews previous literature on ChatGPT. The most controversial issue with ChatGPT is the possibility of students using it as an easy solution to write essays without putting in the necessary effort. Because of this, they won’t acquire the needed knowledge for their course. However, the problem might not be the tool itself, but that the assessment techniques of educational institutions have become outdated However, the paper argues that prohibiting chatGPT is not the way to go. Instead, teachers and students should learn how to use the tool to their advantage.[11]

As mentioned times before, ChatGPT has it obvious limitations. A controversial view in this paper written by Thorp, the author of the article is not necessarily worried about the use of ChatGPT in education as ‘it did well finding factual answers, but the scholarly writing still has a long way to go’. He thinks it pushes academics to design their courses in such a way that they are not easily solved by AI. The author is more worried for the influence of chatGPT on the world of literature.[12]

While ChatGPT has its limitations, it has the potential to replace humans in routine tasks such as homework grading, potentially spelling the end of traditional essay writing assignments. "ChatGPT User Experience: Implications for Education" explores the user experience of ChatGPT and its potential impact on education. He also states that using AI tools to perform certain tasks ‘should be a part of the educational goals in the future’. The author concludes that teachers need to change their assignments to make it harder for students to use AI, but also notes that using AI in assignments to engage students in learning is a viable option. [13]

To summarize, ChatGPT is a powerful tool with many functionalities, from writing texts and composing music to solving mathematical problems and playing games. However, it is not without its limitations, such as over-optimization, algorithmic biases, and the potential for incorrect responses due to insufficient training data. The future research priorities include exploring the ethical implications of AI-generated content, detecting and addressing bias in its outputs, improving the interpretability of ChatGPT and investigating the potential of ChatGPT in domains beyond language.

In academic education, ChatGPT has the potential to revolutionize the way we access and use information, but it also raises concerns about academic integrity, student engagement, and the role of teachers in the learning process. Further research is needed to develop secure methods for online assessments and determine the rightful author of AI-generated content. Despite the challenges, AI systems like ChatGPT have the potential to democratize access to education and make it more inclusive.

Problem statement

In the present study, we investigate the influence of ChatGPT on students in reaching certain learning objectives. Therefore, our problem statement is as follows:

How could the use of ChatGPT affect students in reaching the learning objectives of the course 4WBB0 at the TU/e?

Objectives:

  • Upcoming use of AI in academic settings,
  • how to design this so that it will be accepted by teachers
  • How AI can improve learning
  • Acceptance towards AI in the classroom
  • Looking at ChatGPT as main example
  • Teachers at the TU/e, their thoughts about AI (in their courses)

Users

Target group

The target group of this project are the teachers of the TU/e who designed and teach the course engineering design (4WBB0) and students who followed this course. In this research the perspective of the teachers and students on the use of chatGPT in this course is reviewed. The input of students will be used to get a better image of how students intend to use chatGPT. The input of the teachers will be used to understand the learning objectives of the course engineering design and to determine the effect of chatGPT on those learning objectives. The end product will be a deliverable useful for the course coordinators of engineering design.  

Requirements

Teachers want their students to deliver autonomous work, ChatGPT in academic settings can help with this. Teachers want to transfer their knowledge as effectively and efficiently as possible, using the current state of art technologies. AI technologies, like ChatGPT can help with this, but what teachers require is yet to be determined. The requirements will be determined by distributing surveys among students of the TU/e and conducting interviews with teachers, based on the survey results, in combination with literature studies. Those results will then be combined into a piece of well-considered advice for the teachers at the TU/e, consisting of guidelines for a course for the teachers of how to make exercises and tests for students that can benefit from using ChatGPT, but not be made solely with ChatGPT. Which they will be able to use to guide them through the landscape of ChatGPT use in academic education and give them tools to keep transferring their knowledge to students.

Hypothesis

Articles about the use of ChatGPT in (academic) education suggest that it is preferred to adopt ChatGPT in education as opposed to forbidding it. In this research, this hypothesis will be tested by conducting interviews and sending out surveys to teachers and students of the technical university Eindhoven. The interviews and surveys will give in insight into the effect of ChatGPT on the learning objectives of courses and whether they will be achieved if ChatGPT is used with a focus on the course engineering design.

The course engineering design has 13 learning goals. Each one could be affected by the use of ChatGPT. Considering the literature and the known capabilities of ChatGPT, a hypothesis for each learning goal can be made.

  • Execute a generic design process

ChatGPT would not be able to execute a generic design process, therefore this specific learning goal would not be affected by chatGPT.

  • Formulate a design goal

As chatGPT is an AI language model it is capable of formulating a design goal, provided that the users ask the right question. Therefore this learning goal could be affected.

  • Define the functional and technical specifications

As chatGPT is an AI language model it is capable of defining the functional and technical specifications. However, the user would have to need to provide an analysis of the design problem, the target audience, and the desired outcomes in order to get valuable specifications. This learning goal could be affected.

  • Generate an elaborate list of realization possibilities for the different functions of the design

As chatGPT is an AI language model it is capable of formulating an elaborate list of realization possibilities for the different functions if the requirements and specifications of the design are given. Therefore this learning goal could be affected.

  • Select a number of design concepts from an extensive list of realization possibilities

As chatGPT is an AI language model it is capable of making selecting some designs that fit the design goal the best. Therefore this learning goal could be affected.

  • Make a final design choice between a number of concepts

As it is hard to give chatGPT a complete overview of the context and specifications needed for the design a final design choice would be have to made by the student. Therefore this learning goal is not affected.

  • Develop a detailed design that meets the specifications

As it is hard to give chatGPT a complete overview of the context and specifications needed for the design a complete detailed design is hard to generate. However, it could give the student a start with the design and only need some adjustments. Therefore this learning goal is not affected

  • Develop and execute a test plan for the prototype

As the chatGPT is not a physical AI it can’t execute a test plan. Therefore this learning goal will not be affected.

  • Evaluate a prototype based on test results and give an advice for redesign

ChatGPT is capable of this and therefore this learning goal could be affected.  

  • Reflect on the design and on the design process

As it is hard to explain chatGPT the whole design process a group has been through, reflecting on it would be difficult. Therefore this learning goal would not be affected.

  • Write a design report describing the design process, the foundation of the choices made and the evaluation of the delivered design

ChatGPT is capable of writing separate parts of the report and this learning goal could therefore be affected.

A quick overview of the hypothesis
Can It be done by chatGPT: Yes No
Execute a generic design process X
Formulate a design goal X
Define the functional and technical specifications X
Generate an elaborate list of realization possibilities for the different functions of the design X
Select a number of design concepts from an extensive list of realization possibilities X
Make a final design choice between a number of concepts X
Develop a detailed design that meets the specifications X
Develop and execute a test plan for the prototype X
Evaluate a prototype based on test results and give an advice for redesign X
Reflect on the design and on the design process X
Write a design report describing the design process, the foundation of the choices made and the evaluation of the delivered design X


General hypothesis:

Most of the learning goals of the course engineering design could be affected by the use of ChatGPT, therefore the learning goals of the course might not be achieved by the students who use ChatGPT.

Method

To address the research question of the study, a mixed-methods approach was adopted. This involved the use of both a survey and interviews to obtain a comprehensive understanding of the participants' opinions and experiences with ChatGPT in the context of Engineering Design.

The survey was sent to university students who had completed the Engineering Design course at TU/e, with the aim of gaining insights into their usage and perception of ChatGPT. The survey included questions related to the frequency of ChatGPT usage, perceived usefulness of the tool, and its impact on reaching the learning objectives of Engineering Design.

Furthermore, a series of interviews were conducted with both a professor from the Engineering Design department and frequent users of ChatGPT: which will be called experts. The interviews aimed to provide a deeper understanding of the participants' opinions and experiences with ChatGPT, specifically in the context of Engineering Design. The interviews with the student experts aimed to capture their motivation for using ChatGPT, how they use the tool, and their perception of its impact on the Engineering Design learning goals. The interview with the professor aimed to capture the impact they believe ChatGPT has on the learning objectives, and their overall opinion on the use of ChatGPT.

Together, the survey and interviews aimed to obtain a comprehensive understanding of the participants' opinions regarding the use of ChatGPT in the Engineering Design course and its impact on achieving the learning objectives within this course.

Survey

The survey consists of questions about the participants' general knowledge and experience about ChatGPT, and how this tool could affect them reaching the learning goals of Engineering Design.

Participants

The survey was distributed to students who are enrolled at the TU/e and who have finished the course Engineering Design, which are 2nd year and higher students. There were a total of 44 responses.

Procedure

The outline of the survey was as follows (see appendix 2): a brief section about their general knowledge of ChatGPT, their opinions and experience with using the chatbot as an academic tool, and finally, a more in-depth section with questions about hypotheticals of using ChatGPT within the course Engineering Design and what the affects of it would be on achieving all the learning goals. The first two sections were to measure the proficiency of the participants, and the latter to measure the effects of ChatGPT on achieving certain learning goals.

Interview with ChatGPT experts

Student experts who frequently use ChatGPT were interviewed to gain a better understanding of how and why they use ChatGPT, and how this usage could impact achieving the learning goals of Engineering Design.

Participants

The focus was on university students who were all enrolled at the TU/e and had successfully completed the Engineering Design course. A total of five participants were recruited, one female and four males, ranging from the ages 20 to 23 (Mage= 21, SD = 1.79, 20% female). The participants were sampled by the convenience sampling method, all the participants had to be students at the University of Eindhoven, and they should have followed the course Engineering Design, since this was a crucial part for our research.

Procedure

The outline of the interview was as follows: a brief section about their general knowledge of ChatGPT, their opinions and experience with using the chatbot as an academic tool, and finally, a more in-depth section with questions about hypotheticals of using ChatGPT within the course Engineering Design and what the affects of it would be on achieving all the learning goals. The first two sections were to measure the proficiency of the participants, and the latter to measure the effects of ChatGPT on achieving certain learning goals. We asked open questions that were mentioned in the interview script, see appendix 3. This was followed by transcribing the interview and discussing the interview by adding codes to the transcript. Afterward, similar codes were placed in certain themes. These themes were then analyzed carefully, which gave a certain conclusion of the interview.

Interview with university professor

In addition, an interview was conducted with an associate professor from TU/e who teaches the course Engineering Design. This interview aimed to provide insights into the perspectives of educators regarding the use of ChatGPT and its impact on learning outcomes, particularly in the context of Engineering Design.

Participants

The participant for this interview was Joris Remmers, a social professor of Mechanical Engineering, whose role within Engineering Design is to organize assessments and set up the organizational structure of it. He is responsible for the assessment setting and rubrics.

Procedure

The outline of the interview was as follows: a brief section about his general knowledge of ChatGPT, his opinions and experience with using the chatbot as an academic tool, and finally, a more in-depth section with questions about hypotheticals of using ChatGPT within the course Engineering Design and what the affects of it would be on achieving all the learning goals. The first two sections were to measure the proficiency of the participant, and the latter to measure the effects of ChatGPT on achieving certain learning goals. We asked open questions that were mentioned in the interview script, see appendix 4. If the participant did not give a clear answer, we asked follow-up questions to get more in-depth answers. This was followed by transcribing the interview and discussing the interview by adding codes to the transcript. Afterward, similar codes were placed in certain themes. These themes were then analyzed carefully, which gave a certain conclusion of the interview.

Thematic analysis

All the interviews have been analyzed with the use of thematic analysis. Thematic analysis makes use of finding patterns in the answers to the interview questions. The thematic analysis was conducted via Dedoose[14]. Codes for themes in the interview were made based on the apparent feelings the interviewees have about ChatGPT and its use. Three main groups were established: Features, what interviewees use it for and why they use it; feelings, what the general feeling of interviewees is about ChatGPT; and warnings, what interviewees think is a drawback of ChatGPT. In table 'Codes for the themes' all the codes' descriptions are given.

Codes for the themes
Overarching theme Code Discription
- Academic usage A mentioning of how ChatGPT can be used in academic circumstances.
- Improved capabilities The notion that ChatGPT is better at doing a certain job than conventional technology or humans.
- Inevitable use When someone mentioned ChatGPT would be used now or in the future, regardless of actions that would be taken against it.
- New learning goal When a new possible learning goal was metioned that arose due to ChatGPT.
- Pluripotent When a interviewee mentioned that they think ChatGPT can do everything or that it has no boundries.
Features Coding Help When ChatGPT was used to write code, help with understanding code, debugging code or optimizing code.
Features Creativity When ChatGPT got used to conduct a creative task where it has to recombine ideas into a new one like coming up with new ideas for a brainstorm or write a caption for a picture.
Features Ease of use When it is mentioned that ChatGPT was easy to use.
Features Fun When ChatGPT was used to mess around, so there was no goal for using it except for momentairy happiness.
Features Providing information When ChatGPT was used to gain information.
Features -Guidance When ChatGPT was used to gain information on how to do a certain task.
Features Summarizing When ChatGPT was used to create a summary af a piece of text or a complex subject.
Features Time saving When the reason for using ChatGPT was given to be saving time compared to other methods of doing the same task.
Features Writing help When ChatGPT was used to help write, for example rewrite a text, improve its grammar or flat out writing it all.
Feelings Neutral The general opinion of about ChatGPT was given to be neutral, the interviewee was not possitive nor negative about the use of it.
Feelings Possitive outlook When the interviewee mentioned they saw a possitive effect of ChatGPT in the future.
Warning Better to do it yourself When the interviewee mentioned they thought a person can do the job better than ChatGPT
Warning Depends on user knowledege The capabilities of ChatGPT depend on how the user uses it. THe more knowledge the suer has, the more ChatGPT can do. In the same way, it is not as usefull in the hands of a inexperienced user.
Warning Hinderance to learning When it was mentioned that current learning goals would be unable to be achieved, since CahtGPT would do the work.
Warning Steep learning curve It is hard for new users to properly learn ChatGPT at the start, a lot of learning needs to be done before ChatGPT can be used prpperly.
Warning Limited When it was noted that ChatGPT could not do something.
Warning Unpersonal When it was mentioned that ChatGPT created a gap between the user and its goal.

These codes are applied to the interviews when their description matches what is mentioned. What follows from it is thematic analysis code occurrence table.

Results

User procifiency

[insert table/visualisation on the relation between use and proficiency (learning goal attainment?)

To get a clear image of the results of the interviews it is wise to compare the difference in answers between students who were very proficient in using chatGPT an students who used it now and then for more superficial use.



Survey analysis

*The 39 answers of the survey can roughly be categorized in the following categories, with the occurence count for each category indicated per relevant question.*

Students' answers to the question "How likely do you think you would be able to reach the following learning objectives of Engineering Design with the use of ChatGPT?"

Interview analysis

A very concise summary of the interviews with the ChatGPT experts and professor can be seen in the table below: these tables depict whether these participants think it would be likely to reach the mentioned learning goal with the use of ChatGPT.

Interview ChatGPT experts
Learning objective F M G N Q
Execute a generic design process Not doable Not doable No Yes No
Formulate a design goal Help with formulation Yes Yes Yes No
Define the functional and technical specifications Would be good for this Hard but possible Yes Yes No
Generate an elaborate list of realization possibilities for the different functions of the design Best done by the group Hard but possible Yes Yes No
Select a number of design concepts from an extensive list of realization possibilities Best done by the group Best done by the group Yes Yes No
Make a final design choice between a number of concepts Best done by the group Best done by the group No Yes No
Develop a detailed design that meets the specifications Help with formulation Would be good for this Yes Yes No
Develop and execute a test plan for the prototype Probably not doable Can help but not detailed Yes Yes No
Evaluate a prototype based on test results and give an advice for redesign Help with formulation More effort than normal No Yes No
Reflect on the design and on the design process Would definitely use Not doable Yes Yes No
Write a design report describing the design process, the foundation of the choices made and the evaluation of the delivered design Would definitely use Only for parts of the report No Yes No
Interview with professor
Learning objective Y/N Explanation
Execute a generic design process Yes Students still need to do all the steps themselves, but ChatGPT can be of assistance
Formulate a design goal No This is a creative part and requires out-of-the-box thinking
Define the functional and technical specifications Yes This is objective knowledge that can be looked up
Generate an elaborate list of realization possibilities for the different functions of the design No This is the most creative part - ChatGPT is not innovative enough
Select a number of design concepts from an extensive list of realization possibilities Yes Text-book Google work, so ChatGPT can do it, since it can follow strict procedures
Make a final design choice between a number of concepts No ChatGPT can be used to structure the concepts, but students ultimately need to reflect on their own preferences
Develop a detailed design that meets the specifications Partly This part requires thinking, combining results, solving equations and doing the actual work; ChatGPT can help with parts of it, but not entirely
Develop and execute a test plan for the prototype Yes Can be done by ChatGPT
Evaluate a prototype based on test results and give an advice for redesign Yes ChatGPT can compare data easily, especially if it's about software design
Reflect on the design and on the design process No This requires your own thoughts and experiences
Write a design report describing the design process, the foundation of the choices made and the evaluation of the delivered design Yes It's wonderful for (improving) report writing

[Insert visual to compare these table]

Thematic analysis


Thematical analysis code occurance from the interviews conducted with students who are proficient users of ChatGPT.


Interpretation

Learning goals

The interviewees give conflicting signals over the approach to the learning goals of the course. Two extremities can be found even, where one student would use ChatGPT for almost every step of the process and the other notes ChatGPT lacks the information of the course and group progress that a student may have in their head. Moreover, the survey results show there are always at least 30% of students that think a learning objective can be reached with ChatGPT and at least 23% that think they cannot.

On top of that, students tend to favor the potential of ChatGPT differently for each learning objective respectively. This indicates students' view on the AI's capabilities differs significantly per student, which in turn is supported by the comments of the most extreme interviewees. Whereas one students says the model lacks knowledge to fullfill tasks for engineering design usefully, another claims it is able to do most of the work for the course if you give it enough information.

[Users that indicate many learning goals can be reached also are very frequent users and vice versa]

This discrepancy could be explained by two factors combined: the skill of the student in using ChatGPT and the skill of the user in the learning objectives of the course. A student with less knowledge of the learning objectives and high proficiency with ChatGPT will be significantly more likely to view the technology as useful, than a student which can already quite efficiently obtain some learning objectives and is less proficient with ChatGPT.

Interviews ChatGPT expert

With the thematic analysis general notions about ChatGPT became apparent. The three main reasons ChatGPT is used are for gaining information, help with writing and saving time. One interviewee said that they used ChatGPT as "a search engine instead of google", This goes to show, that these reasons are linked with one another.

It is interesting to note that all interviewees were aware of drawbacks. The most mentioned disadvantage of ChatGPT was that it was limited in its use. Interviewees mentioned that it could "Only do straightforward stuff" and "It should be used wisely". This last remark is due to what another interviewee said: "[ChatGPT] hallucinates and is not up to date". The not up to date part refers to the data ChatGPT is trained on [the youngest data it was trained on comes from 2011 (source?)]

Interview university professor

The interviewee was overall positive about his experience with ChatGPT and even encourages his students to use it. The main benefits were: improving students' report writing, using it as a more efficient search engine, and helping with speeding up doing ordinary tasks, such as googling for literature.

However, according to him, the main drawback of the chatbot is that it is a very linguistic tool. He emphasized the fact that ChatGPT was not creative enough and could not generate innovative ideas due to its limited database.

These statements are backed up by his thoughts on whether it would be likely to reach the learning objectives of Engineering Design with the use of ChatGPT. When the objective requires more creative thinking, out-of-the-box solutions and own experiences, it was quickly decided that ChatGPT alone would not be sufficient to achieve this goal. On the other hand, more standard goals which require objective knowledge or a strict procedure, could be easily done by ChatGPT.

Approach, milestones and deliverables

Planning

Here follows a Gantt chart of all our deadlines to be finished at 12pm on Sunday of the corresponding week. The letters indicate the group member responsible for making the deadline and the scheduling for the specific task.

Gantt chart
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
Approach, milestones and deliverables Ni
Problem statement and objectives Q
Users and requirements G
Summaries of literature F
Literature review M
Surveys Na
Surveys analysis All
Interview preparation teachers All
Interview preparation students Q
Organize interviews teachers Ni
Organize interveiws students All
Conduct interviews teachers Ni/M/F
Conduct interviews students All
Interview analysis All All
ChatGPT panel All
State of the art F
Advise statement Na Na
Discussion F
Future research Q
Reflection ? Ni
Finalization of wiki M
Presentation Q
Chair meetings N
Keep wiki up to date M
Minute taker Q F G Na M Q F

Approach

  • Literature study
  • Interviews, surveys

Deliverables

  • Advice for future

Background literature

[15]This paper regards Teacher’s Perceptions of Using an Artificial Intelligence-Based Educational Tool for Scientific Writing. This study investigated how teachers perceived an AI-enhanced scaffolding system developed to support students’ scientific writing for STEM education. Teachers are worried: What if the introduction of AI will gradually reduce our role in the classroom. What should we do? Should we support AI?  They feared that the AI would reduce their role to assistants and they also questioned the accuracy and reliability of the information generated by the system. For AI to be successfully integrated into STEM education, it is necessary for the roles and relationships between students and teachers to be redefined and for educators to be fully trained on best practices of using AI pedagogical techniques. There is still a trend among educators to hold negative impressions on educational technology. By changing teachers’ current negative perceptions of educational technology, the acceptance of AI as a new type of educational tool and its implementation in schools is possible. The younger generation of teachers, who have more experience with educational technology as both educators and as students, is more interested in exploring new digital technology and potentially incorporating technology into their lessons. Experience with AI in any of these contexts may reduce teachers’ reluctance to use AI for educational purposes.


[16]This paper regards Education in the Era of Generative Artificial Intelligence (AI). The article "Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning" discusses the potential benefits of using generative AI, such as ChatGPT, in education. The article suggests that AI can help teachers and students by providing personalized feedback, facilitating communication and collaboration, and creating engaging and interactive learning experiences. The article also highlights some of the challenges of using AI in education, including issues related to data privacy and the need for human oversight to ensure that AI is used in a responsible and ethical manner. The article concludes by calling for more research and experimentation to better understand the potential benefits and challenges of using AI in education.


[17]The article "Teachers' Perspectives on Artificial Intelligence" explores the attitudes and perceptions of teachers towards the use of artificial intelligence (AI) in education. Based on a survey of K-12 teachers in the United States, the article found that while many teachers are interested in using AI to improve their teaching practice, they also have concerns about the potential impact of AI on student learning, privacy, and job security. The article highlights the need for more education and training on AI for teachers, as well as the importance of involving teachers in the development and implementation of AI tools in education. The article concludes by calling for further research on how to best integrate AI into education in a way that benefits both teachers and students.


[18]The article "Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education" explores the potential benefits and challenges of using generative AI, such as ChatGPT, in engineering education. The article suggests that AI has the potential to enhance engineering education by providing personalized feedback, facilitating communication and collaboration, and creating interactive learning experiences. However, the article also highlights some of the challenges of using AI in engineering education, including issues related to bias, the need for human oversight, and the limitations of AI in capturing the full complexity of engineering problem-solving. The article concludes by calling for further research and experimentation to better understand the potential benefits and challenges of using AI in engineering education, as well as the need for ongoing dialogue between educators, engineers, and AI developers to ensure that AI is used in a responsible and ethical manner.

[19]This paper starts with a general introduction to AI, how it developed over the years, and what these developments entail.

  • AI in program coding (1950s)
  • AI in rules-based expert systems (late 1970’s)
  • AI-grounded automatic data processing systems (mid-1980’s)
  • Machine learning integrated AI (mid-2000’s)

In education, AI is still in the early stages, because there is more focus on the development of AI than on the application of AI in new fields. This research states that there will be two distinct major effects of AI on education. First, education needs to prepare itself for the fast changes in competence needed for jobs, as some jobs will disappear but most jobs will keep changing over time. Second, there will be changes in the pedagogical techniques needed in the classroom. AI will be able to relieve the teacher from some of the work. However, the teacher will need additional tools to understand the statistical results from the AI (for example from a statistical analysis of the performance of the student). This paper goes on by talking about the future of AI in education. The Author states that there needs to be a big shift in education toward more personalized education. AI will be able to determine the learning style of an individual student. AI will help teachers in content delivery and other instructions, but in the future real life, human teachers might become obsolete.

[20]This paper talks about AI in education, the need for it, and its benefits and challenges of it. According to the author, AI is a development that has many promising applications in education. The examples that are named are; personalized education, automatic grading systems, and predictive analytic tools. These applications will relieve the teachers from those tasks which gives them more time with the students. The paper also talks about challenges that come along with AI, like concerns about safety, security, and privacy.

[21]In this paper, the authors review the use of AI in current education, the technical aspects of AI in education, the role of AI in education and the impact of AI in education. The topic of education is spread out into three sub-topics; administration, instructions and learning. The takeaway of this paper is that with the help of AI, teachers will become more efficient which will in result increase the quality of education. Next to that with AI, it will be possible to create a more personalized education for students. AI will have a major impact on education as the tasks AI will be able to do tasks that are not originally designed for computers.

[22]This paper focuses on the viewpoint of the students and their acceptance of AI. The author uses the technology acceptance model (TAM) to analyze the response of students to AI. This is because the perceived usefulness and the perceived ease of use of AI seem to have a big impact on the acceptance of AI in education. The author sees a future in AI in education as it is a cost-effective way to the shortage of teachers. However to make sure that the AI teaching assistants will be successful the teachers need to be trained to work with them. As research points out that if the teacher is uncomfortable with the AI, the student is less likely to adopt it. Based on the results of this research the author recommends research in this area as it is not known how students at different levels of education will react to the adoption of AI as a teacher.

[23]The authors in this paper review the ethical perspective of AI with a focus on AI in education. The biggest problem that the authors see is the privacy issue of AI. Next to that they also see a challenge in the trustworthiness of AI. It is often that AI is presented as a better alternative than a human, as it is supposedly not biased. However, the designers of AI are biased and can make mistakes. Therefore can we even trust AI more than humans? Another issue that the author comes across is the difficulty of addressing the ethical challenges of AI. Therefore it is important to make the designers of AI aware of the ethical dilemmas that come along with new technology and make them aware of their responsibilities. To achieve this it is important that there mandatory teaching or at least education available around this topic.

[24]In this articile, F. Ouyang and P. Jiao reflect on the three paradigms of artificial intelligence in educaiton: AI-directed, AI-supported, and AI-empowered.

  • AI-directed (learner-as-recipient): AI is used to represent knowledge models and direct cognitive learning.
  • AI-supported (learner-as-collaborator): AI is used to support learners while they work as collaborators with AI
  • AI-empowered (learner-as-leader): AI is used to empower learning while learners take agency
Table 1. Three paradigms of AIEd
Paradigm Theoretical underpinning Implementations AI techniques
1. AI-Directed Behaviorism Intelligent Tutoring Systems (ITSs): software that tracks students' work, adjusts feedback and provides hints along the way) AI based on statistical relational techniques
2. AI-supported Cognitive, social constructivism Dialogue-based Tutoring Systems (DTSs); Exploratory Learning Environments (ELEs) Bayesian network, natural language processing,

Markov decision trees

3. AI-empowered Connectivism, Complex adaptive system The human-computer cooperation; Personalized/adaptive learning The brain-computer interface, machine learning, deep learning

[25]P. Lameras and S. Arnab explored and analyzed what Artificial Intelligence means in Education (AIED) in their article "Power to the Teachers: An Exploratory Review on Artificial Intelligence in Education". The main take-away is that adaptivity and personalization are the innovation that AIED can offer to help students to learn and develop skills that are relevant to their own needs and experiences. However, it is important to help teachers to develop necessary digital competencies and skills for using AIED applications and tools in ethical and informed ways to enhance the student learning experience and attainment of learning outcomes. The findings of this review can contribute to developing a better understanding of how artificial intelligence may enhance teachers' roles as catalysts in designing and visualizing AI-enabled learning. As a result, more useful AI-systems specialized in pedagogy will be developed.


[26]As Artificial Intelligence is becoming more apparent in education, it changes the way of teaching. Craig Coates, an entomologist at Texas A&M University, combated the cheating by reorienting his course towards writing and discussion, combined with adaptive courseware, also referred to as intelligent tutoring systems; and by using a tool that uses algorithms and analytics to grade submissions, the switch was a success. Advocates say this lets students study at their own pace and frees up the instructor’s time in class to shore up students’ knowledge.


[27]The study conducted by S. Yang and H. Bai "The integration design of artificial intelligence and normal students’ Education" outlines four major problems of normal education (school for training teachers) development and explores the idea of using artificial intelligence technology to solve them. The main issues and the proposed solutions are:

Problem Solution
Lack of teaching practice experience Intelligent tutors/robots: it can make a special learning plan for students according to their interests, habits and learning needs. It can also be a simulation student of upcoming teachers to gain experience.
Outmoded curriculum Expert system: can accurately classify normal students and highlight personalized teaching.
Lack of independent learning Deep learning system: artificial intelligence can help to analyze the data of learners, organize the content according to the data, and carry out intelligent push. Teachers can create teaching resources to meet the needs of curriculum design and students.
Lack of innovation in teaching methods Combination of all of the above.

[28]Artificial intelligence and natural language processing systems are becoming increasingly prevalent in various forms of writing. These natural language processing (NLP) systems are trained on large databases of text to produce and refine statistical models that generate natural language responses. However, NLP systems do not "know" the meaning of the text they generate and can make factual and reasoning mistakes, which raises issues of accountability for authors using these systems. The use of NLP systems also raises questions of transparency in regards to authorship credit and contributions, and must be acknowledged in the text and references section of the manuscript.


[29]The first chapter of this book aims to determine the knowledge and skills that remain for humans in an AI-driven world. It addresses AI's impact on education and highlights the need to develop awareness and understanding of AI. The goal is to create an educational environment where AI supports learners and teachers and prepares students for an AI-dominant future.


[30]This article discusses recent developments in policies for medical journal publishing and editing in Korea. The article highlights that many editors in Korea are also publishers and therefore need to keep up with current trends and policies in the industry. The article focuses on six main policies that have emerged, including the use of artificial intelligence tools in publishing, preprint publications, open peer review, model text recycling policies, the updated 4th version of the Principles of Transparency and Best Practice in Scholarly Publishing, and the recommendation to include country names in human studies titles. The article also mentions that the use of AI in writing has increased, including in peer review and plagiarism detection. This shows the current and expected relevance AI has in modern scientific writing.


[31]OpenAI launched a preview version of ChatGPT. It is part of the GPT (Generative Pre-trained Transformer) technology that can generate human-like text based on internet data. The release of ChatGPT sparked discussions on the impact of AI on education, with some saying it could render the student essay obsolete and others expressing concerns about cheating and false information. Despite its promise of transforming education, ChatGPT was criticized for its lack of understanding of meaning and content and its association with environmental racism and the interests of tech elites. Some educators called for better assignments that can't be written by an algorithm and for teaching students to use AI ethically. Access to ChatGPT was blocked by some US education departments due to concerns of dependence on technology, loss of privacy, and misuse of AI. This article opens up a range of important issues that are relevant across the educational field.


[32]This paper provides a review of AI applications in higher education, with most of the research being quantitative research from computer science and STEM fields. So this review focuses more on AI utility over the entire academic institute rather than academic integrity and utility by students. Four areas of application were identified: profiling and prediction, assessment, adaptive systems and personalization, and intelligent tutoring systems. The conclusion highlights the need for further exploration of ethical and educational approaches in AI applications in higher education.


[33]In the paper “OpenAI ChatGPT Generated Results: Similarity Index of Artificial Intelligence (AI) Based Model“, university director R. Ventayen researched how new AI powered technology enables cheating in the academic community. Since constructing an entire paper or essay using ChatGPT is considered a violation of academic integrity, the chatbot’s ability to pass similarity tests was researched on prompts with the headlines of the university’s publications. Their results show that ChatGPT provides acceptable similarly indices, and fails to provide the same citations or sources as the similar papers with the prompted headlines. Moreover, Quillbot paraphrases more content, which t, which provides plagiarism-free content.


[34]This paper focuses foremost on the irreversibility of GPT-3 and its answers and ideas. It shows its shortcomings in dealing with semantic, mathematical and ethical questions and consequences for the writing industry. E.g, the big amount of written things that these tools can provide far exceeds the limited physical storage for them and marketing will abuse these tools heavily. We need to be critical with what these tools produce and how we use them. For our question regarding academic integrity, the problems with correct citations become apparent and academic texts produced by ChatGPT are often irreversibly produced from ideas of researchers.


[35]This article shows how new developments in AI can solve 81% of university-level mathematics questions and even come up with new questions. This poses great advantages to the acquisition of new learning materials but the implications of this also addresses significant pedagogical challenges, with these models being available to students as well.


[1]The aim of the article is to explore the academic and administrative applications of Artificial Intelligence. Artificial Intelligence Applications (AIA) are not only assisting education academically and administratively but also enhance their effectiveness. AIA provides help to teachers in various types of tasks in the shape of Learning Analytics (LA), Virtual Reality (VR), Grading/Assessments (G/A), and Admissions. It minimizes the administrative tasks of a teacher to invest more in teaching and guiding students. AIA adds a significant contribution to enhance student learning, minimize the workload of a teacher, grade/assess the students effectively and easily, and to help in a lot of other administrative tasks.


[36]This report is an overview of research on AI applications in higher education between 2007 and 2018 through a systematic review. The combination of results gives us four areas of AIEd applications in academic support services, and institutional and administrative services: 1. profiling and prediction, 2. assessment and evaluation, 3. adaptive systems and personalisation, and 4. intelligent tutoring systems. The conclusions reflect on the almost lack of critical reflection of challenges and risks of AIEd, the weak connection to theoretical pedagogical perspectives, and the need for further exploration of ethical and educational approaches in the application of AIEd in higher education.


[37]This article presents the focus of latest research in AIEd on reducing teachers’ workload, contextualized learning for students, revolutionizing assessments and developments in intelligent tutoring systems. It also discusses the ethical dimension of AIEd and the potential impact of the Covid-19 pandemic on the future of AIEd’s research and practice.


[38]It investigates educational implications of emerging technologies on the way students learn and how institutions teach and evolve. Pinpointing some challenges for institutions of higher education and student learning in the adoption of these technologies for teaching, learning, student support, and administration and explore further directions for research.


[39]This article anaylises a sample of 132 articles published betwee 2007-2017 about the scientific production on artificial intelligence in higher education. It concludes that, although artificial intelligence is a reality, the scientific production about its application in higher education has not been consolidated. Honestly not that interesting.


[40]The paper is about a pedagogical experiment in which undergraduates were assigned to "cheat" by using text-generating AI software (GPT-2) to write their final class essay. The students were asked to reflect on the ethics of using AI in this way, such as what counts as plagiarism, and how working with AI could change their perspectives on writing, authenticity, and creativity. The results showed that composing with GPT-2 opened up the students' perspectives on the ethical use and evaluation of language models, and that their insights on these issues were connected to broader conversations in the humanities about writing and communication. The author of the paper shares the students' experiences and reflections on the experiment.

Survey

https://forms.office.com/Pages/DesignPageV2.aspx?subpage=design&FormId=R_J9zM5gD0qddXBM9g78ZOciNcKFkhVJsg7I7H6kiF5UNzNXQzhaOEpWSjZJSDg1WUZBN1lWTkkwUS4u

Bibliography

  1. 1.0 1.1 Ahmad SF, Alam MM, Rahmat MK, Mubarik MS, Hyder SI. Academic and Administrative Role of Artificial Intelligence in Education. Sustainability. 2022; 14(3):1101. https://doi.org/10.3390/su14031101
  2. Arimetrics (2022) What is ChatGPT. Retrieved from https://www.arimetrics.com/en/digital-glossary/chatgpt
  3. Gao, Leo; Schulman; Hilton, Jacob (2022). "Scaling Laws for Reward Model Overoptimization". arXiv:2210.10760 [cs.LG].
  4. Lakshmanan, Lak (December 16, 2022). "Why large language models like ChatGPT are bullshit artists". becominghuman.ai. Archived from the original on December 17, 2022. Retrieved January 15, 2023. The human raters are not experts in the topic, and so they tend to choose text that looks convincing. They'd pick up on many symptoms of hallucination, but not all. Accuracy errors that creep in are difficult to catch.
  5. van Dis, E. A., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023). ChatGPT: five priorities for research. Nature, 614(7947), 224-226.
  6. Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity?. arXiv preprint arXiv:2212.09292.
  7. Dehouche, N. (2021). Plagiarism in the age of massive Generative Pre-trained Transformers (GPT-3). Ethics in Science and Environmental Politics, 21, 17–23. https://doi.org/10.3354/esep00195
  8. Cotton, D., Cotton, P., & Shipway, J. R. (2023, January 10). Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT. https://doi.org/10.35542/osf.io/mrz8h
  9. Pimentel, C. Is ChatGPT a threat to education? For banking model of education, yes.
  10. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? (2023). Journal of Applied Learning and Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.9
  11. García-Peñalvo, F. J. (2023). La percepción de la Inteligencia Artificial en contextos educativos tras el lanzamiento de ChatGPT: disrupción o pánico. Education in the Knowledge Society, 24, e31279. https://doi.org/10.14201/eks.31279
  12. Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science, 379(6630), 313. https://doi.org/10.1126/science.adg7879
  13. Zhai, X. (2022). ChatGPT User Experience: Implications for Education. Social Science Research Network. https://doi.org/10.2139/ssrn.4312418
  14. “Home | Dedoose”. https://www.dedoose.com/
  15. Kim, N. J., & Kim, M. K. (2022). Teacher’s Perceptions of Using an Artificial Intelligence-Based Educational Tool for Scientific Writing. In Frontiers in Education (p. 142). Frontiers.
  16. Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning. Available at SSRN 4337484.
  17. Lindner, A., Romeike, R., Jasute, E., & Pozdniakov, S. (2019). Teachers’ perspectives on artificial intelligence. In 12th International conference on informatics in schools,“Situation, evaluation and perspectives”, ISSEP.
  18. Qadir, J. (2022). Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education.
  19. Alam, A. (2021). Possibilities and Apprehensions in the Landscape of Artificial Intelligence in Education. 2021 International Conference on Computational Intelligence and Computing Applications (ICCICA). https://doi.org/10.1109/iccica52458.2021.9697272
  20. Limna, Pongsakorn and Jakwatanatham, Somporch and Siripipattanakul, Sutithep and Kaewpuang, Pichart and Sriboonruang, Patcharavadee, A Review of Artificial Intelligence (AI) in Education during the Digital Era (July 2022). Advance Knowledge for Executives, 1(1), No. 3, 1-9, 2022, Available at SSRN: https://ssrn.com/abstract=4160798
  21. Alam, A. (2021b). Should Robots Replace Teachers? Mobilisation of AI and Learning Analytics in Education. 2021 International Conference on Advances in Computing, Communication, and Control (ICAC3). https://doi.org/10.1109/icac353642.2021.9697300
  22. Kim, J., Merrill, K., Xu, K., & Sellnow, D. D. (2020). My Teacher Is a Machine: Understanding Students’ Perceptions of AI Teaching Assistants in Online Education. International Journal of Human–Computer Interaction, 36(20), 1902–1911. https://doi.org/10.1080/10447318.2020.1801227
  23. Borenstein, J., & Howard, A. (2020). Emerging challenges in AI and the need for AI ethics education. AI and Ethics, 1(1), 61–65. https://doi.org/10.1007/s43681-020-00002-7
  24. Fan Ouyang, Pengcheng Jiao (2021). Artificial intelligence in education: The three paradigms. Computers and Education: Artificial Intelligence, Volume 2, 100020, ISSN 2666-920X. Retrieved from https://doi.org/10.1016/j.caeai.2021.100020.
  25. Lameras, P., & Arnab, S. (2021). Power to the Teachers: An Exploratory Review on Artificial Intelligence in Education. Information, 13(1), 14. MDPI AG. Retrieved from http://dx.doi.org/10.3390/info13010014
  26. Beth McMurtrie (2018, August 12). How Artificial Intelligence Is Changing Teaching. The Chronicle of Higher Education(1). Retrieved from https://www.su.edu/conservatory/files/2018/09/How-Artificial-Intelligence-is-Changing-Teaching.pdf
  27. Shuai Yang & Haicheng Bai (2020). The integration design of artificial intelligence and normal students' Education. Journal of Physics: Conference Series, Volume 1453: Conf. Ser. 1453 012090. https://iopscience.iop.org/article/10.1088/1742-6596/1453/1/012090
  28. Mohammad Hosseini, Lisa M. Rasmussen & David B. Resnik (2023) Using AI to write scholarly publications, Accountability in Research, DOI: 10.1080/08989621.2023.2168535
  29. Churi, P.P., Joshi, S., Elhoseny, M., & Omrane, A. (Eds.). (2022). Artificial Intelligence in Higher Education: A Practical Approach (1st ed.). CRC Press. https://doi.org/10.1201/9781003184157
  30. Huh, S. (2023). Recent Issues in Medical Journal Publishing and Editing Policies: Adoption of Artificial Intelligence, Preprints, Open Peer Review, Model Text Recycling Policies, Best Practice in Scholarly Publishing 4th Version, and Country Names in Titles. Neurointervention. https://doi.org/10.5469/neuroint.2022.00493
  31. Williamson, B., Macgilchrist, F., & Potter, J. (2023). Re-examining AI, automation and datafication in education. Learning, Media and Technology, 48(1), 1–5. https://doi.org/10.1080/17439884.2023.2167830
  32. Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1). https://doi.org/10.1186/s41239-019-0171-0
  33. Ventayen, R. J. M. (2023). OpenAI ChatGPT Generated Results: Similarity Index of Artificial Intelligence-Based Contents. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4332664
  34. Floridi, L., & Chiriatti, M. (2020). GPT-3: Its Nature, Scope, Limits, and Consequences. Minds and Machines, 30(4), 681–694. https://doi.org/10.1007/s11023-020-09548-1
  35. Drori, I., Zhang, S. X., Shuttleworth, R., Tang, L., Lu, A., Elizabeth, K. E., Liu, K. X., Chen, L., Tran, S., Cheng, N., Wang, R., Singh, N. K., Patti, T. L., Lynch, J., Shporer, A., Verma, N., Wu, E., & Strang, G. (2022). A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. Proceedings of the National Academy of Sciences of the United States of America, 119(32). https://doi.org/10.1073/pnas.2123433119
  36. Zawacki-Richter, O., Marín, V.I., Bond, M. et al. Systematic review of research on artificial intelligence applications in higher education – where are the educators?. Int J Educ Technol High Educ 16, 39 (2019). https://doi.org/10.1186/s41239-019-0171-0
  37. Chaudhry, M.A., Kazim, E. Artificial Intelligence in Education (AIEd): a high-level academic and industry note 2021. AI Ethics 2, 157–165 (2022). https://doi.org/10.1007/s43681-021-00074-z
  38. Popenici, S.A.D., Kerr, S. Exploring the impact of artificial intelligence on teaching and learning in higher education. RPTEL 12, 22 (2017). https://doi.org/10.1186/s41039-017-0062-8
  39. Hinojo-Lucena F-J, Aznar-Díaz I, Cáceres-Reche M-P, Romero-Rodríguez J-M. Artificial Intelligence in Higher Education: A Bibliometric Study on its Impact in the Scientific Literature. Education Sciences. 2019; 9(1):51. https://doi.org/10.3390/educsci9010051
  40. Fyfe, P. How to cheat on your final paper: Assigning AI for student writing. AI & Soc (2022). https://doi.org/10.1007/s00146-022-01397-z


Appendix

Logbook

Week Name Total Breakdown
1 Famke 8.5h Group discussion (2h), Target group and requirements (1/2h), Studied papers (4h), Wrote summary for papers (2h)
Gabriëlle 7h Group discussion (1h), Studied papers (4h), Wrote summary for papers(1.5h), Target group and requirments(1/2h)
My 8.5h Group discussion (2h), Setup of wiki (0.5h), Studied papers (4h), Wrote summary for papers (2h)
Naud 9h Catch up (1h), Gathering information for appeal ERB (2h), Gathering relevant papers (1h), Studied papers (4h) Writing summary of papers (1h)
Niels 8h Group discussion (2h), Studied papers (3.5h), Wrote summary for papers (1.5h), Made Gantt chart (0.5h), made first agenda (0.5h)
Quincy 7.5h Group discussion (2h), Introduction & Problem statement and objectives (1/2h), studied papers (4h), wrote summary for papers (1h)
2 Famke 5.5h Group discussion (1h), Target group and requirements (1/2h), Searched for literature (2h), Wrote summary for papers (2h)
Gabriëlle 7h Group discussion (1h), Searched for literature (1/2h), Studied papers (3h), Wrote summary for papers (1.5h), Group discussion (1/2h), Prepared for tutor meeting(1/2h)
My 7h Group discussion (1h), Searched for literature (1/2h), Studied papers (2h), Wrote summary for papers (1h), Read and organized already studied literature (2.5h)
Naud 4.5h Group discussion (1.5h), Filled in ERB review form (3h)
Niels
Quincy
3 Famke 3h 20m Tutor meeting (20 min), Group discussion (1h), Worked on survey (2h)
Gabriëlle Tutor meeting (20 min), worked out target group (40 min)
My
Naud Group discussion (1h), Filled in ERB review form (1h), Changed research question and stakehoders (0.5h)
Niels
Quincy
4 Famke Tutor meeting (20 min), Group discussion (2.5h), Working on state of the art (4h)
Gabriëlle
My
Naud Tutor meeting (20 min), Group discussion (2.5h), ERB form correspondence (0.5h), updating wiki with survey questions(1h)
Niels
Quincy
5 Famke
Gabriëlle
My
Naud
Niels
Quincy


Survey questions

Research into ChatGPT

In order to review the potential of using ChatGPT in academic education, we would like to know your experiences with the technology as well as your opinion on it. We will focus on the course Engineering Design, a mandatory course where you go through a complete design process, from the first idea to the realization of the product. We will look at how ChatGPT can be used for this course.

This survey is conducted for the course Project Robots Everywhere (0LAUK0), and its results will be used to shape further investigation in academic regulations regarding AI. The results of the questionnaire will be anonymous and it will take approximately 5-10 minutes to complete.

Do you give consent to participate in this study?

  • Yes
  • No

Knowledge of ChatGPT

Have you heard of ChatGPT before?

  • Yes
  • No

Do you know the capabilities of ChatGPT?

  • Yes
  • No

Could you name some of these capabilities?

  • Open question


To give you an idea of what ChatGPT is, please read this:

ChatGPT is a large language model developed by OpenAI. It stands for “Generative Pre-training Transformer” and uses deep learning technology to generate text and respond to questions in a human way.

The model is trained on a large amount of text data, making it capable of understanding context and semantics. This makes ChatGPT a popular choice for chatbots, virtual assistants and other applications where human interaction is required.

Some examples of what ChatGPT can do:

- Write emails

- Write essays and academic reports

- Write poetry and song lyrics

- Compose music

- Answer (test) questions and solve problems

- Generate lines of code based on a prompt

- Answer customer queries

Usage of ChatGPT

Have you used ChatGPT before?

  • Yes
  • No

How often have you used ChatGPT?

  • Only once or twice
  • Monthly
  • Weekly
  • Daily

For what have you used ChatGPT?

  • Open question

How would you rate your experience with ChatGPT in general?

  • Rating from 0 to 5

How would you rate your experience with ChatGPT as an academic tool?

  • Rating from 0 to 5

ChatGPT in academic education

Have you ever used ChatGPT for academic purposes? If yes, for what course(s)?

  • Open question

Would you say usage of ChatGPT should be considered your own original work (so no plagiarism)?

  • Yes
  • No

When should the use of ChatGPT be considerd as plagiarism?

  • Open question

Looking back, would you think ChatGPT could be a good addition to the course engineering design?

  • Yes
  • No

Please, if possible, elaborate on why? (Elaboration on previous question)

  • Open question

For what do you think ChatGPT CAN be used in the course engineering design?

  • Open question

For what do you think ChatGPT SHOULD be able to be used in the course engineering design?

  • Open question

Learning objectives of Engineering Design

How likely do you think you would be able to reach the following learning objectives of Engineering Design with the use of ChatGPT? (Very likely - Very unlikely)

  • Execute a generic design process
  • Formulate a design goal
  • Define the functional and technical specifications
  • Generate an elaborate list of realization possibilities for the different functions of the design
  • Select a number of design concepts from an extensive list of realization possibilities
  • Make a final design choice between a number of concepts
  • Develop a detailed design that meets the specifications
  • Develop and execute a test plan for the prototype
  • Evaluate a prototype based on test results and give an advice for redesign
  • Reflect on the design and on the design process
  • Write a design report describing the design process, the foundation of the choices made and the evaluation of the delivered design


Additional Learning Objectives

How likely do you think you would be able to reach the following learning objectives with the use of ChatGPT? (Very likely - Very unlikely)

  • Gaining insight in designing a simple computer program
  • Giving and understanding simple mathematical proofs
  • Understand how basic electrical and electronic circuits work


Feelings about ChatGPT

What are your feelings regarding ChatGPT? (Strongly disagree - Strongly agree)

  • I see ChatGPT as having/could have had a positive influence on my engineering design project
  • I would have seen ChatGPT as a usefull tool for my engineering design project
  • I see ChatGPT as a shortcut for tedious tasks I would have/had to perform in the course engineering design
  • I will use/would have used ChatGPT in the course engineering design regularly
  • I feel like I would have learned something when using ChatGPT for the engineering design project

Interview script Teachers

Interviewee: [name] Date: [date]

Interviewer: [name] Location: [location]


Introduction

Hi, my name is ___ and I’m conducting this interview as part of our project for the course Project Robots Everywhere.

The goal is to review the potential of using ChatGPT, an AI chatbot, in academic education, so we would like to know your experiences with this technology as well as your opinion on it. We will mainly focus on the course Engineering Design and will look at how ChatGPT can be used for this course.

  1. What is your name?
  2. What is your function / role within Engineering Design? (lecturer, tutor, etc.)


Knowledge and experience

Before we go into the specific course, we’d like to know what your knowledge and experience is regarding ChatGPT

3. Have you heard of ChatGPT before?

4. Do you know its capabilities?

a. If yes, give examples

[Explain capabilities, show examples]

Some examples of what ChatGPT can do:

  • Write emails
  • Write essays and academic reports
  • Write poetry and song lyrics
  • Compose music
  • Answer (test) questions and solve problems
  • Generate lines of code based on a prompt
  • Answer customer queries

5. Have you ever used ChatGPT before?

a. If yes, what for?

b. Have you ever used ChatGPT as an academic tool?

c. How would you rate ChatGPT as an academic tool?

d. How would you describe your overall experience with ChatGPT?


ChatGPT in academic education

Now we will go more in depth about the potential usage of ChatGPT in academic education, and more specifically, Engineering Design.

6. What is your opinion on using ChatGPT as an academic tool?

7. What is your expectation for the future of academic research with these AI technologies on the rise? Do you believe the course should change to reflect this?

8. How do you think chatbots like ChatGPT would influence students’ ability to write? Does this also affect their ability to write code?

9. Does using ChatGPT count as plagiarism in your opinion?

10. Would you be open to using / allowing your students to use ChatGPT in your course?

a. If yes, how would you implement it? Would you actively encourage it

b. If no, why?

11. Would the learning objective of Engineering Design still be obtainable when using ChatGPT?

a. Which ones will become “useless” since automation is becoming more generalized? (for example, writing simple code)

b. What would you change about the learning objectives? (make them more complex?)

c. How would you adjust the assignments to fit the new criteria?

12. What would you think of giving students an assignment in which they are required to use ChatGPT?

13. In your opinion, does ChatGPT make the education of students better, or does it hinder them in their development?


Survey input

t.b.d.


Theory

ChatGPT is a large language model developed by OpenAI. It stands for “Generative Pre-training Transformer” and uses deep learning technology to generate text and respond to questions in a human way.

Some examples of what ChatGPT can do:

  • Write emails
  • Write essays and academic reports
  • Write poetry and song lyrics
  • Compose music
  • Answer (test) questions and solve problems
  • Generate lines of code based on a prompt
  • Answer customer queries


Possible questions

  • How do you think chatbots like ChatGPT would influence students’ ability to write?
  • What would you think of giving students an assignment in which they are required to use ChatGPT?
  • Does using ChatGPT count as plagiarism in your opinion?
  • In your opinion, does ChatGPT make the education of students better, or does it hinder them in their development?


Interview Script Student "Experts"

Interviewee:[name] Date:[date]

Interviewer:[name] Location:[location]


Introduction

Hi, my name is ___ and I’m conducting this interview as part of our project for the course Project Robots Everywhere.

The goal is to review the potential of using ChatGPT, an AI chatbot, in academic education. You have been selected because you are well known about ChatGPT and its uses, so we would like to know your experiences with this technology as well as your opinion on it.

  1. How long have you been studying at the TU/e?
  2. How long have you been using ChatGPT


Knowledge and experience

  1. For what purposes do you use ChatGPT?
  2. Why do you use ChatGPT?
  3. For which of these purposes do you find ChatGPT works the best?
  4. How do you feel about your experiences with ChatGPT?


ChatGPT in academic education

Now we will go more in depth about the potential usage of ChatGPT in academic education, and more specifically, Engineering Design.

  1. What is your opinion on using ChatGPT as an academic tool?
  2. Does using ChatGPT count as plagiarism in your opinion?
  3. Did you complete the course Engineering Design?
  4. Would you, and when yes, how would you use ChatGPT to obtain the following learning objectives of Engineering Design?
    1. Execute a generic design process
    2. Formulate a design goal
    3. Define the functional and technical specifications
    4. Generate an elaborate list of realization possibilities for the different functions of the design
    5. Select a number of design concepts from an extensive list of realization possibilities
    6. Make a final design choice between a number of concepts
    7. Develop a detailed design that meets the specifications
    8. Develop and execute a test plan for the prototype
    9. Evaluate a prototype based on test results and give an advice for redesign
    10. Reflect on the design and on the design process
    11. Write a design report describing the design process, the foundation of the choices made and the evaluation of the delivered design


  1. Do you think you would actually learn these objectives while using ChatGPT?
  2. In your opinion, does ChatGPT make the education of students better, or does it hinder them in their development?