In Cambridge, as across the higher education sector, innovations in generative artificial intelligence (AI) such as ChatGPT are giving rise to concerns among staff and students, as well as perceived opportunities to enhance educational experiences and practices. Concerns are particularly associated with academic integrity and implications for assessment; perceived opportunities include supporting students in developing strategies for academic reading and argumentation. During the Long Vacation of 2023, a short survey was disseminated via membership of the General Board’s Education Committee to faculty / departmental directors of teaching (and equivalent). The survey aimed to elicit short accounts of respondents’ perceptions and some accounts of initiatives within departments and faculties to explore beliefs and practices of staff and students relating to the use of generative AI in teaching, learning and assessment.
This paper does not represent an analysis of the affordances and limitations of generative AI in higher education. Rather, it presents an initial, inductive thematic analysis of responses at a particular point in time and indicates significant variations in beliefs and in familiarity with technologies such as ChatGPT. These insights will inform the further development of support for inclusive approaches to teaching, learning and assessment at Cambridge over the short and medium term (one to three years).
This paper was prepared by Cassie Lowe and Meg Tait, CCTL, November 2023.
Read: Cambridge staff perceptions of generative AI
The paper is also presented in HTML format below.
Survey: aims and respondents
The survey invited respondents to
- share information about efforts to learn more of students' and/or staff perceptions or concerns relating to the use of generative AI in teaching, learning and/or assessment in their context (for example via surveys, focus groups, working groups)
- comment on questions, concerns and/or opportunities relating to generative AI and teaching, learning and/or assessment in their contexts.
Recognising that the technology is developing rapidly, as is familiarity with its affordances and limitations, the survey was not intended to generate findings which are generalisable; rather, it was intended to generate a snapshot of beliefs and concerns. Analysis of responses will help to inform support over the next two to three years for inclusive educational practice at Cambridge.
In total, 47 responses were received from respondents across 27 departments and all six Schools and from the Institute of Continuing Education: 6 responses were received from Arts & Humanities; 9 from Biological Sciences; 2 from Clinical Medicine; 8 from Humanities & Social Sciences; 9 from Physical Sciences; 13 from Technology. (A breakdown of responses by institution is appended.)
Learning of students' and/or staff perceptions
Question 1: Are you aware of efforts within your institution to learn more of students' and/or staff perceptions or concerns relating to the use of generative AI in teaching, learning and/or assessment in your context (for example, via surveys, focus groups, working groups, other)? If so, please describe briefly any initiatives which you think may help the collegiate University to better understand / support good practice.
Theme One: nothing to report
Some respondents were either unaware of discussions or did not believe there were any current discussions happening within their department / faculty / College relating to the use of generative AI in teaching, learning and assessment. Many said that they were not ‘aware of any’ discussions at the departmental / College level but some had read institutional guidance. Some simply wrote ‘no’ to question one.
Theme Two: informal, local and wider discussions
Many respondents described informal conversations amongst colleagues or local-level discussions around generative AI and Large Language Models (LLMs) such as ChatGPT. Some referred to conversations around LLMs as a recurring theme within departmental / faculty committees. One respondent noted that a School-wide survey was in circulation and another noted a departmental survey to solicit opinions. Another respondent noted that they had been having discussions on the topic at a sector-level network. Few respondents provided any or very much information on how they are engaging students in the discussion at a formal level; however, one respondent stated that they had been having discussions with students at the Staff-Student Consultative Committee and students had fed back that they would like to use it to ‘help with writing style’ and ‘help with programming’. A few noted that consulting students will be the next steps following staff discussions. Some respondents also spoke through examples of where students have been engaged in using the tool (see theme four: proposals for uses / examples of practice).
Theme Three: assessment (e.g. exams, plagiarism concerns)
There were multiple instances of respondents intending to return to exams and/or consider the format of exams, and specifically in two cases ‘closed-book exam[s]’ and/or in-person exams, in order to ‘mitigate risks’. In a similar vein, a respondent stated that their course is mostly exam-based and so will not be impacted by the rise of generative AI / LLMs. Many noted that colleagues were expressing concern, and the tone of responses indicated a degree of anxiety about the impact of LLMs on assessments (however, see theme six for positively framed responses).
One respondent indicated that their department had responded to the growth of generative AI and potential use of LLMs for assessments by including a large, red, bold-typed statement on the undergraduate Moodle VLE examinations section: ‘Content produced by AI platforms, such as GPT-4 and ChatGPT, would not be original work and so would be considered a form of academic misconduct to be dealt with under the University’s disciplinary procedures.’ Another respondent put forward several possible solutions to the assessment risks of plagiarism, such as ‘the use of testing software that does not allow students to leave the test or import information into the online test’ to remove possibilities of copying and pasting or using external software. The same respondent also suggested using monitored computer rooms for testing and returning to ‘old-style’ testing with handwritten exams.
Theme Four: proposals for uses / examples of practice
A number of respondents described or proposed what they perceived as educationally effective uses of generative AI (see also theme six: positively framed responses). Some respondents indicated that they are starting to discuss and explore its possible uses in classroom-style settings. One respondent indicated that they would be leading a unit on ethical tools and generative AI this coming academic year and another had used ChatGPT with their students, getting them to critically evaluate the quality of ChatGPT’s responses to questions. Another respondent suggested that LLMs such as ChatGPT have their place as useful resources for students, for example providing useful summaries and introductions for new subject areas. Similarly, another respondent remarked that ChatGPT can be asked to provide some useful sources for additional reading in areas the students would like to explore further. Another respondent indicated that they are involved in a research project that is exploring how students can be encouraged to use generative AI critically; they indicated they would like to share findings through the appropriate University channels. Finally, another respondent remarked that ChatGPT is already being used by both staff and students to aid with research and that these tools will become as ‘ubiquitous as calculators’; they argued that engaging with technologies such as ChatGPT and understanding how to use them effectively is important for all members of the University community.
Theme Five: requests for further institutional guidance
Colleagues have requested further institutional guidance, and some have provided ideas of what they might like to see to help guide them on their use of generative AI in education. There were requests for University of Cambridge experts to speak on the subject internally, guidance for students on how they might use it to support their learning and avoid plagiarism, a repository of case studies of using generative AI in T&L practice, further guidance on raising awareness of potential threats generative AI poses, guidance for staff on how to use it effectively in their research and teaching (from ideation, to structuring ideas, to grammar). In contrast, one colleague noted that they knew of a policy at an external institution that embraced ChatGPT as an educational tool and that they did not agree with this. One colleague asked for a university-wide policy, another cautioned against anything that would stipulate university-wide demands on practice.
Theme Six: positively framed responses to the introduction / possibilities of generative AI and LLMs
A number of responses to Question 1 were positive in tone. Some respondents noted that AI has the potential to improve teaching and learning and presents an ‘excellent tool if used responsibly’. Some respondents remarked that high-preforming students are using it like Wikipedia to get introductions to otherwise dense material in order to build their foundational knowledge. Others suggested that through using tools such as ChatGPT, students' grammar will improve in turn as a result of the LLM correcting their grammar, perceiving the tool as presenting learning opportunities for students.
Questions, concerns and/or opportunities
Question 2: what questions, concerns and/or opportunities relating to generative AI and teaching, learning and/or assessment are you aware of in your context?
Theme One: risks (academic integrity, learning, admissions)
A number of respondents expressed their thoughts on potential risks and causes for concern, from admissions to exams and the longer-term impact on learning. Many indicated their concerns for the risks of academic integrity, such as students using LLMs to produce work and submitting this as their own; such responses often inferred a need ‘to return to in-person, supervised examinations’. Others described themselves as feeling ‘helpless’ to stem what they perceived as a likely tide of students submitting plagiarised work, remarking that ‘a “no” won’t stop students from cheating if they are intent on doing so’; others wrote of a need to ‘protect the process of teaching’ from this ‘unequivocally toxic’ tool. For some respondents, advances in generative AI put ‘coursework in jeopardy’. However, one respondent argued for ‘mov[ing] on from simple essay assembly from lecture notes as a training and assessment tool’; they indicated an interest in exploring oral exams and case studies. With these risks and sense of inevitability in mind, one respondent asked: ‘how do we account for the use of powerful (AI) writing aids?’. A number of respondents feared a negative impact on learning if students were to become overly reliant on generative AI to ‘think’ for them, rather than developing their own skills in critical thinking and information synthesis: ‘it is hard to see this sort of AI having anything other than a deleterious effect on practical human intelligence’. Another respondent drew attention to consequences for the academy: with scientific papers now being produced by generative AI, they asked, ‘what is left for us as researchers and educators?’
Theme Two: potential mitigations of risks
Some respondents seemed to be thinking of workarounds for the risks summarised above. For example, a number of respondents suggested that coursework assessment might be accompanied by, or substituted with, oral examination; however, it was noted that this is a labour-intensive option and therefore impractical at scale. Another respondent suggested that students be asked to keep and produce variants of the essays produced, for example first, second and third drafts alongside a plan of reading / structure to show how the essay has been developed over time. Respondents also noted the opportunities presented by LLMs for students (discussed further below) and suggested that to mitigate the risks students should be trained on appropriate use of tools such as ChatGPT, to show both the technology’s strengths and its weaknesses. One respondent provided a case study from another institution that seeks to mitigate the risks posed to assessment: ‘one of [the] assignments is that the students require [ChatGPT] to produce a text. The students then annotate the text, including the sources that were relied upon in it’.
Theme Three: opportunities and possibilities
Respondents commented that the risks presented by generative AI are to be taken seriously; however, a number identified what they considered to be opportunities to enhance teaching, learning and assessment. One commented that ‘ChatGPT is a great prompt to think about diversifying assessment’ to explore options and advantages. Another remarked, ‘I would like to be in a position where I can fully encourage the use of ChatGPT and similar generative AI tools for learning. They are exciting tools – and they will be used anyway’. Others indicated that universities need to understand ‘the changing needs for education in the future’ and that trying to fight integrating generative AI tools such as LLMs into higher education ‘will make the way we teach and run the University obsolete in less than 10 years' time’. Other respondents noted that generative AI tools will be a feature of life after university, as companies start to integrate them into their operations, and so it is important to train students how to use them effectively: ‘AI will increasingly be used to produce any written material... so finding a way to exclude it from exams may not actually be appropriate if we are aiming to assess real-world skills.’
Beyond the rationale (or pressures) to integrate generative AI into education, respondents also highlighted what they saw as the positive opportunities which tools such as ChatGPT present. These included increased productivity of students and staff, where tools can assist with writing, data analysis, coding and act as ‘an interlocutor for ongoing processes of research thinking – asking it questions, floating ideas, etc. Which can be a useful contribution to the research process.’ Others noted that generative AI can provide a digest of literature for a specific topic and that students can use tools to organise their thoughts and materials gathered into a coherent structure. A respondent noted that generative AI can undertake the ‘manual skills that were previously considered essential to scholarly work’ and remarked that consequently, new possibilities are available to scholars. Others remarked that using generative AI for grammar-checking may be particularly beneficial to students with SpLDs and those for whom English is not their first language.
Further comments?
Question 3: If you would like to comment further on AI and teaching, learning and / or assessment, please do so.
Just over half of respondents offered further thoughts in response to Question 3.
Theme One: looking back or standing still
A number of responses seemed to suggest that the University should try to fortify existing practices against the implications of generative AI for teaching, learning and, in particular, assessment. Some respondents indicated that ‘cop[ing] with’ the capabilities of generative AI necessitated reverting to previous practice: ‘back to the exam hall, folks. There’s nothing worse than timed written exams under close invigilation – except for all the other modes of assessment.’ Another respondent called for examination environments without access to internet and advised that the University should advocate for a greater use of oral examinations. A further respondent argued that the institution needs to create ‘faster and more robust ways to deal with cheaters’ and that examiners need better support. Another requested that the University should supply and test ‘resources for detection of plagiarism and renew its honour code’; they considered a student ‘writing with AI [as] a betrayal of trust’ and felt that this position should be expressed by the University.
Theme Two: looking forward and seeking opportunities
A majority of those who responded to Question 3 indicated that they are looking to adapt practice and move forward with developments in technology, in order to improve how students are assessed and enable students to develop skills which would be required in the future. One respondent countered calls to revert to closed-book, invigilated exams, arguing it would be ‘inappropriate (but a fix) to go back to unseen exams in gyms’. They explained their reasoning on two grounds: ‘Will the University fund sufficient internet-blocked machines to permit typed exams and is this really the skillset we should be imparting?’ Another respondent argued that, in order to go forward, ask ourselves ‘why we assess and what the purpose of this exercise is, how does it relate to the goal of transferring and creating knowledge’. This same respondent continued: ‘just like with other new technologies like the book print or the internet [educators] will find positive ways to include LLMs in their education structures.’
Several respondents argued that we must embrace technologies through incorporating them into teaching and assessment so that both educators and students can use them ‘skilfully and critically’. Another argued that should the University not embrace generative AI and consider ways to engage students in learning how to use it skilfully, Cambridge will fall behind other universities in a competitive field. This same respondent also argued that the introduction of generative AI means that colleagues might have to rethink their assessment strategies to incorporate more practical opportunities. There is a strong sense of optimism amongst some respondents, who perceive that generative AI presents greater opportunities for teaching, learning and research and that ‘we will learn to adapt and utilise these capabilities’. Others expressed less optimism, commenting that they are not sure how to address impacts on assessment, but commenting that it will be difficult to ‘ban’ the use of generative AI use; they argued that strategies to support safe use are required. Other respondents noted potential benefits for staff and student workloads: for staff, one commented, generative AI can save time on administration, such as drafting recommendation letters and outlines; another saw possibilities to enhance students’ learning.
One respondent noted that they had asked ChatGPT to summarise questions, concerns and/or opportunities relating to AI and teaching, learning and assessment (Question 2). They described its response as ‘quite sound – for the most part!’ and quoted it:
- ‘Personalized learning: ChatGPT can adapt to individual students' needs and learning styles, providing tailored explanations and resources. It can help address specific questions and misconceptions, making learning more effective and engaging.
- Immediate feedback: Students can receive instant feedback on their work, assignments or queries, helping them identify and correct mistakes promptly. This real-time feedback can accelerate the learning process and encourage iterative improvement.
- 24/7 availability: ChatGPT can be accessed at any time, allowing students to seek assistance or access relevant educational materials outside of regular class hours. This availability accommodates different schedules and promotes self-directed learning.
- Supplemental learning tool: As a supplementary resource, ChatGPT can complement traditional teaching methods by offering additional explanations, examples and insights. It can reinforce classroom learning and encourage independent research.
- Exploring complex topics: ChatGPT can break down complex subjects into more accessible and understandable language. It can help students grasp difficult concepts by presenting them in a clear and straightforward manner.
- Language learning and practice: For language learners, ChatGPT can serve as a conversational partner, providing opportunities for practicing and improving language skills through real-time interactions.
- Research and project assistance: ChatGPT can assist students in their research projects by providing relevant information, suggesting sources and generating ideas. This can be especially valuable for literature reviews and initial exploratory stages of research.
- Encouraging critical thinking: By engaging in discussions with ChatGPT, students can refine their critical thinking skills and learn to ask better questions, consider multiple perspectives, and evaluate information critically.
- Virtual tutor or mentor: ChatGPT can act as a virtual tutor or mentor, guiding students through complex problem-solving tasks and helping them develop problem-solving and analytical skills.
- Accessibility and inclusion: For students with learning disabilities or those who require additional support, ChatGPT can provide an inclusive learning environment, offering personalized assistance and accommodations.
- Practical applications for various disciplines: ChatGPT's versatility allows it to be used across various fields of study, such as language arts, social sciences, STEM subjects, and more, making it relevant for a wide range of courses.’
Concluding remarks
This initial thematic analysis of responses indicates significant variations in perceptions across Cambridge; it is clear from reviewing raw responses to the survey that there is a considerable range of beliefs and levels of familiarity with generative AIs such as ChatGPT within Schools and, indeed, within faculties, departments and Colleges. There is a clear sense that for many respondents the introduction of generative AI, and in particular LLMs, poses a threat or challenge to existing practice. For some, this encouraged a reversion to tried-and-tested unseen in-person exams as a method to assure assessment against plagiarism. Others, who indicated an interest in diversifying assessment, nevertheless also felt uncertain as to how to maintain rigour and integrity. Some respondents offered proposals for how tools such as ChatGPT might be used to support student learning to alleviate administrative burdens. Some expressed a wish for clear guidance from the University, while others argued strongly that such matters must be addressed within faculties and departments. For many respondents, generative AIs invite reflection on fundamental questions of how we understand knowledge, learning and the purpose of studying at university level. One respondent remarked:
In terms of learning and teaching, given the freedom to focus on how to give students knowledge […], I trust that lecturers just like with other new technologies like the book, print or the internet will find positive ways to include LLMs in their education structures
They considered that ‘LLMs will have a very disruptive effect on assessment but without asking and answering the question "why do we assess" I do not think we can have a relevant discussion about this’.
Note
The University’s new Blended Learning Service is developing guidance on AI and Education, including guiding principles for using generative AI and frequently asked questions.