One of our teachers attended the recent BALEAP PIM: EAP and the Academy in the Age of GenAI: Implications for practices and practitioners and today is sharing his reflections on it. It will be interesting to hear about this and compare it with the AI-related talks that took place at the InForm conference that I attended sometime after this PIM took place!

The session will include a few reflections based on current research: focus on 2 conference presentations that Tim attended, on two aspects of how Gen AI and EAP intersect. We will also discuss how the issues raised may impact our professional practice.

We started by looking at statements from a session Tim gave in November 2024 and identified how things have changed since then:

• There were opinions that AI is even more widespread now and established now, and getting better at what it does, with more natural responses.
• There is also more opportunities for creating AI tools e.g. Gemini Gems, so more customisable.
• In terms of policy, it is on the way to become embedded in assessments and curricula here at Sheffield, but across universities policies are very different, there isn’t uniformity.
• We are more aware of drawbacks including environmental damage, ethical issues, reliablity of results, information ‘dumping’.
• Academic research into Gen AI and EAP has grown rapidly in the last two years, is much broader now.

In terms of the BALEAP day, there was a full day of presentations with 8 strands that each included several papers. There was a lot of overlap between strands. We are going to focus on two only:

First presentation: How Gen AI reshapes perceived academic writing ability in EAP contexts by Dr Elhadj Moussa BenMoussa (University of East London)

Tim had seen one of Moussa’s presentations before in the academic literacies BALEAP sig so knew of his research. Moussa’s study looked at student drafts of academic essays, then AI-assisted revisions of the same texts, as well as tutor feedback on draft and edited version. (Tim doesn’t know the level of the students or class they were in.) But they were in-sessional. And recurring patterns were found:

Before AI: writing weaknesses were visible and tutors could identify various issues e.g. grammatical errors, spelling, cohesion.

After AI: the text improved, weaknesses such as those above became hiddent beneath fluent prose but the thinking had not improved.

“The illusion of competence” was the speaker’s phrase. AI texts often appear fluent, coherent, academically structured (in terms of paragraphing) and confident in terms of style and tone. However, they may contain weaknesses in terms of argumentation, critical engagement, source integration, disciplinary reasoning and authorial stance (writer’s voice in other words). The speaker was looking at it from an academic literacies perspective. He found that the text appears stronger but the underlying understanding remains unchanged.

The AI version has added ideas, has better cohesion, higher level vocab. In the AI-assisted version, the element of the student’s perspective on feedback has been lost. If anything, the student draft is more critical even though simpler. The AI version has no argumentation in it. We are assuming an average level of simple, straightforward prompting. Prompting expertise doesn’t contradict the argument here however. The AI assisted version sounds more academic but does it demonstrate deeper knowledge? No, it is just descriptive.

Why does this matter to EAP?

Traditional indicators of proficiency are becoming less reliable for assessment as AI performs these automatically. EAP practitioners are uniquely places to help understand the shift to evaluating reasoning, knowledge, rhetorical decision making, source use and intellectual ownership.

Moussa argues that we need to “make thinking visible” (epistemic visibility was the fancy term used!). We need to make students demonstrate why they made choices, how arguments were developed, how sources were selected, how evidence was interpreted and how conclusions were reached. I.e. shift from what was written to how was knowledge constructed.

As a takeaway, Moussa recommended:

That was the first presentation. The next stage was to discuss for five minutes: what do you think the implications of these ideas are for how we adapt assessments, feedback on written work and classroom activities on EAP (pre-sessional, in-sessional, foundation…)?

Ideas shared in our session:

• focusing on speaking rather than writing as more difficult to fake
• focus on the process rather than the end product but at the same time do we need to go back to in-room exams rather than ongoing/project assessment? Not sure how much focus on process helps.
• Have to accept they are using it and think more about how
• More use of reflection on own learning progress with reference to specific lessons and learning materials
• Given that students will be expected to be able to use AI effectively in future workplaces, it arguably makes sense to assess their ability to use it well rather than how they perform without it
• A variety of assessment types to cross-reference for gauging student abilities
• One of the things is in class helping to develop their analytical skills on the spot e.g. when we do a task in a breakout room, saying to students right now you’ve done that task, how easy or difficult is it (scale of 1 to 5), how do you rate yourself? (1 to 5). Make them think about it. Then, so why was it difficult? Why was it easy? To get them used to providing answers that aren’t AI-based, but are their own analysis of their learning.
• Also for feedback on regular tasks, you can build in that critical process. Moving beyond lanaguge correction to looking at the ideas themselves and encouraging reflection.
• Consciously use AI in a course programme and ask students to use it in a particular way and then they critique the output that they get
• Adapting marking criteria
• For each assessment it has to be stated what the parameters for AI are.
• Bit of a paradox: that shift to arguing, styles, evidence etc sounds as if we have surrendered and are trying to engage students receptively but not productively?
• Or is it more helping the students improve their reasoning etc as they produce the text and then polishing it with AI? If you can help students as much with reasoning and argument as with language, but you have to look at language as well e.g. how to do these things. But if you already have the reasoning formulated in depth, the polishing is less of an issue. The suggestion isn’t to copy and paste!

Focus on Gen AI literacy.

What do we understand by this term?

Ideas shared in our session:

• writing prompts
• being able to evaluate output quality and relevance
• being able to use AI tools to enhance learning and productivity (ethically and appropriately)
• knowledge and flexible application of AI tools for different purposes, i.e. social , academic, vocational activities
• know how to use accurately and appropriately according to circumstances

Here is a definition of GenAI literacy from the literature:

Second presentation: “Student practices and experiences of Gen AI: developing a practice-oriented heuristic for GenAI literacy” by Julio Jiminez, Katherine Mansfield and Richard Paterson from University of Westminster.

Background: There is tension between institutional discourse (polices, integrity, guidance, how we teach it, recommendations) and student practice. Students are already integrating gen AI into their practice but in more nuanced ways than institutional discourse assumes. Policies struggle to keep pace with rapidly changing practices and technologies.

Research question: What do students actually do with GenAI in everyday academic life?

The speakers surveyed 441 students across 66 disciplines.

Findings:

GenAI is used by students for…

• generating ideas (68%)
• planning work (66%)
• finding sources (33%)
• receiving feedback (24%)

Note: Very different percentages between the top two and bottom two. They talked in focus groups with the students as well.

Key themes:

1. Students use GenAI strategically: they pick specific tools for specific tasks; their selection takes into account what they consider the uses and limitations of the tool; their use suggests that rather than simply using tools because they are there, use is based on the fit between the task and the tool.
2. GenAI literacy is emerging: students do critically evaluate the accuracy, reliablity and bias of tools and output and they are developing awareness of what is and isn’t appropriate use. To improve this, there needs to be ongoing decision making which is context-sensitive.
3. Learning is increasingly human-AI mediated: GenAI is used to support planning, drafting and revision. However, students maintain their evaluative control over the outputs. Therefore, academic practice is becoming a human-AI hybrid process.

Overall, this all indicates that students’ GenAI literacy is moving beyond technical proficiency towards strategic, critical and context-sensitive use.

The speakers came up with 4 principles for how these findings could feed into approaches to teaching GenAI literacy:

They came up with this framework (Framework rather than practical steps):

From left to right it moves from more basic to more complex.

Finally, we discussed the following more practical questions based on this session:

1. Does any of your experience of student use of GenAI align with or differ from the findings in any way? How?
2. How well does the heuristic (framework) match how students are taught AI lteracy in an institution or teahcing situation you are familiar with?
3. Can you think of any practical exercises, workshops or courses for sudents that could incorporate the ideas from this session?

Ideas shared:

• Students are given specific information about AI use but it isn’t incorporated into day to day activities. So one participant uses it in classes to help students become more independent users but class time is limited. Need longer sessions where you can demonstrate something and give students time to try it. So it shouldn’t be an add-on anymore.
• Students used gen ai to create diagrams for their presentations and cited them as well as referenced their creations They were relevant and much better to support their speaking than looking for what approximates relevance for their topic – So creating visuals. It worked really well in conjunction with academic referencing/citing. They were responsible enough to cite and add sources if necessary. It helped them support their speaking. This requires increased literacy in terms of knowing about doing that and how to do it. However, with simple prompts they managed very good results.
• Get Gen AI to write an essay in class and then get the students to analyse its weaknesses, such as poor argumentation and use of evidence. This requires criticality in this area.
• Often a massive disconnect between AI-positive message from here and what people experience. There needs to be a more joined up approach.

Tim’s overall reflections on the conference were as follows:

• he doesn’t know as much as he thought about EAP and AI (I think that is probably true for many of us!)
• There is a lot of research going on into EAP and AI
• EAP professionals have an important role to play in the transition to the integration of Gen AI in HE institutions
• Students may be using GenAI in a more strategic, critical way than EAP practitioners assume

My thoughts:

My first take-away is how much student agency there is around AI. That whole strategic use thing. It suggests that it is really important in terms of AI literacy and how we teach use of AI to involve the students’ perspective as much as possible, to better understand their use and be better able to help them shape their use to fit the specific context in terms of what is and isn’t acceptable and helping them refine their use. I suppose the more confident we become about GenAI, the more confident and better able we will be to have these sorts of conversations with students.

Thinking about my specific context, this last academic year has been the first year where we have acknowledged AI rather than ignoring it beyond having a blanket ban which is impossible to enforce. We relied mainly on interactive content put together by the digital team, which we gave students time to complete in class. In the coming year, in contrast, the idea is to have in-class activities that are teacher-led rather than interactive content-led regularly in the first five weeks of the course and then at a few specific points thereafter. We will see how it goes!

My second main take-away is the difference between the student example and the AI revision. I can imagine a student who fed in the studente example and got back the AI example would be like “yeah, great, this is better! I’ll use this!” while their actual message has been lost. Of course, they’d need a clear understanding of both their message and the AI output in order to realise that. So I wonder how we help them become better able to do that. I need more time to think about that! I always need more time to think about things but time is in such short supply…