Dr Neil Dunne is Programme Director for Trinity’s Postgraduate Diploma in Accounting. In this reflection, Neil reaches out to Programme Directors from across the disciplines, inviting them to consider some key learnings from pandemic assessment that they might carry forward into the new normal.

Over the past five years, Trinity’s Postgraduate Diploma in Accounting has launched the accounting career of almost 200 graduates. Upon completion, graduates attain exemptions from professional accounting exams (e.g., Chartered Accountants Ireland, ACCA and others), which helps them navigate the arduous journey towards professional certification. Professional accounting bodies base their accreditation decisions primarily on the content of syllabi and exams. So when COVID shook Ireland in March 2020, my concerns as Programme Director included not only the pivot to online teaching, but also the challenge of assessment in a pandemic.

Semester 2 exams, which were fast approaching, had been written for a closed-book face-to-face context, i.e., the traditional basis for our professional accreditations. I had to consider how we would assess in April 2020 and beyond in a way that would be online and flexible, yet rigorous enough to maintain our extensive accreditation. It was challenging. With hindsight, I have identified four key learnings on how to navigate online assessment, which I hope are useful for all Programme Directors:

Reach out:  The accounting academic community recognized the issues, and really rallied around each other. I consulted colleagues in the Irish and British Accounting and Finance Associations, both informally and through seminar attendance. Here, I learnt some useful innovations, and also that all Accounting Programme Directors were anxious!

I exchanged frequent correspondence with the professional accounting bodies, who conveyed flexibility and empathy, but perhaps understandably refrained from being too specific on what exactly was required of online assessment. Nonetheless, their documents provided a useful foundation for me to decide on how to assess online.

I frequently checked in with our fantastic faculty, who had to amend their exams for an open-book context, and external examiners, who had to review these amended exams. Similarly, I engaged often with our two class reps, who conveyed the perfectly understandable anxiety and concern of students, and who also played a vital role in communicating my decision-making process to other students via their class WhatsApp groups. I cannot praise the class reps highly enough.

Get the details right: Although open-book exams clearly differ from their closed-book variant, the events of March-April 2020 vividly demonstrated this to me. Let’s start with the front page of the exams. We designed a new standardized cover sheet for open-book accounting exams, which combined guidance from professional accounting bodies, some Trinity-specific declarations, and my own ideas. This cover sheet made clear that answers taken verbatim from a textbook, or unsupported by workings, would not be accepted, and that any examples used in answers should be original (rather than textbook-sourced). These regulations served to allow students demonstrate their own original thinking. To minimise student anxiety and uncertainty, we placed this new cover sheet on Blackboard well in advance of the exam session.

Rather than hurriedly arranging online proctoring, which is expensive and often flawed, we aimed for exams where candidate attainment would be unaffected by the presence or absence of invigilation. This ‘prevention is better than cure’ approach to potential plagiarism necessitated rewriting of our April 2020 exams. Our litmus test became: if a question was answerable entirely by reference to the textbook, or did not allow the candidate to demonstrate original thought, we modified it or removed it from the exam. Operationalising this philosophy necessitated migrating from knowledge-based towards applied and scenario-based questions. Additionally, questions now often sought opinions. For example, a question that might previously have read ‘Describe the nature and purpose of alternative performance measures (APMs)’ became:

Whilst reading the ‘Top Accounting’ website, you noticed the following quote:

“APMs are quite misleading for users of accounts, and should be banned”

Required:  Do you agree with this statement? Refer to material you have studied this semester to support your answer.

The first part of the revised question seeks an opinion, thus privileging original thought. The second part requires that opinion to rest on lecture material, thus reducing the ‘Googleability’ factor. In other words, a candidate seeing APMs for the first time during the exam could not just Google ‘APM’s and update their answer sheet accordingly. In contrast, candidates that had engaged with the material consistently throughout the semester could immediately begin to demonstrate their aptitude.

Speaking of answer sheets…. We decided to allow candidates hand-write rather than type their answers, for two reasons: First, students expressed a strong preference for hand-writing, and were wary of exams mutating into an assessment of Word/Excel proficiency (which aren’t Programme Learning Outcomes), rather than core accounting concepts and skills. Second, the requirement to hand-write answers allowed us to more accurately assess the provenance of each script.

Students downloaded the exam from Blackboard each day, hand-wrote their answers, and then, using an app recommended by us, scanned and uploaded their answers back to Blackboard. We allowed students 15 extra minutes to deal with any IT issues, i.e., downloading and uploading the exam. It generally worked well, and facilitated stylus-based marking/annotation. I’d also set up a ‘mock’ assignment a few weeks before the exam session for students to submit their answers and thus gain practice using the app. Although time consuming, this helped iron out any IT issues in advance of the exam.

Navigate the aftermath: Even pre-COVID, we all know that examiners’ real work begins after the exam, in terms of trudging to the Exams Office, collecting our scripts, and then allocating several weeks to grading, exam boards, etc. However, the online exams surfaced some unique extra considerations. First, we had to closely monitor for grade-inflation. A significant spike in results might have problematized our entire approach to online exams. However, overall results in 2020 and 2021 broadly remained in line with prior years. Second, notwithstanding the mitigation measures described in the previous section, the dreaded spectre of plagiarism still loomed large, and faculty had to extend extra effort in identifying excessive similarity of response. Unfortunately in 2020, there were some cases, entailing difficult emails and Zoom calls. In 2021, we had no such cases.

People are understanding! The various stakeholders affected by our assessment decisions were generally very understanding. For instance, students adopted a pragmatic and resilient approach that will serve them well in the accounting profession. College immediately provided invaluable training modules and seminars around the area of online assessment. Trinity Business School accepted that longer assignment-type online exams would not be appropriate, and facilitated our request to hold two-hour online exams instead. Additionally, our Programmes Team provided fantastic support. External examiners willingly reviewed a whole new set of online exams. Professional bodies understood that we were still assessing the same learning outcomes, and indeed any post-COVID accreditation reviews have been successful. Finally, our accounting faculty demonstrated their long-held great concern for both student well-being and the integrity of the accounting profession.

To conclude, COVID has made us all think more carefully about assessment. Although the worst may be behind us, the new normal will also involve online assessment, so hopefully the above points may be useful to colleagues in Trinity and beyond. Professionally, I have certainly been on a journey these past 18 months, and would be delighted to talk through any concerns with colleagues that wanted to reach out.