Impact

Evaluation

The project’s impact has been monitored continuously throughout its duration and will continue to be monitored for the next three years through an evaluation process built into normal university processes such as business planning, the access and participation plan and TEF/NSS action plans which all feature specific targets for closing the BAME attainment gap.

The robustness of methods used

Some limitations have been noted with the methods used to conduct the research. Firstly, the rate of participation varied across the three participating institutions (University of Roehampton, Carshalton College and Queen Mary University of London), but overall there was participation across all the planned activities. These producing findings that although are not generalizable, are the observed findings which inform the recommendations.

Secondly, the use of BAME student consultants for this research needs to be considered with care. There is an ethical dimension to this approach and a responsibility to protect the BAME student consultants from possible backlash for undertaking racialised work. Reflecting the BAME student voice has its merits, but we should not put BAME students at risk by asking them to carry the burden of emotional labour which, is a commpon outcome of undertaking work in this area.

And finally, there were concerns over the absence of a robust coding method, due to project staffing issues, and what could reasonably be expected of student consultants, along with the project timeline and student lifecycle. These factors impacted on the depth of research undertaken. However, an interpretation of the emerging themes and the robustness of their relationship to the research literature has been undertaken in line with appropriate research protocols.

Evidence of success

RAFA2 has put in place solutions that are showing evidence of closing the BAME attainment gap in each of the partner institutions.

Student outcomes: Data submitted in the Roehampton and QMUL Access and Participation Plans (2019) show that for both providers, the BAME to white attainment gap has narrowed and progress has been made over time.  Local data for Carshalton echoes this trend, but attainment figures are not reported because the numbers of students graduating are so small. There has been little change in terms of degree outcome mapped by ethnicity. The attainment gap compared with white students has narrowed for both black and Asian students, though outcomes continue to be worse for black students. 

At both institutions, first and upper-second class awards (as a percentage of total) for black and Asian students have increased at a higher rate compared with white students in both institutions. An analysis (Bunge and Uribe 2017) of students’ ‘final outcome’ (i.e. including students who withdrew as well as those who graduated) has shown ethnicity to be a strong determinant of student success when other factors such as gender and socio-economic status has been controlled for. This study also highlighted that lower attainment for BAME students was more pronounced within the Faculty of Humanities and Social Sciences, departments that sit at the heart of RAFA2’s work.

Positive impact on staff: In terms of increase in awareness of potential causes of the attainment gap and some of the assumptions and practices that hinder identifying solutions. In ownership, staff are more readily accepting and taking responsibility in finding and implementing solutions. This is coupled with an increased desire to want to tackle the issues and make change. As a consequence of RAFA2, staff have requested more opportunities to talk about race in safe and supportive settings.

Positive impact on students: Participants report an increase in confidence and in engagement from the opportunities provided to talk about their concerns. RAFA2 provided enhanced employability opportunities, uplifting events which students reported helped to foster of sense of belonging and inspiration alongside providing them with information and guidance on improving performance in assessment. Through working as partners and co-creators, RAFA2 has enabled the providers to realise their commitment to improving student engagement, developing student agency, political efficacy and supporting student to make a difference that counts.

Assessment: All three institutions have implemented new assessment strategies to improve assessment outcomes for students. Roehampton and Carshalton have updated their assessment and feedback frameworks to strengthen guidance and expectations around assessment strategies and are building on assessment literacy and assessment for learning. QMUL has created a new university assessment strategy, which will require every School in the next two years to undertake first an assessment review and then a redesign in partnership with students for implementation thereafter.

Space to reflect and an opportunity to be creative and innovative in making changes to the curriculum: RAFA2 provided the project team and academic colleagues with space to reflect and learn from participating in the project. Partners reflected on and adapted the methodology to find ways to replicate and sustain RAFA2 in the different settings. Academics reflected on their analysis. RAFA2 provided an opportunity for staff and students to work together to identity local solutions, which include curriculum developments and enhancement initiatives, created in partnership with students.

Contribution to enhanced monitoring: RAFA2 has resulted in institutions analysing ethnicity data more thoroughly – and in a more nuanced way.  The information gathered is being used routinely in action planning, target setting and is being monitored, reviewed and evaluated as part of institutional improvement in student success and outcomes activities. 

Production of materials: RAFA2 provided the resourcing needed to facilitate staff to undertake a full exploration of their own programme data, resulting in the production of actions, targets and materials to inform their practice and the practice of others in the sector. This etoolkit contains examples of the resources developed by the students and staff across the three institutions. 

Identification of solutions RAFA2 has made an impact on all three participating institutions by identified solutions that can be replicated across the sector. Learning can be taken from processes and from the solutions identified in our activities, outputs and outcomes.

Evaluation of activities

We worked with with over 50 students as partners/co-creators/co-developers/co-deliverers, and as student as researchers.

This led to:

• Contributing ideas with ideas acknowledged and acted upon
• Consultation and gathering feedback and opinions in focus groups
• Participation, including meetings with decision-makers
• Partnership, having a key role in decision-making
• Activism, such as identifying problems and solutions, and advocating change
• Leadership which covered co-planning and co-educating
• Having a share in the responsibility of the outcomes

During the project, RAFA2:

• Paid students working on project, such as student consultants and student researchers
• Provided student training in leadership, project management and research
• Facilitated staff CPD, such as raising awareness and exploring what lies behind the data, and developing solutions for implementation
• Examined teaching and learning practice, including changes in assessment and feedback practices
• Student engagement in masterclasses and on belonging, employability, hosting events and the production of materials for staff CPD