The Percent Evaluation method at DRS2020

We delivered a workshop on the Percent Evaluation method at DRS2020! We, Elisavet Christou, Adrian Gradinar and Pinar Ceyhan were particularly excited about presenting this method for the first time and testing aspects of it during the workshop. Percent Evaluation is a practical method to help design researchers explore their evaluation needs and develop an evaluation plan by allocating and calculating a percentage of a research project’s resources for evaluation purposes. We were part of the largest ImaginationLancaster presence at DRS so far, with 4 workshops, 2 conversations and 8 paper presentations and although the virtual format of the conference due to COVID-19 has presented us with some challenges (time-difference, adjusting workshop for virtual delivery), we were delighted to participate in this vibrant design research community conference experience. 

At ImaginationLancaster we are researching ways for embedding evaluation processes into research projects. The inter/multi-disciplinary environment of ImaginationLancaster inspired us to develop this practical method for embedding evaluation processes into research projects and to tailor it to the design research community needs. We have identified the need for design researchers to embed evaluation into their research projects to capture various elements of value, for example impact, change, learning or engagement, while we have also identified several difficulties approaching evaluation. This is often because evaluation is a complex process with various steps and actions. Often researchers think about evaluation at the later stages of a research project’s timeframe, when they need to produce reports demonstrating the impact of their research. Or sometimes, researchers approach evaluation as an audit, a process that needs to happen in order to present evidence of value to others, like funders or stakeholders. 

With Percent Evaluation, we present design researchers with a practical method that helps them explore and plan a project’s evaluation from the early stages and embed evaluation processes into the project’s design. This can help researchers capture robust evidence related to change, impact, learning, or any other important aspect of their project’s value throughout the project’s timeframe while it can also help inform the research itself by creating opportunities for learning, collaboration, feedback and reflection. 

Percent Evaluation lives on website – currently under development – and it’s a three-step method. The steps are 1. EXPLORE, 2. PLAN & ALLOCATE, 3. CALCULATE. Users will be able to create an account and use the method along with its resources as they wish.  

1. EXPLORE, helps you understand more about the WHY, the WHAT and the HOW of your project’s evaluation by answering and reflecting on a set of exploratory questions. This is where even if you don’t know exactly what kind of evaluation you are aiming for, you can spend some time exploring your options and possibilities, through a step-by-step process.  

1. PLAN & ALLOCATE, helps you develop an evaluation plan. Based on your EXPLORE answers you can now add your evaluation activities to a timeline and allocate relevant resources like working hours and costs.  

1. CALCULATE, is where you calculate the percentage of your project’s resources that you allocated to your evaluation plan. Our percentage formula translates the allocated resources to a percentage – supported by a visual representation – of the whole of your project’s resources. There is no right or wrong % as it completely depends on your evaluation needs, plans and available resources. 

The method presents you with a visual representation of your project’s evaluation percentage. Our aim for this method is for users to have the option to export relevant forms, calendars, plans and visualisations, as well as, a full report of your project’s evaluation at the end. 

 At the workshop we tested the first step, EXPLORE, through a live exploratory session followed by a 10-min conversation around some of the benefits and challenges of our method. Overall,  the participants thought that Percent Evaluation is a method that can be useful for design researchers, especially for those who are trying to find creative ways for capturing value or those who are exploring ways to showcase research impact and value to address various research quality and excellence frameworks. As a team, we are looking forward to further developing and testing Percent Evaluation by developing interactions and collaborations within the broader design research community and we hope to be able to present you with a final version of our method in the coming years! 

If you want to learn more about the method or wish to test it within your own research group, please get in touch. 

Elisavet Christou: e.christou@lancaster.ac.uk 

Adrian Gradinar: a.gradinar@lancaster.ac.uk 

Pinar Ceyhan: p.ceyhan@lancaster.ac.uk