Evaluating our Digital Education Services

In 2023, we are launching a new twice yearly Digital Education Services Evaluation survey. This is part of a new service improvement strategy that will combine user feedback, anonymous usage data and statistics taken from Help4u to help us monitor the effectiveness of the services we provide and prioritise service improvement activities based on what we discover. We will be developing this strategy over time as we learn the most effective ways to gather feedback and respond in a targeted, effective way.

Fill in our January 2023 survey now!

We will be sending out 2 surveys per year – one in January and one over the summer.

Goals

This initiative will focus on the effectiveness of the services we provide and the community’s ability to use them. It will not look at learning & teaching practice, which is out of scope here. We want to:

  1. Monitor the effectiveness of our services and our user’s ability to exploit them, over time.
  2. Identify and prioritize areas for improvement and intervention.
  3. Track the effectiveness of those improvements and interventions.

This is simple enough and nothing new, but we’re keen to move toward more structured model for evaluating how well our services are doing and how confident our community is in using them.

The Plan

The survey we’ve just put out (January 2023) is a first step that will, become a strategy for improving what we provide over time. We will take the result we get from these surveys and combine them with 3 other sources of feedback: usage analytics, Help4u statistics and targeted feedback:

Usage analytics are simple figures we can extract from our various tools and platforms that tell us things like ‘How many people logged into My Dundee today?’ or ‘How many times was Turnitin used to manage an assignment compared to Blackboard Assignments?’. Note that we haven’t started processing this kind data on any scale, but as we do, we will ensure users are fully informed about what data we use and how we use it in accordance with our legal obligations and simple good practice.

Help4u Statistics allow us to look at all of the times people have reported any kind of problem with a service. We can sort these ‘tickets’ into useful groups that tell us things we can act on. For example, If the majority of tickets relate to users being unsure how to use the kit in a teaching room, then we know we need to divert some resource toward guides, videos and training.

Targeted Feedback is when we need to dive deeper into a service area. This may be because there a strategic priority or because other feedback tells us there’s a problem, but we don’t know exactly what. This research might take the form of additional surveys, focus groups or interviews.

The idea is that as we process these varied sources of information together, we can get greater insight into what’s working, what isn’t, and why.

Use of Data

The data we collect, and process will only be used to evaluate the effectiveness of our services, allowing us to identify areas for improvement and measure the effectiveness of interventions. This evaluation does not use personally identifiable data.

Results

We will publish the result of our evaluations so that you can see what we’re doing and why. Hopefully, we will be able to build up a long-term story of service improvement over time.

This is a new initiative and we will be learning and developing the strategy over time, so look out for posts in the future as we develop our service evaluation strategy.

Leave a Comment