Comparing Surveys

This guide outlines the survey comparison tool, an Upshot feature which enables you to compare two deployments (instances) of the same survey to assess the differences in response. We have seen this becoming increasingly important to measure changes in behaviour and attitudes as well as tracking progress.

Both Standard and Anonymous Survey Types can be created and used for comparison.

The system will automatically calculate the changing results between two deployment (instances) of the survey for responses that are quantifiable and set as required, such as whole number, decimal number, scale, yes/no, single choice, multiple choice, count and matrix questions. For questions which require text-based responses, the survey will not be able to make quantitative comparisons, however when downloading survey results you will be able to see these text answers side by side.

All comparison results can be downloaded into a CSV file to provide you with the same data displayed on screen in an appropriate format for further offline analysis in a spreadsheet.

Creating a Survey for Comparison
Creating Deployments
Comparing Deployments
Summary of Comparison Results
Filtering the Comparison Results
Webinar
Key Terms

Creating a Survey for Comparison

To begin with, you will need to create the initial survey to be deployed. To do this, select Create Survey from the evidence tab.

It is important that all questions you wish to compare in the survey are required – this will allow the results from each question to be compared in the future instance of the survey.

Once your survey is finalised you will need to set up at least two different deployments to enable it to be compared.

Note: Anonymous surveys can also be compared.

To find out more about Creating Surveys please click here.

Creating Deployments

To set up your first deployment click on Send.

It is worth clearly naming each deployment so when it comes to comparing surveys you know which instances you are comparing. For example - the initial deployment could be called Baseline and the second deployment Follow up. Fill this information in under Name.

You must add at least one participant to your survey at this stage to be able to create a deployment.

Once you are happy with your selections click Send/Generate. For further guidance on the different ways of completing survey responses please click here.

If you select View survey template this will now take you back to the following screen:

To add more responses to your Baseline deployment click view.

From here select Add participants.

To create your second deployment, to enable you to compare responses, go back to your survey template and then select Send:

Follow the same steps as before and name your deployment clearly, e.g. Follow Up, Post Project.

Once all steps are complete you should end up with two deployments:

Comparing Deployments

When comparing two deployments, only responses from participants who have responded to both instances of the survey will be included in the comparison.

Once the second deployment has been sent out and the survey results have been submitted, you are ready to compare the results. To do this click view next to the survey you want to compare the results for.

From the viewing page (once results have been submitted in both deployments) you will have the option to compare survey results.

Summary of Comparison Results

Depending on the type of question asked you will get a comparison question by question. The example below is a comparison of a scale question.

Not only will the system give you a comparison of the overall results, it will allow you to quickly look at individual’s results across the two deployments:

In this instance we can see our recipient’s individual results comparing the first deployment against the second.

Clicking the Download spreadsheet option once you have generated the results of the comparison will download all of the results for further analysis into a CSV file where you can create things such as radar graphs or tables demonstrating the change in results if you wish.

Click here for more information on how you can create graphs and charts from your download.

Filtering the Comparison Results

Users can also choose to filter the responses used in the comparison across two deployments.

For example a user might want to filter the Baseline/Follow Up comparison to only compare:

Survey responses within a certain date period (i.e. for a long running survey only those who completed in the last 12 months).
Responses from attendees with certain demographics (i.e. those of a certain Gender or Age).
Response from attendees who took part in a particular Project, Activity or session.

This can be done from the comparison page described above. At the top of the screen users can apply filters after clicking the + button shown below:

This opens up a range of filters similar to the People Report, with the addition of the date range for Survey Responses received.

When filters have been chosen the comparison for each question will only include people that match that criteria and a message like the below will be shown:

If downloading the comparison results, these will also take into account the filters chosen.

Webinar

As part of the Upshot Community Webinar series in 2020 a thirty minute webinar was delivered called 'Gathering Evidence with Upshot surveys' and this can be seen below. Specific focus around Comparing Surveys can be seen from 27:00 onwards.

Key Terms

Deployment (Folder of survey responses):

This is a folder of attendees’ survey responses. This is where you can group together certain responses to the same survey, for example for an Evaluation survey those responses related to a specific group or activity. Also, the system can run a comparison of different deployments of the same survey. For instance, in surveys to do with Mental Health and Wellbeing it is often important to see individual responses change over time. Different deployments would allow you to measure and then compare this. I.e. Baseline results compared to Follow Up.