Creating Surveys

This guide outlines how to create surveys on Upshot. Surveys can provide key quantitative and qualitative evidence which can assist you in the evaluation of activities or to help build individual case studies. They can also become an even more powerful tool when group/individual responses are compared over time. There are key criteria that need to be followed to ensure the system can complete this.

Note: Since the recording of this video there are now additional Question types available such as 'Matrix', 'Decimal number' and 'Conditional' questions.

For ideas of surveys that could be used and some detail on Accredited Surveys from the sector please click here. 

• Creating a Survey
• Survey Types
• Question Types
• Conditional Questions
• Key Considerations when creating a survey for comparison
• Webinar
• Key Terms

If you would like further support when creating your survey, please contact support@upshot.org.uk

Creating a Survey

To create a survey, go to Evidence > Create Surveys.

This will take you to the Add Survey page. Here you can add a Title (required) and a survey Description (optional).

From here you will choose between the different Survey Type(s) – Standard, Anonymous or Public.

Survey Types

The choice of survey type will affect how your survey can be completed, whether it’s attached to Upshot participant profiles and whether those participants can be identified via the results.

Standard

[Sent/Attached to Upshot participants]

You can choose which Upshot participants to send this survey to and all responses will be linked back to their participant profile.

Standard surveys can either be emailed out to Upshot participants or completed on their behalf if you use paper copies. Organisations can also choose to use the Upshot Mobile App with their participants to complete these.

Anonymous

[Sent to Upshot participants]

You can choose which Upshot participants to send the survey to, but responses will not be identifiable.

Anonymous surveys can only be emailed out to Upshot participants. 

Public

[Anyone with Survey link]

You can generate a survey link and then send this to anyone to answer. Responses will not be linked to Upshot participants.

Because of this you have the option to add some default demographic questions to facilitate reporting, if you select this survey type.

Note (I): As the results are not linked to Upshot participants the survey comparison tool cannot be used with this survey type.

Once the survey type is selected you can add a question from the options on the right.

Please see a description of the question types below.

Question Types

There are a variety of question types available on the system: Text, Whole number, Scale, Yes/No, Single choice, Multiple choice, Count, Date, Matrix.

Considering which question type to use for specific questions is important as this impacts how users analyse can analyse their results later. For example a large set of 'Text' responses might be more difficult to analyse than if participants had answered a 'scale' or 'single choice' question.

Organisations will also find some pre-built Standardised Questions available under the heading of Upshot Global Questions. These questions come from validated and accredited surveys from the sector and can help ensure a consistent approach to measuring impact, rather than the need for organisations to create their own custom questions.

Text

This type of question allows a respondent to give a free text response, helpful for questions that require added detail or to explain a previous answer. These are a powerful tool for qualitative analysis purposes, allowing the respondent to provide in-depth feedback.

E.g. -  'Explain how you feel the session could be improved?'

Whole number

This allows a respondent to state how many, or how much, of something there was. 

E.g. - 'How many times a week do you eat chocolate?'

Decimal number answer

This allows a respondent to state how many, or how much, of something there was but also include up to two decimal points in their response. 

E.g. - 'Please specify the weight (KG): ' 

Note: A respondent could still input a 'whole number' in their response here as well as a number with decimals. E.g. the answer to the above could be 18.00 or 18.25.

Scale

This allows respondent to give a rating/score based on their thoughts and feelings, or perhaps to illustrate how many times a week they completed something. 

The scale can be set to between 0-10.

Scale questions can be particularly helpful for analysis purposes, allowing users to later work out an 'average score' for a particular question.

This is why in some instances, setting up a questions with a range of single-choice options such as 'Strongly Agree/Strongly Disagree', might work better as a scale question, as this would be easier to analyse. In these instances it would make most sense to set up the most positive answer 'Strongly Agree' as the highest score on your scale.

It is often helpful to provide additional help text to explain the scoring system on the scale.

E.g. - 'How would you rate today's session?' 

'How many times a week do you exercise?'

'To what extent do you agree with the statement that 'people from your local area can be trusted''

Yes/No

This can be used for definitive opinion questions and are often used in conjunction with follow up text questions to allow an explanation for the response given.  

E.g. 'Do you plan on attending the session again?'

Single choice

This allows respondents to state a preference out of some pre-determined response options. The theme/content of questions where single choice would be appropriate can really vary, but they can be a key way of allowing organisations to quantify responses around a set of predetermined options.

E.g. - 'What do you prefer to drink?'

'How often do you use public transport?'

Multiple choice

This allows respondents to choose multiple responses in relation to a question, allowing users to analyse how many respondents selected certain options.

E.g. 'Which of the following have you heard of?'

Count

This allows respondents to state the amount of time they completed certain actions in response to the options presented.

In instances like the example below, this can save users from having to create separate questions for all of the relevant options.

E.g. - 'How many times did you use each of these methods of transport to get to school in the last week?'

Date

This allows respondents to add a date. This can be helpful if the date of the event they attended was different to the date they are completing the survey, which will automatically be generated in the results.

E.g. - 'What date did you attend?'

Matrix

Matrix questions allow organisations to present questions that have the same set of responses to a series of statements, such as 'Strongly Agree/Strongly Disagree', in a grid format, which can help to shorten the length of the survey.

'Options' (columns) may represent answers such as 'Strongly Disagree - Strongly Agree' whilst the 'Rows' would represent a series of statements or criteria.

The matrix can be single or multiple choice, by default this will be single-choice but users can select 'Allow multiple responses per row?' when creating.

Questions can also be assigned a numeric value or ' weight' that can aid analysis later, by treating these similar to scale questions. When creating, the user can select 'Assign numeric value for answers?' 

Please note that survey respondents would not see these 'numeric values', they would just be shown to Upshot users on screen and be applied in the background to aid users analysis.

E.g. - 'Please select the option that best describes your experience of each over the last two weeks.'

'For the following mobile phones, which features do you like?'

There is also the option to add or use Standardised Questions, this can enhance reporting capability when analysing specific survey questions. To find out more about Standardised Questions please click here.

Once you have chosen your question type you have the following screen appear on the right.

Here you add your question (and answer options if appropriate): 

Once your question is created this will sit in the layout of your survey on the left-hand side of the screen. 

Questions can be reordered by clicking and dragging them up and down on the left-hand side.

Once you have finished creating the survey you have the option to Save Draft or Save at the bottom of the screen. 

If you select to Save Draft the survey is not yet complete, and you can discuss with your colleagues, re-edit and add questions as necessary. 

Once you choose to Save the survey this is now complete, and you can then Send the survey out or input responses on behalf of attendees.

Note: Once 'Saved' the survey can no longer be edited.

Conditional Questions

Users can choose to make survey questions ‘conditional.’

This means that questions are only displayed based on a participant’s response to a previous question.

This can simplify the survey for respondents, asking relevant extra detail of those that answer in a certain way and streamlining the survey for those that do not.

In the below example an organisation creates a conditional question in Q2, that asks users to give more feedback only if they have answered 'Q1 – How did you find today’s session?' with ‘Very good’.

The Display when . . . in the bottom right-hand corner sets the condition for when this question will appear.

• Conditions can be set based on previous responses to 'Whole number, Scale, Yes/No, Single and Multiple-choice questions.'
• A conditional question must always be placed after the original question it is conditional on. E.g. Q2 must always come after Q1 above.
• The conditional question does not have to immediately follow the question it is conditional on, it just has to be after. E.g. Q4 can be conditional on Q1.
• Conditional questions can be made required if desired.
• Multiple conditional questions can be dependent on the same original question.
• Surveys can have multiple conditional questions dependent on another. E.g. Q2 can be dependent on the response to Q1, and Q3 can be further dependent on the response to Q2. See example below:

• When exporting survey results the conditional questions will always appear in the download.
• When Upshot users view the survey in the system, complete a survey response on a participant's behalf or use the Printable Questions option, the condition will be written out below the relevant question.

Key considerations when creating a Survey for comparison

Upshot can compare different deployments of a survey. Survey comparisons are powerful pieces of evidence that showcase the change in groups or individuals from the work you deliver.  

When creating the survey any questions that you want compared must be made required.

The system can automatically calculate the changing results between two deployment (instances) of the survey for responses that are quantifiable such as whole number, decimal number, scale, yes/no, single choice, multiple choice, count and matrix questions. For questions which require text-based responses, the survey will not be able to make quantitative comparisons, however when downloading survey results you will be able to see these text answers side by side.

Note: Standard and Anonymous survey types can be compared using the Upshot comparison tool. Public surveys cannot be, because the responses are not linked to participant profiles.

For more instruction on Comparing Surveys please read the following guide.

Next Steps

For instructions on Sending and Completing surveys please read the following guide.

Other options once surveys are created sit under Tools. 

Here you have the following options:

• Copy – Selecting this option allows you to copy the survey you have created. This is a helpful feature if you want to keep the majority of the survey intact but want to add/remove questions for sending out again. Rather than having to recreate it completely from scratch.

• Download all results – This downloads the results from all deployments into a CSV file which you can then manipulate in Excel for reporting purposes.

• Retire – this effectively ‘deletes’ a survey from the system. Note this option will only be presented if there are no responses against this survey.

Facilitating Organisation Survey Options

Facilitating Organisations can create surveys to:

1. Survey Delivery Organisation Upshot users
2. Survey Delivery Organisation attendees (Note this is only an option if the Facilitating Organisation has access to Delivery Organisations attendees personal information)
3. Create surveys to be used by their delivery organisation accounts. These can be created at the Facilitating Organisation and 'pushed down' by using the Cascading Surveys feature.

Facilitating Organisations are also advised to read the following guides on Standardised Questions and Cascading Surveys before creating and sending a survey. Please do get in touch if you have any questions.

Webinar

As part of the Upshot Community Webinar series in 2020 a thirty minute webinar was delivered called 'Gathering Evidence with Upshot surveys' and this can be seen below. A focus on 'Creating Surveys' can be seen from 05:00.

Key Terms

Deployment (Folder of survey responses): 

This is a folder of attendees’ survey responses. This is where you can group together certain responses to the same survey, for example for an Evaluation survey those responses related to a specific group or activity. 

Also, the system can run a comparison of different deployments of the same survey. For instance, in surveys to do with Mental Health and Wellbeing it is often important to see individual responses change over time. Different deployments would allow you to measure and then compare this. I.e. Baseline results compared to Follow Up.

Standardised Questions:

Standardised Questions are survey questions that can be used to enhance the reporting available to organisations when used. Standardised questions include Upshot Global Questions, a mix of accredited survey questions and ones from large funders in the sector. Organisations can also create their own Standardised Questions. 

Survey Types:

The choice of survey type (Standard, Anonymous or Public) will affect how your survey can be completed, whether it’s attached to Upshot participant profiles and whether those participants can be identified via the results.