Online Surveys

This is an introduction for federal government program managers on how to collect customer feedback via online surveys. Common types of online surveys include:

• Email—survey is emailed to customers, either as a link to a web-based survey, or questions are included in the body of the email. Often used as a post-transaction survey, as well as for broader customer feedback collection
• Pop-up—“pops up” with a request for feedback after a visitor has landed on your website
• Website—a link on your website to a survey, often used to gather feedback on a website or web-based interaction

Survey basics

1. When should you use an online survey?
2. How can an online survey help you?
3. With whom should you coordinate before launching an online survey?
4. How do you identify potential survey respondents?
5. How can you use online surveys to collect feedback about specific customer interactions?
6. What are the limitations of online surveys? 

Survey design

1. Where can you find tips about survey and survey question design?
2. How long should it take to fill out a survey? 
3. How can technology improve survey questions and overall survey design?
4. What are some factors that influence the reliability of survey responses?
5. Should you collect personally identifiable information? 
6. Should you test your survey prior to launch?
7. What kind of response rate can you expect, and what factors influence response rates?
8. How can you increase survey response rates? 

Operational details

1. How much does it cost to run an online survey?
2. Do online surveys need approval under the Paperwork Reduction Act (PRA)?
3. Do online surveys need to be Section 508 compliant?
4. How frequently should you run a survey?
5. What tips do you have for collecting survey data?

Survey basics

1. When should you use an online survey?

Email surveys are useful for collecting feedback after an online customer service interaction such as an email or chat; after a website visit; or for proactively reaching out for customer feedback. While email surveys are often used to collect feedback after an in-person or telephone interaction, it is a good idea to collect feedback through the same channel as the initial interaction.

Sample email survey questions include:

• Did we provide the information/answer you needed? [yes/no]
• How easy was it for you to interact with us through email?  [1 = not easy to 5 = very easy]
• How satisfied are you with the service you received from us?  [1= very dissatisfied to 5=very satisfied]

Pop-up or website surveys are useful for collecting feedback after a customer has visited your website, or for collecting feedback from customers for whom you do not have contact information. Sample pop-up or website survey questions include:

• Why did you come to our website today? [list answers]
• Were you able to complete your task? [yes/no]
• If you were not able to complete your task, why not? [open-ended]

2. How can an online survey help you?

A plan of action for following up on customer feedback is critical for your survey’s success. You can use the feedback in many different ways, including:

• Training and coaching. Customer feedback from an online survey can sometimes be tied to a specific customer service representative interaction and used for coaching or training purposes. Or, the feedback can be used in aggregate to identify areas for improvement for frontline customer service staff. 
• Improving website or email response processes. This feedback can pinpoint where exactly a customer was dissatisfied with an online experience.
• Improving business processes. Feedback from online surveys could be shared with other parts of an agency to improve processes. For example, survey results could reveal that a customer had to interact with an agency multiple times about the same issue. If this feedback relates to a specific business process, the feedback can be used to make improvements.
• Identifying areas for more in-depth investigation. These surveys can reveal early warning for areas that might need improvements.
• Setting performance goals, establishing performance metrics and setting service standards. If you are using online surveys for this purpose, it is important that your survey follow a rigorous quantitative survey design, and you should contact a statistician or survey expert when creating these surveys.

3. With whom should you coordinate before launching an online survey?

It is helpful to get buy-in from key stakeholders prior to launching a survey in order to ensure that results are acted upon. Examples of key stakeholders are individuals from all teams who might need to make changes based on survey outcomes, including web and program teams. Contact your agency’s PRA office about approval processes, and whether your survey will be eligible for the five-day fast-track PRA approval process.

4. How do you identify potential survey respondents?

For email surveys, organizations often use internal email customer lists. If you don’t have a list, they can be purchased from list brokers. These lists typically include individuals who have volunteered to receive unsolicited emails or to participate in surveys. The more narrowly you define the criteria for participation, the more challenging it will be to find participants. For pop-up or website surveys, visitors can opt-in to participate in the survey after visiting your website. For pop-up surveys in particular, bear in mind that some people use pop-up blockers with their web browsers, which will impact participation.  

5. How can you use online surveys to collect feedback about specific customer interactions?

Because email addresses are usually linked to specific customer accounts, you may be able to connect survey responses to a specific customer interaction, but only as allowed by privacy laws. For example, if a survey request is sent out after a recorded email, chat or phone transaction, you could ask for feedback about the specific customer service interaction. This type of feedback is often used for coaching and training purposes, although connected with demographic details it can also be used to better understand your customer base. Bear in mind that federal agencies are restricted by privacy laws, and you should contact your agency’s legal department to find out more.

6. What are the limitations of online surveys?

Any type of online survey will only reach a subset of your total customer base. For email surveys, your survey population will be limited to those customers who use email and whose email addresses you have. In addition, email surveys are voluntary, with responses only from customers who choose to participate.

Pop-up and website surveys share the same limitations as email surveys, except the survey is not limited to customers whose mail addresses you have, but rather to visitors to your website. In addition, with pop-up and website surveys, you often have less information about and less control over who actually participates in the survey (since any visitor can take the survey).

Back to Top

Survey design

1. Where can you find tips about survey and survey question design?

See our survey and question design guidance. 

2. How long should it take to fill out a survey? 

The number of questions in a survey should be the minimum needed to achieve your feedback collection goals. Ideally, it should take less than 5 minutes and no more than 10 minutes to complete a survey. Shorter surveys also have a higher response rate than longer surveys. In addition, the abandonment rate increases with every additional question.

3. How can technology improve survey questions and overall survey design?

Many survey tools enable survey customization for better results. For example, using conditional or branching logic, you can hide, skip, or display questions based on responses to a previous question. Response biases can be reduced by randomizing question order or randomizing response order within a given question. Some survey tools allow “answer piping,” where answers from previous questions are embedded into subsequent questions.

In addition, some survey software can send an email alert when a customer provides extremely negative feedback. These alerts allow leadership to act quickly to address the issue, or to use that feedback for immediate coaching and training purposes. Also, some survey technologies can also redirect customers to a new URL or pop-up windows with additional information. 

4. What are some factors that influence the reliability of survey responses?

Shorter surveys often yield more reliable responses than longer ones (as do questions asked at the beginning rather than end of a survey). Reliability also depends on how close to the interaction the survey is completed, and on the wording of questions. Feedback collected immediately after an interaction is much more accurate than feedback collected later. See the survey and question design guidance for more information on factors that influence survey reliability.

5. Should you collect personally identifiable information? 

Collecting some personally identifiable information can be useful for segmenting your customer base, but only as allowed by privacy laws. In particular, collecting contact information can be helpful if you plan to respond directly to an individual complaint or dissatisfied customer. If you would like to take advantage of the five-day fast-track PRA assessment process, you should not collect any personally identifiable information, although you may still be able to ask for certain demographic information (e.g., age and location). 

6. Should you test your survey prior to launch?

Testing a survey is important for ensuring that the feedback you receive meets your survey goals, the survey questions are easy to understand and answer, and to check for technical or formatting errors. It is helpful to have a pilot launch with individuals whose background are similar to those who will be taking your survey, and to have the test run on different computers. After the pilot launch, you should find out what participants were thinking while filling out the survey questions. Survey tests should be run far enough in advance of survey launch to correct any errors that may arise.

7. What kind of response rate can you expect, and what factors influences response rates?

Online survey response rates vary greatly, with a carefully targeted email survey typically getting a 10-15% response rate. Pop-up surveys response rates tend to be significantly lower.

The biggest factor influencing response rates is the strength of the relationship between the customer and the agency, and the interest that the respondents have in the survey topic. Other factors that decrease response rate include long surveys and sensitive questions. Well-constructed surveys, with questions relevant to respondents, will also increase response rate. ln addition, different customer segments will have different response rates, and certain demographic groups may have more time to fill out a survey. Finally, offering incentives also increases the response rate, and the impact of these incentives is greater if respondents have a low interest in the survey topic. Contact your legal department if you are considering paying your respondents.

8. How can you increase survey response rates? 

For email surveys, response rates drop off significantly after the initial launch of survey (roughly half the responses to a particular solicitation will come within one day after launching your survey), so a follow-up or reminder email can be useful. Some other ways to increase response rates for email surveys are timeliness (sending the survey out as soon as possible after an interaction to ensure relevance), as well as personalization. Indicating how long it will take to fill out a customer survey reduces abandonment, and shorter surveys have a lower abandonment rate than longer surveys. Offering incentives also increases the response rate, and the impact of this is greater if the relationship between the respondent and agency is weak, or if respondents have a low interest in the survey topic. Contact your legal department before paying participants.

Back to Top

Operational details

1. How much does it cost to run an online survey? 

Costs vary, but online surveys are typically one of the least expensive survey options, especially compared to traditional survey methods such as phone or mail surveys. A very basic solution for a very small number of survey contacts (e.g., less than 100 participants) can sometimes be free, or as low as $20 month. Things that contribute to cost include the size of the distribution list, how customized your survey will be, the tools and templates you want to use, the level of integration needed, and type of analysis you will be doing, and the overall sophistication of the survey product you need. More sophisticated solutions provide more functionality and yield more insightful results, but also cost more.  Check to see whether your agency has a survey tool internally or a license to an external tool. In addition, find survey tools that are available to the federal government through federal-compatible Terms of Service Agreements.

2. Do online surveys need approval under the Paperwork Reduction Act (PRA)?

Yes, in most cases, online surveys to collect customer feedback require PRA approval, but this does depend on how you plan to use the survey and conduct your analysis. For example, if you are collecting information to identify early warning signs or to get a general sense of customer feedback, then your online survey may be eligible for the five-day fast-track PRA approval process. Contact your agency’s PRA officer to find out more.

3. Do online surveys need to be Section 508 compliant?

Yes, online surveys run by agencies need to be Section 508 compliant.  Some off-the-shelf survey applications are compliant, so be sure to check for that when researching survey tools. For more information, go to www.section508.gov. 

4. How frequently should you run a survey?

Survey frequency should be determined by your survey goals, how frequently you plan to act on the data you collect, and your resources for running the survey.

5. What tips do you have for collecting survey data?

Each survey tool can yield data in a different format (although most surveys tools do provide data in MS Excel). If you need data in a particular format, make sure that you can get it in that format prior to launching the survey.

Back to Top

Examples

• Department of Labor Web Site Customer Satisfaction Survey
• Department of Education Internet Customer Survey
• Environmental Protection Agency Website Customer Satisfaction Survey
• Office of Personnel Management Customer Satisfaction Survey

Resources

• Usabiliity.gov
• Basics of Survey and Question Design

»Back to How To Collect Customer Feedback

Content Lead: Alycia Piazza
Page Reviewed/Updated: September 11, 2012