Blumen Research

Actionable surveys

A great article by John Goodman and Patty David from TARP Worldwide Inc. on why customer satisfaction surveys are not actionable was recently written for Quirks.com. I want to focus on the methods they bring forward to create a more actionable survey and expand on those ideas.

Ask direct questions about what went wrong: Pre-populate your questionnaire with a list of things that could have gone wrong.

This is a strong point and according to the authors it helps in uncovering as many as three times more problems. I assume that it provides a base and starting point for clients or consumers to analyze problems they are currently having, a great starting point.

Integrate survey data with other sources: Placing the research data alongside the contact data along with key data describing what the company has done with the client creates a more relevant analysis.

As more CRM applications open application programming interfaces (API) it is imperative on market research to tie this data with customer satisfaction analysis providing companies with a more in depth analysis compared to standard demographics. Sales forces and Oracle both have methods to integrate data with third party applications.

Suggesting specific actions: Start to come up with suggested solutions for at least the top problems within the report.

Where I thought the authors were going to go with this point is to suggest actions that the client or consumer can take right after the survey is complete. If during the development of the survey a company can already establish what some of the core issues that may arise, it would be worth while providing direct solutions automatically emailed to the client after the completion of the survey based on their responses.

By providing clients with preemptive solutions before the entire analysis is conducted company can show a desire to improve and further develop strong client relationships.

Creating economic correlation to research data: Quantify the cost of taking action or failure to take action.

By placing a dollar amount on the issues that will be located in the Improvement area of the Importance Satisfaction model companies will have a better understanding of the impact in improvement on their bottom line.

For the full article check out Quirks.com (free registration is required).

Derived importance in the Importance Satisfaction Model

In the post covering the value of the importance satisfaction importance model I mentioned three areas that have to be taken into account before drawing results from the model. The notion that respondents will correctly identify importance levels for each product or service attribute will now be up for debate.

From the last post:

It is up for debate if customers can correctly identify importance levels for each attribute. When asked an importance level for several attributes what client would not want to indicate that each one is important. You have my business now focus on everything. While not all clients take this approach it is still a situation that occur. On the flip side of having clients indicate importance levels you can use various regression models to put together derived importance levels.

While it true that each respondent would want each and every issue taken care of in a timely and effective manor, they can also distinguish which ones are extremely important and which ones are important. The fewer questions located on the questionnaire the easier it will be for the respondents to identify important aspects. It would take a bit of testing to establish that number, but anywhere around 10 - 15 questions would be fare to nail importance. Over that number and you might be entering into the derived importance territory.

Derived importance

The first step is carefully designing the actual survey. Survey length even if you are going over the 15 questions should still be taken into account so the respondent does not opt out at a certain point or suffer from survey fatigue which will affect satisfaction and derived importance results.

While deciding on the number of questions to include you can conduct qualitative research with a subset of the total target to refine the list of attributes to be included in the survey. The shorter the survey, the more respondents you are bound to have which increases the likelihood of establishing statistically valid results for within segments, just in case it is still difficult to establish the attributes to importance based on the model.

Calculated derived importance is done by correlating the overall satisfaction rating with the satisfaction level for each attribute or by conducting a multiple linear regression and standardizing the coefficients.

Improvement Actions and the Importance Satisfaction Model

In order to raise the value of customer satisfaction where through a customer satisfaction index or in percentage you need to focus on the areas that requires improvement and maintains the areas of excellence. In an ideal situation with unlimited budgets each company would be able to focus entirely on every aspect of their business at each point in time to increase customer satisfaction. Since we never deal with ideal situations it would be worth while to tackle the headaches and maintain high satisfaction levels in areas that your client base is already ecstatic with. To correctly establish business operations to focus improvements you can utilize an importance satisfaction model.

Importance satisfaction models are divided up into four quadrants representing four areas of your business. On the x-axis is the importance level as established by the client base and on the y-axis is the level of satisfaction for each attribute. To divide up the model into the four quadrants, the customer satisfaction median is used as the central point dividing the y-axis while the median of the importance levels is used as the central point dividing the x-axis.

Four quadrants can be defined as follows:

Excellent

Attributes located in this area are ranked high in importance and high in satisfaction. It is the responsible of the company to maintain these levels of satisfaction to keep the client coming back and hopefully refer new a couple of new clients in.

Improvement

The work begins now to get this area improved. This quadrant contains all attributes with high importance but low satisfaction levels which in turn affects total customer satisfaction results and will impede a company’s growth.

Surplus

Clients are highly satisfied with these attributes, but don’t view them as being important. The performance of the company within this area will exceed the client’s expectation and should of course be maintained.

Fix, but don’t fret

These are attributes with low satisfaction and low importance. Companies don’t have to pay to much attention to these attributes as client’s don’t view them as being important, however client’s will also be pleasantly surprised by improvements in this area.
Our four quadrants make up the model and is a useful tool to guide management to make decisions on the business operations to focus effort and resources. During times where cost reductions are required management can turn to the surplus area.

As with most models there are considerations that have to be taken into account. I will outline them here and focus on each one in later posts.

1.  It is up for debate if customers can correctly identify importance levels for each attribute. When asked an importance level for several attributes what client would not want to indicate that each one is important. You have my business now focus on everything. While not all clients take this approach it is still a situation that occur. On the flip side of having clients indicate importance levels you can use various regression models to put together derived importance levels.

2.  Since the survey is conducted using ordinal scales (Very satisfied, Not satisfied etc.) it is also debatable whether you can represent the median for satisfaction or importance results as a definitive number.

An area that I will examine will be utilizing sliding scales within surveys, which may fix this issue.

3.  After the model is complete and the four quadrants are established will it then be wise to focus efforts on attributes with the To Be Improved section with higher importance levels or lower satisfaction levels?

Within the subsequent pots I will focus on these areas within the importance satisfaction model, but at the same time it is ready to be utilized by management.

Analyzing your competitor’s recommendation levels

Running customer satisfaction surveys on your current client base while not a completely straightforward task with respect to building out the survey and the analysis is at least straight forward with respect to building out the client database. But after completing your survey and establishing the relevant metrics leading to high recommendation levels and economic growth, every company should come back and consider the recommendation levels of their top competitors.

Within every industry there will be scores of competitors with ranging market share. Understanding the core issues leading to high recommendation levels and underlying reasons leading to lower customer satisfaction and recommendation levels of your top competitors will provide your company with the know how to bite into at least a small portion of their market share and possibly focus on a business operation within your own company that otherwise might have gone unnoticed.

Now the main question is how do we target our competitor’s customers to analyze their satisfaction and recommendation levels?

To gather scores for your top competitors there are essentially two methods.

The easiest approach is for businesses that compete for the consumer’s or client’s “share of wallet”, meaning that the buyer may be using several vendors to purchase their products or service or will purchase one category from you and another from a separate vendor.

In this case you can analyze your current share of wallet of your existing client base and within the same survey ask your clients’ which other vendors they use, using a radio button and how likely it is that they will recommend the vendor to a friend or business contact.

The second option which will apply to most companies is a supplement to your customer satisfaction analysis, a short top competitor analysis. For this case you will want to keep the survey short and sweet as the respondents currently have no vested interest in assisting your company. To gather the required data you need to survey a representative sample of the entire customer base, which you should already know since you target the same customer base. Also, make sure to have an accurate and representative sample based on geographic location.

While targeting the customer base for your market you should have a max of four questions on the questionnaire.

1. Using a radio button style question, ask the respondent who their top three vendors are from a list of about the top ten vendors in the industry.

2. In your opinion who is the top vendor in the market?

3. How likely is it that you would recommend [vendor] to a friend or business contact?

4. What was the reason for providing your recommendation score?

After compiling all your responses and ensuring results are statistically significant you will have a birds eye view of the competitive landscape and the recommendation levels for your top competitors as well as why the respondent provided the score.

Taking a look at the entire customer base is more difficult and time consuming than analyzing your own client customer satisfaction which is why the survey itself should be short enough to gain a quick view and a high response rate.