Communication Measurement: QA

Monday, November 28, 2005

Q: How do you start developing a communication measurement strategy?

A: Start with a SWOT analysis (Strengths, Weaknesses, Opportunities and Threats). Consider such elements as how well your current communication program will:
Support business strategy;
Reach and meet the needs of various stakeholder groups;
Have an appropriate mix of channels (type of channel, direction of flow, timeliness, etc.);
communicate the right messages;
have the right organizational staffing, reporting relationship and financial resources to do all the above effectively.

As you brainstorm with others on your SWOT analysis, you will discover that on many aspects of your program, you don't know enough to categorize something as a strength or a weakness. You may not be sure of the distribution of many channels, you may not know what your stakeholders are interested in knowing about or how well they understand subjects that senior leadership might want them to know about. All of these will suggest content areas that should be included in your measurement process.

A typical communication measurement process starts with some objective assessments of what you are communicating. Objective analysis tools include inventories, content analysis and reading grade level tests. Once you are clear on what communication you are sending out, it's time to ask your audience for their input, using some qualitative research first (executive interviews and focus groups) to identify areas for quantitative analysis (typically surveys).

Q: Besides readership surveys and focus groups, how else can a company effectively measure its customer communication?

A: The ultimate measure is sales, but the trick is to find ways of tracking your communications against sales in a way that eliminates the effect of other factors. One way is to pre-test various approaches to communication with different, demographically identical segments of your customer audience.

For example, a California utility company had been sending out a brochure to encourage customers to call for a free home energy audit. (California energy utilities are required to reduce the per capita consumption of energy, so the audit finds ways for a customer to use less energy.) Response rates from the current brochure had started decreasing. The marketing communication manager pre-tested mock-ups of several different new brochures with focus groups. He then printed small quantities of the two most preferred versions and mailed them to different random samples of his audience. He mass-produced the brochure that resulted in significantly more customer calls.

Another approach is to measure the effectiveness of different types of communications in generating sales leads. A very simple way is to list a different phone number or post office box for potential customers to reach you in each different communication piece, news release, advertising, direct mail, Web site, etc. Your computerized phone system can track how many calls come through on different phone lines, even if all the lines are answered by the same group of people. You can then calculate the average number of leads divided by the cost of each channel of communication. If your tracking system is a little more elaborate, you could go a step further and also track what percentage of leads from each communication results in sales. You could then calculate the revenue generated from each type of communication divided by its cost. If your company has a system for tracking customer questions or concerns, you could monitor the number of customer calls on various topics, change your communications to better address those issues, and then track whether the number of questions on those issues goes down.

Q: How often should you measure communication?

A: The answer depends on what you're measuring, if you've had time to make any changes since the last measurement and how large your audience is. Typically, surveys are conducted no more than once every 12 to 24 months. However, if there are aspects of your culture or a publication you are actively trying to change, you may want to supplement the large surveys with mini-surveys on key measures administered to small samples of your audience more frequently, perhaps quarterly or even monthly. During a time of massive change, you might even survey more frequently to measure the impact of specific changes or announcements.

On the other hand, if you have a relatively small audience of only several thousand, conducting frequent surveys with a large enough sample to be statistically reliable would mean surveying the same people several times a year, which is not recommended.

Q: How can I measure how well supervisors are communicating with their employees?

A: First, be clear on whether you need "soft," qualitative findings or "hard," quantitative data. For example, if you want to track improvement or compare different supervisors, you will need the kind of statistically reliable results available from a survey. Even if you need hard numbers, initial focus groups with supervisors and employees will help you ask the right questions and pick the right measurement method. Focus groups might discover that the problem isn't communication skills at all, but that supervisors haven't been given the content to communicate. This information can be found by asking the supervisors themselves how well-informed they feel. Another thing to measure is the frequency of staff meetings. It's possible to have supervisors who know how to conduct meetings, but who rarely do. Track this by asking employees how frequently they have staff meetings and how often they should be conducted.

Second, determine the purpose of your information gathering. If the goal is to assess the general level of supervisory communication, you can include questions on an employee survey administered to a random sample of employees and use that information to prioritize communication training content. If you want to assess the communication effectiveness of individual supervisors, you would instead need to conduct a survey of all employees to obtain enough responses to tell how well supervisors in different organizational units are doing. Better yet, surveys can be administered in work groups to assess the skills of the group's own manager. 360-degree feedback on each supervisor from peers, subordinates and bosses is often used as part of the performance appraisal process. When conducting a 360-degree appraisal, it is important to ask managers to evaluate themselves as well. Comparing the results of how well they think they communicate with how their subordinates rate them often highlights very interesting and surprising discrepancies.

You can also ask employees and supervisors their perceptions of the relative importance of various supervisor communication skills for doing their jobs well. Different skills may be more important in some parts of your organization or for different types of jobs. The priorities for training will then be the skills that are rated as very important, yet are not frequently or well demonstrated. In addition, a statistical technique called regression analysis can help determine the relative importance of various communication behaviors.

Q: How will the results be fed back to employees and supervisors?

A: Typically, the results of a broad survey will be fed back to all managers at once, and then to all employees. However, if part of the purpose of the measurement is to improve communication within work groups, it can be even more useful to fee back results in a "cascade" from top management to individual work groups. First the president would share his or her results in a meeting with the vice presidents. They would also discuss what actions will be taken to improve weaknesses and make more use of the strengths. These vice presidents' direct reports feed back the results to their staffs. When the assessment cascades downward, each new group of evaluators feels safe in being candid because they have already experienced the benefits of providing this type of information to their own bosses in a safe environment.


This article was taken from Strategic Communication Management.

posted by upay at 12:11 AM 0 comments