I was interviewed a few weeks ago (along with PRSA Chair Gerry Corbett and Council of Public Relations Firms’ Senior Vice President Matt Shaw) by The Business Journals for an article on “How to respond to bad publicity.” The three tips at the bottom of the article come right from what research teaches us about crisis response:

1. “If you act fast, you have a better chance of controlling the story and the outcome.
2. “Treat employees, community leaders and officials as key contributors to helping you solve the problem.
3. “Never miss out on the opportunity to learn from the mistake. The public is far less forgiving when you’ve had this problem before and you let it happen again.”

Frank Ovaitt is President and CEO of the Institute for Public Relations.

To ensure that my students are learning the most contemporary approaches, I frequently reference and use the white papers available on the website of the Commission on PR Measurement and Evaluation. Sometimes, I reference and link to a white paper when I am lecturing or explaining a concept. Occasionally, I require my students to read a paper and summarize or critique its research approach. Many of my PR colleagues at other universities have told me they also use the Commission white papers in their research and campaigns courses.

Based upon feedback from some educators who teach PR research and evaluation, I’ve written a white paper that recommends and describes how many of the Commission white papers can be used in the classroom. I’ve organized my paper by research topics, such as social media and experimental design, so that readers can quickly find resources on topics important to them. I’ve also included a synopsis of recent books on PR research, measurement, and evaluation.

I hope you find this paper helpful as you plan and teach research, measurement, and design. Please let me know if you think other papers should be added to the list. I also welcome your feedback for ways the Commission can assist you in the classroom.

Read Resources for Public Relations Educators for Teaching Research, Measurement, and Evaluation

Dr. Julie O’Neil, a professor of strategic communication, teaches undergraduate and graduate courses in public relations, integrated marketing, and research methods at Texas Christian University.

Klein now admits that those on the left were not necessarily less enlightened; they just tended to stick to their views. In other words, they were likely to refute a true statement if it didn’t align with their beliefs. The correct answers to the economic questions were more closely aligned with conservative views, giving left-leaners a disadvantage in showing off their economic intelligence.

A follow-up survey in 2011 tested whether conservatives and libertarians were equally unenlightened about statements that challenged their views. He concluded that regardless of political leaning, “the more a statement challenged a group’s position, the worse the group did.” In other words, the correct conclusion is that confirmation bias exists, not that left-leaning people are unenlightened.

There’s a great deal we can learn from Mr. Klein’s screw-up that we can apply to our own research.

Questions must be valid for your research purpose. Mr. Klein’s biggest mistake was how he analyzed responses to questions that were not intended to serve his research purpose. The questions from the original survey related to economic issues but were not designed to test economic knowledge.

Avoid seeing something that’s not there. Mr. Klein went digging for information that didn’t exist in the in the original survey. He was looking for it, so he saw it. A good researcher constantly seeks as much meaning as possible, explores different theories and ideas, and analyzes the data from different angles.  But it’s just as important to not see things that aren’t there.Study design is critical in isolating variables so that you can get rid of the clutter that may contribute to an incorrect conclusion.

Repurpose with caution. Repurposing data can be an attractive option, because it’s often less expensive than doing your own original research. It’s especially efficient when looking for basic information and basic relationships. But when you are asking a more complicated question, it’s probably best to design your own study around the research question you are trying to answer.

Klein’s experience is a great reminder of why skepticism and transparency are key when you are delving into the results and conclusions of a study. When a claim seems some combination of outlandish, controversial or counter-intuitive, it’s a good idea to check the methods, the intended use of the data, and the history of the organization doing the research.

In the end, Mr. Klein’s mistakes weren’t costly, besides some initial embarrassment and the energy it took to respond to critics. The criticism led him to correct the mistakes made in the first publication and ultimately give us insightful conclusions based on a well-designed study that matched a well-defined research question.

Kara Capelli is a graduate of The George Washington University’s Graduate School of Political Management, in Strategic Public Relations. She is currently employed as a Public Affairs Specialist for the U.S. Geological Survey.

PRDepiction, a blog devoted to the depiction of public relations in film, TV, radio and books, across all media, has just been launched. In addition to recent productions and books, it lists many of the earliest films and novels that incorporate PR characters.

Although not the first movie about publicity, Bing Crosby’s starring role as the PR man for Hawaii in the 1937 Waikiki Wedding gave the business a glamorous, musical, fun perspective.

In fiction, J. Ward Moorehouse in the first volume of John Dos Passos’s USA Trilogy, published in 1930, is the earliest PR character identified so far. He is portrayed in the novel as an Ivy Lee-type of PR man before the First World War, but becomes less ethical and respectable as the trilogy reaches its final volume. The blog has also identified a novel from 1936 with the title of Public Relations, written by Lewis Lefko. Its secondary title is “a novel of the utilities field.”

The website is the collaborative effort of academics and practitioners in Australia, Ireland, UK and US such as Jane Johnston (Australia), Conor McGrath (Ireland), Philip Young, Scott Davidson and Tom Watson (all UK) and Cheryl Lambert, Karen Russell, Tom Hove (all US). They welcome suggestions of additions, especially from outside the English language. PRDepiction will also feature research and press articles/blogs on the subject.

The URL is http://prdepiction.wordpress.com.

Tom Watson PhD is Professor of Public Relations at Bournemouth University in England. He runs the annual International History of Public Relations Conference and is a member of the Commission on PR Measurement & Evaluation.

Communicating in an environment of risk is one of the more difficult roles for senior communications strategists.

In larger organizations risk assessment often is managed by Enterprise Risk Management (ERM), which sets the risk agenda based on its discipline-specific quantitative tools and processes.  But communicators may find that ERM tools rarely predict reputation risk and often fail to provide a framework for effective mitigation through communications.

An ERM expert may compare the death rates from nuclear power to those from smoking and conclude nuclear power presents low risk to the enterprise. People who live near a nuclear plant may feel dread [or the inability to control an event that may result in catastrophe] that is significantly out of proportion to the fatality rates. Facts do not trump emotion, as communicators know instinctively.

Into this chasm of confusion step researchers who focus on risk communication, a field that has applicability to communicators by examining how people hear messages in certain situations, and the implications for the choices they make. Among other benefits it provides a fact base for the use of emotion.

Professor John Maule is one such researcher. He directs the Centre for Decision Research at Leeds University Business School. He and colleagues analyze perception and the efficacy of methods of communication in risk situations, seeking answers to questions that are essential for strategic communications.

In a recent article Maule cites research that identifies five key attributes for effective risk communication: competence, objectivity, fairness, consistency and acting in good faith. He cites research that suggests communicators should understand how each stakeholder group looks at risk, and at the world. This may require research in the form of traditional focus groups and surveys, or through metadata from social intelligence that unpacks the public dialogue and what it means for a company, product or issue.

He points to a body of work that questions whether we should frame a risk qualitatively or quantitatively for better effect. For example, researchers suggest that we value human life proportionally. The higher the percentage of a group that might be impacted by a risk the more concern we are likely to feel. So, we might feel greater concern when 5,000 people out of 25,000 are affected by a disaster than 5,000 out of one million.

Maule presents a wide-ranging body of research (with a valuable bibliography with which to dive deeper) with some key findings for communicators.

– Approach risk communication as a two-way dialogue between your experts and the public that results in a common understanding of the risks.

–Understand that past experience has a strong influence on whether people will assign the same level of risk you do to a situation. (As a young Kansan I experienced many tornado warnings that did not impact me. But ignoring such warnings could have proved fatal had I been in Joplin, Missouri last spring.)

– Risks that are hard to visualize may not be as compelling. Maule discusses examples of organizations over-predicting risks such from hackers and under-predicting risks from system overloads, when the latter may be a far more common experience.

– Risk communicators should understand and monitor psychological issues such as dread and lack of familiarity attached to their issues, products and programs so they can better predict and communicate risks to the general public.

He divides risk into four types or sources of threats: to well-being, infrastructure, customers and to the public. Have you done communications scenario planning around each type of risk? Are you using social intelligence to track the likelihood and velocity of these types of threats? Do you understand the psychological issues associated with each major risk issue facing your organization?

There are tools that help the communicator understand the emotions inherent in issues in the public dialogue and identify messaging that addresses the concerns. Communications that ignore the emotional underpinnings of an issue may be viewed as cold, uncaring and irrelevant.

On the other hand, communicators who actively monitor and listen, and who engage with the ERM team to bring an outside-in perspective, are more likely to manage risk appropriately, thus protecting the reputation of their organization.

Articles used for this post: “Risk Communication in Organizations,” a chapter by AJ Maule in the forthcoming forthcoming paperback edition of “The Oxford Handbook of Organizational Decision Making” by Gerard P. Hodgkinson and William H. Starbuck, Oxford University Press, 2008; and “Translating Risk Management Knowledge: The Lessons to be Learned from Research on the Perception and Communication of Risk” by AJ Maule, in Risk Management: An International Journal 2004, 6 (2) 17 – 29.

Linda Locke, principal at Reputare Consulting, subscribes to journals from which she draws inspiration and concepts for clients of her corporate reputation consulting practice. If you wish to suggest a research article for her review, contact her at Linda.Locke@ReputareConsulting.com, or DM her on Twitter @Reputationista.

The authors already have experienced consternation and “push back” from those who advocate CSR in any and all circumstances.  Many of these critics are from the communication industry, a major group that has long advocated CSR.  To hear communicators talk, one would think that any CSR program would be a good thing and would be welcomed by stakeholders and the larger society.  It seems reasonable to assume that companies seeking to “do good” would be looked at more positively, but the authors have found that this is not always the case.

The authors examined routes to CSR value: the “direct route” and “indirect or stakeholder route”.   In the direct route, the company finds programs that lead to cost savings or increased revenues.  For example, companies can invest in sustainability and have a favorable bottom line impact.  This is the position supported by “shared values” advocates (I blogged about this in September 2011).  Shared values suggests that companies seek to maximize their own self interest so the best way to get a company to engage in CSR is to find an area that both maximizes their value and also contributes to the good of society.

The “stakeholder or indirect route” is the more traditional route advocated by the communications profession.   This involves the company finding some way to contribute to society.  Stakeholders then judge the program. The expectation, of course, is that stakeholders will praise the company for “doing good”.  This does not always occur.  As a result, there are a lot of companies that have wasted a lot of money.

One of the authors of the book, Daniel Korschun, is a colleague of mine at the LeBow College of Business at Drexel University and a Fellow of the Center for Corporate Reputation Management that I lead.  We talk regularly about reputation management in general and CSR in particular.  My view has always been that CSR success is based on two primary variables: 1) that the CSR activity is consistent with the strategy of the company; and 2) that it is meaningful to the stakeholder for whom it is intended.  The authors support this view, but offer a far more refined basis for the success or lack of success of CSR programs.   The findings can be summarized by three themes: Understanding, Usefulness and Unity.

Understanding:  Stakeholders filter the CSR program against their understanding of the company and their expectations of the firm.  If the CSR programs do not make sense to the stakeholder, i.e., the program is from a company with a poor reputation, the reaction is likely to be suspicion rather than praise.  The initiative in this case can create a negative ROI.  A company with a poor reputation should begin correcting its problems and avoid trying to jump-start their reputation with a large-scale CSR program.

Usefulness: Stakeholders look to their own needs and interests first and ask “what’s in it for me”.  While a CSR program may benefit society at large, the program is likely to be more successful if stakeholders perceive a benefit to themselves.  An example here is the disappointment of many pharmaceutical firms that invested heavily in African River Blindness only to find little or no interest in the United States.  The program was of high value to those on the African Continent, but had little usefulness to Americans.

Unity: When Understanding and Usefulness are in play for stakeholders, they interpret that the CSR efforts align the company’s values with their own.  This, the researchers found, is a “powerful predictor” of CSR success.  The concept of unity is similar to relevance.  That is, the belief that the organization is “like me” or is important to my life.  It is more important than differentiation in a highly complex, competitive marketplace.

When one looks at CSR in this light, one sees a route to decision-making that is more logical and systematic than currently available.  All potential CSR programs can be judged against the “3Us”, and the program can better be aligned with the company’s business strategy and the interests of stakeholders.

Communication researchers should take heed.  We need more research on behaviors and less on opinions that lead us to make decisions that may not be valid.  For example, one well known research study by a major PR firm regularly finds that the vast majority of people would rather do business with a company that is socially responsible.  What a revelation!  They likely also want to be friends with people who love their mothers.  The issue is not what people say, but rather what they do.  Unless we can better predict behaviors, our research will have little practical value.

This is a book well worth reading by all those who are interested in corporate social responsibility.  While it may disappoint some who want companies to do any and all social activities possible, it serves as a guide for executive decision-making.  This book should be a welcome read for those who want more research, science and rigor in communication advice.  It demonstrates that the art of communication needs a good dose of science to assure that its perspectives are not only correct, but also produce the ROI that companies should expect from their investment.

Elliot Schreiber, Ph.D. is Clinical Professor of Marketing and Executive Director of the Center for Corporate Reputation Management at the LeBow College of Business, Drexel University, Philadelphia, as well as CEO of Brand and Reputation Management LLC.

Sir Winston Churchill is widely—and famously—quoted as having said, “Statistics are like a drunk with a lamppost, used more for support than illumination.”   Embedded in that comment is a common—and valid—concern about an overdependence on data and the danger posed when data is used as a substitute for critical thinking.

If Sir Winston were seated across my desk at this moment, I’d make a confession to him:  guilty as charged.  I have used research many times simply as a means of supporting my opinions and conclusions, rather than as a predictive tool.  Illumination is great, but I, too occasionally need something to lean against while I am fumbling for the light switch.

When it comes to measurement and evaluation, I think a lot of us begin there.  And when management knows that our opinions are actually supported by something, that’s a fine start.   But to the point of the lamppost analogy, if we are going to be effective counselors, we can’t just validate—we have to illuminate.

So, how do we make that transition?  Here are a few thoughts:

1. Repetition, repetition, repetition. As we conduct focused research on the same topic over the course of time, the positive outcomes of that research are cumulative.  We build on what we know.  This also allows us to develop multiple research methodologies to drill deeper into the issues.  It is especially helpful in conducting formative research—the research that actually helps shape the strategy behind programs and campaigns rather than simply evaluating their effectiveness.

Here’s an example:  Some years ago, a group of public relations students was asked to assist a community blood bank in reversing the trend of declining blood donations. The students began by performing research to ascertain why blood donation levels were so low.   The blood bank believed (based on current issues and observations) that donors were fearful of contracting a disease, such as AIDS, from the blood donation.  The blood bank also speculated that a newly opened plasma center that was paying for blood had drawn donors away from the blood bank. At the time the students were called in to assist, the blood bank was planning a massive education, awareness and motivational campaign.

The students commenced the research process by polling 400 community members with specific questions about their attitudes and concerns regarding blood donation and the blood bank.  Fascinatingly, not one respondent said he/she believed it possible to contract a disease by donating blood.

The students then conducted focus groups with various community segments and reinforced the survey findings:

• That the publics who would or did sell their blood to the plasma center had never nor would ever consider giving blood at the blood bank, and, conversely—

• That those publics who would or did give blood at the blood bank would never consider selling their blood to the plasma center.

Further, the survey and focus groups revealed the true cause of the decline in donations:  respondents said there was insufficient information and publicity surrounding mobile blood drives.  Understanding this enabled the blood bank to avoid a costly and unnecessary campaign to raise awareness about the safety and importance of blood donation and instead focus on resolving a much simpler publicity problem.

This experience illustrates the bigger point about repetition of effort:  Because the students had time to build on that research, and because they developed multiple methodologies, they were able to truly illuminate the deeper issue.

2. Hang out with your data.

A true understanding of data is often a process, not an event.  It comes as we examine data over time, and in different ways.  It’s also helpful to share data with other people, because they, too will see that data in different ways.  And when others pick up on something we’re not seeing, that can be an important step towards illumination.

3.  Give your data a “voice” so that it speaks compellingly to management and clients.

• Make it relevant. Research findings, like people, get more attention when they are saying something interesting.  Measure things important to clients and management.
• Tell a story. Well-told stories engage audiences.  Lay out your measurement process as a story arc:  “Here’s what the problem was, here’s how we tackled it, here’s what we found out, here’s what reassured/surprised/amazed us, and here’s what we think it means.”
• Be clear about what you are trying to accomplish with the data. People don’t want a good ending spoiled in the first 5 minutes of the movie, but they do want to know what the movie is about before they buy the tickets.

Christina Darnowksi, Research Director at PRSA and a former research consultant, notes:  “When I was a consultant, people would often come into my office, show me a survey they or someone else had already used, and say, ‘I want to use this to survey a group.’  I’d then say, ‘But what are you trying to accomplish?’  When I heard the answer, I’d know that they really needed a completely different kind of data collection instrument.  It’s important to take great care in crafting that instrument.”

The lesson?  If we are merely looking for support, just about any lamppost—or any vertical surface—will do.  But, if we are really after illumination—data that will point the way to doing things better—we have to be very clear about what we’re trying to accomplish.

Susan Walton is Associate Chair of the Department of Communications at Brigham Young University.

So it is with the public relations profession in China.  Several years after my last trip there, I journeyed back with the encouragement and support of multiple Institute for Public Relations (IPR) Trustees – most notably Mark Hass and Bob Grupp.  As IPR adopts an increasingly global stance, I wanted to further develop a sense of how the science beneath the art of public relations™ might support a growing profession in China.

I must offer my warmest thanks to all parties who met with me to share their time and knowledge – among them representatives of the China International Public Relations Association, the Central University of Finance and Economics School of Culture and Communication, APCO Worldwide, Burson-Marsteller, Edelman, GE, Johnson & Johnson and Weber Shandwick.

What did these friends, old and new, teach me?

First, that China has a fast-developing indigenous public relations industry, with a growing number of local practitioners working on the client side and in Chinese and multinational agencies.   The local agencies themselves have global ambitions and seek to establish reputations that will allow them to fulfill those ambitions.

Second, the Chinese public relations community hungers for research and expert knowledge from many sources.  Already, they have sought permission and translated IPR research papers.  IPR in turn wants to acquire and share more knowledge about China’s current research and practices with professionals and academics worldwide.

Third, that some of IPR’s priority research topics resonate well in China.  Of particular interest is our dive into the social science of social networking.  China’s own social media platforms are huge, vibrant and some say better.  Forward-looking academics and practitioners also feel that our organizational communication initiatives could be of value to huge Chinese companies.

Fourth, that universities in China are held in high regard and may be natural allies in IPR’s efforts to build research-based knowledge.  In many countries, practitioners believe they are ahead of academics in their thinking about professional practice.  Not so, it seems, in China.

Fifth, any time you enter a new environment, it’s time to reexamine your assumptions about how things work.  One of my contacts suggested that while public relations strategy often can be global, implementation in China must be local.  And while some ethical practices in China disturb Western practitioners, there is evidence that the reverse is also true.

Taking all of these lessons to heart, what comes next?  Certainly the exchange of more research and content, with translations to make the work accessible to more practitioners.  Plus the exchange of invitations, introductions and speaking opportunities.   Perhaps even forming an IPR China advisory group, gathering the in-country representatives of organizations represented on our Board.

When employees like the company they work for, they feel a stronger sense of belongingness and commitment; they are less prone to leave and they look forward to contributing to the success of the company everyday. Another aspect is the critical role of employees as communication assets for the organization. What employees say about the organization is often perceived to be more credible and authentic than messages from senior management or the public relations team.

So how does organizational leadership at all levels influence employees’ perception of the organization’s reputation? What types of leaders are helpful in developing favorable attitudes of employees toward the organization?

Research has identified two major leadership styles in the organizational setting, namely, transformational leadership and transactional leadership. Transformational leaders are characterized by creative, interactive, visionary, empowering and passionate communication behavior. They encourage subordinates to voice opinions and provide them with support, coaching and mentoring. On the other hand, transactional leadership is an exchange process based on rewarding and disciplining employees on the quality of their performance. With this method, the leader elicits an agreement on what needs to be done and rewards followers in exchange for satisfactorily carrying out their assignments. Transactional leaders exert control through policies, rules, bureaucracies and so on, which is regarded as more authoritative.

In 2011, an on-line survey was conducted with 700 randomly selected employees of different levels from a Fortune 500 energy company, and 167 employees completed the survey.  Results show that transformational leadership style, which is relationship-oriented and people-oriented, positively influences employees’ overall evaluation of the organization in terms of emotional appeal, products and services, financial performance, vision and leadership, work environment and social responsibility. Employees who rate their leaders high on the transformational leadership index tend to rate the organization high on reputation. Employees who feel motivated, empowered, encouraged, trusted and cared for by their managers are more likely to trust, like and respect the company. By contrast, transactional leadership demonstrates a significant negative effect on employee perception of organizational reputation. The organization is less favorably evaluated by employees when managers focus only on profit-oriented approaches and exert a greater degree of control.

According to the present study, what determines the employees’ views toward the company is how they feel they are treated, whether they have enough say in decision-making and whether they feel they are doing important work. Transformational leaders can bring this scenario to fruition by empowering the employees. Transactional leaders can be effective and even necessary under certain circumstances, but they are less likely to generate trust and commitment. Over-reliance on rewards and punishment also makes employees feel that they are not fully trusted.

Although scholars argue that transformational and transactional leadership are not mutually exclusive, good leaders should know how to switch between styles based on the situation (e.g., the environment or strategy). But in terms of generating employees’ favorable attitudes, evaluation of the organization and building good internal reputation, transformational leadership should take precedence.

Based on existing literature and findings of the current study, 10 communication characteristics of transformational leadership have large positive impact on shaping employees’ favorable perception of the organization:

1. Leaders showing interest and concern in followers’ needs, individual feelings and well-being; seeing the individual as a whole person rather than as just an employee.
2. Leaders listening to followers through two-way communication.
3. Leaders maintaining good relationships with followers.
4. Leaders being role models.
5. Leaders communicating a compelling and inspiring vision.
6. Leaders creating clearly communicated high performance expectations to show trust in their followers.
7. Leaders stimulating and soliciting new perspectives and ideas, and challenging the status quo.
8. Leaders developing their followers’ skills and capabilities.
9. Leaders delegating power and autonomy to make job-related decisions.
10. Leaders engaging employees in departmental or organizational decision-making.

Clearly, internal reputation building is not a simple matter. It is a complex phenomenon tied to leadership behavior, communication and other contextual factors such as organizational culture and structure.

For organizational management and public relations leaders, the current study suggests that transformational leadership style, which is strategic, inspiring, interactive, empowering, democratic and relational-oriented, should be advocated not only because it affects employees’ motivation, productivity and performance, but also because of its role in building favorable internal reputation.

In the age of social media, employees are increasingly empowered to communicate with others and initiate dialogues in the public domain. How the employees perceive the organization determines what they publicly say, and their opinions consequently become the basis for how other stakeholders and stockholders perceive the organization’s reputation. Therefore, communication professionals should work with top management to build a favorable internal reputation, which in turn generates external intangible assets for the organization.

For a detailed review of the research paper, please go to http://www.instituteforpr.org/research/awards/ketchum/winners/2010-2/.

Linjuan Rita Men is a Ph.D. candidate at the University of Miami and 2010 winner of the Ketchum Excellence in Public Relations Research Award.

The discussion expanded in so many interesting directions that I will not even going to attempt to summarize; you will be better off reading the entire thread. I think it comes to the following question: Which of the following should public relations measure: (a) outputs (b) outtakes (c) outcomes (d) business results (e) all of the above?

Read on and let us know what you think.

Forrest Anderson (Independent Communications Research and Strategy Consultant): I would argue it is preferable to measure outcomes rather than outputs, because business objectives are outcomes. Organizations do not employ internal or external communications to achieve outputs unless they lead to some kind of behavior – a positive vote, increased sales, reduced employee turnover, higher share price. What organizational leaders are interested and what communications disciplines should try to achieve are behavioral objectives or outcomes. So, when you can, that’s what you should measure. Outputs are only a proxy for the real thing – outcomes.

David Michaelson (David Michaelson and Company LLC): I think Forrest has the right answer here. Outputs are merely a mechanism for achieving outcomes. Public relations is not conducted to achieve placements. It is conducted to affect change. Outputs are only a tactic to get to the end result. Outputs are part of the process to reach an objective. They are not the objective itself.

Frank Walton (RF Binder): Yes – to all the comments from my colleagues. I’d provide a slightly different frame. I’m not sure that measuring outcomes is “better” than measuring outputs. But they are two different things – and reside on two different places in the organizational value chain. Measuring outputs is an Operations Metric; by measuring outputs you can demonstrate increased efficiency, show resource allocation, etc. Measuring outcomes, as my colleagues note, can document contribution toward meeting organizational objectives.

PR people are what they count.

PR people (individually and as a profession) need to ask what they’re doing when they go to work. (As Aristotle tells us, without categories, all is lost – so here goes:) Is PR (and all that PR people do) there to reduce friction in the overall system so that the “real value drivers” succeed? Or is PR one of the “real value drivers”? One is not base, and the other noble, but they’re not the same thing.

Fraser Likely (Likely Communication Strategies Ltd): I would argue that Outcomes are not more important. From a Chief Communication Officer (CCO) perspective, they are as important as two other measures.

1. Outputs and outtakes are important. They are important to the PR/Communication Department. They are measures of productivity, be it in-house or consultant. They also lead to measures of efficiency and cost-effectiveness (for example: comparing communication channel to communication channel; or communication activity/product to communication activity/product). Getting to an outcome requires knowledge of where to expend resources. These measures are important for the CCO to manage properly. But, no one else in the organization cares about these measures.

2. Outcomes are important, as Forrest and David have enunciated well. They are important for the PR/Communication Department to be able to show its contribution to the objectives of another department (HR for employee change; Marketing for customer change; Board for corporate reputation changes; etc.; etc.). These are contributions to business line objectives, not necessarily to corporate objectives. Achieving a measurable outcome from a planned communication program in support of another department’s strategy and objectives is important for the CCO to be able to demonstrate value in strategy execution. Only the immediate client department cares about these measures.

3. Outgrowths (or outflows as Ansgar Zerfass would call this third order of PR/Communication Department measurement) are important. They are important for the PR/Communication Department to be able to show its contribution to the strategic management of the organization itself (Board and Executive Committee for intended strategy formulation, strategic plan and emergent strategy formation; risk planning; large scale change management programs; issues/crisis management; etc.). These are contributions to the full strategic management process, including on-going decisions around corporate objective setting. They include for the CCO such leadership actions as direct advice, training, messaging, etc. They include for the PR/Communication Department the provision of service lines and service offerings, the organization of resources (make or buy) and the capabilities and competencies of staff (including consultants). Achieving these benefits (perceived and/or real) against cost (as a cost centre) are important for the CCO to be able to demonstrate value in corporate strategic management. The CEO, Board and Executive Committee all care about these measures.

Therefore, from the perspective of a CCO, these three levels of measurement are ALL important for the successful management of a PR/Communication Department.

It is not either or, it is all.

Mark Weiner (PRIME Research North America): In my experience, it’s almost impossible to separate outputs from outcomes if we try to make an ROI assessment: both are required. But in my opinion, Frank’s is the best response I’ve ever read to this age-old question. “Efficiency” is a key ROI metric and if outputs lead to outcomes, streamlining outputs leads to a better ROI on outcomes. So maybe outcomes are not more important than outputs, per se, but rather outcomes are more important than they are now in the minds of most PR people.

Katie Paine (KD Paine & Partners): Short answer is, that if you walk into an Executive Leadership Team and talk about Outputs, and the Social Media/Marketing/ or guy in the next office comes in and shows business outcomes, he/she will get a raise, and you’ll be lucky to keep your job.

Mark Weiner: But are most social media guys doing any better than anyone else? The ad guy, the marketing guy and most other guys cannot or do not make a case for outcomes on everything they do, and even if they do, it’s usually no better than the case made by PR.

Katie Paine: Actually, they are, but that’s primarily because they come from a different perspective. In Social Media the outcome is a registration, or a follower or something that shows engagement because primarily the desired outcome is an increase in to the marketable universe, and presumably if you take the time to register for something, or to follow someone on Twitter, you’re saying “yes, I’m interested enough to be marketed to” – The social media equivalent of Outputs – i.e. the number of Tweets sent of Facebook status updates are irrelevant unless there is a response.

David Geddes (InfoTrend): This raises the question of whether the social media people are getting an easy pass on what constitutes an outcome. Shouldn’t an outcome be something that drives business results? If I were a CMO, I would not give much credence to a count of registrations, Twitter followers, or anything similar until you can link this to desired business results. Read Avinash Kaushik on this subject.

Sean Williams (CommunicationAMMO): I’m not sure there is a question about that, David – I interpret Katie’s observation is that showing engagement is an outcome … a communication outcome that may or may not lead to business results, but that doesn’t necessarily matter.

Some of the “pop research” (done often by firms with a vested interest in positive results) argues that followers spend more than non-followers, or that companies higher on the social media engagement index have better business results than those lower. As many of you know, I’m quite skeptical about how generalizable those “findings” are…

Not so long ago, the demand from companies (such as Goodyear) was to show financial impact for social media – true ROI – but now it seems OK to go with more intermediate results: engaged followers, perceived value, participation, etc. Does that constitute some sort of “pass?” Maybe, but no easier a pass than mainstream media placements get – companies still think they’re important regardless of measurable impact on the business (I know of a few that I cannot name who’ve bailed on measuring media because the C-Suite doesn’t require it. It accepts on the face that media placements are good for the firm.)

David Geddes: My bias is to hold social measurement up to the standards – sometimes implicit and sometimes explicit – in the writing and blogging of Avinash Kaushik. He makes it very clear that there is a lot of junk measurement out there. Shouldn’t we set a high standard when it comes to measuring what “engagement” is? We should strive to get people to be, to borrow Avinash’s term, measurement ninjas. I forget his derogatory term for those who count things merely because they are able to count them.

Sean Williams: Point taken. The trick is that complying with a really high standard can be expensive (i.e., marketing mix modeling) – also, in social media it seems to be more of a marketing-led measurement: looking for captive audiences to sell to rather than develop strong relationships with…

Katie Paine: I disagree. Is PR responsible for selling cars for GM or getting people to consider knocking on the showroom door? For the social media folks I’m working with, some can track results directly to online sales, but many, who don’t sell on line or require a more consultative sell, consider getting a registration or a follow or even a like shows both awareness and potential consideration. They then turn that “lead” over to the online marketing people who complete the sale.

Brad Rawlins (Brigham Young University): I believe that there are three potential outcomes from public relations efforts: cognitive (awareness, understanding, remembering), affective (attitudes, opinions, feelings) and conative or behavioral. You can’t jump straight to behavioral without measuring the cognitive and affective indicators that would predict some behavior. So, I believe measuring outcomes includes these indicators as well as behavioral.

I’ll give you a couple of examples. A while ago I helped a large regional bank conduct a communications audit. I talked them into including a cultural audit. The results indicated that their channels were pretty effective in getting their message out (outputs), and that a large majority were aware of their key messages (the customer comes first), which is where we would have stopped with the communication audit. But the cultural audit gave us evidence that the employees didn’t believe any of the key messages because the bank had demonstrated its values through rewards that reinforced a strong sales culture, and no-one really believed you would get ahead in the bank by catering to what customers really needed rather than selling them as many bank services as possible.

So, including measurement of affective variables with the cognitive variables informed us about the behavioral variables. They were aware and understood the key messages, but didn’t believe them and weren’t going to behave accordingly.

Another example is a recent survey that I helped conduct with pipeline companies. They are required to survey people who live next to pipelines to make sure they are aware of the risks and how to report pipeline failure. We replicated the section that measured how they get informed and the level of their awareness and understanding, but we added in questions of trust to see whether their efforts to inform had a positive impact on their relationship with these communities (it did). It was an easy way to show how output and cognitive measures could contribute to an important nonfinancial outcome such as trust.

Forrest Anderson: Brad, really good comment!

Mike Daniels: I’d second Forrest’s approval of Brad’s comment – it’s a really very helpful categorization for framing client conversations (and expectations) around optimizing measurement programs to meet specific business/organization goals. With your permission, Brad, I would like to reference this structure in the intro to a workshop on using the AMEC Valid Metrics outputs to outcomes grids that Ruth and I are leading in Lisbon.

I also think Frank’s defense of the importance of outputs in measuring operational efficiency shouldn’t be underestimated, at least from a tactical perspective. It’s often the first (and very visible) win from an outputs measurement program, and can have significant business outcome impact in reducing costs and enhancing productivity.

In the end, though, business requirements self evidently differ from organization to organization, and over time. Just as advertising can be focused at generating non-sales outcomes, depending on where the business stands and the challenges it faces at a given time, so PR and communications will have different priorities for outcomes and output measurement from time to time. Measurement programs need to be flexible enough to accommodate these real world dynamics…

Bert Regeer (Royal Dutch Shell): In line with Fraser, this is what happens with us in practice. Our annual communications program has a series of campaigns, driving strategic messaging (fuelling the conversations around our company positively) and mitigating issues (reducing the critical and negative conversations around our company). If we are successful, we will see that behaviors of key influencers, customers will change (outcomes)…and that is our part of creating business value.

But before we reach that stage we want to see whether our message are actually being picked up (so media analysis looking at outputs and NGOs reducing energy levels around us or the industry as a whole) and whether we see that opinions among what we call special publics are changing – and that is the measure whether our efforts have an impact on outtakes…

In other words, all levels are useful – the ultimate impact is in the outcome, but we’ll need the other stages as well to check whether our efforts are picked up and resonate…

Sean Williams: Just to build a bit on Bert’s and Fraser’s excellent comments: The CEO recognizes that there are many inputs to both revenue and expense, and doesn’t expect PR (for the most part) to isolate its contributions (though he/she does expect us to contribute…). It’s therefore reasonable to create objectives for outputs and outtakes that are within our direct control. Thus, our performance against those objectives becomes very important and worthy of measurement/evaluation. If we have the resources to expend to determine our precise contribution (the business results/outcomes of that activity), good on us!

Julie O’Neil (Texas Christian University): Outcomes are preferable to outputs. But, as many have articulated, outcomes (i.e., increase in sales, donations, votes, attitude changes, etc.) can be long-term in nature. Outputs tend to be short-term snapshots of public relations efficiency and effectiveness in reaching the desired outcomes. For example, I recently worked with a university that wants to increase its academic reputation among key groups. To accomplish this long-term objective, the public relations department first altered its messaging strategy and placement. I helped the university conduct an audit to discern the presence and tonality of these key academic messages. That’s step one: the outputs. This audit helped the department alter its messaging strategy for the second half of the year. The next step will be to link these outputs via correlation with longer-term outgrowths or outcomes. Did people process those academic messages? Do they believe them? More importantly, will the university receive applications from a higher-caliber student and change opinions regarding its academic stature among key constituencies (outcomes) in the upcoming year?

Forrest Anderson: Rereading the exchange reminds me of why I enjoy being a member of the Commission. Where else can you find discussions like this?

I started with the point of view that outcomes were the most important to measure, but Frank’s and Fraser’s comments quickly turned me to their position that other metrics are important as well, because they help us evaluate the success of the process intended to lead to the outcome.

Then Brad came in with his results from adding culture questions to a communications audit and finding misalignment between what the organization was communicating and what it was doing. This brought a whole new dimension to the discussion because not only was it useful in terms of managing the organization, but it was an unexpected discovery. To me this suggests that not only are the measures important but also the context in which we measure. This, perhaps, falls into Fraser’s third category of “Outgrowths.”

Fraser Likely: I believe that there is a consensus from the first round, or close to a consensus, that measuring outputs is indeed important.

Here, I would also include measuring the effectiveness of different media channels in distributing the communication product that would become the communication Output. I’d also include Outtake measures here, which are once the Output has been received, was it paid attention to, was the message in the Output understood and did the receiver act on the receipt of the Output (did they seek out other information; did they share the message; did they become engaged with the message).

These are all communication production and distribution effectiveness measures. They are important because they are milestones that indicate the potential effectiveness of a more lengthy communication campaign. They also are important because they can lead to cost-effectiveness, efficiency and ultimately productivity measures.

Certainly, the CCO needs to know how a campaign is going (are the right mix of channels being employed; are the message source and message appropriate; are intermediaries mediating our message in the right way). The CCO also needs to know where and how to spend operational monies and thus be able to manage resources.

OUTCOMES

I believe we should be talking about Communication Effects here rather than using the term outcomes. Brad perfectly outlined the three categories of communication effects from communication campaigns. Outcomes has come out of the Program Evaluation field. Communication campaigns support revenue generating, expenditure reduction or risk reduction programs. Typically, these programs are run by other functions; Marketing; HR; IR; etc. We support these programs with relationship building and communication campaigns or projects. It is these programs conceived by other functions that produce outcomes: short-term; intermediary; and long-term. It is from these outcomes that a ROI measure can be made. A ROI measure can not be made on communication effects (communication outcomes if you will). This is in line with what David Geddes said in round one: ” ” … shouldn’t an outcome be something that drives business results.”

OUTGROWTHS

I believe that measuring communication OUTPUTS (with channels and Outtakes) and communication EFFECTS/OUTCOMES is not enough. Primarily, Output/channel/Outtake measures have importance only within the PR/C department itself. Communication Effects/Outcomes are important to the PR/C department and to the department’s internal client (Marketing; HR; etc.). The most important measure, though, is the quality of the learning that comes from the communication and relationship building operational and campaign work. That learning includes: intelligence; analysis; scenario building; strategy; advice; and counsel. The greatest measure of the value of the PR/C department and of the Chief Communication Officer is the quality of the intelligence; analysis; scenario building; strategy; advice; and counsel proferred as part of the organization’s strategic management processes: strategy formulation; strategy execution; and emergent strategy formation. This is the measure that has the most value to the management team and Board.

In conclusion, I believe all three levels of measures are important. But, the most important is the ability to measure the third and show results. Of course, there aren’t results at the third level unless there are results at the first two levels.