Last week I made a case for the need for an action-oriented set of priorities for the public relations research, measurement, and evaluation function. Our goal at the managerial level, is to quantify public relations’ contribution to meeting organizational goals and building organizational value. To achieve that goal, we – research measurement, and evaluations professionals – need to attend to issues of a measurement framework, standards, and data quality.
This week I address three the first related topics: a measurement framework, metrics, and standards.
1. A measurement framework.
First, we must adopt a common measurement framework based communications theory and public relations theory. Note that I do not say a single measurement framework. A starting point is a model based on activities, outputs, engagement, outtakes, outcomes shown in the diagram below (for additional discussion see Michaelson and Stacks 2010). These models have their origins in information theory (Shannon and Weaver 1949, Berlo 1960). Any similar model is suitable provided that it cover the panorama from activities to business outputs. Feel free to adapt this model to your needs if it proves useful.
All public measurement programs – including those that are limited to measuring outputs – should be expressed in this broader framework. Let’s expand the perspectives of our clients, internal or external. Many are using this model, or something close, already. Failure to address all phases of the communications process opens the door to output-centric measurement programs, which are long distance from organizational results.
Why is this essential? First, this framework aligns our work as public relations professionals with organizational goals and building value. Second, this framework builds on foundations in communications theory, information theory, cognitive psychology, and the social sciences more broadly. Third, this model aligns with models used in sales, marketing, and advertising. Finally, failure to address all phases of the communications process opens the door to output-centric measurement programs, which are long distance from organizational results.
Associated with the measurement and evaluation framework, we need a set of standard and recommended metrics adapted to stakeholder audiences and public relations goals for to these audiences. But not just any metrics suffice; we need metrics that have what a statistician calls reliability and validity.
Validity is not a term that can be loosely applied to any set of metrics. “Validity refers to whether a measure is actually measuring what you defined it to measure” (Stacks, 2002, 130). Today, we are a low and qualitative stage of validity – content validity – a “measurement validity that is based on other researchers or experts evaluations of the measurement items contained in a measure” (Stacks, 2006). In other words, everyone is taking an educated guess at the appropriate metrics. This applies, incidentally, to the recently-developed AMEC Valid Metrics, reasonable as they seem.
A measure cannot be valid without being reliable. “Reliability is the ability of a measure to measure the same thing comparably over time. A reliable measure is one that is stable – it does not fluctuate without reason” (Stacks, 2002, 131). Our field lacks reliability measures for many types of public relations measurement, particularly media measurement (traditional and social alike).
As part of our research and measurement agenda, we need to advance towards the more rigorous standard of construct validity, “a statistically tested form of measurement validity that seeks to establish the dimensionality of a measure” (Stacks, 2006). Validity needs to be demonstrated; it is not just a label.
The public relations field needs to develop standards for its research, measurement, and evaluation function. We have an important resource in a paper “Standardization in Public Relations Measurement and Evaluation” and a summary slide deck by Institute Research Fellows David Michaelson and Don Stacks thoroughly treating the topic. Why are standards essential? To cite David Michaelson:
With a standardized system of comparative evaluation, public relations professionals will be able to gauge the absolute performance of specific programs and specific program elements. In addition it will allow the comparative performance of prior and competitive programs. And, finally, allow us to compare that performance within industry and category, as well as the performance of the program relative to other industries or categories.
Quite practically, standards will save us time, effort, and allow us to deliver better research, measurement, and evaluation.
What standards do we need? How should these standards be developed? The International Standards Organization delineates the standards development process as follows:
- Consensus. The views of all interests are taken into account: manufacturers, vendors and users, consumer groups, testing laboratories, governments, engineering professions and research organizations.
- Industry wide. Global solutions to satisfy industries and customers worldwide.
- Voluntary. International standardization is market driven and therefore based on voluntary involvement of all interests in the market place.
The Institute for Public Relations Commission on Measurement and Evaluation has joined with the Council of Public Relations Firms and AMEC in an initiative to develop social media measurement standards, and with the Council of Public Relations Firms on standards and best practices in measuring return on investment. We hope this will be a model for further standards initiatives.
Be prepared for RFPs, especially those emerging from purchasing departments, to require adherence to industry standards, best practices, and truly valid metrics. We have the opportunity now to develop our own standards. Failing this, standards will be imposed from the outside.
Participate in the discussion and let me know what you think by commenting below. You can also reach out to me directly; my contact information is on the InfoTrend company web site. And look for more detail on these topics in coming weeks.
David Geddes, Ph.D., is Chief Consulting Officer at InfoTrend, Inc. (http://www.infotrend.com), a business research firm offering predictive analytics for public relations, marketing, and branding.
Berlo, D. K. (1960). The Process of Communication. New York, New York: Holt, Rinehart, & Winston.
Michaelson, David and Stacks, Don. (2010). A Practitioner’s Guide to Public Relations Research, Measurement and Evaluation. New York: Business Expert Press.
Michaelson, David and Stacks, Don. (2011). Standardization in Public Relations Measurement and Evaluation. Presentation at the 9th Institute for Public Relations Summit on Measurement, Philadelphia, September 18, 2011.
Rothwell, J. Dan. (2010). In the Company of Others: an introduction to communication, 3rd Edition. New York: Oxford University Press.
Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication. Urbana, Illinois: University of Illinois Press.
Stacks, Don. (2002). Primer of Public Relations Research. New York: Guilford Press.
Stacks, Don, ed. (2006). Dictionary of Public Relations Measurement and Research. Institute for Public Relations. Accessed October 10, 2011 at http://www.instituteforpr.org/wp-content/uploads/PRMR_Dictionary.pdf.