Sunday, July 25, 2021

How To Evaluate Survey Findings and Reports With a Discerning Eye


Research surveys and survey reports have become important marketing tools for many kinds of B2B companies, including those that offer marketing technologies and various kinds of marketing-related services. Many B2B companies are conducting or sponsoring surveys, and they feature survey reports in their marketing programs. As a result, many B2B marketers are now both producers and consumers of research-based content.

Survey reports can be valuable sources of information about business trends and practices, emerging technologies, customer attitudes and a wide range of other subjects. But survey results and reports can also be unreliable and/or misleading. 

The availability of free or inexpensive and user-friendly survey tools has made it easier for marketers to create and conduct surveys. Unfortunately, these same tools also make it easy to design and conduct surveys that don't produce reliable results.

As producers, marketers obviously want potential customers to view their survey results and reports as credible and reliable. And as consumers, marketers are increasingly using the results of surveys when making important decisions. So it's important for them to carefully evaluate the survey reports they encounter. As consumers, it's always a good idea to approach any survey report with a critical eye because as Mark Twain wrote, "There are three kinds of lies:  lies, damned lies, and statistics."

In my work, I review lots of survey reports. I make extensive use of survey results and other research studies when I'm developing content for clients, and I frequently discuss survey findings in this blog. Over the years, I've developed a mental checklist of things I look for when reviewing a survey report.

I'm planning to devote three posts to this topic. In this post, I'll discuss some of the basic things I look for when I'm reviewing a survey report. My next two posts will discuss issues that can affect the validity of survey findings and/or the credibility of survey reports.

My Starting Mindset

Whenever I begin reviewing a survey report produced or sponsored by a business enterprise, I assume the survey was conducted to support a marketing agenda. Having a marketing purpose doesn't necessarily mean the research is flawed, but it does put me on alert for indications of bias in the design of the survey and/or in the presentation of the findings.

Survey Methodology

Many survey reports will briefly describe the research in the introductory section of the report, but a thorough report will also include a detailed description of the methodology used in the research.

For a survey of business professionals relating to a business subject, I look for the methodology description to include at least the following:

  • Sample size (the number of responses the survey received)
  • When the survey responses were collected
  • How the survey responses were collected (e.g. online, telephone)
  • How potential survey participants were selected
  • If appropriate, how survey respondents were qualified
Respondent Demographics
Most of the survey reports I encounter describe the demographic attributes of survey respondents at least to some extent. When I'm reviewing a survey of business professionals regarding a business topic, I usually want to see a breakdown of the following demographic characteristics.
  • Job role/job function 
  • Industry verticals/types of companies represented
  • Company sizes represented
  • Geographic locations of respondents
Respondent demographics can be important for interpreting survey findings and assessing the relevance of those findings. For example, I recently published a post describing the findings of a survey by Gartner regarding marketing data and analytics. In that survey, 83% of the respondents were with companies having $1 billion or more in annual revenue. So the findings from this survey may be highly relevant for marketers in large enterprises, but somewhat less instructive for marketers in small and mid-size companies.

Use of a "Representative Sample"
Surveys are frequently used to capture insights about a defined population of individuals by collecting data from a small sample of that population. This approach only works, however, if the survey respondents constitute a representative sample of the larger population.
Survey sampling is a complex topic, and it's impossible to describe it fully in a blog post. The most important thing to remember is this:  If the survey respondents aren't a representative sample, the findings of the survey cannot be "projected" to the larger target population. In essence, the findings are only valid for the group of people who responded to the survey. 
Therefore, when evaluating survey findings, it's always important to determine whether the survey respondents constitute a representative sample. When a survey uses a representative sample, the survey report should include a detailed explanation of the sampling process in the survey methodology description.
Very few of the surveys I review are based on representative samples. Such surveys can still be useful, but they can also be misleading because some report authors ignore this limitation. A well-prepared survey report will make this limitation clear, as Gartner did in its Marketing Data and Analytics Survey 2020 by including the following language:
"Disclaimer:  Results from this study do not represent global findings or the market as a whole but reflect sentiment of the respondents and companies surveyed."

Coming Up
In my next two posts, I'll be discussing other issues that can affect the validity of survey findings and the credibility of survey reports.

Image courtesy of Marco Verch via Flickr (CC).

Sunday, July 18, 2021

How Senior Business Leaders View the Role and Performance of Marketing

Source:  CMO Council

Last month, the CMO Council published a report that provides several important insights regarding how senior corporate executives view the role and performance of marketing in their organization. Rate the State of Marketing:  A C-Suite Scorecard was based on a survey of 120 senior management executives in a variety of leadership roles.

Thirty-eight percent (38%) of the survey respondents were with companies having more than $1 billion in annual revenue, and 18% were with companies having annual revenue of $501 million to $1 billion. Survey participants worked in more then 17 industry sectors, and 84% were with companies headquartered in North America.

The CMO Council study addressed two broad issues:

  1. What do senior leaders see as the primary roles and functions of marketing in their company?
  2. How do senior executives rate the performance of their marketing organization?

How Senior Executives View the Role of Marketing

The CMO Council survey included several questions designed to capture the views of senior business leaders about the primary purpose of marketing in their organization. The question that produced the most revealing insight related to performance metrics.

The survey asked senior executives to identify the top five metrics or KPI's they would use to measure the value, contribution and performance of marketing. An overwhelming 80% of the respondents selected revenue and sales growth as one of their top five KPI's. The second and third most frequently identified metrics were customer acquisition and profitability and customer satisfaction and retention, both of which measure essential drivers of revenue growth.

Other survey responses also demonstrate the importance senior company leaders place on revenue growth. For example, the survey asked participants to identify the top five areas of marketing operations in their company that need improvement. The following table shows the five areas most frequently selected by survey respondents.








As the table shows, demand generation and pipeline was the number one area of marketing that senior executives believe needs improvement.

The CMO Council also asked survey participants to identify the five most essential roles of the chief marketing officer in their executive team. More than half of the survey respondents placed the following roles in their top five:

  • Customer experience advocate and champion (62% of respondents)
  • Digital transformation/marketing automation leader (54%)
  • Brand reputation custodian and value creator (51%)
  • Maestro of communications and demand generation (51%)
In contrast, fewer than half of the survey respondents included the following CMO roles in their top five:
  • Primary revenue builder and growth strategist (40% of respondents)
  • Architect of innovation and business expansion (34%)
  • Go-to-market authority and pricing expert (12%)
It shouldn't be surprising that most senior business leaders believe their CMO must play a leading role in customer experience and the digital transformation of marketing, given the undeniable importance of both of these issues. However, the responses to this question suggest that many senior executives still believe the most vital roles of the CMO are centered around marketing communications. This perception may limit the CMO's ability to maximize revenue growth, which requires strategies that extend beyond marketing communications.

How Senior Executives Rate Marketing's Performance
The CMO Council survey also included several questions that were designed to capture how senior business leaders feel about the performance of marketing in their company. In the introduction to the survey report, the authors write, ". . . CMOs will be more than happy with grades received in the new C-Suite Scorecard of marketing value and effectiveness benchmarked in this report . . ."
My take on these survey results is not quite that enthusiastic. If I were asked to assign a letter grade to marketing based on the totality of the survey findings, I would give marketing a "B," meaning good but with clear room for improvement. Here are two survey findings that illustrate my reasoning.
Marketing's Performance in 2020 - On a 10-point scale, 6% of the surveyed executives rated the performance of their marketing teams in 2020 as exceptional (9 or 10), and another 40% rated marketing's performance as very good (7 or 8). However, 45% of the survey respondents graded marketing's performance last year as only moderate (5 or 6).
Marketing's Ability to Lead Growth in 2021 - Seventeen percent (17%) of the surveyed business leaders said they were extremely confident that their marketing function can lead a growth recovery in 2021. However, 52% of the survey respondents said they were only moderately confident in marketing's ability to lead growth this year, and another 29% reported being only somewhat confident.
The bottom line here is that marketing leaders still have work to do to earn the highest levels of confidence from other senior business leaders.

Sunday, July 11, 2021

How Effectively Are Companies Managing Content Operations?


Within the last few weeks, two research reports have been published that provide several important insights about how marketers are managing content-related activities and processes. One report is by the Content Marketing Institute ("CMI"), and the second is by Altimeter.

Content marketing has become nearly ubiquitous among both B2B and B2C companies, and the volume of content required to "feed the beast" has been growing exponentially. The need to produce and deliver content that is personalized and contextually relevant in near real time has also increased dramatically. As a result, we are seeing more research that addresses various aspects of content operations.

In this post, I'll review some of the major findings from the research conducted by CMI and Altimeter.

The CMI Research

Source:  Content Marketing Institute

CMI recently published the results of the 2021 Content Management & Strategy Survey (the "CMI Survey"). The CMI Survey was fielded in April of this year and produced 263 responses.

Eighty-three percent of the survey respondents were with B2B or B2B/B2C companies. Forty percent were with companies having 1,000 or more employees, and another 43% were with companies having 100-999 employees. Seventy-nine percent of the respondents were located in North America.

Seventy-eight percent of the respondents in the CMI Survey said their organization takes a strategic approach to managing content, and 81% strongly or somewhat agreed that their company views content as a core business strategy.

The strategic approach is evident in the factors the survey respondents take into account during their content planning process.

  • 61% said they always or frequently use structured, repeatable content production processes
  • 49% said they always or frequently consider the need to deliver consistent experiences throughout the customer journey
  • 26% said they always or frequently consider the need to deliver personalized experiences
CMI asked survey participants about the content development aids they are currently using, and the following table shows the nine aids most frequently identified by survey respondents.













The CMI survey also revealed that marketers are using a variety of technologies to support their content management work. More than half of the survey respondents reported using the following technology tools.
  • Email marketing software (85%)
  • Social media publishing/analytics (84%)
  • Customer relationship management (73%)
  • Content management system (72%)
  • Content distribution platform (51%)
The picture is more mixed when it comes to how effectively marketers are using their technology tools. For example, 31% of the survey respondents described their company's proficiency with using content-related technologies as expert or advanced, but 42% said they aren't using their technology tools to their full potential.
CMI also asked survey participants about the content management challenges they are facing. Half or more of the respondents cited the following three challenges.
  • Communication among teams so everyone is on the same page (58%)
  • Enough staff skilled in content strategy (53%)
  • Using user experience (UX) design to improve the overall experience a customer has with our organization (50%)
Only 15% of the survey respondents strongly agreed that :  "Our organization extracts meaningful insights from data and analytics derived from the consumption of content." And only 41% somewhat agreed with that statement. So, it's surprising that the ability to use data effectively didn't show up as a major content management challenge. It's possible, of course, that CMI didn't include a data-related challenge in the list provided to survey participants.

The Altimeter Research

Source:  Altimeter
 Altimeter (a Prophet company) recently published the results of The 2021 State of Digital Content study. This research was based on a survey of 375 senior content team executives and practitioners in companies having at least 1,000 employees. Respondents were located in the United States, the UK, Spain, Germany and China. Survey respondents were with companies operating in five industry verticals - banking/finance, healthcare, manufacturing, retail and technology.
The Altimeter study focused on a wide range of content-related topics. In this post, I'll focus on the survey findings that are most related to content management.
Ownership of Content Strategy - Altimeter's research found that in a plurality of the companies represented in the survey (31%), multiple business functions in the company "own" content strategy. With this approach, a company actually has multiple content strategies, which can result in a disjointed customer experience. Fifteen percent of the companies represented in the survey used a hybrid model where senior leaders from relevant functions jointly develop a holistic content strategy. The content team then owns the production of content based on the holistic strategy.
Content Production Model - Fifty percent of the respondents in the Altimeter survey said they use a centralized, dedicated team to create content for any business function in the company that needs it. This model can be described as an "internal agency" or "center of excellence" approach. But 32% of the survey respondents said they are setting up multiple content creation centers within their company to meet the ever-increasing demand for content. These centers are typically focused on different products, regions or functions.
Major Content Challenges - Altimeter asked each survey participant about the biggest challenge they face in creating and delivering content. The following table shows how the survey respondents answered this question.











The Takeaway
The findings of these two research studies provide ample evidence that managing content operations is still a work-in-progress at most business organizations.

Top image courtesy of EpicTopTen.com via Flickr (CC).

Sunday, July 4, 2021

How to Retool Personalization for a Privacy-Conscious World

(The following post is an edited mash-up of two of my most popular "non-COVID" posts from last year. Delivering personalized content and experiences remains a top priority for many B2B marketers, and recent research has found that effective personalization is still a challenge for many marketers. In a survey of digital marketing leaders conducted by Gartner in November and December of last year, 63% of the respondents said they still struggle with delivering personalized customer experiences. Therefore, this topic is as timely today as when my original posts were published last year.)

The value of personalization in marketing has been largely unquestioned for nearly two decades. Today, most marketers view personalization as essential for success, and many companies have made personalization a top priority. But the marketing environment is changing, and that means it's time for marketers to rethink their personalization strategy.
In a report published in November of 2019, Gartner predicted that by 2021, one-third of marketers will reduce spending on personalization, and by 2025, 80% of marketers who have invested in personalization will abandon their efforts due to lack of ROI, the perils of data management, or both. These predictions were both surprising and controversial because they ran counter to most of the  conventional wisdom about personalization.
While I doubt that eight out of ten marketers will completely abandon personalization over the next five years, it's clear that marketers are facing a personalization conundrum. On one hand, numerous studies conducted over the past several years have reported that consumers and business buyers want - and are willing to provide personal information in order to receive - personalized offers, messages, and experiences.
But a growing number of studies also show that consumers and business buyers don't always welcome personalized marketing and will react strongly when they perceive that personalization goes too far. In one recent study, for example, 38% of survey respondents said they would stop doing business with a company that sent them "creepy" personalized messages. Source:  "Rethink Personalization for Maximum Impact" (Gartner, Inc., 2019).
Most marketing pundits and many marketing leaders argue that the key to increasing the effectiveness of personalized marketing is more personalization. They contend that marketers should collect and use more data about customers and prospects, make personalization more specific, and use it more frequently, in more channels, and for more types of interactions. The popularity of this view explains why hyper-personalization and personalization at scale have recently attained buzzword status.
The "more personalization" argument is based on the idea that increased personalization will produce more relevant messages and experiences, and that the increased relevance will make those messages and experiences more compelling. The fundamental flaw of this approach is that it fails to account for the significant shift in public attitudes toward personalization that's occurred over the past few years.

The Shadow of Cambridge Analytica
Since the Facebook-Cambridge Analytica scandal became public knowledge a few years ago, we've been bombarded with media coverage about how companies collect and use our personal information. Facebook's data privacy policies and practices have been widely criticized, but other large tech firms such as Alphabet/Google have also been the subject of multiple media stories and Congressional hearings.
All of this has made the public more acutely aware of how much personal data companies are collecting and how they are using that data to target and personalize advertisements and other marketing communications.
Note:  The data practices of large technology companies have also been addressed by several respected scholars. If you'd like to see an example of these discussions, get a copy of The Age of Surveillance Capitalism by Shoshana Zuboff. Dr. Zuboff is the Charles Edward Wilson Professor emerita at the Harvard Business School. Her book is thorough and sobering - if somewhat strident - but at over 700 pages, it's not a quick or easy read.
The heightened public awareness is impacting personalization in two ways. First, as members of the public have become more knowledgeable about how companies are using personalization in marketing, they have become desensitized to it. They no longer see personalized messages or content as extraordinary. So, many of the more widely-used personalization tactics and methods make less of an impact today than they did in the past. As the old saying goes, "Familiarity breeds contempt."
More importantly, as people have learned more about how companies are collecting and using personal information, they've also become more disturbed about those practices. Today, when someone receives a personalized message, he or she is likely to think first about what enabled the personalization. What does this company know about me? How did the company obtain that information?
The conundrum facing marketers is clear. Most consumers and business buyers say they want and value personalized offers, messages, and experiences. At the same time, however, both consumers and business buyers are becoming more concerned about privacy, and they are increasingly distrustful about how companies are obtaining and using their personal information.
Under these circumstances, the "more personalization" strategy may do more harm than good. So, what's the alternative? 
Personalization has been the subject of numerous research studies over the past few years, and these studies provide a good picture of what's required for personalized marketing to produce maximum results. There are three major components of an effective personalization strategy.

Make Personalization Useful
The first requirement for effective personalized marketing is that it must deliver meaningful and pragmatic value to the recipient. A 2018 study by Gartner/CEB documents the business value of personalization that is perceived by customers and prospects to be helpful. I've previously discussed this research, so I won't repeat that material here. For a more detailed description of the Gartner/CEB study see this post.

Make Personalization "Relationship-Appropriate"
The second component of an effective personalization strategy is to use a level of personalization that is appropriate for each customer or prospect. By appropriate, I mean that the level of personalization should match the real-world status of the relationship. A message or offer sent to a long-time customer can and should be more personalized than a first outreach to a new prospect.
To be effective, personalized marketing must be based on genuine insights about your customers and prospects. When you take personalization beyond such insights, it becomes inauthentic and will tend to be perceived as presumptuous. Corporate Visions recently conducted a field trial involving this principle, and you can read more about that research in this post.

Get Meaningful Permission for Personalization
Much of the concern about data privacy and personalization revolves around the issues of transparency and control. Many consumers and business buyers aren't confident they know what personal data companies are collecting about them or how that data is used. And many feel they don't have any meaningful control over those data practices.
Several recent research studies have shown how important transparency and control are for customers and prospects. For example, in a 2019 survey of 3,000 people in the U.S., Canada, and the U.K., The Harris Poll asked participants about the importance of several data privacy practices. The following table shows the percentage of survey respondents who rated four transparency and control practices as very important or absolutely essential:

















These research findings point the way to the third important component of an effective personalization strategy. In a world where privacy concerns are heightened, meaningful permission is critical to successful personalized marketing. If the personalization research tells us anything, it tells us that most consumers and business buyers will welcome and value personalized content when it is helpful, authentic, and based on permission that is willingly and consciously given.

So, how can marketers gain this kind of permission? There are three key steps.

Use Personalization "Programs" In most cases, personalization efforts should be organized into discrete programs, each of which is designed to provide a specific type of value to a specific type of customer or prospect. This approach leads marketers to focus on the purpose of personalized marketing from the recipient's perspective.

Invite ParticipationInvite your customers and/or prospects to "subscribe" to personalized content on a program-by-program basis, and reassure them that subscribing to one program won't open the floodgates to other marketing communications.

Be TransparentIt's important to be "radically" transparent in your invitation about the details of the personalization program. The main objective of the invitation is to persuade customers or prospects to participate in the program. So it should include:

  • Why the program will be useful and valuable for the recipient
  • What personal information will be used, and how the information will be used
  • How the personalized content will be delivered (format)
  • How frequently the personalized content will be delivered
  • The duration of the program
  • A clear statement that the recipient has the option to "unsubscribe" at any time
It's About How - Not Whether - to Personalize 
The issue for marketers is not whether to personalize marketing content and customer experiences. The evidence is clear that customers and prospects want and appreciate the increased relevance that personalization can provide. The real issue is how to deliver personalization in a way that respects privacy. By making personalization helpful, authentic, and based on meaningful permission, marketers will reap the maximum benefits of personalized marketing.
Top image courtesy of Phil Wolff via Flickr (CC).