The Promise and Peril of Generative AI

Source:  Shutterstock

Generative AI has the potential to drive a once-in-a-generation step-change in business performance and productivity, but a recent, first-of-its-kind scientific experiment demonstrates that generative AI can also be a double-edged sword.

When used correctly for appropriate tasks, it can be a powerful enabler of competitive advantage. However, when used in the wrong ways or for the wrong kinds of tasks, generative AI will diminish, rather than boost, performance.

This Thursday, November 30th, will mark the one-year anniversary of OpenAI's public release of ChatGPT, the generative AI application based on the company's GPT large language model. For the past year, generative AI has been the hottest topic in marketing and one of the most widely discussed developments in the business world.

Several surveys conducted this year have consistently shown that most marketers are using - or at least experimenting with - generative AI. For example, in the latest B2B content marketing survey by the Content Marketing Institute and MarketingProfs, 72% of the respondents said they use generative AI tools.

The capabilities of large language models have been evolving at a breakneck pace, and it now seems clear that generative AI will have a profound impact on all aspects of business, including marketing. Some business leaders and financial market participants argue that generative AI is the most significant development for business since the internet.

Given this importance, it's not surprising that generative AI is becoming the focus of scholarly research. One of the most fascinating studies I've seen was conducted by the Boston Consulting Group (BCG) and a group of scholars from the Harvard Business School, the MIT Sloan School of Management, the Wharton School at the University of Pennsylvania, and the University of Warwick.

Study Overview

This study consisted of two related experiments designed to capture the impact of generative AI on the performance of highly skilled professional workers when doing complex knowledge work.

More than 750 BCG strategy consultants took part in the study, with approximately half participating in each experiment. The generative AI tool used in the experiments was based on OpenAI's GPT-4 language model.

In both experiments, participants performed a set of tasks relating to a type of project BCG consultants frequently encounter. In one experiment, the tasks were designed to be within the capabilities of GPT-4. The tasks in the second experiment were designed to be difficult for generative AI to perform correctly without extensive human guidance.

In both experiments, participants were placed into one of three groups. One group performed the assigned tasks without using generative AI, and one used the generative AI tool when performing the tasks. The participants in the third group also used generative AI when performing the tasks, but they were given training on the use of the AI tool.

The "Creative Product Innovation" Experiment

Participants in this experiment were instructed to assume they were working for a footwear company. Their primary task was to generate ideas for a new shoe that would be aimed at an underserved market segment. Participants were also required to develop a list of the steps needed to launch the product, create a marketing slogan for each market segment, and write a marketing press release for the product.

The participants who completed these tasks using generative AI outperformed those who didn't use the AI tool by 40%. The results also showed that participants who accepted and used the output from the generative AI tool outperformed those who modified the generative AI output.

The "Business Problem Solving" Experiment

In this experiment, participants were instructed to assume they were working for the CEO of a fictitious company that has three brands. The CEO wants to better understand the performance of the company's brands and which of the brands offers the greatest growth potential.

The researchers provided participants a spreadsheet containing financial performance data for each of the brands and transcripts of interviews with company insiders.

The primary task of the participants was to identify which brand the company should focus on and invest in to optimize revenue growth. Participants were also required to provide the rationale for their views and support their views with data and/or quotations from the insider interviews.

Importantly, the researchers intentionally designed this experiment to have a "right" answer, and participants' performance was measured by the "correctness" of their recommendations.

Given the design of this experiment, it should not be surprising that the participants who used generative AI to perform the assigned tasks underperformed those who did not by 23%. The results also showed that those participants who performed poorly when using generative AI tended to (in the words of the researchers) "blindly adopt its output and interrogate it less."

The results of this experiment also raise questions about whether training can alleviate this type of underperformance. As I noted earlier, some of the participants in this experiment were given training on how to best use generative AI for the tasks they were about to perform.

These participants were also told about the pitfalls of using generative AI for problem-solving tasks, and they were cautioned against relying on generative AI for such tasks. Yet, participants who received this training performed worse than those who did not receive the training.

The Takeaway

The most important takeaway from this study is that generative AI (as it existed in the first half of 2023) can be a double-edged sword. One key to reaping the benefits of generative AI, while also avoiding its potential downsides, is knowing when to use it.

Unfortunately, it's not always easy to determine what kinds of tasks are a fit for generative AI . . . and what kinds aren't. In the words of the researchers:

"The advantages of AI, while substantial, are similarly unclear to users. It performs well at some jobs and fails in other circumstances in ways that are difficult to predict in advance . . . This creates a 'jagged Frontier' where tasks that appear to be of similar difficulty may either be performed better or worse by humans using AI."

Under these circumstances, business and marketing leaders should exercise a significant amount of caution when using generative AI, especially for tasks that will have a major impact on their organization.

(Note:  This post has provided a brief and necessarily incomplete description of the study and its findings. Boston Consulting Group has published an article describing the study in greater detail. In addition, the study leaders have written an unpublished academic "working paper" that provides an even more detailed and technical discussion of the study. I encourage you to read both of these resources.)

[Book Review] A Thought-Provoking Guide to Creating a Successful Content Marketing Strategy

 "Your content will never provide competitive advantage. But your content strategy just might."

Robert Rose, Content Marketing Strategy

Source:  Kogan Page

Content marketing has become a core ingredient of the B2B marketing mix over the past two decades. Today, virtually all B2B companies use content marketing in some form.

But despite this widespread use, many B2B marketers aren't satisfied with the performance of their content marketing programs. In the 2022 edition of the content marketing survey by the Content Marketing Institute and MarketingProfs, only 29% of the B2B respondents said their organization was extremely or very successful with content marketing. 

One key to effective content marketing is having a well-designed content marketing strategy. In the 2022 CMI/MarketingProfs survey, 64% of the "most successful" respondents reported having a documented content marketing strategy. Only 19% of the "least successful" respondents said they had a documented strategy.*

Because having a sound strategy is so critical to content marketing success, Robert Rose's new book is a must-read for marketers. Content Marketing Strategy:  Harness the Power of Your Brand's Voice (Kogan Page, 2023) provides an authoritative guide to the formulation of an effective content marketing strategy.

Robert Rose is the Chief Strategy Advisor of the Content Marketing Institute and the CEO and Chief Strategy Officer of The Content Advisory, a content marketing consulting firm. He has been working in marketing and content strategy for nearly 30 years.

What's In the Book

As its title suggests, Content Marketing Strategy is about the strategic management of content marketing. More specifically, the book describes a process that marketing leaders can use to manage content marketing as a core business function that requires a full-fledged business strategy.

Robert Rose clearly spells out his perspective on the topic of content marketing strategy in the quotation shown above and in the Introduction of the book where he writes that marketing needs to operate as a media company and that the primary function of marketing is to ". . . create valuable experiential media-driven products for audiences that can be monetized in several ways . . ." (Emphasis in original)

In Chapter 1 of the book, Rose echoes Michael Porter's thinking on strategy when he defines content marketing strategy as ". . . a marketing discipline that is the sum of all the activities required to enable a business to consistently communicate in a way that creates tangible value for target audiences. It is what enables a brand to have not only a voice but also something to say that is worth listening to."

Rose uses Chapter 2 to introduce the fundamental elements of a successful content marketing strategy. He argues that an effective strategy has three pillars - Communication, Experiences, and Operations. Then, he goes on to write that these three pillars encompass five core categories of activities, which he labels purpose, model, frame, value, and audience.

In Chapters 3-8 of the book, Rose discusses each of these five categories of activities in detail. For example, he covers purpose in Chapter 3 ("Designing a Strategic Purpose"), model in Chapter 4 ("The Content Marketing Operating Model"), and audience in Chapter 5 ("Understanding Audiences").

In Chapter 9, Rose describes how marketing leaders and their teams can formulate a content marketing strategy that incorporates the principles and frameworks he discussed earlier in the book. Rose called this process "story mapping," and it includes four steps.

1. Identify all the conditions that must exist for your content marketing effort to be successful.
2. Identify all the obstacles that could prevent the Step 1 conditions from existing.
3. Determine which of the Step 1 conditions are absolutely essential and which of the Step 2 obstacles are "showstoppers."
4. Specify when the Step 1 conditions will be accomplished.

My Take

The widespread popularity of content marketing has spawned an impressive volume of literature. Yesterday, I conducted a search at Amazon using the term "content marketing strategy." I found 51 books having that term in the title or subtitle, and I stopped counting after only four pages of search results.

Many of these books probably contain useful information and insights. However, if you want to better understand how to manage content marketing as a strategic business function - and why that approach is critical for successful content marketing - you should put Content Marketing Strategy near the top of your reading list.

Robert Rose can speak authoritatively about content marketing. In his position with the Content Marketing Institute and as a consultant, Rose has had a front-row seat to observe the evolution of content marketing for more than a decade. He's seen what does and doesn't work, and he describes several of these experiences in his book.

Content Marketing Strategy is well-organized and clearly written. The book isn't exactly "light reading," but Rose's writing style makes the content approachable and easy to understand. He also includes several illustrations in the book, and these illustrations make it easier for readers to comprehend some of Rose's most important concepts.

B2B marketers can easily find books, white papers, ebooks, and other resources about content marketing. Robert Rose's book is one of the most thoughtful and thought-provoking books I've encountered on this vitally important topic. I strongly recommend it.

* The "most successful" respondents were those who said their organization was extremely or very successful with content marketing. The "least successful" respondents were those who characterized their organization's content marketing efforts as minimally or not at all successful.

[Research Round-Up] B2B Highlights From the Fall 2023 Edition of "The CMO Survey"

Source:  "The CMO Survey" (Christine Moorman, 2023)

(This month's Research Round-Up is devoted entirely to the Fall 2023 edition of "The CMO Survey." This research has been conducted semi-annually since 2008 and consistently provides a wealth of valuable information about marketing trends, spending, and practices.)

The findings of the Fall 2023 edition of "The CMO Survey" were released in late September. "The CMO Survey" is directed by Dr. Christine Moorman and is sponsored by Deloitte LLP, Duke University's Fuqua School of Business, and the American Marketing Association.

The Fall 2023 survey results are based on responses from 316 senior marketing leaders at for-profit companies based in the United States. Over two-thirds of the respondents (67.7%) were affiliated with B2B companies, and 95.6% were VP-level or above. The survey was in the field July 26 - August 17, 2023.

Dr. Moorman and her colleagues typically produce three reports for each U.S. edition of the survey.

• "Highlights and Insights Report" - This is a relatively brief and graphically rich report that provides mostly overall survey results and analyzes those results and major marketing trends.

• "Topline Report" - This report provides response data at the aggregate level for all survey questions.

• "Firm and Industry Breakout Report" - This report provides response data by four economic sectors (B2B product companies, B2B services companies, B2C product companies, and B2C services companies), 15 industry verticals, company size, and percentage of online sales. This report is lengthy but it provides the most detailed view of the survey data.

In this post, I'll be discussing the responses of B2B marketers exclusively, unless otherwise indicated. The percentages and other numerical values in this post are the mean of applicable survey responses, also unless otherwise indicated.

Marketers Are More Optimistic About the Economy

For the past several years, "The CMO Survey" has asked participants for their views on economic conditions, and the Fall 2023 edition of the survey was no exception. It asked participants to rate their level of optimism regarding the overall U.S. economy on a 100-point scale, with "0" being the least optimistic, and "100" being the most optimistic. 

The following chart shows how B2B marketers rated their optimism in the six surveys conducted over the past three years.

As this chart shows, B2B marketer optimism is at its highest level since the February 2022 survey, and B2B marketers are significantly more optimistic now than they were in the two preceding surveys (Spring 2023 and September 2022).

The Fall 2023 survey also asked participants if they were more or less optimistic about the U.S. economy compared to the previous quarter. The following table shows how B2B marketers responded.

The results in this table also indicate that B2B marketers have become more optimistic since last spring. In the Fall 2023 survey, 46.4% of respondents with B2B product companies and 47.1% of respondents with B2B services companies said they were more optimistic compared to the previous quarter. In the Spring 2023 survey, only 32.0% of the respondents with B2B product companies, and 24.7% with B2B services companies reported being more optimistic. 

Anemic Growth in Marketing Spending

The increased optimism regarding economic conditions has not produced a substantial increase in marketing spending . . . at least not yet. In fact, the survey shows that the growth of marketing spending has been anemic for more than a year.

For the past several years, the survey has asked participants by what percent their marketing spending changed in the preceding 12 months. The following chart shows how B2B survey respondents answered these questions in the six surveys conducted over the past three years.

B2B marketers are relatively optimistic about the growth of marketing spending over the coming year. In the Fall 2023 survey, respondents with B2B product companies said they expect overall marketing spending to increase 6.8% over the 12 months following the survey, and those with B2B services companies expect a 6.6% increase over the same period.

Unfortunately, spending growth predictions by respondents to "The CMO Survey" have not been particularly accurate in the past. For example, in the September 2022 survey, respondents (B2B and B2C) predicted that marketing spending would increase by 8.8% over the following 12 months. In the Fall 2023 survey, respondents reported that their marketing spending had increased by only 2.6% over the preceding 12 months.

*****

The Fall 2023 edition of "The CMO Survey" includes data regarding several other topics, and, like earlier editions, it provides a wealth of valuable insights for B2B marketers. I encourage you to read the full survey report.

Why B2B Marketers Need to be Careful With Purpose Marketing

Many marketing pundits argue that companies should make environmental and social "purpose" an integral part of marketing communications. Read on to learn why you should approach purpose marketing cautiously.

Purpose marketing can be defined as the use of messaging in external communications that expresses a company or brand's core mission and values. It includes messaging that highlights how a company or brand is positively impacting the lives of employees and customers and/or society as a whole. The term also refers to communications that spell out where the company or brand stands on important social issues.

Numerous research studies have purported to prove that consumers and business buyers now place great importance on the social responsibility track record of the companies or brands they do business with. Many of these studies also indicate that potential buyers now expect companies and brands to "take a stand" on important social issues.

As a result, many marketers have become enamored with purpose marketing. Industry media outlets such as Advertising Age, Adweek, Marketing Week, and The Drum have frequently published articles describing the purpose marketing campaigns and other social responsibility efforts of well-known brands such as Dove, Nike, Gillette, and Patagonia.

In reality, however, the case for purpose marketing isn't nearly as clear-cut as the hype would suggest. Several recent research studies have painted a more nuanced picture of purpose marketing. These studies suggest that marketers should approach purpose marketing cautiously and thoroughly evaluate potential benefits and risks before they launch a significant purpose marketing program.

Here are a few of the highlights from two surveys conducted earlier this year.

The Bentley University-Gallup Survey

The 2023 Bentley-Gallup Business in Society study was based on a survey of 5,458 U.S. adults (ages 18 and older). The sample for the survey was weighted to be demographically representative of the U.S. adult population. The maximum margin of sampling error for results based on the entire survey sample was + or - 1.8 percentage points at the 95% confidence level for response percentages around 50%. The survey was conducted May 8-15, 2023.

Less than half of the survey respondents (41%) said that businesses should take a public stance on current events. This was a decline of seven percentage points from the 2022 edition of the survey.

The main driver of this decline was waning support from respondents who identified as Democrats. In the 2022 survey, 75% of Democratic respondents said businesses should take a public stance on social issues. That percentage dropped to 62% in the 2023 survey.

The researchers asked survey participants about 11 categories of issues (e.g. racial issues, gun issues, LGBTQ+ issues, etc.). Of these 11 issue categories, climate change (at 55%) and mental health (at 52%) were the only two that a majority of survey respondents said businesses should take public positions on.

Perhaps not surprisingly, younger respondents, Asian, black, and Hispanic respondents, and respondents identifying as Democrats were more likely to say that businesses should take a public stance on social issues.

The Ipsos Survey

The Ipsos study was a survey of 1,096 U.S. adults (ages 21 and older). The survey was conducted February 8-9, 2023, and the findings were described in a paper titled, What the Future:  Purpose.

The findings of this research revealed that consumer attitudes regarding brand purpose are more nuanced than generally believed. For example:

• 85% of the survey respondents said global or national brands should play a role in solving global problems, but 51% said companies should remain neutral on social issues.

• Two-thirds of the respondents agreed that purchasing sustainable products made a difference for the environment, but only 52% said they were willing to pay more for products that are manufactured sustainably.

One of the most striking findings in the study related to the importance of brand purpose in purchase decisions. Ipsos asked survey participants which of 12 factors were most important when they were deciding which brands or products to purchase. The following table shows that the factors relating to brand purpose (shown in red) were near the bottom of the list in terms of importance.

Does Purpose Marketing Work?

Other recent research has suggested that many purpose marketing programs have failed to make a meaningful impression on potential buyers.

In a 2022 online survey of more than 2,000 U.S. adults by GfK, over half of the respondents could not name (unaided) a single brand that is "taking care of the environment and fighting climate change," "promoting diversity and inclusion," or "giving back to the community."

The authors of the survey report drew this conclusion:  "Despite all the billions of dollars spent on purpose-driven campaigns, brands have not achieved top-of-mind awareness for this crucial work. In some cases, purpose marketing has become a kind of 'green noise' - a constant hum about virtuous brand behavior in which few messages or actions stand out . . ."

The Bottom Line

These recent studies provide a more balanced view of buyer attitudes regarding the appropriateness and value of purpose marketing. The findings show that while values matter to potential buyers, there is little consensus regarding whether companies and brands should take public stances on social issues.

That's why marketers should approach purpose marketing with a great deal of caution.

If you'd like to learn more about the benefits and risks of purpose marketing, take a look at this "deep dive" article that I published in January of last year.

Top image courtesy of Paul Mison via Flickr (CC).