I’ve been watching AI-powered writing platforms for years and never felt they could replace a human. But when I heard all the hype, and saw some examples, of what the ChatGPT AI writing model could do, I got more worried.
ChatGPT (Generative Pre-trained Transformer creates clear, well-written prose that includes human-sounding insights and real-world examples. For example, asking it why Steve Martin and Martin Short are so funny, its answer included
“Steve Martin and Martin Short have a comedic chemistry that stems from their mutual respect and admiration for each other's talents. They have been friends for many years and have worked together on various projects, including the films "Three Amigos" and "Father of the Bride."
That’s a good start on a popular culture blog. But can it do technically oriented marketing collateral and thought leadership? I decided to compare the draft I wrote of a brief security-focused piece for a client with what the robot could do.
I first asked ChatGPT to answer the basic questions the client had asked me to cover, then asked two more times, adding more specifics (such as addressing specific technical topics and how the client’s solution would address the security challenges.)
Advantage: ChatGPT
In response to my first request, in under 30 seconds ChatGPT had scanned the Web and done a thorough job identifying and describing addressing the same basic themes I had written about in my draft. The text was clear and direct, though not imaginative.
In what might have taken me an hour or more ChatGPT did a very good first pass at compiling and presenting the results of a Web search of the conventional wisdom about this topic. This is a clear time saver and a good start to writing a marketing piece – but only a start.
Advantage: Human
First, the Chat GPT response was short – about 200 words, compared to the 900 or so I had written, which included more context and “how to” advice.
Second, (as OpenAI, the creators of the model advise up front) the model has not been trained on the most recent content from the Web. I asked it, for example, about the security implications of Apple’s M! chip. It found and spit back information about the chip’s security advantages, but didn’t find a more recent story about a hardware (and thus unpatchable) security flaw.
Most importantly, the ChatGPT content was not customized for the needs of my client, which has an offering in this space they need to discreetly promote. When I asked ChatGPT to mention my client, it hit the reader over the head with overtly promotional language from their Web site. Not the subtle, “help the reader first” approach most of my clients prefer.
Don’t Unplug Your Humans – Yet
ChatGPT doesn’t claim to do more than present a quick, clear high level summary of knowledge already on the Web. That means it could (with a fact check and some minor polishing) produce very short “today’s state of the market” content. This instant summary of the conventional thinking on a topic can eliminate an hour or two of upfront research on the part of a writer.
But by its very nature, ChatGPT looks far less capable of taking large amounts of less structured information (from, say, a brainstorming session between a product manager and a marketing manager) to fine-tune content for lead generation or thought leadership. That is often where much of the most important work happens in my writing assignments.
So right now ChatGPT can’t replace humans in creating complex, insightful content that requires drawing and correlating insights from multiple, messy human sources. Or can it? If someone out there has pushed the marketing envelope farther than I did with Chat GPT let me know and I’ll push up my retirement date.