Pros and Cons of AI in Marketing: What 6 Months of Testing Have Taught Us
- Irene Schouwenaar
We haven’t made a secret of the fact that we’ve been exploring the use of AI marketing tools. (Who in our industry hasn’t?) But back in April, we decided to start pitting our human team against generative AI tools in a series of experiments that took examining the pros and cons of AI in marketing to a whole new level.
It’s been six(ish) months since we first started exploring AI in digital marketing. There’s clearly a long way to go. But we figured it was time to start sharing what we’ve been doing — and what we’ve learned so far. With apologies in advance for a bit of self-promotion, here’s what we’ve been up to lately.
Which AI Marketing Tools Did We Test (and How)?
We’re still in the very early stages of collecting data on AI pros and cons. It took us a bit of time to organize what testing AI tools would look like, and still more to build out our public-facing test beds. (Check out Blog by Humans and Blog by Robots to view some of our ongoing trials — not everything we’re testing fits into a blog template.)
- Blog copy
- Blog designs
- Blog SEO
- Ebook copy
- Email copy
- Infographics
- Case study copy
- Landing page/Thank You page copy
- Landing page code/layouts
- Meeting summaries
- Jasper
- CopyAI
- ChatGPT
- Parrot AI
- Hubspot Content Assistant
- Perplexity
- Google Bard
- Grammarly
- Lexica
- Looka
- Brandmark
- TensorArt
- Graphy
We started simple, with limited variables. Beginning with blog posts, we’d create two versions of each asset: one written, designed, and optimized by humans and one with one of those elements executed by a generative AI tool. We did long blog posts, short blog posts, case studies, ebooks, and so on — each with only one AI-generated component.
The first experiment posts went live in late July, so we’re holding off on sharing data on the pros and cons of AI in digital marketing until we have time to see some clearer trends. In the meantime, here are a few of the lessons we’ve learned while testing AI tools.
1. The Road to Hell is Paved with AI Prompts
For a technology that’s supposed to do all the heavy lifting for us, using generative AI to create marketing content is a surprising amount of work. The team on the AI side of our project spent a lot of time honing their prompt game in an effort to make the tools behave, but the results frequently left something to be desired. Issues include:
- Overly dramatic copy, even when we prompted AI to write in a professional tone (Has ChatGPT been watching daytime TV?)
- Missing information, especially when we prompted AI to include specific text or data (Want to cite that stat? Jasper has other plans.)
- A mind of its own — Sometimes AI just straight up ignored our instructions (telling it to limit copy to a specific word count was like telling a toddler to go to bed)
TPM’s Take: Is AI good or bad at copy? The jury’s still out. The more information you can include in a prompt, the closer you’ll get to your goals. (But closer is a relative term.) There’s a definite learning curve to even the most intuitive tools.
2. Parroting Can Be a Useful Skill
We’ve already talked about a bunch of the limitations we discovered while testing AI tools’ ability to write blog copy. Original content is not AI’s forte. But AI really shines when you ask it to restructure or summarize something that already exists. This didn’t feel terribly valuable at first — until we realized how this skill could be used when repurposing or promoting content. Use cases we’re exploring include:
- Social copy or paid ads to boost existing assets
- Email nurture sequences linking out to blog posts or other assets
- Landing page copy
- Generating blog posts based on ebooks and vice versa
- Summarizing the key points of meetings or calls
TPM’s Take: One of the main benefits of AI is its ability to condense large quantities of information into bite-sized pieces. There are definitely efficiencies to be found here, provided you’re willing to invest the time in learning how to prompt effectively, and you have some strong assets to work from.
3. Graphics and Art Are Not the Same Thing
- A person wearing a sort of steampunk-looking outfit, at a computer, in space
- A vaguely misshapen woman sitting beside a laptop that someone else appeared to be working on
- Same woman as version 2, with a less distorted face but with no arms
- A woman and a man, facing but not using a laptop, but staring at me with the creepiest gaze ever
- A relatively human-looking woman, using a laptop (yay!) while laughing hysterically
To be clear, I was not on the AI team. These examples are just from me trying out different tools. But the reports they gave on testing AI tools for image generation sounded pretty much the same.
TPM’s Take: Will marketing teams be replaced by AI? I’m just quoting our AI team directly here: “Still very far from being able to use for marketing purposes” and “Great at making art, not great for making graphics.”
4. Someone REALLY Needs to Make an Infographic Tool
Probably the most frustrating experience testing AI tools so far was one where the team attempted to create an infographic using AI tools. Our writers did some research and compiled a series of statistics and data points that we felt would be a solid basis for an infographic. Our human designers made their version, and the AI team… spent a lot of time trying to find a tool up to the task.
After hours of fruitless searches and false starts, they used a tool that could generate basic pie charts and bar graphs to make some “graphics” and a separate tool to place all those graphs on a background. The result was fine, but not especially exciting — especially compared to the human team’s version. In the technology’s defence, the AI team produced their infographic in about ⅕ the time the human team took once they figured out a solution. (The only problem is that the AI version LOOKS like it took ⅕ the time.)
TPM’s Take: There are a lot of good things about AI for content marketing, but this isn’t one. If we were developers instead of marketers, we’d be all over this clear gap in the market. There are simply no good tools to create an infographic, even if you’ve already got the data ready to go.
Understanding the Pros and Cons of AI is an Ongoing Journey
AI technology — and our ability to use it — is growing in leaps and bounds. But testing AI tools against experienced human marketers clearly revealed the pros and cons of AI for content marketing. The future of AI in marketing is probably a hybrid one, where repetitive and mundane tasks are offloaded to generative tools, and marketing teams deliver the creative juice.
The TPM team will continue experimenting with new and evolving AI tools to identify how they can help us supercharge our marketing efforts. We’ve also launched a hybrid AI tech stack, offering consulting on AI tools and low-cost campaigns boosted by efficiencies we’ve already uncovered. Contact us to learn more.
