MediaNews Group has shut down all of its comment sections. And earlier this year, Gannett ended online commenting for most of its news sites. B2B media organizations have gone up and down over the years with their comments sections. But done right, those sections can bring engagement, audience and even revenue. Enter AI and Graham Media Group’s First Dibs Comment Bot.
One of my favorite sections of The Washington Post is their weekly Free For All comments page. This past Saturday, comments included not spelling out acronyms in their first use, ignoring Canada Day, National Geographic’s layoffs, and calling the late Alan Arkin’s character (Yossarian) in Catch 22 “paranoid” in his obituary. (“The obituary had inadvertently reversed the entire meaning of the book and movie.”)
But this Free For All is obviously closely guarded. While comments sections can be positive and conversation-enhancing—especially on webinars—they can also become unruly and time-heavy to moderate. Looking at past articles on the topic this morning, I found adjectives like “toxic,” “attacks,” “vile” and “personalized.”
“It’s something we’re all grappling with,” Boston Globe editor Brian McGrory said last year on NiemanLab, but “there is an expectation that people should have a say.” Added Gabriel Escobar, editor and VP of the Inquirer: “Engagement and the time that people spend on the site is an important digital metric, and if people are allowed to comment, they’re obviously on the site longer.”
While more places shut down their comments, one entity is working hard to continue their comments safely and successfully. And they want to do it through AI and ChatGPT. Say hi to the First Dibs Comment Bot.
Graham Media Group’s Dustin Block recently gave a presentation of their First Dibs project at the AI & Local News Challenge Demo Day sponsored by NYC Media Lab. Here are some details from that video presentation and Sophie Culpepper’s story on NiemanLab.
Now is the time. Comments can be valuable. The “cookie-pocalypse” is here, making first-party data critical for publishers. “Commenting is a huge driver for user registration and on-site registration [for us],” Block said. [But] without quality guidance, “users disengage because of lack of quality discourse. And getting talent to participate in comments… It has to be safe and quality conversation.”
It’s nicer than you might think. The First Dibs Comment Bot posts the first comment on stories to guide conversations, grow participation and drive users deeper into engagement funnels, Block said. It’s crafted just to post questions and be friendly. It finds the theme of the story and asks an open-minded and positive question making for consistently good prompts. Including “please” in a prompt for a question “substantially changed the results,” Culpepper wrote, “and explicitly asking the AI to keep questions positive ‘really helps the conversation.’”
It loses the “hallucinatory.” One of the obvious criticisms about ChatGPT is that it can sometimes seemingly make stuff up. “Hallucinatory” has been the most popular word to describe these misadventures. But because it is being told here to just phrase questions, not statements, “it’s less prone to hallucination.” “It hasn’t asserted any facts, so it hasn’t given any wrong information,” Michael Newman, Graham’s director of transformation, said. (He would have fit perfectly in my story on interesting job titles last week.) No hallucinations or missteps so far.
It treats sensitive stories well. “We’re very concerned about sensitive stories,” Block said. “We didn’t think the bot would handle that well. We’ve been a little surprised at how well it does.” On a sensitive story about a heroin overdose, the bot asked: “What steps should law enforcement take to ensure the safety of both officers and individuals who may be experiencing a drug overdose during an arrest?” “Not bad—probably better than our human editors would do, certainly faster,” Block said. “It has repeatedly been able to take these difficult stories and at least not be wildly out of line for what we’re looking for,” and do it much faster.
You still need people. “Moderation remains key for us,” Block said. “The bot would be moderated [by our] external moderation team.” Feedback has also “led to a practice of asking the bot to generate five different prompts, instead of just one,” Culpepper writes, “and allowing the writer to pick the prompt they like best, Newman said. He’s interested to see whether this tweak ultimately helps drive more human engagement.”
Revenue is the goal. Comments mean engagement, and engagement can mean dollars. Graham Media Group wants to grow logged-in users by 10%, doubling the average revenue per user. The same goes for their user registrations. In the Q&A, Block emphasized how comments can mean much more to their “logged-in users,” as opposed to those just visiting. And that can also mean dollars.
Time will tell. First Dibs is set to go live in August and will be well-looked after. “We don’t want to put this out in the world if it’s going to do something that even offends the person who’s reviewing it before it posts up on the site,” Newman said.
