AI implementation in retail and ecommerce has reached an inflection point where individual experimentation must give way to organizational strategy, but the path from proof-of-concept to production remains fraught with challenges around data security, cost justification, and realistic expectations about automation.

“We have many employees running lots of experiments, using AI to do really interesting things,” said David Cost, chief digital officer at Rainbow Shops, during a panel discussion at The Lead Summit in New York City. “As they get some success with those experiments, they then want to spread them, and find out that they don't really have the skill set to build a structure around it."

AI automation still requires human accountability

Organizations should invest in process mapping and documentation before pursuing AI automation.

"People will say, 'We want to automate this, we want this to be faster,' but if you were to ask them what the exact process is they want to automate, you'll get a lot of ambiguity," said Serge Moreau, former vice president of technology at Tommy John, noting this exercise often reveals inefficiencies and unnecessary steps that should be eliminated rather than automated.

At Tommy John, Moreau said he built a QA framework that reduced testing time from a full day to two hours, a transformative improvement for the development lifecycle. But the success came from first documenting the complete process, identifying the people managing it behind the scenes, and only then applying automation and AI.

Maintaining human oversight remains critical, particularly in areas involving customer interactions.

"Leveraging an AI agent to manage your orders or to set your pricing is probably something that you shouldn't do from an ecommerce perspective," Moreau said. "Where is the liability if you're having agents dealing with humans or customers, and they just order it through ChatGPT? Where does the onus happen?"

Scaling AI means balancing innovation with control

For Columbia Sportswear, a heritage brand founded in 1938, AI isn't the strategy itself but an enabler of customer acquisition strategy.

"If I am to shepherd a new generation and continue to build on this trajectory that we've already built, introducing new generations over this long period of time, I have to get into AI. I have to show up there because that's how consumers are shifting from a shopping experience standpoint," said Mimi Ruiz, vice president of ecommerce at Columbia Sportswear.

Columbia Sportswear has implemented a three-pillar governance structure for AI initiatives that includes guiding principles, a monthly governing committee, and an approval process for every AI use case.

"We approve every single use case of AI, new technology, and that is part of our really deep care towards consumer data, making sure we have a closed model that we're protecting what is proprietary to our brand," Ruiz said.

The structure does create backlogs and bottlenecks, Ruiz acknowledged, but the company prioritizes data protection and clear outcomes over pushing out dozens of proofs-of-concept.

The challenge is only growing more acute as AI solutions evolve at breakneck speed. Tools and models can change significantly in just a few months, making it difficult for companies to keep pace with both the technology itself and the compliance requirements surrounding it.

Larger organizations must balance speed of innovation against regulatory compliance and data security. The governance overhead is necessary but requires dedicated resources and cross-functional coordination between IT, legal, finance, and business units.

AI delivers more value as a copilot than a replacement

The tension between innovation and operational reality also shapes how retailers measure AI success internally, particularly when it comes to labor savings and automation claims.

Every attempt to use AI to reduce headcount has failed at Rainbow Shops, Cost revealed, while initiatives focused on augmenting human capabilities have succeeded spectacularly.

The company's most telling experiment involved customer service automation. A vendor claimed their AI solution handled 40% of customer service workload without human interaction, measuring ticket closures in their system, according to Cost. But when Rainbow examined actual customer service agent hours and case volumes, the impact was negligible.

"If I showed you the data before and after, you wouldn't know that there was any automation whatsoever," Cost said. The reason: Customers would interact with the AI agent, become unsatisfied, hang up, and immediately call back to speak with a human agent. The vendor's metrics showed closed tickets; Rainbow's data showed no reduction in human labor costs.

Organizations need to define success metrics that align with business outcomes, not vendor-provided statistics. Cost justification requires measuring actual labor hours and productivity gains, not just system-level activity metrics.

This was originally featured in the Retail Daily newsletter. For more retail insights, statistics, and trends, subscribe here.