Black Friday planning is already under way. Campaigns are being mapped out, offers are taking shape and code-freeze dates are beginning to appear in calendars.
The freeze is often treated as the point when risk settles down. Your development team stops shipping major changes, the platform becomes more stable and attention turns to trading through peak.
But a code freeze only controls the changes your own team makes.
It does not freeze the AI, product data, content or third-party services shaping the customer journey. Payment providers continue releasing updates. Stock, prices and delivery information keep moving. Promotions change. AI models, prompts and knowledge sources can be updated.
Your team may stop shipping. The rest of the ecommerce ecosystem does not.
AI now sits across the ecommerce journey
A year ago, most conversations about ecommerce AI focused on chatbots and product recommendations. Those still matter, but AI is now being used much more widely.
It can influence on-site search, guided shopping, personalisation, customer service, product content and even parts of the checkout journey.
These features rarely work alone. An AI assistant may rely on live product data, stock availability, pricing, delivery information, returns policies and several third-party services to give a useful answer.
That means the AI experience can change even when nobody has released new website code.
A chatbot that gave the correct delivery information when it was tested may become outdated after cut-off dates change. A product assistant may recommend an item that has since sold out. AI search may still return results, but those results may no longer reflect current prices, offers or availability.
The interface works. The customer experience does not.
What your code freeze does not freeze
A well-managed code freeze remains an important part of Black Friday planning. It reduces the chance of a late internal release creating new problems and gives teams a more stable platform to test.
What it does not do is protect the whole customer journey from change.
After your final internal release, updates may still be made to:
- Product prices, descriptions and availability
- Promotions and discount codes
- Delivery services and cut-off dates
- Returns and customer service information
- Payment methods and fraud controls
- Consent, reviews and personalisation tools
- AI prompts, models and underlying data sources
AI is particularly exposed because its answers often depend on several of these moving parts.
The AI itself does not always need to change for its answer to become wrong. Sometimes the information around it has changed instead.
The problems normal checks may miss
Automated and functional testing remain essential. They can confirm that a page loads, an integration responds or a defined journey can be completed.
What they cannot always tell you is whether an AI-generated answer is accurate, relevant or genuinely useful.
An automated check might confirm that a chatbot returned a response without recognising that it gave the wrong returns information.
It may verify that AI search displayed products without understanding that none of them matched what the shopper actually requested.
It may confirm that a recommendation appeared while missing that the suggested product was unavailable in the customer’s required size.
This is why AI-powered journeys need more than a technical pass or fail. They also need to be evaluated from the customer’s point of view.
Three Black Friday AI risks worth testing
1. Outdated or inaccurate information
A customer asks whether their order will arrive before Christmas or whether a Black Friday purchase can be returned in January.
The chatbot responds confidently, but its answer is based on an older policy.
From a technical perspective, it worked. From the customer’s perspective, the retailer may have made a promise it cannot keep.
2. Search and recommendations that miss the point
A shopper asks for a waterproof winter coat under £100, available in their size.
The AI returns water-resistant fashion jackets, products above budget or items that are no longer available.
The results are technically related to the query, but they have not met the customer’s need.
3. A good AI experience followed by a broken journey
A product assistant helps someone find the right item, but the journey falls apart when they move into basket or checkout.
Their selected preferences are lost. The promotional price does not appear. The recommended delivery option is unavailable. Their preferred payment method does not work on their device.
The AI interaction may have been helpful, but the customer still cannot complete the purchase.
Black Friday AI testing should not happen once
The answer is not one enormous test just before code freeze.
AI-powered journeys should be checked at several points.
Before the freeze, test the feature and the complete journeys around it while there is still time to fix significant problems.
After the final production release, check the experience again in the environment customers will actually use.
During peak, repeat targeted checks when promotions, prices, delivery rules, product data or third-party services change.
Monitoring may tell you that conversion has fallen. Testing the live journey helps explain what customers are actually experiencing.
Why real-user evaluation matters
Internal teams naturally test the scenarios they expect. Real customers do not behave that neatly.
They phrase questions differently, make typos, change their minds, misunderstand instructions and use combinations of devices and behaviours that may not appear in a predefined script.
The strongest approach combines automated checks and functional QA with structured evaluation by real users.
Automation helps establish whether the defined components and pathways work. Human evaluation helps reveal when something technically functions but makes little sense to the customer.
Digivante’s JourneyEval AI evaluates AI search, chatbots, product assistants and AI-powered customer journeys with real users, helping teams uncover irrelevant results, confusing outputs and unexpected friction.
Your code freeze is not the safety net
A code freeze reduces one source of risk. It does not remove every source of change from the customer journey.
The real safety net is visibility.
It is knowing how the experience behaves before the freeze, checking what customers encounter afterwards and having a way to investigate quickly when AI, content, data or third-party services change.
Because during Black Friday, confirming that the website is technically live is not enough.
Customers still need to find the right product, receive accurate information, understand the offer and complete their purchase.
Find the issues your code freeze won’t catch
Digivante’s Black Friday Readiness Audit checks critical customer journeys using real testers on relevant devices, helping ecommerce teams identify high-impact problems before peak.
You receive clear, prioritised findings showing where customers encountered issues and what is worth addressing first.
Book your Black Friday Readiness Audit.
Frequently asked questions
What is Black Friday AI testing?
Black Friday AI testing evaluates AI-powered ecommerce features and the journeys around them before peak. This can include AI search, chatbots, product assistants, recommendations and AI-assisted checkout experiences.
Does a code freeze stop AI systems changing?
Not necessarily. AI models, prompts, product information, policies and third-party services may continue changing after internal development releases stop.
Can automated testing validate an AI customer experience?
Automated testing can confirm that defined functions and pathways work. It may not recognise whether an AI-generated answer is accurate, relevant or useful, which is where real-user evaluation adds value.
When should ecommerce teams test AI before Black Friday?
Teams should test before code freeze, after the final production release and again when important content, data, promotions or external services change during peak.
