In an age of digital opportunity and agile development, ecommerce and digital product managers know that first mover advantage and continuous improvement are key to competitive success.

But it’s an omnichannel world where nothing stays still. Websites and apps are moving targets for bug detection and usability. As release cycles shorten and hardware and software evolve, customer experiences become more sophisticated. Make a change here and something might break there. Everything is related to everything else, and everything is in constant motion.

With so much change and pressure to innovate, testing for and resolving issues can become a firefighting activity for damage limitation. A lack of resources, knowledge, processes and tools makes it challenging to impose a systematic and preventative testing regime.

For digital and ecommerce managers, hitting a pragmatic CX balance between rapid progress and high product quality is a constant challenge. And it’s one that we’re very familiar with at Digivante.

In this ebook, we review the key challenges for digital product owners in hitting launch deadlines without compromising the customer experience. We set out the key testing priorities to help you launch digital products and features that achieve and sustain the best possible conversion rates, while allowing continuous innovation and improvement to proceed at pace. And we suggest ways to improve your current testing approach and processes in line with your business priorities.

Digital quality challenges for leading retail ecommerce brands

Usability challenges

A few years ago, Marks and Spencer’s website relaunch promised a new era for their online shoppers, offering cutting-edge digital experiences and easier paths to purchase. On release, it proved to be rife with issues and usability problems. Millions of registered customers were forced to create new accounts while others reported a complete inability to check out. The £150 million rebuild saw an 8.1% drop in online sales and caused a 1.5% drop in share price

Peak outages

High profile brands lose business every year through system crashes, because websites remain untested and un-optimised for the constant surges in traffic that are “business as usual” in modern online retail. In 2019, American retailer Costco’s Thanksgiving outage reportedly cost them $11 million. A Black Friday crash for UK electrical retailer Curry’s led them into a PR nightmare, when customers took their frustrations to social media.

New ecommerce functionality is complex but vital

The latest digital features and designs bring more and more value to customers. Launching them early brings kudos and attracts customers. Responding fast to competitor innovation is key to minimising churn. Personalisation, conversational AI and gamification deliver memorable user experiences, increased loyalty, more dwell time and greater conversion. They introduce new digital techniques behind the scenes and rely on complex algorithms and integrations. That introduces risk in QA terms and increases pressure on testing.

Territory expansion means adapting to local norms

The digital global opportunity brings distant and lucrative new markets tantalisingly close for eager, challenger businesses. But can your website and apps really deliver the conversions you want in the regions you need to target? Optimising for cultural and linguistic differences is an ongoing challenge.

Brands who want to cut through in new markets need to know that their offerings are working on the ground. They need the ability to continually tweak and optimise them to deliver the best results, working with potentially new and unfamiliar customer behaviours.

Mobile is growing bigger than ever

More and more customers around the world transact only using mobile devices – they’re always on and always to hand for an immediate browse or purchase. Testing for mobile is vital, across the full range of devices and OS in use across all the territories where you operate. New mobile devices and upgrades are released all the time and many consumer demographics aspire to upgrade frequently. Brands need to make sure their testing keeps pace with customers’ mobile usage.

Desktops aren’t dead yet

Mobile might be the headline growth area, but many customers use desktop, laptop and tablet devices. The digital world never stands still. All these devices and their OS are constantly evolving, and new browsers are being launched. Digital product managers can’t afford to sacrifice testing on these more traditional devices. Many users have more than one device – they expect the same quality of experience from your digital products at every touchpoint.

Agile development is relentless

Ecommerce development sprints are shortening. They introduce functionality in more rapid cycles, designed to help brands keep their competitive edge. In these conditions, the nature of testing has to change. It needs to uncover high-impact issues earlier and faster, to lower the risk of late fixes and increased costs. Testing must be more agile, to keep in step with the agile development cycle. That means finding more testing resources that can run testing fast, to support the software development lifecycle and its deadlines.

The impact of reactive testing

In a reactive testing situation, who knows what’s really causing customer leakage? Fixing glaring faults is just the tip of the iceberg. Elsewhere, customers are quietly switching off and switching their loyalty to another brand without you even knowing what tipped the balance for them.

That means you can’t see trends or early warning signs of poor CX. Flawed design and function become embedded and repeated. The business won’t let you delay releases or launches to take stock and unpick the mess, because you need to keep improving your proposition to match or exceed competitor innovation.

In this environment, conversion optimisation activity is about firefighting. And that’s a big problem when your team’s objectives are tied to proactively improving conversion, customer satisfaction, active users and churn targets.

Protecting and optimising your conversion and ecommerce revenues

Testing priorities for ecommerce and digital product managers

How do you tackle these challenges for your digital products and make sure your conversion rates are hitting target? We recommend seven key areas of testing focus for ecommerce and digital product managers, to manage and sustain high quality CX and to keep pace with ongoing development and innovation.

1. Functional testing

Functional issues are bugs that prevent vital user journeys from being completed. They lead to baskets being abandoned and customers getting stuck in loops or being kicked off websites altogether, never to return. Inappropriate redirects, 404 pages, incorrect filter results and checkout issues can all seriously damage revenue and reputation.

Functional testing identifies these issues using non-disruptive methods and then orders them using impact analytics. Testers working at scale against scripts, on a range of real-world devices and browsers, can quickly uncover where certain browser, device or OS-specific bugs are present.

2. Usability testing

Website usability testing addresses how easy it is to navigate the site and how efficiently customers can perform desired actions. It shows up errors that customers typically make as they attempt to make those actions. These experiences contribute to the level of satisfaction a site delivers.

Usability testing flags illogical, confusing or unclear elements. It identifies reactions to colour and other design choices that may be relevant for future optimisation. Iterative usability assesses the impact of changing designs and customer journeys on ever-evolving feature-rich ecommerce sites and apps.

3. Localisation testing

When your brand focuses on international opportunities, you need devices on the ground and real people with real local knowledge to help you spot issues that will impact conversion. You need local currency and payment solution testers to check integrations, which often cause problems.

Testing the quality of your language translations is key, so you don’t confuse users with clumsy, ambiguous wording. Localisation testers can report on culturally inappropriate, insensitive or offensive content carried over from other sources, which might otherwise alienate users and could be a PR disaster at launch.

4. Performance testing

It’s no good creating exciting website features and user experiences if you can’t deliver them fast and seamlessly. Research shows that more than half of visitors will leave a web page if they’re forced to wait longer than three seconds.

Retail red letter days like Black Friday and Cyber Monday, or unexpected news events and the rise of overnight trends, can bring sudden and unpredictable levels of custom to your app or website. Human performance testing checks it can cope with high levels of traffic. A mass test on the payment section of a site can mimic the effect of a flash sale or other rush to your website.

5. Exploratory testing

Exploratory testing gives qualified, professional testers the freedom to follow their hunches, creativity, product and industry knowledge to uncover bugs and potential usability issues. It allows teams to replicate real-world behaviour and decision-making beyond a script, to take a thought wherever it leads them, as it might a customer.

Exploratory testing creates a constant feedback loop between testers and developers. It requires a high level of skill, and it’s particularly important in an agile development environment.

6. Conversion analytics

If you can identify and prioritise the high impact conversion issues to investigate, you can tackle the biggest revenue leakages fast. Analytics are invaluable. Digivante’s Convert optimisation tool uses your Google Analytics ecommerce data to identify where specific devices or browsers are converting worse than others. It clearly identifies where you’re leaving easy revenue on the table.

Convert analyses user sessions and transactions through your site or app to identify lost conversions. It categorises them by browser, device and OS and shows how many users would experience the issue. This means you can tailor your testing approach to maximise ROI.

7. Live monitoring

For the most critical user journeys in a fast-changing ecommerce environment, daily checking can prevent small glitches scaling up into major revenue lost and customer churn. Use a small, specialist third party testing team to check key journeys every day and report back.

Live monitoring gives you peace of mind that the most valuable user journeys are working as designed on your app or ecommerce site. It gives you early warning if a change somewhere else has had a knock-on effect that needs fixing urgently.

Exploratory testing helped to resolve fluctuating sales for Etsy

Member merchants launching flash sales and unexpected interactions between hardware and software were just two of the small but high-impact events that caused sales fluctuations for Etsy worth millions of dollars in sales on their bottom line. When they couldn’t exactly account for or understand the impact of those changes, they couldn’t optimise rapidly enough to keep improving UX and protect their revenue.

Etsy moved to a system of continuous deployment, with a strategy of constant regression and exploratory testing, feeding back into developers to fix high priority issues.

hand made jewellery

Approaches and tactics to make it happen

Scale and speed through test outsourcing

Maintaining focus and output in all these testing areas is demanding for in-house testing teams. How do you handle surges in testing requirements, as you approach a launch deadline? The tests need to happen but there are only so many staff in the business and hours in the day.

Outsourcing to professional testers can be a cost-effective way to give challenger brands access to the size and quality of testing resource that hyperscale companies have in-house. A partner like Digivante can make hundreds of testers available on one project, delivering the equivalent of hundreds of days testing in 24 hours.

This solution can offer excellent professional and sector expertise, as well as the wide range of physical hardware devices needed to conduct thorough tests. With an international reach and 24/7 capacity, outsourced coverage eclipses most in-house capabilities.

Professional testing companies should deliver real world testers who match the target audience of your site – this can include age, gender and other demographic information, so that your testing feedback reflects the people whose behaviours and opinions matter most to your bottom line.

The right partner will be able to mobilise a large-scale, customised test group quickly, with the right device and browser combinations. For peace of mind around quality and confidentiality, make sure your testing partner provides professional, qualified testers who sign an NDA for each project.

More hands on deck with team augmentation

Digital testing requirements don’t tend to be consistent and even. It’s often hard to predict the resources you’ll need for particular projects. If you employ a full complement of in-house testers, they may not be fully utilised. But if you hold back from hiring, you can hit problems when you don’t have people at hand to ensure product quality.

On way to cut through the problem is by working with a team augmentation partner like Digivante. We can provide testing specialists to boost your internal test teams on demand, so you don’t have to carry the overhead. Augmenting your teams with our experts can bring you extra resources exactly where you need them to plan, deliver, and automate digital testing in any target area.

Make the most of automation by using blended testing

Test automation can help reduce time to market when you release new features and functionality. We recommend a pragmatic approach to test automation: do it selectively, where it will save you time and money.

An expert testing partner like Digivante will advise you on the best opportunities to automate tests at the right time, for the best overall outcome. We balance this with manual or crowdsourced testing, where it delivers better value and results.

Digital experiences are a battleground for competitive differentiation and market leadership

Successful ecommerce stories are about continually refining experiences, features and conversion, based on deep customer insight.

Feature-rich websites and innovative apps, constantly updated with new and exciting functionality deliver PR talking points, new opportunities for marketing engagement, and ultimately, more and more frictionless sales. In this age of agile development, there’s also a constant opportunity to make incremental changes that make a huge difference to the bottom line.

With so many moving parts, there’s an increased risk of bugs creeping in, slowing systems and frustrating customers who can’t do what they need to. It’s vital to be on top of testing, to ensure your digital product quality keeps pace with the speed of continual delivery and deployment.

Ecommerce and digital product managers need to understand and champion the critical elements of testing. They need cost-effective support for ongoing cycles of functional testing and the agility and resources to focus non-functional testing as needed.

Using your existing testing resources and augmenting them with an expert testing partner, you’ll never again be left wondering where customers went and why. You’ll have a clear vision of what you need to do next to maximise sales, revenue and optimise conversions.

Find out more about Digivante’s services and how we can help you deliver great customer experiences. Contact one of our specialist team today.

Share this story