Website testing: The Complete Guide for 2020

Everything you need to know about website testing, the secret behind the world’s best online experiences

Why do we do website testing?

The internet is an integral part of our lives. With more than four billion active users, it helps us access information, make decisions and do business on a global scale.

Thanks to the growing number of mobile devices, your customers can now log on from anywhere, at any time. This trend shows no signs of stopping with the number of global smartphone users predicted to hit 3.8 billion by 2021 and a third of millennials spending four to five hours a day online and on mobile.

web1

To stand out in this competitive and complex marketplace, you need to provide an informative, user-friendly and accessible digital experience to every user.

To achieve this, you need website testing.

What is website testing?

Website testing is an umbrella term for processes which test websites and other software applications for unwanted issues across a broad range of devices. It includes functionality, usability, accessibility and performance testing.

Change your business in 30 minutes - book a Demo

The benefits of website testing

Graph

Website testing improves the quality of your website and applications, bringing a broad range of benefits to your business.

Increased revenues

How much do bugs cost your business? The answer may surprise you. When Amazon crashed on its Prime Day in 2018, it lost an astounding $34 million per hour.

While your business may not be on the same scale as Amazon, bugs are widely regarded as the ‘conversion killers’ of the online world. If your website’s functionality doesn’t work as it should do, crashes, throws up a 404 error or is hampered by slow load speeds, for example, your visitors won’t hang around on your site.

Website testing identifies any issues with your website. With global e-commerce sales predicted to reach more than $6.5 trillion by 2023, your revenue now depends on your digital experience more than ever before.

Boost your brand

Companies now live and die by their online reputation. The broken trust caused by a security breach, functional defect or poor user experience can be costly to repair.

Website testing identifies such issues to protect your brand reputation and boost your website performance. It allows you to monitor your online presence using a range of methods.

Website testing also identifies other problems that have an adverse effect on your user experience. For example, if your website is littered with broken links, slow image loading or isn’t optimised for mobile and other such issues, your business falls through the cracks.

Expedite your time-to-market

Website testing gives your developers the necessary feedback to proactively fix bugs, create a maintainable codebase and optimise your digital experience – all before launch.

Instead of firefighting bugs, your developers can now set the online world on fire by providing an enhanced and up-to-date digital experience.

As a result, and combined with the agile process, website testing accelerates your time-to-market, so you race ahead of the competition.

Cut costs

There are plenty of positive cost-saving connotations thanks to website testing. For example, by identifying and resolving any software or hardware bottlenecks, you can vastly reduce your tech costs. Your development process is also streamlined, so you can add value at every point of your digital experience.

What’s more, if your website works, your customer service team will deal with fewer issues from online customers. This leaves them free to field other queries from your customers and reduces the number of resources you need.

The challenges of website testing

Website testing is not simple. To realise its vast array of benefits, you need to address a diverse range of issues.

Push and pray? Or bottleneck?

It’s not uncommon for businesses to have internal teams conduct their website testing. This is an increasingly difficult task however, as the way we develop website applications is changing into an increasingly iterative process.

Such practices are having a negative impact. For example, research reveals that agile and DevOps adoption is increasing, but bugs are not getting fixed any faster.

This is because tech teams are under mounting pressure to conduct website testing on a more regular basis. Consequently, exploratory testing is also often skipped, as teams rush to push a website through the development process.

This leaves many businesses with an impossible decision: push to production and pray there aren’t any issues with the website or create testing bottlenecks where your entire development process stalls.

Neither option is a solution. You need to find another way.

Cross-device and cross-platform testing

Cross-device and cross-platform testing is a crucial phase of quality assurance. Research reveals that testing is now conducted on more browser types than ever before, and that 77% of mobile testing uses a mix of both real devices and emulators.

With an endless combination of modern and legacy devices and platforms for users to choose from, there is simply not enough time to provide the coverage you need for robust website testing.

You don’t know what you don’t know

The quality of your digital experience impacts your revenue. But, without the right metrics and insights, you could struggle to understand your website’s issues.

You need a firm grasp of your website’s good (and bad) features to build a watertight business case for any additional QA. However, this can often feel like a chicken and egg situation, where you need to conduct additional testing to prove the need for additional testing.

Getting real user feedback

User feedback provides you with the necessary insights into your online customer journeys. It can help you create an online experience that’s not just relevant to your users, but also an intuitive and pleasurable one – and hopefully an experience that your users will rave about.

User feedback can also help you understand the ROI of your user experience. Companies that follow a user research-driven website development process can also increase their KPIs by an average of 135%. 

But how can you get reliable user feedback? Many businesses don’t know where to start, or don’t have the time to carry out such comprehensive usability testing – or both.

Security and performance

What data security measures do you have in place? Is pen testing enough? Do your pages load fast enough for your users?

These are just a handful of questions you need to address to ensure your website is secure and running at peak performance. 

This is no mean feat – but a very necessary one. The average cost of a data breach in 2020 is predicted to exceed $150 million.  

Web users also expect a more robust and interactive web experience. For example, some 30% of site visitors expect a page to load in one second or less, while 18% expect a page to load instantaneously.

Website Testing Pillar Page CTA

The importance of early detection

Fixing errors and bugs late on in your development process is a very expensive and complex process.

Why? Let’s look at the lifecycle of a bug from identification to fix when it’s in production. 

If a bug is reported by a customer, that customer has to report it to a call centre, the assistant takes down the necessary details and then it is sent to their manager. The bug is passed to the tech department, where a developer works on it. But the developer often cannot reproduce the bug in your test environment. So, it goes all the way back to the source and the cycle starts again.

What’s more, an undiagnosed bug or error on your live website may cause an ongoing instability, and you could lose customers without understanding the root cause. Any such errors could also cause a domino effect, where you fix one thing only to unleash a raft of new bugs and errors on your website.

What can you do? You could introduce a regular code reviewing process to help with quality assurance and optimise your coding cycle from the outset. However, to truly mitigate the impact of bugs on your website, you need to implement website testing as early as possible into your development lifecycle.

“AN UNDIAGNOSED BUG OR ERROR ON YOUR LIVE WEBSITE MAY CAUSE AN ONGOING INSTABILITY, AND YOU COULD LOSE CUSTOMERS.”

What are the common website testing methods?

We’ve talked about why you need to complete robust website testing. But how should you carry out such tests, and who should you get to do it?

#1 Do nothing

Method: You ignore website testing altogether. 

Pros: You avoid the initial overheads associated with website testing.

Cons: This approach will most likely result in website outages, high maintenance costs, lost data, missed business opportunities and, consequently, colossal brand damage. While you might save money in the short-term, the long-term impact to your business could be devastating.

SHOCKINGLY, 2% OF BUSINESSES ADOPT THIS HIGH-RISK APPROACH AND COMPLETE NO ONLINE TESTS AT ALL.

#2 Get your developers to do it

Method: You get your internal developers to test your website on the side. 

Pros: You can save money on hiring costs and training of additional staff.

Cons: Most developers are expert problem solvers; they are not trained or (often) adept at creating issues for your website. They have the wrong mindset for testing and will miss issues.

Also, your valuable developer resources are now focused on tasks outside of their remit if you adopt this strategy. So, the functionality of your website could start to falter as your developers neglect their core duties.

Change your business in 30 minutes - book a Demo

#3 Use your web development agency

Method: An external agency is developing your website anyway. So, you ask these external developers to do the testing at the same time.

Pros: You already have a working relationship with the agency, and you free up your internal resources from testing your website.

Cons: Whether they are external or internal developers, a website developer is not a specialist website tester. You will also pay a high premium for this service, and your web development efforts could get left behind as testing uses up these resources.

#4 Outsource to a consultancy

Method: You outsource to a high-profile consultancy firm.

Pros: Such large organisations have the vertical capability, global coverage and quality of operations to conduct thorough website testing, with minimal disruption to your internal development teams. For example, Accenture has an estimated 34,500 dedicated application testing service resources.

Cons: Such large businesses often come with a high cost of service, lack of flexibility and lack of consistent innovation.

#5 Bring in the contractors

Method: You hire a specialist website testing contractor(s).

Pros: Contractors are useful short-term resources, where you get the expertise you need.

Cons: You will have to complete the necessary hiring processes and training. This can be costly from both a time and cost perspective. Consultants are also only available on a short-term basis (usually of the order of a few months up to, potentially a couple of years). So, this is not a long-term solution that also lacks consistency.

#6 Crowdsource

Method: Your website testing is crowdsourced, where testers are located at different places. Crowdsourced teams monitor project tasks, identify issues, and report bugs in real-time so (internal or external) QA engineers can fix any problems and maintain a seamless user experience.

Pros: Fast test results, such testers excel at basic and visual tests, high number of testing hours and high device coverage from testers located all around the world.

Cons: Not all tests are suitable for crowdsourcing and you still need someone to fix the identified bugs.

#7 Hire a professional QA team

Method: You hire a specialist QA team to work with your tech teams and carry out your website testing.

Pros: Domain expertise, round-the-clock production hours, labour savings, cost savings, efficiency and time savings.

Cons: You need to find a trusted partner that will give you the guaranteed quality of service you need. Some businesses also do not want to give a third-party external control of their website testing or access to their IP.

Ultimate Website Testing Checklist Pillar Page CTA

What’s the best website testing strategy?

There are many different ways to conduct your website testing, and you’re probably on your way to make a choice based on your business requirements and priorities.

Here are some common factors you may (or may not) want to base your decision on:

Cost: Is website testing worth it?

When discussing his company’s attitude to profitability, Apple CEO Tim Cook once said: “I do not consider the bloody ROI.” However, unlike Apple, you probably want to get some value for money out of your website tests.

A revenue calculator is a vital tool to help you do just this. It monitors the impact of website testing on your ROI. For example, let’s assume you have a critical bug on your site that hits Android users during checkout. This is damaging your sales, but you haven’t noticed because customers haven’t complained, or your call centre hasn’t passed this on. It simply becomes part of the background average conversion on your site – because, unlike retail store owners who deal with customers on a face-to-face basis, you might not see the problem.

Website testing not only helps you find and fix these issues, but a revenue calculator also allows you to quantify the positive impact on your top line.

Make sure you use a revenue calculator to quantify the impact of website testing on your revenue.

In-house versus external: Which one provides the best business value?

The choice between an external and in-house team often comes down to the amount of value you want to extract from your website testing strategy.

In many cases, an in-house strategy is a false economy due to a range of hidden costs. For example, by outsourcing your software testing, you don’t need to invest in any high-end testing software and hardware. Your HR costs may also be reduced, as the third-party service providers would take responsibility for the recruitment and training of professional testers.

By using an internal team, you could also increase your indirect and operational costs as you have to set up a dedicated quality assurance department to bring the necessary, specialist skills in house. You also become responsible for quality management and could lose focus on your core competencies as a business.

An external website testing team can add value to your organisation in ways internal resources cannot.

Using partners: How can you find a good one?

There are a huge number of software testing companies out there offering outsourced website testing services – but how do you know which one is best for your organisation?

One of the difficulties of outsourcing your website testing is finding a trusted partner that you’re willing to give the responsibility of potentially high value, business-crucial projects. When researching partner companies, ask for recommendations and client testimonials.

Look for external website testing companies with a solid portfolio and who are regarded as QA professionals by their clients.

Collaboration between testers and developers

With the adoption of Agile and DevOps, testers need to collaborate with developers for faster and bug-free build deployments. Your testers will need to learn development skills as well to have good collaboration with the development team.

As a result, you may find your testers helping with some of your web development work and your developers may also end up writing a few test cases.

An agile methodology can help developer-tester collaboration. You could divide your teams up for individual sprints to encourage your developers and testers to cooperate, while still expediting high-quality deployments.

Website testing does not exist in silo. Your developers and testers need to work together.

Getting the best domain knowledge

You need the right tools, techniques and team to get the things done. In other words, you need to closely monitor quality throughout the entire testing process, as your business’s reputation may be at stake if work is not up to standard.

Look for quality website testers with expert knowledge.

Consider confidentiality and control

When the pressure’s on, and there are strict deadlines to meet, it can prove tricky to effectively maintain communication with a third party, and even tougher to make sure that your testing is delivered on time.

From a confidentiality perspective, you must also ensure your testers do not give away your valuable IP. Imagine the damage that could be done if a tester leaked your latest product launch. A three-way contract can effectively protect against this issue.

You need to maintain your corporate confidentiality and effectively communicate with your testers to meet your deadlines.

Security matters

During a typical testing process, you may need to share highly confidential information, such as customer databases, design documents or source codes with your testers.

Therefore, you need to ensure that the testing team works within your firewall to retain your corporate confidentiality. What’s more, such measures will ensure any unscrupulous testers cannot exploit any vulnerabilities in your systems.

Check the security measures your testers use and keep them within your firewall.

Test coverage

Every feature or chunk of functionality on your website need to be tested on as many device and platform combinations as possible to ensure a seamless experience.

API testing is another important component to consider. Many software development processes are shifting towards a service-oriented architecture (SOA) approach with the implementation of public APIs. However, API testing is complex and requires a high level of coding skills – but it is very important to add API testing in your software testing lifecycle to ensure bug-free APIs.

A robust website testing strategy is required across multiple devices and platforms, including APIs.

What are the different types of website testing?

There are various types of website tests. Here’s a rundown of the key website and app testing methods you need to understand.

Functional testing

Ever visited a website that doesn’t do what you want it to do? Did you go back to that website? Probably not.

Functional testing makes sure your website behaves in the expected way. It checks for bugs and other issues, to ensure your conversions and digital performance are maximised.

Typical checks:

  • Forms – Do they work? And do they work on desktop and key mobile devices?
  • Live URLs – Are your redirect pages working? Do you have a custom 404 page?

Localisation testing

Localisation testing ensures your website is available to a global audience. It covers linguistic, technical and visual checks to overcome translational and dialect issues. Some localisation testing services also suggest changes to improve your localisation optimisation.

Typical checks:

  • Translations – Do you need to introduce any specific dialects or special characters?
  • Local details – Have you got the correct contact information for each location? Do you accept different address formats to cater for different locations?
  • Performance – Is your website available and functions over a range of locations?

Payment testing

Payment testing ensures your customers can complete their purchase, regardless of their location, browser and device. Payment integrations are also checked to ensure no transaction falls at the last hurdle.

Typical checks:

  • Payment options – Do your payment options work across multiple devices and platforms?
  • Coverage – Are the appropriate payment options available for specific locations?
  • UX – Is the payment process seamless?

Web & mobile conversion quality index CTA

Performance testing

More than half of visitors leave a page if they’re forced to wait longer than three seconds.

Performance testing provides an accurate picture of your website’s load times, identifying potential issues and bottlenecks. There are many different types of performance tests, including:

Load test: a simple test that is usually conducted to understand the behaviour of your website under a specific expected load. This could test what happens when a number of concurrent users perform a specific number of transactions within a set duration on your website.

Such tests provide response times for all your business-critical transactions. If your database, application, and other servers are also monitored, then a simple load test could also reveal any bottlenecks in your network.

Stress test: these tests normally try to break your web applications by incrementally increasing the number of users on your site. This kind of test can determine your website’s robustness in times of extreme load.

Spike testing: as the name suggests, a spike in the number of users is created to help understand the behaviour of your web application when an unexpected spike in users occurs. It may, for example, go down or may be able to handle such sudden load changes.

Endurance/soak testing: this tests your website’s response to a continuous, expected load. This type of performance test can also determine if there are any memory leaks in your applications.

Typical checks:

  • Site speed – What is your average load time? Is this the same for desktops and mobile sites? Do return visitors get quicker speeds?
  • Compatibility – Does your website perform across multiple browsers, platforms and devices?
  • Images – Do these images load correctly on every device and browser?
  • Simultaneous users – Have you stimulated HTTP requests that could come from a high volume of users to see how your site reacts?

Usability testing

Usability testing helps you understand how real users interact with your website and make changes based on the results. This helps you evaluate the overall user experience by measuring how easily a user can accomplish a series of tasks on your website (also known as the user journey).

Typical checks:

  • Navigation – Does every user journey work? Is each user journey a seamless experience? Are there different user journeys for different user types?
  • Search – Can your users find the right pages when searching?

Accessibility testing

Up to 21% of adults require accessibility features in one form or another to perceive, understand, navigate and interact with the internet.

Accessibility testing ensures your website is accessible to all users. Whether a user has an impairment or a slow internet connection, accessibility features let them know their rights matter, and their business is valued.

Typical checks:

  • Accessibility – Have you included image alt text, audio alerts, visual controls, transcripts, colour coding, etc.?

Security testing

Security testing ensures your website is free from any vulnerabilities, threats and risks that could cause a big loss to your business. Types of security testing include:

Vulnerability scanning: Automated software scans a system against known vulnerability signatures.

Security scanning: Network and system weaknesses are identified and solutions for reducing these risks are consequently suggested. Both manual and automated scanning can be performed.

Penetration testing: An attack from a malicious hacker is simulated and the target system is analysed to check for potential vulnerabilities exposed by the attempted attack.

Risk assessment: The security risks inherent to the organisation are assessed and categorised. Recommendations to reduce these risks are also provided.

Security auditing: This is an internal inspection of applications and operating systems for security flaws. An audit can also be done via line by line inspection of code.

Ethical hacking: Unlike malicious hackers who steal for their own gains, the intent is to expose security flaws in the system.

Posture assessment: This combines security scanning, ethical hacking and risk assessments to show an overall security posture of a website.

Typical checks:

  • Authentication – How do you verify your users?
  • Confidentiality – What are your privacy settings?
  • Compliance – Do you comply with regulations like the GDPR?
  • Authorisation – How do you limit/control user privileges? What security measures are in place to protect your website from online threats?

Crowdsourced testing

Crowdsourced testing uses remote testers who perform website testing through an online platform. This provides greater test and device coverage, cost efficiencies and can usually be delivered far quicker than traditional methods.

Typical checks:

  • Everything – Crowdsourced testing can be used across all the types of testing in this section. However, most businesses use it to increase coverage during functional and localisation tests.

Can we not just automate website testing?

With more than 2,000 devices and 800 browser-OS combinations available, website testing is now a major undertaking. It may be tempting to automate your tests to get rid of the hard work – but this could be a false economy.

What is automated testing?

Automated testing relies on pre-scripted tests that run automatically to compare actual results with expected results. In many ways, the term ‘testing’ is misleading because automation tests actually just check your website performs as it should, for a selection of pre-defined scenarios.

Automated testing can carry out repetitive tasks and regression tests after your website is updated.

Change your business in 30 minutes - book a Demo

What’s wrong with automated testing?

In the right circumstances, automated testing is a valuable tool to complement your manual testing methods.

However, automated testing cannot pick up on the little (and sometimes big) problems that real people will see. Everything from word choice to colour matters, and this is what human testers will naturally highlight.

What’s more, if you run 100 automation scripts and 25% fail, you will still need human tests to figure out what’s gone wrong. So, you will still need to pay for human testing resources, negating the business case for automated testing. You also need to hire individuals to write these automated test scripts, which places a further drain on your time and resources as a business.

Manual testing allows for human observation, which is more useful when boosting the UX and usability of your website. It is also more suited for user interface testing, exploratory tests and ad hoc testing – where a QA specialist randomly tests your website, trying to break it and discover flaws.

How do you get started with website testing?

Website testing isn’t a one-off transaction; you need to adopt a long-term solution for your business.

So, how do you kick off your website testing strategy?

Outsourced or internal testing?

The decision to outsource versus employ an in-house testing team is one which depends on the specifics of your business and/or its projects. It’s impossible to give an algorithm that every company should use to decide what to do when making this decision.

However, the situation isn’t as black and white as it seems. Depending on the project you have in mind, you could use either an external or internal team – or you could use both. A small group of in-house testers could complement an outsourced company, for example. Or you could outsource a small project in its entirety to see if this approach works for your business.

Whatever choice you make, make sure you have weighed up the pros and cons, and the effect any testing projects could have on your internal teams and their respective projects.

Is outsourced website testing secure?

Confidentiality is a real concern when using remote testers. Leading organisations in the field such as Digivante have been delivering this testing for many years without ever having a breach of confidentiality. This can be enforced through three-way contractual protection between the customer, the testing organisation and the testers themselves. 

In addition, some organisations do not just utilise freelance testers, but also have large teams of remote testers that are permanent employees and can therefore provide full professional indemnity insurance protection.

What should you expect from a testing company?

If you decide to work with an external testing company, make sure they offer you actionable insights, can cater to your specific business needs, over a range of solutions and provide a solid portfolio of work.

Good testing companies also focus on more than just finding defects. They develop enduring evidence-derived goals which aim to boost your revenue and, in turn, your ROI in the foreseeable future. Think of them less like website testing companies and more like a business development resource.

How much should you pay?

No two companies needs are the same, of course. But any company worth their salt will carefully tailor their website and app performance testing services to match your business goals and provide a very clear plan and pricing structure to reflect breadth and depth of testing need to achieve those goals. 

Accelerate Your Conversion Rate in 24 hours Pillar Page CTA

What happens next?

As you reach out to different website testing companies, find out what services they offer, request testimonials and examples of work from previous clients, and ask for comparable quotes on specific projects.

The best website testing companies will provide you with a quality service for many years, and will be recognised by their clients as proven QA professionals.

web1

Book a call

TALK TO A SOLUTIONS CONSULTANT

Find out more about how Digivante can help you optimise your digital performance, speak to a solution consultant today.

SPEAK TO A CONSULTANT