Introduction to Testing


Author	Alena Galysheva
Last Updated	03.02.2025

Basic Terminology

Test

A test is a broader concept that encompasses the entire process of verifying and validating software to ensure that it behaves as expected. The purpose of a test is to evaluate the overall functionality, performance, security, or other aspects of the software. A test may include multiple test cases, configurations, test data, and procedures to assess different aspects of the software under various conditions.

Test target

Test(ing) target is a particular element of the software that is the focus of testing activities. Individual functions, modules, integrations, or the system as a whole can be test targets depending on the scope and objectives of testing.

Test case

A test case is a specific set of steps or instructions or specific inputs that can be used to test a particular function or feature of a software application. A test case may be designed to pass or to fail. It usually includes the prerequisites, input values, and expected results when after running a test. Test cases exist in both manual and automated testing. Some of them are extremely simple, consisting of just a few steps, while others are more complex and may even include branches.

Test scenario

A test scenario is a high-level description of how a particular function or feature of an application should work. It provides a general outline of the steps to be taken for testing that function or feature and the expected results. For example, "User can log in using their credentials" is a test scenario that can be used for a social media website. A test scenario can contain multiple test cases. In this example, there could be several test cases checking valid and invalid credentials (e.g., empty field(s), incorrect username or password, using suspended user account credentials, entering a script in the fields, etc.).

Test step

A test step is a single action or operation within a test case. It represents the smallest unit of a test and outlines a specific task or interaction with the software under test. A test step should be precise and unambiguous to allow for systematic and reproducible testing. Test steps describe the actions to perform, the input to be provided, and the expected outcome. Each step is marked pass or fail based on the comparison result between the expected and actual outcome.

For example, "Enter valid username and password that are not present in the database, then click 'Sign In' button" is a test step, and "Login attempt fails. The user remains on the login page, and the email and password fields retain the previously entered values, an error message is displayed" is the expected outcome. While the example test step is long, it may be impractical to break it down into smaller steps like "Enter a valid username that is not present in the database", "Enter a valid password that is not present in the database", "Click the 'Sign In' button".

Test plan

A test plan is a document that outlines the scope, approach, and schedule of planned testing activities in the project. Typically, a test plan includes the overall objective of the testing effort, a detailed outline of how testing will be conducted, a list of features, applications, or components to be tested, testing tools used, and plans for scheduling and resource allocation across all testing stages for both testers and developers.

Test suite

A test suite is a collection of test cases related to each other. Test cases can be grouped by type of test, execution time, related features, modules or areas of application. For example, a test suite "Login Page" can have multiple test cases like "Login with valid credentials", "Login with invalid credentials", and "Login with empty fields", which are grouped based on the functionality being tested (a login page).

Testing Techniques

Testing techniques and testing types explanations are borrowed heavily (and at times completely copied) from the article Types of Testing: Different Types of Software Testing in Detail by Kalpesh Doshi at BrowserStack. Please go check it out, it is a good read.

Software testing techniques are methods used to test a software product. The most common software test techniques are black, white, and gray box testing, which are used to test the test target at different levels of detail, from a high-level view of the system's functionality to a detailed view of its internal workings.

Black Box Testing

Black box testing is a type of software testing technique that focuses on testing the functionality of a software system without any knowledge of its internal workings. This technique is used to test the system from a user's perspective, focusing on testing whether the system works as expected and meets the user's needs. The tester only knows the inputs and expected outputs of the system and does not know how the system works internally.

White Box Testing

White box testing t is a software testing technique that focuses on testing the internal workings of a software system. This technique is used to test the system from a developer's perspective, focusing on testing the system's internal logic, performance, and efficiency. The tester has complete knowledge of the system's internal details, including the code and its implementation.

Gray Box Testing

Gray box testing is a software test technique that combines the approaches of both black box and white box testing. This technique is used to test the system from a developer's perspective, where the focus is on testing some of the system's internal workings but not all. The tester has partial knowledge of the system's internal details, and this technique is often used in system integration testing.

Types of Testing

There are different types of testing that can be performed within the previously described testing techniques. Testing types are used to test different aspects of the software product and to ensure that it meets the stakeholders' requirements. The examples of types of testing may or may not be related to a certain traffic visualizer application.

Functional Testing

Functional testing is a type of software testing where the system's functionalities are tested by feeding them input and examining the output. The purpose is to ensure that the software behaves as expected, meeting specified requirements. It mainly deals with what the system does.

Unit Testing

Unit testing is a software testing type in which individual units/components are tested in isolation from the rest of the system to ensure that they work as intended. A unit refers to the smallest testable part of a software application that performs a specific function or behavior. A unit can be a method, a function, a class, or even a module. They can be run in isolation or in groups. Unit tests are typically written by developers to check the correctness of their code and ensure that it meets the requirements and specifications.

Example of Unit Testing

A developer has scripted a password input text field with its validation (8 characters long, must contain special characters.); makes a unit test to test out this one specific text field (has a test that only inputs 7 characters, no special characters, empty field).

Advantages of Unit Testing

Early detection of bugs
Simplifies debugging process
Encourages code reusability
Improves code quality
Enables continuous integration

Integration Testing

Integration testing is a testing type in which different modules or components of a software application are tested together as a group to ensure that they work as intended and are integrated correctly. The main aim of integration tests is to identify issues that might come up when multiple components work together. It ensures that individual code units/ pieces can work as a whole cohesively.

Integration testing can be further broken down to:

Component Integration Testing: This type of testing focuses on testing the interactions between individual components or modules.
System Integration Testing: This type of testing focuses on testing the interactions between different subsystems or layers of the software application.
End-to-End Integration Testing: This type of integration testing focuses on testing the interactions between the entire software application and any external systems it depends on.

Example of Integration Test

A software application consists of a web-based front-end, a back-end program that processes data, and a database that stores data. Integration tests would verify if the feedback submitted in the front-end in the text form is processed by the middleware and added to GitLab as an issue.

Advantages of Integration testing

Early detection of issues
Improved software quality
Increased confidence in the software
Reduced risk of bugs in production
Better collaboration among team members
More accurate estimation of project timelines

System Testing

System testing is a testing type that tests the entire software application as a whole and ensures that the software meets its functional and non-functional requirements. System testing is typically performed after integration testing. During system testing, testers evaluate the software application's behavior in various scenarios and under different conditions, including normal and abnormal usage, to ensure that it can handle different situations effectively.

Example of System Testing

A software application consists of a web-based front-end, a back-end program (API) that processes data, and a database that stores data. The system test for this scenario would involve the following steps:

The user accesses the front-end interface of the traffic visualizer.
The user queries for traffic data of an area.
Back-end program receives the request and processes it, including parsing the request and fetching data from the database.
Database received the query and returns the data.
Back-end program receives the data, filters and formats it, and sends a response back to the front-end.
Requested data is received by the front-end and displayed to the user.

Advantages of System Testing

Identifies and resolves defects or issues that may have been missed during earlier stages of testing
Evaluates the software application's overall quality, including its reliability, maintainability, and scalability
Increases user satisfaction
Reduces risk

End-To-End Testing

End-to-end (E2E) testing is a testing methodology that tests the entire software system from start to finish, simulating a real-world user scenario. The goal of end-to-end testing is to ensure that all the components work together seamlessly and meet the desired business requirements. Most often people use the term system testing and end-to-end testing interchangeably. However, both of them are different types of testing.

System testing is a type of testing that verifies the entire system or software application is working correctly as a whole. This type of testing includes testing all the modules, components, and integrations of the software system to ensure that they are working together correctly. The focus of system testing is to check the system's behavior as a whole and verify that it meets the business requirements.

End-to-end testing, on the other hand, is a type of testing that verifies the entire software application from start to finish, including all the systems, components, and integrations involved in the application's workflow. The focus of E2E testing is on the business processes and user scenarios to ensure that they are working correctly and meet the user requirements.

Example of End-to-End Testing

E-commerce transaction: End-to-end testing of an e-commerce website involves testing the entire user journey, from product selection to payment, shipping, and order confirmation.

Traffic visualizer: End-to-end testing of a traffic visualizer application involves testing the entire user journey: opening the traffic visualizer, selecting an area for exploration, requesting real-time traffic information, receiving real-time traffic flow, data being displayed on the map, and the user seeing real-time traffic flow data.

Advantages of End-to-End testing

Allows you to test real world scenarios
Comprehensive testing
Improves quality

Monkey Testing

Monkey testing is a testing type where the tester tests the application with random inputs to analyze if the application breaks. The objective of monkey testing is to verify if an application crashes by giving random input values. There are no special test cases written for monkey testing.

Example of Monkey Testing

A tester accesses the web-base front-end and randomly clicks on the page and navigation links, and submits random inputs to the fields.

Advantages of Monkey Testing

Does not require extensive knowledge
Ensures reliability
Used to identify bugs that cannot be discovered through traditional methods
Cost Effective

Smoke Testing

Smoke testing is a testing type that is conducted to ensure that the basic and essential functionalities of an application or system are working as expected before moving on to more in-depth testing.

Example of Smoke Testing

Smoke testing for login will check whether the login page is accessible and if the users can log in.

Advantages of Smoke Testing

Quick Feedback
Early detection of defects

Acceptance Testing

Acceptance Testing verifies whether a software application meets the specified acceptance criteria and is ready for deployment. It is usually performed by end-users or stakeholders to ensure that the software meets their requirements and is fit for purpose.

Example of Acceptance Testing

Conducting tests to meet if an app meets the requirements of the user. For a traffic visualizer application, acceptance testing would involve testing the application for displaying accurate real-time traffic, seeing traffic hotspots, filtering traffic data, being able to use the web application on mobile devices, etc.

Advantages of Acceptance Testing

Increased stakeholder engagement
Reduced risk
Reduced development costs
Improved customer satisfaction

Sanity Testing

Sanity testing is a testing type that is performed to quickly determine if a particular functionality or a small section of the software is working as expected after making minor changes. The main objective of sanity testing is to ensure the stability of the software system and to check whether the software is ready for more comprehensive testing.

Example of Sanity Testing

After fixing a bug that caused a crash in a mobile application, you can perform sanity testing by opening the app and ensuring that it does not crash anymore.

Advantages of Sanity Testing

Quick and efficient
Saves time and effort
Cost-effective

Non-Functional Testing

Non-functional testing evaluates aspects other than specific behaviors or functions of the system. It assesses the system's performance, reliability, scalability, and other non-functional attributes. This type of testing is equally critical as it ensures the software's overall quality and user experience.

Security Testing

Security testing is a type of software testing that assesses the security of a software application. It helps to identify vulnerabilities and weaknesses in the system and ensure that sensitive data is protected.

Examples of security testing include penetration testing, vulnerability scanning, and authentication testing.

Advantages of Security Testing

Improved system security
Protection of sensitive data
Compliance with regulations

Performance Testing

Performance testing is a type of software testing that assesses the performance and response time of a software application under different workloads. It helps to identify bottlenecks in the system and improve the performance of the application.

Examples of performance testing include load testing, stress testing, and volume testing.

Advantages of Performance Testing

Increased customer satisfaction
Better scalability
Improved user experience

Load Testing

Load testing is a type of performance testing that assesses the performance and response time of a software application under a specific workload. It helps to identify the maximum capacity of the system and ensure that it can handle the expected user load.

Examples of Load Testing

Simulating the expected average number of concurrent users accessing the traffic visualizer website at the same time or performing multiple queries within the same time frame on the database and tracking how the system responds to it.

Advantages of Load Testing

Improved system reliability
Better scalability

Stress Testing

Stress testing is a type of performance testing that assesses the performance and response time of a software application under extreme workloads. It helps to identify the system's breaking point and ensure that it can handle unexpected workloads.

Examples of Stress Testing

Simulating thousands of users accessing a website simultaneously or performing millions of queries on a database and tracking how the system responds to it.

Advantages of Stress Testing

Improved system reliability
Better preparedness for real-world scenarios
Better scalability

Volume Testing

Volume testing is a type of testing that assesses the performance and response time of a software application under a specific volume of data. It helps to identify the system's capacity to handle large volumes of data.

Examples of Volume Testing

Inserting large amounts of data into a database or generating large amounts of traffic to a website and tracking how the system responds to it.

Advantages of Volume Testing

Improved system reliability
Better scalability

Scalability Testing

Scalability testing evaluates the software's ability to handle increasing workload and scale up or down in response to changing user demands. It involves testing the software system under a range of different load conditions to determine how it performs and whether it can handle increasing levels of traffic, data, or transactions.

Examples of Scalability Testing

Testing a website by gradually increasing the number of simulated concurrent users accessing the website and tracking how the system responds to it.

Advantages of Scalability Testing

Optimize system performance
Better scalability

Endurance Testing

The goal of endurance testing is to identify how well a software system can handle a workload over an extended period of time without any degradation in performance or stability. It involves simulating a normal or average workload or traffic scenario over a period of a few weeks to months.

Examples of Endurance Testing

Testing a website for performance with normal or average user traffic over an extended period.

Advantages of Endurance Testing

Identifies long-term performance issues
Reduces downtime
Enhances user experience

Usability Testing

Usability testing is focused on evaluating the user interface and overall user experience of a software application or system. It involves testing the software with real users to assess its ease of use, learnability, efficiency, and overall user satisfaction.

Exploratory Testing

Exploratory Testing is a software testing type that is unscripted, meaning that the tester does not follow a pre-defined test plan or test case. Instead, the tester relies on their own expertise, intuition, and creativity to explore the software and find defects.

Example of Exploratory Testing

A tester testing for different actions, such as tapping different buttons, swiping screens, and inputting different types of data, etc. The tester might look for crashes, freezes, errors, and unexpected behaviors throughout the exploration.

Advantages of Exploratory testing

Exploratory testing allows testers to be more flexible.
Exploratory testing can often be more time-efficient
Used to test real world scenarios

UI Testing

User interface testing is a type of software testing that focuses on testing the graphical user interface (GUI) of an application. The purpose of user interface testing is to ensure that the application's GUI is functioning correctly and meets the requirements and expectations of end-user

Examples of User Interface Testing

Identifying visual bugs in the layout, design, color scheme, font size, and placement of buttons.

Advantages of User Interface Testing

Identifying visual bugs
Reduced development costs
Increased productivity
Increased usability

Accessibility Testing

Accessibility testing is a type of testing that is focused on evaluating the accessibility of a software application or system for users with disabilities. It involves testing the software with assistive technologies, such as screen readers or voice recognition software, to make sure that differently abled users are able to access and use the software application effectively.

Examples of Accessibility Testing

Testing a website with a screen reader to ensure that the website is compatible with screen readers and its content is accessible via text-to-speech.

Advantages of Accessibility Testing

Accessibility benefits not only those with specific needs but enhances the overall user experience for everyone.

Compatibility Testing

Compatibility testing evaluates the compatibility of a software application or system with different hardware, software, operating systems, browsers, and other devices or components.

Browser Testing

Cross browser testing is a type of software testing that ensures a web application or website works correctly across multiple browsers, operating systems, and devices. It involves testing the website's functionality, performance, and user interface on different web browsers such as Google Chrome, Mozilla Firefox, Microsoft Edge, Safari, Brave, and Opera, among others.

Examples of Cross Browser Testing

A tester testing on Google Chrome, Firefox and Brave can identify issues that might arise in a particular browser.

Advantages of Cross Browser Testing

Increased customer satisfaction
Enhanced brand reputation
Early detection of issues
Improved website traffic and conversion

Cross Platform Testing

Cross-platform testing is a testing type that ensures that an application or software system works correctly across different platforms, operating systems, and devices. It involves testing the application's functionality, performance, and user interface on different platforms such as Windows, macOS, Linux, Android, iOS, and others.

Examples of Cross Platform Testing

A tester performs cross-platform testing to ensure that the web-based front-end works correctly on different devices, such as desktops, tablets and smartphones.

Advantages of Cross Platform Testing

Improved software quality
Competitive advantage
Improved market reach

Other Types of Testing

Regression Testing

Regression Testing is a software testing type that ensures that changes or modifications to an existing software application do not introduce new defects or negatively impact existing functionality.

API Testing

API testing is the process of testing the functionality, reliability, performance, and security of an application programming interface (API). An API consists of protocols, routines, and tools for building software applications.

Data Driven Testing

In this type of testing, testers use different data sets to validate the software system's behavior under different conditions.

Localization Testing

This type of testing ensures that the software system can be easily adapted to different demographics. Language in different language versions of the website is tested here.

Parallel Testing

This type of testing involves executing the same test cases on multiple systems simultaneously. It helps in identifying performance issues and defects that are specific to certain configurations.

Testing and Agile

Note: Everything coming after this point is partly author's personal opinion and derived from personal experience working as a tester, so take it with a pinch of salt.

Agile testing is not treated as a separate phase but rather as an ongoing activity carried out in collaboration with developers. Team Overflow of WIMMA Lab 2023 used the following testing approach:

Epic Level Testing

Identify specific testing targets at the beginning of the sprint and initiate test planning.
Perform regression testing for user stories completed in the previous sprint.
Conduct a comprehensive review of all features at the end of the sprint.
Execute smoke testing to ensure the stability of the software application.
Automate regression and smoke testing processes.

Feature Level Testing

Test the feature once all related user stories are completed.
Perform integration and end-to-end testing.
Validate the feature against the specified requirements and acceptance criteria.
Conduct accessibility testing.

User Story Level Testing

Validate each user story against its acceptance criteria.
Manually test each user story.
Include at least one acceptance criteria for scenarios where the expected outcome is "this should not happen" or "this should not work."

Manual Testing and Automated Testing

Manual testing is a type of testing that involves a human tester performing the testing process manually, where the tester executes the test cases without the help of any automation tools. The tester manually verifies the functionality of the software by following a set of steps and checking the results against the expected outcomes. Manual testing is usually done in the early stages of the software development life cycle (SDLC), such as during the testing phase or even during the design phase. Manual testing can be time-consuming and can require a lot of resources, but it is useful for identifying defects and issues that may not be easily detected by automation testing.

Automation testing, on the other hand, involves using automation tools to execute test cases. Automation testing is usually done after the manual testing is completed, and the software is stable enough to be tested automatically. The tester uses automation scripts to automate repetitive tasks, and the tools can perform the test cases much faster than manual testing. Automation testing is useful for testing large and complex systems and for testing software that requires repetitive testing. Automation testing can save time and resources, but it can be expensive to set up and maintain the automation framework.

The decision to use manual testing or automated testing depends on various factors, and each has its own strengths and weaknesses. Here are some considerations:

Use Manual Testing When

Exploratory Testing: Manual testing is excellent for exploratory testing, where the tester actively explores the application to identify defects.
Usability Testing: For aspects like user interface (UI) and user experience (UX) testing, manual testing provides a more human perspective.
Early Development Stages: In the early stages of development when the application is not stable, manual testing can adapt to frequent changes.
One-Time or Infrequent Testing: For scenarios where the cost of automation would outweigh the benefits, such as for small projects or one-time tests.

Use Automated Testing When

Repetitive Tests: Automated testing is ideal for repetitive tests that need to be executed frequently, such as regression testing.
Large Volume of Data: When dealing with a large volume of data, automated tests can quickly process and validate extensive datasets.
Performance Testing: For simulating many concurrent users accessing the system simultaneously, automated tests are more efficient.
Stability of the Application: Once the application stabilizes, and the features are not changing frequently, it makes sense to invest in automation.
Regression Testing: Automated testing is excellent for running repeated, frequent tests, especially after code changes.
Load Testing: Automated tools can simulate thousands of virtual users to check the system's behavior under different loads.

Ultimately, it's best to use a combination of both manual and automated testing often in a software project.

Tasks and Responsibilities

As a tester, navigating your role as a team member can be challenging. Since it's typically impossible to test everything in the software product, it's essential for you to be able to prioritize your tasks and know your area of responsibility.

Tester Responsibilities

Bug Identification: Uncover and report bugs to contribute to product improvement.
Test Case Creation: Craft comprehensive test cases to ensure thorough coverage.
Test Execution: Methodically execute test cases to assess product functionality.
Documentation: Document test cases for future reference and clarity.
Product Assessment: Conduct qualitative and quantitative evaluations aligning with customer requirements.

Not Tester Responsibilities

Bug Resolution: Leave the fixing of identified bugs to the development team.
Unit Testing: Allow developers to handle the intricate details of unit testing.
Black Box Testing: General black box testing responsibilities fall outside your domain.
Requirement Specification: Avoid direct involvement in drafting feature requirements.
Feature Acceptance Criteria: Let the team define and finalize feature acceptance criteria.
User Story Acceptance Criteria: Similarly, let the project manager handle user story acceptance criteria.
Overall Software Quality: While you identify bugs, the responsibility for overall software quality lies beyond your scope.

Maybe Tester Responsibilities

Root Tracking: Consider exploring root causes of identified issues if time permits.
API Testing: Discuss with developers whether testing team or development team should be responsible for API testing.
Feature Alignment: Work with project manager to ensure features align with the specific requirements of user stories.
Security Testing: If skilled, engage in security testing; otherwise, leave it to dedicated security experts.

Advice to Testers

Understand App Goals: Instead of just testing, immerse yourself in the app. Grasp its goals, understand each feature's purpose, and align your testing strategy accordingly for efficient results.
Master Time Management: Prioritize tasks wisely, recognizing that you can't test everything. Efficient time management is crucial for effective testing.
Acknowledge Your Importance: Recognize the significance of your role. Developers may overlook aspects, and your work is vital for verifying and validating software quality.
Clarify Ambiguities: Know your testing target well. If uncertain about functionality, ask questions. A clear understanding ensures precise testing.
Foster Collaboration: Avoid a tester versus developer mentality. Work collaboratively as a team to deliver a high-quality end result.
Organize and Document: Keep your work organized and documented. This practice aids navigation in a growing test base and enhances communication within the team.
Craft Detailed Bug Reports: When reporting bugs, provide detailed information. Help your team and developers by eliminating guesswork about your testing environment.
Define Test Objectives: Understand the questions your tests aim to answer. Your testing techniques serve as a repository of answers, and defining the questions enhances their effectiveness.
Maintain Positivity: Approach testing with a positive mindset. Expect challenges, and by seeking critical bugs, you may uncover additional issues and contribute positively to the team.