What is a Test Coverage

Idea of test coverage is to give you confidence while quickly verifying whether the change will break things.

Outside-In Testing

Starting from high-level tests and working way down to lower-level concerns.

You start high level by implementing a broad feature, then you drill down to lower levels that are failing because the test for the lower level feature is missing.

Once you get to the lowest level, you drill back up to feature level to ensure entire testing suite is working.

Different Types of Tests

Unit test

Tests a single unit in isolation.

• a class
• A unit test will create an object, invoke a method, and check a result
• It answers the question "does my code do what I intended it to do?"

Even if all unit test with 100% coverage do what they are supposed to do, there is no guarantee that the complete system works as designed.

Unit Test is Outside In

you don't need to go through the entire application to validate a component

Integration test

Tests the combination of two components in the system

• It is focused on the relationship between the components, not the components themselves
• It answers the question "do these components work together as intended when connected"

System test

Tests the whole software system

• It answers the question "does this software work as intended?"

Acceptance test

Automated way for the customer answer the question "is this software what I think I want?"

• It is kind of a system test

All automated tests are limited by axiom "End-to-end is further than you think" - meaning, eventually a human has to sit down in front of a computer and look at user interface to make sure something is working as expected

Outside-In

• high level tests - perspective of user or customer of the web application
• high level interactions - going to website and adding items to shopping cart and submitting order
• flow of the application and large steps (behaviors throughout the entire application)

E2E Testing

E2E testing is a technique that tests the entire software product from beginning to end to ensure the application flow behaves as expected. It defines the product's system dependencies and ensures all integrated pieces work together as expected.

E2E tests from the end user's experience by simulating the real user scenario and validating the system under test and its components for integration and data integrity.

vs Acceptance

Acceptance test is a phase in testing which may include E2E as part of itself.

Acceptance testing is more geared towards user and visual aspect of the application whereas E2E is more focused on complexity of the application and involves all layers of the application to interact with each other during the test execution.

Differences

Unit tests are faster and easier to write, faster to run, and easier to diagnose. They don't depend on "external" elements like a file system or a database, so they are much simpler/faster/reliable. Most unit tests continue to work as you refactor (and good unit tests are the only way to refactor safely). They absolutely require that your code be decoupled, which is hard, unless you write the test first. This combination of factors makes the Red/Green/Refactor sequence of TDD work so well.

System tests are hard to write, because they have to go through so much setup to get to a specific situation that you want to test. They are brittle, because any change in the behavior of the software before can affect the sequence leading up to the situation you want to test, even if that behavior isn't relevant to the test. They are dramatically slower than unit tests for similar reasons. Failures can be very difficult to diagnose, both because it can take a long time to get to the point of failure, and because so much software is involved in the failure. In some software, system tests are very difficult to automate.

Integration tests sit in between: they are easier to write, run, and diagnose than system tests, but with broader coverage than unit tests.