User Acceptance Criteria Explained

What are Acceptance Criteria?

A development project can only be considered a success if the product owner is happy with the delivered product. How this happiness is measured is the essence of the acceptance process. Ultimately the product owner will be satisfied if the product does exactly what they wanted it to do. An acceptance criteria document is how the developers can demonstrate that they have achieved the success criteria for this goal and met the exit criteria.The acceptance criteria are essentially the tests that an application must pass to demonstrate that it has met all the user requirements. This article provides a view of the importance of acceptance criteria from the project management perspective. We will look at:

• When they are needed
• Why they are important
• Who should define them
• When they would be written
• How they would be written

When are Acceptance Criteria needed?

Their use marks the point of development where the user story is finished satisfactorily. User stories are typically produced as natural language descriptions that are open to interpretation rather than precise mathematically based expressions. The user stories represent the product owners' requirements expressed in a form that they can understand. These requirements may consist of a set of conditions that the user requires the end product to satisfy.  It may also include established standards that the end product must comply with. The acceptance criteria define how these requirements will be shown to have been satisfactorily implemented in the delivered product.

Why are Acceptance Criteria Important?

The criteria need to reflect what the customer actually wanted rather than what the developers thought they wanted. This is the opportunity to identify that the user story was correctly interpreted into the product's functional requirements. Typically, user stories can be vague and open to interpretation if not correctly defined. It is possible for functional requirements to be compliant with user stories but not reflect their intent. The acceptance criteria lay down precisely what is expected. This is essentially the difference between verification and validation:

• The agile development process includes testing to make sure the product works correctly. This is the verification process.
• The acceptance testing makes sure the product does what the customer wants it to do. This is the validation process.
• The acceptance testing demonstrates that the approval conditions have been met.

1. Typically, project verification criteria include:

• All code must be reviewed and tested.
• All unit tests must pass.
• Code coverage must exceed the minimum requirement.
• Code must comply with static code rules.

2. Typically, project validation criteria are simply approval conditions that must be met.

• The effort expended on project verification generally significantly outweighs the effort spent on validation.
• While developers understand that the verification lays the foundations for validation, the product owner may not appreciate the dependency.
• The validation process using the acceptance criteria must provide the product owner with the confidence that the product does what they want it to do.

3. Correct criteria deliver benefits

• Software-based projects most often fail because of poor requirements. 
• Ambiguities, omissions, and errors in the user requirements translate directly to a failed product.
• Product owners are not typically experts in requirements definition. 
• Acceptance criteria provide a mechanism for all stakeholders to fully understand product requirements, identify issues, and resolve them before the development process is compromised. 

Who defines the Acceptance Criteria?

Ultimately the product owner is responsible for defining the criteria. They must reflect the end customers' requirements. It must be done by someone independent from the development process. This is to avoid assumptions and design decisions made by developers influencing the approval conditions. However, the development team's input can help identify unverifiable criteria or highlight criteria that conflict with user requirements or are not derived from a user requirement.

A typical process for an agile development process would be:

• The product owner adds a user story to the sprint backlog
• The product owner creates acceptance criteria for the user story
• Stakeholders review user stories and associated criteria
• Development team review user stories and related criteria
• Development implement user story in the product
• The test team implement acceptance criteria as acceptance test cases
• Acceptance testing validates the developed product

When should the Acceptance Criteria be produced?

The approval conditions can be produced at any point in the development lifecycle up until the point when acceptance testing is required. However, the best value comes when acceptance conditions are written and agreed upon before developing use cases. The user stories used by the development team are often open for interpretation. Developers make assumptions that may not necessarily align with the product owners' expectations. Having defined user conditions for acceptance in place during the requirements development phase can provide guidance to developers. They also clearly set out the exit criteria for the development phases' end goals and the definition of done that drive task scheduling and prioritization. This benefits the product owner in ensuring the development delivers what they require. This also helps the development team in establishing the critical requirements and functionality scope of the product. The conditions for acceptance also clearly define the endpoint for the product development, essential in planning iterative development methodologies. If development follows an agile-based methodology, user acceptance criteria will need to be reviewed as part of the sprint process. This will ensure any impact from reprioritization decisions made in scrums affecting user stories is reflected in the user stories acceptance criteria.

How Should Acceptance Criteria be Written?

• They must be written so that they are verifiable. They must be in a form that can be translated into automated acceptance test cases that result in a pass or fail condition.
• They must be clear, concise, and unambiguous. All stakeholders, from end-user to developers, must have a common understanding.
• They must be derived from the user's perspective. They should test what the user sees and not be influenced by how the product is implemented.
• They must be sufficiently general so they do not influence how the product is implemented.
• They must be technology agnostic. They should not change if the end product is migrated to a different platform or programmed in a different language.

Examples of unverifiable acceptance criteria for user stories include:

• The user interface must be intuitive,
• The calculation of the shopping cart total must be fast,
• The amount of memory used should be minimized.

Typically, a user story, unless very simple, will need multiple conditions for acceptance to fully demonstrate it has been satisfied. Suppose a user story generates an excessive number of acceptance criteria. In that case, this provides a good indication that the user story is too complex and should be decomposed into several shorter stories. A good rule of thumb is no more than three or four acceptance criteria per user story. This provides an excellent opportunity to review user stories' quality and help make development more efficient by keeping user story complexity and sprint size manageable. This aids scrum productivity and reduces the likelihood of development errors.

Acceptance Criteria Structure

1. User Story Derived Criteria

Where user stories are written using the user/action/outcome format, corresponding user stories acceptance criteria can be written using a given/when/then form. In this case, the user story is written to define that the intended user wants to perform an intended action that produces the following outcome. The corresponding acceptance criteria are then written to define that 'given' an initial state, 'when' the intended user takes an action 'then' the following outcome occurs.

These conditions for acceptance can be defined as a test scenario with the following attributes:

• The user story the approval conditions relate to
• The product behavior that is being tested in the scenario
• The 'given' conditions that define the initial state of the scenario being tested
• The 'when' conditions that define the specific action that the user makes
• The 'then' conditions represent the expected outcome of the 'when' action
• If there are multiple options for the given, when or then conditions, the approval conditions may comprise multiple statements linked with terms such as 'or,' 'and,' 'if.'

Acceptance criteria for user stories should cover both positive and negative scenarios.

• Given an initial state when the user takes a predefined action, then the correct outcome occurs.
• Given an initial state when the user takes an undefined action, then no outcome should occur.

An example of this is the user stories for a customer adding items to an eCommerce shopping cart. The acceptance criteria for user stories should include a user attempting to add zero items and the expected actions of adding one or more items.

2. Verification Checklists

Suppose a user story is defined as a checklist. In that case, user approval conditions can be written for each checklist entry to verify that the list item has been satisfied.

3. Feature Testing

Product requirements may include a list of features not expressed as user stories or a checklist. User acceptance criteria can be produced as natural language test cases for each element.

Acceptance Criteria Rules

• All user requirements, functional and non-functional, defined in the product use cases must be within the approval conditions' scope.
• Every user story, checklist, and other user requirement formats must have at least one condition for acceptance.
• They must be defined and agreed upon before the associated user requirement is implemented.
• They must be verifiable with a clear pass/fail indication.
• They must not influence implementation decisions.
• They must not be written solely by the developers. All stakeholders should be involved, and the product owner must have responsibility.

Acceptance Criteria Examples

User story 1: As a returning website visitor, I want to log into my account

The scenario for the Acceptance Criteria is testing that a logged-out user can log in.

• As a logged-out user
• I want to log in to the website
• So that I can access my account

The Acceptance Criteria for this user story is created using the given/when/then scenario-oriented approach

Scenario: User logs in using valid credentials

• Given I'm a logged-out user
• And I'm present on the sign-in page
• When I enter the correct username and password credentials into the fields on the webpage
• And I click the login button
• Then I am logged into my account

User story 2: I want to change my password

The scenario for the Acceptance Criteria is testing that a logged-in user can change their password.

• As a logged-in user
• I want to enter a new password
• So that I can change my password

The Acceptance Criteria for this user story is created using the given/when/then scenario-oriented approach

Scenario 1: User enters a new valid password

• Given I'm a logged-in user
• And I'm present on the change password page
• When I enter the correct old password credentials and a valid new password into the fields on the webpage
• And I click the change password button
• Then the password for my account is changes
• And I receive an email notification of the change

Scenario 2: User enters a new invalid password

• Given I'm a logged-in user
• And I'm present on the change password page
• When I enter the correct old password credentials and an invalid new password into the fields on the webpage
• And I click the change password button
• Then the password for my account does not change
• And I receive notification that the new password is invalid

Scenario 3: User enters an invalid old password and a new valid password

• Given I'm a logged-in user
• And I'm present on the change password page
• When I enter incorrect old password credentials and a valid new password into the fields on the webpage
• And I click the change password button
• Then the password for my account does not change
• And I receive notification that the old password is incorrect

More examples are available on the General Services Administration (GSA) Office of the Chief Technology Officer (CTO) team's website.

Conclusions

Acceptable criteria fulfill two distinct purposes: 

1. They provide assurance to the product owner that their product does what they want it to do. 
2. They also provide explicit instruction to the development team about the development's end goals and reduce ambiguity in the user's functional requirements.

This article provides a view of how acceptance criteria aid the software release management process. Enkonix will be happy to help you with this process at any stage, just send a message and we’ll contact you with new ideas.