Share

Stop Writing Requirements Nobody Can Test

Vague requirements survive reviews—until someone has to verify them. See how words like “fast,” “robust,” and “where possible” can be turned into measurable engineering requirements using EARS and better requirement-quality analysis.

Stop Writing Requirements Nobody Can Test

There is a particular kind of requirement that looks perfectly acceptable right up until somebody has to prove that it was met.

It usually sounds something like this:

The system shall provide a fast and user-friendly interface.

Nobody objects when it appears in the first draft.

The customer understands what they meant.

The systems engineer understands what they meant.

The software team thinks they understand what it means.

The requirement receives an ID. It goes into the specification. Eventually, it gets baselined.

Six months later, somebody in verification asks a very simple question:

What constitutes a pass?

And suddenly nobody agrees.

Is 500 milliseconds fast?

Two seconds?

Five?

Does "user-friendly" mean an inexperienced operator can use it without training? That an experienced operator can finish a task quickly? That the buttons are large? That the interface looks modern?

Someone says, "We can work that out during testing."

That is exactly backwards.

If the test team has to decide what the requirement means, the requirement was never finished.


The Tester Is Often the First Person Forced to Ask the Right Question

There is a useful habit when writing requirements:

Pretend the system already exists.

Now imagine handing the requirement to someone who did not attend the stakeholder meetings, did not participate in the design discussions and cannot ask you what you intended.

Their job is to produce objective evidence that the requirement has been satisfied.

Could they do it?

If the answer is no, you probably do not have a verification-ready requirement yet.

ISO/IEC/IEEE 29148 treats verifiability as a fundamental characteristic of a well-formed requirement. NASA guidance similarly calls for requirements to be clear, unambiguous, measurable where appropriate and capable of being verified. NASA also recommends identifying the verification approach while requirements are being developed rather than waiting until the product is complete [1], [2].

And technically, "testable" is not quite the whole story.

Not every requirement should be verified by a physical test.

Depending on the requirement, verification may use:

  • test,
  • analysis,
  • inspection,
  • demonstration, or
  • some combination of them.

NASA explicitly distinguishes these verification methods [2].

So the better question is not simply:

Can I test this?

It is:

What objective evidence would convince us that this requirement has been satisfied?

That question exposes bad requirements remarkably quickly.


The Five Words That Should Make You Nervous

Some words are not automatically forbidden.

But when they appear in a requirement, they should make you stop.

Fast.

User-friendly.

Robust.

Minimal.

Where possible.

They sound reasonable because they communicate intent.

The problem is that they usually do not communicate an acceptance boundary.

And engineering ultimately needs a boundary.

The product either satisfies the requirement or it does not.

Let's take these one at a time.


1. "The System Shall Respond Fast"

Imagine a monitoring application used to supervise industrial equipment.

The first requirement says:

REQ-101: The diagnostic dashboard shall load quickly.

Everyone understands the desire.

Nobody can verify the requirement.

"Quickly" compared with what?

What starts the clock?

What stops it?

Does it apply when there are five connected devices or five hundred?

Does the network matter?

Does "loaded" mean that the page frame appears, or that current diagnostic data is actually available?

So perhaps the team makes its first improvement:

REQ-101: The diagnostic dashboard shall load within 2 seconds.

Better.

But we are not finished.

Two seconds under what conditions?

Now we get somewhere useful:

REQ-101: When the operator opens the Diagnostic Dashboard while the system is connected to up to 50 field devices, the application shall display the current status of all connected devices within 2.0 seconds.

Now the verification engineer has something to work with.

There is a trigger:

The operator opens the dashboard.

There is a defined system response:

Current status is displayed.

There is an operating condition:

Up to 50 connected devices.

There is a measurable performance limit:

2.0 seconds.

A test procedure can define the exact configuration, instrumentation, repetitions and pass/fail logic.

That is a very different situation from being told to verify that the system is "fast."


Notice What We Did Not Specify

We did not say:

The application shall use eight worker threads and a Redis cache to display the dashboard within 2 seconds.

Unless those technologies are genuinely imposed constraints, that would start dictating the solution.

Good requirements generally specify what must be achieved, along with the conditions and performance boundaries necessary to prove it, without unnecessarily deciding how the design must achieve it [1], [2].

Measurable does not mean over-specified.

That distinction matters.


2. "The Interface Shall Be User-Friendly"

This one appears everywhere.

REQ-205: The maintenance interface shall be user-friendly.

The problem is not that usability is unimportant.

It is exactly the opposite.

Usability is important enough that we should define what good usability actually means for this system.

ISO 9241-11 treats usability in the context of actual use rather than as a vague visual property of an interface [3].

So ask:

User-friendly for whom?

A first-time consumer?

A trained technician?

An operator using the system eight hours a day?

Someone wearing gloves?

Someone working under time pressure?

Then ask:

What are they trying to accomplish?

Suppose the real stakeholder concern is that technicians take too long to perform a common maintenance configuration.

Instead of:

The maintenance interface shall be user-friendly.

we might end up with separate requirements such as:

REQ-205: The maintenance application shall permit a trained maintenance technician to complete the standard sensor-replacement software workflow within 3 minutes, excluding physical removal and installation of the sensor.

And perhaps:

REQ-206: The maintenance application shall permit at least 90% of representative trained maintenance technicians to complete the standard sensor-replacement software workflow without assistance.

Those requirements are not perfect merely because they contain numbers.

The numbers still need justification.

Why three minutes?

Why 90%?

Who counts as a representative technician?

What training is assumed?

Those questions should be answered through stakeholder analysis and validation.

But now we are discussing engineering.

The original phrase "user-friendly" allowed ten people to imagine ten different systems.

The revised requirements begin turning the stakeholder's intent into observable outcomes.


A Number Is Not Automatically a Good Requirement

This is worth emphasizing.

Engineers sometimes learn that vague requirements are bad and respond by attaching arbitrary numbers to everything.

That is not requirements engineering.

That is numerically precise guessing.

If "fast" becomes "500 ms," someone should be able to explain why 500 ms matters.

Perhaps an operator must respond within a certain time.

Perhaps another subsystem expects the result.

Perhaps the limit comes from an external interface.

Perhaps user testing found that longer delays interrupt a workflow.

Perhaps 500 ms is simply the engineering margin selected against a one-second stakeholder expectation.

Whatever the reason, capture it.

NASA's requirements guidance specifically recommends documenting rationale and assumptions because the reason behind a requirement can otherwise disappear as the system evolves [2].

A requirement without rationale can survive long after the reason for it has vanished.

Then the number becomes sacred simply because nobody remembers where it came from.


3. "The System Shall Be Robust"

Here is another favourite:

REQ-310: The communication system shall be robust.

It sounds technical.

It is still nearly useless as a requirement.

Robust against what?

Packet loss?

A disconnected cable?

High traffic?

Corrupted input?

Low voltage?

Temperature?

An unavailable server?

Unexpected operator input?

Component failure?

"Robust" often hides an entire family of requirements.

Suppose what the stakeholder actually means is:

"A short network interruption shouldn't make us lose data."

That can become something much more useful:

REQ-310: If network connectivity is unavailable for no more than 60 seconds, then the gateway shall retain all telemetry generated during the interruption.

Then:

REQ-311: When network connectivity is restored following an interruption of no more than 60 seconds, the gateway shall resume transmission of retained telemetry within 5 seconds.

Maybe another requirement defines buffer capacity.

Another defines what happens after a longer outage.

Another handles corrupt data.

Another covers environmental survivability.

That is the point.

"Robust" may have been hiding six requirements inside one adjective.


This Is Where EARS Becomes Useful

The structure of those revised requirements is not accidental.

EARS — the Easy Approach to Requirements Syntax — was developed by Alistair Mavin and colleagues at Rolls-Royce and first published at the IEEE International Requirements Engineering Conference in 2009.

The idea is deliberately lightweight: gently constrain natural-language requirements using a small number of recurring patterns rather than allowing every requirement to become an improvised English sentence.

The generic EARS structure is based around clauses such as:

While <precondition>, when <trigger>, the <system> shall <response>

EARS includes patterns for ubiquitous behaviour, state-driven behaviour, event-driven behaviour, unwanted behaviour and optional features. The original work specifically targeted recurring natural-language problems including ambiguity, vagueness, complexity and untestability [4], [5].

For example:

When the operator selects START, the controller shall initiate the startup sequence.

The word when forces us to think about the trigger.

Or:

While the controller is operating in Maintenance Mode, the controller shall inhibit automatic equipment startup.

Now the applicable state is explicit.

Or:

If the controller detects an invalid configuration file, then the controller shall reject the file and generate a configuration-error event.

Now an unwanted condition and the required response are visible.

That structure makes requirements considerably easier to reason about.

But there is an important limitation.


EARS Cannot Rescue a Requirement That Has No Engineering Content

Consider this:

When the operator requests system status, the application shall display the information quickly.

That can follow an EARS-style structure and still be a bad requirement.

The trigger is clearer.

The system is identified.

The response is identified.

But "quickly" still has no acceptance criterion.

EARS helps structure the thought.

It cannot invent the missing engineering decision.

That still belongs to the engineer and the stakeholder.

This is an important distinction whenever templates, automated tools or AI are introduced into requirements engineering.

A syntactically beautiful requirement can still say almost nothing.


4. "The Product Shall Have Minimal Weight"

Now consider:

REQ-402: The portable unit shall have minimal weight.

This sounds like a requirement.

It may not actually be one.

"Minimal" compared with what?

If lower mass is simply better, then what we have may be a design objective.

The engineering team might be expected to minimize mass while balancing cost, durability, battery capacity, manufacturability and thermal performance.

That belongs naturally in a trade study.

A requirement needs a boundary.

If there is a genuine stakeholder constraint, write the constraint:

REQ-402: The portable unit shall have a mass not exceeding 2.0 kg, including its battery and standard mounting hardware.

Now 1.9 kg passes.

2.1 kg fails.

The designer is still free to produce a 1.5 kg solution if doing so makes sense.

This distinction prevents a surprisingly common problem: turning desirable optimization goals into impossible requirements.

"Minimize power."

"Minimize weight."

"Maximize reliability."

"Reduce cost as much as possible."

Those are useful design objectives.

They are terrible acceptance criteria unless someone defines the boundary.


5. "Where Possible"

And then we arrive at perhaps the most dangerous phrase of all:

REQ-501: The system shall preserve operator settings where possible.

Where possible?

Who decides that?

The systems engineer?

The software developer?

The supplier?

The person running the acceptance test?

IBM's requirements-quality guidance describes phrases such as "where possible," "if necessary" and "as appropriate" as escape clauses because they make it unclear when the requirement actually has to be satisfied [6].

Imagine a supplier saying:

"We determined preserving the settings wasn't possible in this configuration."

Did they violate the requirement?

The sentence gives you no objective answer.

So find the real condition.

Perhaps the intent is:

REQ-501: If input power is lost, then the controller shall preserve each user-configurable parameter designated as persistent in the controlled Configuration Data Dictionary.

And:

REQ-502: When the controller completes startup following a loss of input power, the controller shall restore the most recently stored value of each persistent user-configurable parameter.

Now we can discuss exactly which parameters are persistent and exactly what behavior is expected.

There is also an interesting EARS lesson here.

EARS legitimately uses Where for an optional-feature pattern.

For example:

Where the wireless communications option is installed, the gateway shall provide the wireless diagnostic interface.

That is very different from:

The gateway shall provide wireless communications where possible.

The first sentence defines a condition.

The second creates an excuse.


The Real Test: Can You Write the Verification Logic?

Before approving a requirement, try completing this sentence:

"We will know this requirement has been satisfied when..."

If you cannot finish it without inventing information that does not appear in the requirement, its rationale, its referenced data or its defined verification method, something is missing.

For:

The dashboard shall be fast.

you cannot finish the sentence.

For:

When the operator opens the Diagnostic Dashboard while the system is connected to up to 50 field devices, the application shall display the current status of all connected devices within 2.0 seconds.

you can.

You immediately know that a candidate verification could measure elapsed time from the defined operator action to the defined observable response under the specified configuration.

The detailed test procedure still has work to do.

But the requirement now contains enough engineering content for verification planning to begin.

NASA goes so far as to recommend developing a requirements verification matrix that identifies how each "shall" requirement will be verified [2].

That is a useful discipline because it forces verification problems into the open while the requirements are still editable.


Do Not Wait for the Test Team to Find This

By the time a requirement reaches formal verification, ambiguity has already had months or years to spread.

A designer interpreted "fast" one way.

The software team interpreted it another.

A supplier optimized against a third interpretation.

The test team eventually asks for a number.

Then the project discovers that agreeing on the number now requires a design change.

This is why requirement quality is not a documentation problem.

It is a design-risk problem.

The words in the specification influence architecture, interfaces, procurement, estimates, schedules, test equipment, procedures and acceptance decisions.

A vague requirement postpones a decision.

It does not eliminate the need for that decision.

Eventually somebody has to decide what "fast" means.

The only question is whether that happens while editing a sentence or while explaining a failed acceptance test.


Can Requirement Quality Be Checked Automatically?

To a degree, yes.

And this is one of the better uses of automation in requirements engineering.

A machine does not need deep knowledge of your system to notice that a requirement contains phrases such as:

  • quickly,
  • adequately,
  • sufficient,
  • user-friendly,
  • as appropriate,
  • where possible,
  • minimal,
  • optimized,
  • easy,
  • seamless.

It can also look for missing units, missing limits, compound obligations, unclear actors, weak verbs, passive constructions and other patterns associated with poor requirement quality.

This idea predates today's generative AI.

Researchers have explored automated detection of "requirements smells" as a lightweight supplement to human reviews. In one industrially evaluated approach, Femmer and colleagues reported that automated smell detection could uncover many practically relevant defects, while also showing imperfect precision — exactly why automated analysis should assist reviews rather than replace engineering judgment [7].

A broader ACM systematic mapping study identified hundreds of studies applying natural-language processing to requirements-engineering activities, with quality-defect detection forming a major area of research [8]. More recent systematic reviews continue to find significant interest in automating requirements analysis and quality assessment [9].

The opportunity is obvious.

The computer can find the suspicious words.

The engineer decides what they should mean.


Where Ngenaire Fits

This is exactly the distinction we are trying to preserve in Ngenaire.

The goal is not to have AI silently rewrite an engineering specification and declare it "good."

That would simply replace one source of ambiguity with another.

Instead, Ngenaire provides requirement-quality analysis so engineers can surface potential quality problems while the requirements are still being developed.

Requirements can be written using EARS-style patterns. Quality Analysis can be run against the requirement set. Ngenaire's AI-assisted critique can identify potential issues and provide category-based feedback and suggested rewrites for engineer review. Requirements can then live in the same engineering workspace as verification plans, test cases, execution evidence and the Requirements Verification Traceability Matrix rather than being disconnected from the evidence that will eventually prove them [10], [11].

That creates a much more useful workflow:

Write the requirement.

Analyze the wording.

Challenge the ambiguity.

Apply an EARS structure where appropriate.

Define the missing performance boundary.

Identify the verification method.

Trace the requirement to the evidence that will prove it.

And importantly, keep the engineer in the loop.

If Ngenaire flags:

The system shall respond quickly.

AI can reasonably identify that quickly is vague.

It might suggest a structure such as:

When <trigger>, the system shall <response> within <time limit>.

But it should not pretend to know whether the correct limit is 100 milliseconds, 2 seconds or 30 seconds.

That answer comes from the engineering context.

Automation should expose the missing decision.

It should not fabricate it.


A Requirement Quality Review That Takes 30 Seconds

Before releasing your next requirement, ask six questions.

1. What makes this requirement applicable?

Is there an event, operating state, mode, configuration or failure condition?

2. Who or what is responsible for the response?

Is the system element clear?

3. What must actually happen?

Could two competent engineers interpret the expected behavior differently?

4. Where is the acceptance boundary?

Is there a quantity, range, tolerance, defined state, referenced characteristic or other observable criterion?

5. Under what conditions does that boundary apply?

Load, environment, configuration, operating mode, input range and initial state can completely change the meaning of a performance requirement.

6. How will we prove it?

Test?

Analysis?

Inspection?

Demonstration?

If nobody can propose a credible verification method, do not assume the verification engineer will solve it later.

Fix the requirement.


Stop Rewarding Requirements for Sounding Technical

This may be the root of the problem.

A lot of bad requirements sound good.

The system shall provide robust communications.

Sounds professional.

The application shall provide an intuitive user experience.

Sounds customer-focused.

The design shall minimize power consumption.

Sounds engineering-driven.

The processor shall respond rapidly where possible.

Sounds flexible.

None tells us enough.

A good engineering requirement does not need to sound impressive.

It needs to survive contact with design and verification.

There should be a point at which an engineer can look at objective evidence and say:

Pass.

Or:

Fail.

And everyone understands why.

That does not mean every engineering decision should be reduced to one number. It does not mean eliminating judgment, operational context or stakeholder validation.

It means refusing to hide unresolved engineering decisions inside comfortable adjectives.

Because "fast" is not a speed.

"User-friendly" is not a usability criterion.

"Robust" is not a failure definition.

"Minimal" is not a limit.

And "where possible" is not an acceptance condition.

If the tester has to invent the meaning, you did not write a requirement.

You postponed a conversation.

Have it now.


References

[1] ISO/IEC/IEEE, Systems and Software Engineering—Life Cycle Processes—Requirements Engineering, ISO/IEC/IEEE 29148:2018, 2nd ed., Nov. 2018.

[2] National Aeronautics and Space Administration, NASA Systems Engineering Handbook, NASA/SP-2016-6105 Rev. 2, Washington, DC, 2016.

[3] International Organization for Standardization, Ergonomics of Human-System Interaction—Part 11: Usability: Definitions and Concepts, ISO 9241-11:2018, Mar. 2018.

[4] A. Mavin, P. Wilkinson, A. Harwood, and M. Novak, "Easy Approach to Requirements Syntax (EARS)," in Proc. 17th IEEE Int. Requirements Engineering Conf. (RE '09), Atlanta, GA, USA, 2009, pp. 317–322, doi: 10.1109/RE.2009.9.

[5] A. Mavin and P. Wilkinson, "Ten Years of EARS," IEEE Software, vol. 36, no. 5, pp. 10–14, 2019, doi: 10.1109/MS.2019.2921164.

[6] IBM, "Escape clause," IBM Engineering Requirements Quality Assistant Documentation, accessed Aug. 26, 2026.

[7] H. Femmer, D. Méndez Fernández, S. Wagner, and S. Eder, "Rapid quality assurance with Requirements Smells," Journal of Systems and Software, vol. 123, pp. 190–213, 2017, doi: 10.1016/j.jss.2016.02.047.

[8] L. Zhao et al., "Natural Language Processing for Requirements Engineering: A Systematic Mapping Study," ACM Computing Surveys, vol. 54, no. 3, Art. no. 55, pp. 1–41, 2021, doi: 10.1145/3444689.

[9] M. A. Umar and K. Lano, "Advances in automated support for requirements engineering: a systematic literature review," Requirements Engineering, vol. 29, pp. 177–207, 2024.

[10] Ngenaire, "Ngenaire — AI-Augmented Engineering," Ngenaire, accessed Aug. 26, 2026.

[11] Ngenaire, "Explore the sample project," Ngenaire Documentation, accessed Aug. 26, 2026.

Subscribe to Ngenaire Engineering Blog

Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe