NLP Strategy

The idea of this strategy is to apply natural language processing techniques to improve the testing of REST APIs. Given an OpenAPI specification, some additional rules are extracted based on the human-readable part of the specification. Consider, for example, a particular specification that describes the parameters of a particular request. We may, for example, have the parameter language to which is associated a description in which the accepted values are made explicit. Similarly, we may have a count parameter to which, again, is associated a description field in which it is specified that the maximum value is 50. In these two cases, a normal automated testing tool could miss these characteristics and thus create unsuitable or failing test cases.
In addition to this, it is also possible that natural language descriptions within the specification can identify inter-parameter dependencies that would otherwise not be detected.

It should be noted that the set of rules extracted through NLP techniques may also contain some incorrect ones due to natural language ambiguity. To mitigate this problem, this strategy includes a validation phase that verifies the NLP-extracted rules with the API implementation, creating and executing validation test cases, and discards rules that fail this validation. During this rule validation phase, validity checks are performed. In some cases, validation may also lead to corrections to the extracted rules. Rule validation consists of two phases: the static pruning phase (in which rule combinations are discarded by statically analysing the compatibility between the rules) and the dynamic pruning phase (in which rule combinations are dynamically validated by sending requests to a distributed instance of the API and checking the resulting HTTP responses).

Once this is done, a fine-tuning phase is performed to further assess the validity of the rules. In this phase, a strategy is applied that reproduces the successful requests identified in the previous phase by applying a series of mutations to further validate each potentially valid rule. For example, to validate rules of the required type, the request is reproduced twice, with and without the required parameter. If the request without the parameter is also successful, it is discarded. To validate if-then rules, the request is played twice, once applying the predicate and once applying its negation. If the first request succeeds and the second is rejected, the rule is validated as valid. In addition to validating rules, fine tuning can also attempt to repair the inter-parameter dependency rules Or, OnlyOne, AllOrNone and ZeroOrOne.