Testing the "REST"
Software testing is almost the "last line of defence" in the software lifecycle between the software vendor and the customer. Testing is the phase that completely validates your architecture, design, and development. If not done properly, testing can carry bugs that can directly impact an organization's revenue. So huge stakes hinge on this part of the lifecycle
Web Services have now become a mainstream technology, and are steadily reaching maturity as a technology. The proliferation of Web Service technologies can be gauged from the amount of commercial and enterprise implementations that are currently taking place.
The certificate of adoption of Web Services can be gauged from the fact that two of the leading open system technologies, i.e., J2EE and the .NET Framework have now included Web Services in their standard offerings.
With all the attention that Web Services have been getting in the past year, there are remarkably few sources of information about testing Web Services, in particular REST-based Web Services. We'll try to provide some insight into the techniques and strategies for testing REST-based Web Services.
Web Services Styles (SOAP versus REST)
the most popular Web Service implementation styles for most projects would boil down to deciding between SOAP (Simple Object Access Protocol) and REST (Representational State Transfer).Enterprise
the most popular Web Service implementation styles for most projects would boil down to deciding between SOAP (Simple Object Access Protocol) and REST (Representational State Transfer).
SOAP-based Web Services
SOAP-based Web Services are based on a set of standard specifications backed by the World Wide Web (W3C) and OASIS consortia. This is the primary reason for the success of this implementation style. It has all the technology vendors agreeing to the basic set of standards that allows technologies to become interoperable. There are also other reasonably mature standards like WS-Security that address the non-functional aspects.
SOAP-based Web Services are based on a set of standard specifications backed by the World Wide Web (W3C) and OASIS consortia. This is the primary reason for the success of this implementation style. It has all the technology vendors agreeing to the basic set of standards that allows technologies to become interoperable. There are also other reasonably mature standards like WS-Security that address the non-functional aspects.
SOAP-based Web Service implementations are based on three important constituents.
The Simple Object Access Protocol
SOAP is a stateless, lightweight, XML-based, one-way message exchange protocol. It basically acts as the medium of communication between the service consumer and the service producer. SOAP specifications define an extensible framework by which application specific information can be exchanged.
SOAP is a stateless, lightweight, XML-based, one-way message exchange protocol. It basically acts as the medium of communication between the service consumer and the service producer. SOAP specifications define an extensible framework by which application specific information can be exchanged.
SOAP isn't specifically tied to a particular protocol but it's been widely used with HTTP. One of the biggest advantages of SOAP is its ability to handle high levels of complexities. It can deal with basic data types, structs, even arrays.
SOAP doesn't address the semantics of the application-specific messages used for communication between the service consumer and the provider. The SOAP skeleton contains a SOAP envelope with an optional SOAP header element and a compulsory SOAP body element. Any processing error is sent as a SOAP fault in the SOAP message.
The SOAP header element is optional. However it's used to send authentication and session information. Although this doesn't lie within the SOAP protocol, it does provide the flexibility that's exploited by standards like WS-Security. The header element is meant to encapsulate extensions to the message format without having to couple them to the payload or modify the fundamental structure of SOAP.
The SOAP body element is mandatory and contains the application specific information that has to be exchanged between the consumer and the provider. This payload could be a remote procedure call, a XML document that's interchanged.
Web Services Description Language (WSDL)
The Web Service Description Language is an XML-based interface definition and description format used to describe and publish Web Services in a standard manner. The WSDL definition lets consumers identify the location of the service, the request messages that the service requires, and the corresponding output message format that the consumer will get.
The Web Service Description Language is an XML-based interface definition and description format used to describe and publish Web Services in a standard manner. The WSDL definition lets consumers identify the location of the service, the request messages that the service requires, and the corresponding output message format that the consumer will get.
WSDL lets service providers expose only their interfaces describing the location and the message formats and completely hiding the underlying implementation. So as long as the service interaction is on the basis of the defined WSDL as the interface, the service provider can keep changing the implementations without it affecting the consumer.
Universal Description Discovery and Integration (UDDI)
The UDDI specifications let businesses publish their services in private and public registries so businesses and people can find and transact with one another easily and dynamically. UDDI lets a business:
The UDDI specifications let businesses publish their services in private and public registries so businesses and people can find and transact with one another easily and dynamically. UDDI lets a business:
Describe its business and its services
Discover other businesses that offer desired services
Integrate with these other businesses.
UDDI is made up of three different elements. Each of the elements isn't required for a listing so not all listings in UDDI registries contain all the elements. But the more elements in a listing, the more easily a service can be found and bound to.
The three elements are:
A "white pages" - This contains the basic contact information for each Web Service listing. It generally includes basic information about the company, as well as how to make contact.
A "yellow pages" - This has more details about the company and includes descriptions of the kind of electronic capabilities the company can offer to anyone who wants to do business with it. It uses commonly accepted industrial categorization schemes, industry codes, product codes, business identification codes, and the like to make it easier for companies to search through the listings and find exactly what they want.
A "green pages" This is what allows someone to bind to a Web Service after it's been found. It includes the various interfaces, URL locations, discovery information, and similar data required to find and run the Web Service.
REST-based Web Services
In the Web Services world REpresentational State Transfer (REST) is a key design idiom that embraces a stateless client/server architecture in which the Web Services are viewed as resources and identified by their URLs.
REST leverages the successful aspects of existing Internet protocols. To cap off the REST model, the URI and an HTTP verb (GET, POST, PUT, DELETE) are used together to issue a request to a REST Web Service to access and manipulate the server-side state, or data record, identified by the URI. The actual XML message is contained in the HTTP request and security is provided by HTTPS, which is the secure version of HTTP.
The concepts of REST are further described below.
Representation
Resources are first-class objects Resources are obtained as complete representations.
Resource
A Web page is a representation of a resource Resources are described by URIs. Consumers can retrieve a specific representation of the resource.
State
"State" represents the state of the application state as represented by a collection of resources The state is always maintained and traversed to and fro between the server and the client
Since REST Web Services ride the ubiquitous HTTP protocol and piggyback on the Web infrastructure, their presence is omnipotent and can be polymorphic in nature. This can be explained by the various number of REST-style services that currently exist on the Web. These include a client/server style, stateless and cached with a uniform interface, even code on-demand that is exemplified by Java applets.
You can locate resources by a Universal Resource Identifier (URI), and the operations that might be performed against those resources are defined by the HTTP specification. The core operations include:
GET - This operation returns a state representation of the identified resource. You can determine the state by a number of contextual factors such as who is submitting the request, the parameters of the operation (passed in as either HTTP headers or query string parameters), and the current session state maintained by the service provider.
POST - This operation performs some form of application-specific update to the identified resource. The behavior of this operation is completely up to the service implementing it. The data returned by this operation is also completely up to the application.
PUT - This operation creates a new resource at an identified location (URI). The operation input must include a state representation of the resource. It is up to the service to create a resource based on that state representation.
DELETE - The DELETE operation destroys a resource at the identified location (URI).
Testing Web Services
As we've seen, Web Services both REST as well as SOAP are going to be components that expose their functionality over HTTP. Hence we could potentially expose our services over an intranet or the Internet, which implies that the visibility of these Web Services is generally pretty high. There are, of course, some critical differences in the significance of intranet-based Web Services as opposed to the ones that will be exposed to the public domain. Intranet-based services lie within the boundaries of the enterprise and hence can be controlled and monitored with access restricted only to employees. However, an Internet-based service can be accessed by anybody and so we have far lesser control over those services. It's pretty much the same as the difference between an intranet and the Internet itself.
As we've seen, Web Services both REST as well as SOAP are going to be components that expose their functionality over HTTP. Hence we could potentially expose our services over an intranet or the Internet, which implies that the visibility of these Web Services is generally pretty high. There are, of course, some critical differences in the significance of intranet-based Web Services as opposed to the ones that will be exposed to the public domain. Intranet-based services lie within the boundaries of the enterprise and hence can be controlled and monitored with access restricted only to employees. However, an Internet-based service can be accessed by anybody and so we have far lesser control over those services. It's pretty much the same as the difference between an intranet and the Internet itself.
We'll look briefly at the various aspects of testing Web Services in general and then how it's done for REST-based services.
Functional Testing or Black Box Testing
This testing is carried out to ensure that the Web Service functions per the design and the requirements. These only deal with proposed inputs and expected outputs. Service test cases are generally mapped to business use cases that outline all the permutations and combinations of inputs and outputs.
This testing is carried out to ensure that the Web Service functions per the design and the requirements. These only deal with proposed inputs and expected outputs. Service test cases are generally mapped to business use cases that outline all the permutations and combinations of inputs and outputs.
REST-based Web Services are pretty unique since they basically correspond to the various operations outlined above. We suggest a couple of different possible scenarios to address the challenges involved.
The application to be tested could have an implementation of strict REST-based Web Services. We'll elaborate on this further with an example, say a product catalog in which items can be created, viewed, deleted, and updated. This is pretty similar to the CRUD operations that would happen for the items in the underlying database.
When a new item is created it is achieved by the PUT operation and it in turn creates a new URI like www.xyz.com/items/itemnew01
When an item is to be viewed it is achieved by the GET operation by using the URI www.xyz.com/items/showitem?itemid=I001. This returns the URI www.xyz.com/items/itemnew01
When an item has to be edited it is achieved by the POST operation by using the URI www.xyz.com/items/itemid=I001
When an item has to be deleted it is achieved by the DELETE operation by using the URI www.xyz.com/items/itemid=I001 In such cases, while the former can be functionally tested by normal Web site testing mechanisms and we can use the normal Web testing tools that are available today.
Now if the application doesn't have strict REST-based Web Services, the Web Service responses could be slightly different. For example, it could return an XML over HTTP, instead of an URI representation of a resource. So when an item is to be viewed it could return an XML instead of www.xyz.com/items/itemnew01. This particularly tends to happen to promote interoperability, so that non-browser-based clients can access these Web Services too.
This case is a bit more complex. Currently there aren't many tools available that can cater to this case.
Currently most testers build up their test cases and the input data in Excel sheets and verify it against the expected output XML. It probably makes sense to build a small tool that can do the following:
Read the input parameters from the Excel sheet
Invoke the appropriate service.
Store the response provided by the service
Compare the response against the expected response using tools like XMLDiff and provide reports
Non-Functional Requirements Testing
The other fundamental aspect of architecture and design is to ensure that the software meets all the non-functional aspects like security, reliability, availability, and scalability. These facets are generally the most ignored by software designers as well as testers. These prove costly mistakes because a functionally perfect system is of no use if it can't be used by a specified set of users in a predictable response time. There are also serious ramifications if the system isn't secure and allows unauthorized users access.
The other fundamental aspect of architecture and design is to ensure that the software meets all the non-functional aspects like security, reliability, availability, and scalability. These facets are generally the most ignored by software designers as well as testers. These prove costly mistakes because a functionally perfect system is of no use if it can't be used by a specified set of users in a predictable response time. There are also serious ramifications if the system isn't secure and allows unauthorized users access.
Hence non-functional testing is as significant as functional testing.
Security Testing
As previously mentioned, Web Services are software components that could be exposed over the Internet. In such scenarios security becomes a primary concern, Authentication and authorization concerns have to be tested thoroughly. In most scenarios for REST-based services, authentication is taken care of by using the HTTPS protocol. Most enterprises prefer to add an additional layer of authentication using either basic HTTP authentication or a complex token-based security mechanism. Basic authentication generally deals with access control lists. So testing should cover the aspects of "malicious testing," where the system is tested for controlled access by a specific and ratified set of users.
As previously mentioned, Web Services are software components that could be exposed over the Internet. In such scenarios security becomes a primary concern, Authentication and authorization concerns have to be tested thoroughly. In most scenarios for REST-based services, authentication is taken care of by using the HTTPS protocol. Most enterprises prefer to add an additional layer of authentication using either basic HTTP authentication or a complex token-based security mechanism. Basic authentication generally deals with access control lists. So testing should cover the aspects of "malicious testing," where the system is tested for controlled access by a specific and ratified set of users.
Another common security lapse in a REST Web Service is its vulnerability to SQL injection attacks. The similarity between the various HTTP requests to the CRUD database operations is indicated in the section above. It leaves a poorly designed Web Service highly vulnerable to SQL injection attacks. So the test strategy should contain test cases to cover this angle as well.
Amazon's private "shared-secret" token is a complex implementation of security that uses HMAC signing to show proof of possession of the public token. Although this kind of security implementation in REST Web Services diffuses the basic advantages of REST by reinventing the wheel, this security model exists. Testing these security models is a lot more complex and is more along the lines of testing a PKI infrastructure including the token issuing authority. Authorization is generally designed using filters in combination with the resource control lists. Although this is no different from standard authorization testing mechanisms, it's imperative that the testing strategy includes tests for authorization. This ensures that a user can access only what he's supposed to access.
Another common area of testing is the audit trail of requests and responses. Since most REST Web Services are GET-based or based on information retrieval rather than altering the state of the resource on the server, it becomes very important to audit these requests. So the testing strategy should cover the audit trail mechanisms of these Web Services too.
Regression Testing
A regression test is normally a cut-down version of a functional test. Its aim is to ensure that the Web Service is still working between builds or releases. It assumes that some area of functionality worked in the past and its job is to check that it still does. However, versioning is difficult to achieve transparently in REST-based Web Services since each resource has its own representation. So if a resource changed in different builds or releases, it would have a new URI. However, less strict versions of REST Web Services allow a pure XML over HTTP and POST/PUT to be used interchangeably. They also have a version field either as part of the payload or query string. This lets them introduce versioning from a deployment perspective so different versions of the same service can co-exist while using the same URI. Although this violates basic principles of Roy Fielding's theory, unfortunately there are a lot many implementations out there. Hence regression tests need to account for versioning test cases when multiple versions of the same service will co-exist in the application ecosystem.
A regression test is normally a cut-down version of a functional test. Its aim is to ensure that the Web Service is still working between builds or releases. It assumes that some area of functionality worked in the past and its job is to check that it still does. However, versioning is difficult to achieve transparently in REST-based Web Services since each resource has its own representation. So if a resource changed in different builds or releases, it would have a new URI. However, less strict versions of REST Web Services allow a pure XML over HTTP and POST/PUT to be used interchangeably. They also have a version field either as part of the payload or query string. This lets them introduce versioning from a deployment perspective so different versions of the same service can co-exist while using the same URI. Although this violates basic principles of Roy Fielding's theory, unfortunately there are a lot many implementations out there. Hence regression tests need to account for versioning test cases when multiple versions of the same service will co-exist in the application ecosystem.
Load Testing
The aim of load and stress testing is to find out how well your Web Service scales as the number of clients accessing it increases. The load testing is generally done to find the acceptable benchmarks associated with the Web Service and identify the Service Level Agreements (SLAs) that can be specified with this service. These benchmarks then become the SLAs of the service. Load tests for REST-based Web Services are primarily based on the same concept as load testing Web applications or Web sites. However there are subtle differences that have to be looked at carefully. REST Web Services are XML-based services. So while you get the flexibility of XML, you also get the baggage that comes with it. XML is bulky and takes up more bandwidth. Large XML messages tend to take up lots of memory. These XML messages also have to be parsed and using DOM parsers for large XML documents often leads to low memory situations. So it's imperative that the load testing strategy includes testing the performance of the services under an extreme load, particularly large XML payloads.
The aim of load and stress testing is to find out how well your Web Service scales as the number of clients accessing it increases. The load testing is generally done to find the acceptable benchmarks associated with the Web Service and identify the Service Level Agreements (SLAs) that can be specified with this service. These benchmarks then become the SLAs of the service. Load tests for REST-based Web Services are primarily based on the same concept as load testing Web applications or Web sites. However there are subtle differences that have to be looked at carefully. REST Web Services are XML-based services. So while you get the flexibility of XML, you also get the baggage that comes with it. XML is bulky and takes up more bandwidth. Large XML messages tend to take up lots of memory. These XML messages also have to be parsed and using DOM parsers for large XML documents often leads to low memory situations. So it's imperative that the load testing strategy includes testing the performance of the services under an extreme load, particularly large XML payloads.
Conclusion
We have articulated the basic architectural Web Service paradigms, namely SOAP-style Web Services and REST-style Web Services. We looked at various aspects of testing REST-based Web Services. It helps to develop a testing strategy or methodology for REST-based services. We also mooted the idea of designing a tool based on available out-of-the-box components to test the services. We concluded that testing REST-based services is reasonably easier than SOAP-based services, where additional testing of standards, interoperability, and versioning is required. Testing is also required to see if the SOAP-based Web Services adhere to WS-I standards.
We have articulated the basic architectural Web Service paradigms, namely SOAP-style Web Services and REST-style Web Services. We looked at various aspects of testing REST-based Web Services. It helps to develop a testing strategy or methodology for REST-based services. We also mooted the idea of designing a tool based on available out-of-the-box components to test the services. We concluded that testing REST-based services is reasonably easier than SOAP-based services, where additional testing of standards, interoperability, and versioning is required. Testing is also required to see if the SOAP-based Web Services adhere to WS-I standards.
References
No comments:
Post a Comment