The PowerShell command to remove an entire directory and its contents ( including sub folders and files) is below

cmd.exe
1
rm -Rf pathToDirectoryToBeRemoved/

R flag denotes to run “rm” command recursively . “f” flag denotes to run in forcefully. We can even replace “f” with “v” for verbose mode and “i” for interactive mode.

Note: Above command can also be used to delete files which have long path ( more than 260 characters)

Previous two blog post talked about how we can use mountebank for stubbing where responses are in json format . They can be accessed (here) and (here). We can use same approach for stubbing SOAP services using XML as well. In this post, I will explain how we can provide XML response using Mountebank .

Let us have a quick look into the files created. Before we begin, folder structure of various file as below

folderstructure

Imposter.ejs

The main Imposter file is

Imposter.ejs
1
2
3
4
5
{
"imposters": [
<% include Port4547.json %>
]
}

Port4547.json

This file specifies which port number to use and what all stubs needs to be created is as below

Port4547.json
1
2
3
4
5
6
7
8
9
10
11
12
{
"port": 4547,
"protocol": "http",
"stubs": [
{
<% include XMLStubGET.json %>
},
{
<% include XMLStubPOST.json %>
}
]
}

XMLStubGET.json

This is the first stub for this example and it looks for any request coming with the method “GET” and path “/Blog.Api/[0-9]+/CustomerView” , where [0-9]+ is regular expression of any numeric

XMLStubGET.json
1
2
3
4
5
6
7
8
9
10
11
12
13
"responses": [
{
"inject": "<%-stringify(filename, 'ResponseInjection\\GetXMLStub.js') %>"
}
],
"predicates": [
{
"matches": {
"method" : "GET",
"path" : "/Blog.Api/[0-9]+/CustomerView"
}
}
]

XMLStubPOST.json

This is the second stub for this example and it looks for any request coming with method “POST” and path “/Blog.Api/XMLexamplePOST/[0-9]+” , where [0-9]+ is regular expression of any numeric .It also needs a body as InsertCustomer1

Note: If you have body in multi-line, then make sure to enter “\n” for new line

XMLStubPOST.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
"responses": [
{
"inject": "<%-stringify(filename, 'ResponseInjection\\GetXMLStub-POST.js') %>"
}
],
"predicates": [
{
"matches": {
"body" : "<Action>Insert</Action><Record>Customer1</Record>",
"method" : "POST",
"path" : "/Blog.Api/XMLexamplePOST/[0-9]+"
}
}
]

GetXMLStub.js

Below js file create a response based on template mentioned and return the response with proper status. Please note that, we are not using “Json.Parse” here as we did for previous examples involving json.

GetXMLStub.js
1
2
3
4
5
6
7
8
9
10
function GetTemplateResponse (request, state, logger) {
response = "<%- stringify(filename, 'StubTemplate\\CustomerDetails.xml') %>"
return {
statusCode : 200,
headers: {
'Content-Type': 'application/xml; charset=utf-8'
},
body: response
};
}

GetXMLStub-POST.js

GetXMLStub-POST.js
1
2
3
4
5
6
7
8
9
10
function GetTemplateResponse (request, state, logger) {
response = "<%- stringify(filename, 'StubTemplate\\RecordAdded.xml') %>"
return {
statusCode : 200,
headers: {
'Content-Type': 'application/xml; charset=utf-8'
},
body: response
};
}

CustomerDetails.XML

This is the template for the first stub - GET example

XML
1
2
3
4
5
6
<customer>
  <FirstName>John</FirstName>
  <LastName>Citizen</LastName>
  <Address>Some St, Some State, Some Country</Address>
  <Email>Test@test.com</Email>
</customer>

RecordAdded.xml

This is the template for the second stub - POST example

XML
1
2
<Status>Added</Status>
<Record>Customer1</Record>

After creating above files and keeping them as per directory structure is shown above, it is time to start mountebank

mb –configfile SOAP-XMLStubExample/Imposter.ejs –allowInjection

Note: Give the right path to Imposter.ejs . If you need to debug Mountebank, you can use below command at the end “ –loglevel debug”

Now trigger a get request to http://localhost:4547/Blog.Api/3123/CustomerView.

This should match with our first predicate and should return the response mentioned

Mountebank_XML_Response_

Now trigger a POST request with a body . If predicates are matched, then it will respond with expected response as below

PostManRequest

In Nut shell, creating a XML response is similar to creating json response. There are only minor differences in the js file which creates the response. The main difference is the omission of Json.Parse and also changing the response headers.

Above examples can be cloned from my GitHub repository here. After cloning the repository to local, just run RunMounteBankStubsWithSOAPXMLStubExampleData.bat file. Postman scripts can also be found inside PostmanCollections Folder to testing this

On corporate world, most of the times, the access required for installing applications and connecting to internet will be limited. There can be scenarios where access to install Mountebank using npm will not be available. In those circumstances, we can just unzip the zip file downloaded from self contained archive links in mbtest.org

Below code snippet can be used for extracting the zip files on the fly , so that it can be used for running test cases on any machine.

Pre-Requisite

  • .Net 4.5 is needed
  • Add reference to below dll to solution
    • System.IO.Compression.dll
    • System.IO.Compression.FileSystem.dll

Example

Below example is based on (copied from) msdn

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
    Public void ZipFile()
    {
        string startPath = @"c:\example\start";
        string zipPath = @"c:\example\result.zip";
        string extractPath = @"c:\example\extract";

        ZipFile.CreateFromDirectory(startPath, zipPath);
    }

    Public void UnZipFile()
    {
        string startPath = @"c:\example\start";
        string zipPath = @"c:\example\result.zip";
        string extractPath = @"c:\example\extract";

        ZipFile.ExtractToDirectory(zipPath, extractPath);
    }

This is an extension to my previous blog about how we can use mountebank to create a stubbed response based on a template file . You can read about it here.  In this step by step example, I will explain how we will use mountebank to modify the response based on the request . Before we start, please ensure you are familiar with Part1 of the excercise. If you need to know more about mountebank and how to use mountebank , please read through how to install mountebank  and service virtualisation using mountebank.

As in previous example, let us create Imposter.ejs and 4547.json . Contents of the Imposter.ejs is as below

Imposter.ejs
1
2
3
4
5
6
7
8

{
 "imposters": [

 <% include 4547.json %>

 ]
}

Contents of 4547.json is as below

4547.json

4547.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14

{
 "port": 4547,
 "protocol": "http",
 "stubs": [

{
 <% include  CustomerNotFound.json %>
 },
 {
  <% include  CustomerFound.json %>
 }
 ]
 }

Now create CustomerFound.json

CustomerFound.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

 "responses": [
 {
 "inject": "<%-stringify(filename, 'ResponseInjection\\GetCustomerFound.js') %>"
 }

 ],
 "predicates": [
 {
 "matches": {
 "method" : "GET",
 "path" : "/Blog.Api/[0-9]+/CustomerView"
 }
 }
 ]

As we can see from above, if there is request which matches the predicates , then response will be dictated by the GetCustomerFound javascript file kept inside directory ResponseInjection. Predicate used here is a GET request which have a matching path of /Blog.Api/[0-9]+/CustomerView.

Contents of GetCustomerFound.js is

GetCustomerFound.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
function GetTemplateResponse (request, state, logger) {

response = JSON.parse("<%- stringify(filename, 'StubTemplate\\CustomerFoundView.json') %>");
 var ext =require('../../../StubResponse/ResponseInjection/extractrequest');

var reqdata = ext.extractor(request);

 response.data.customerID=reqdata.CustomerID;

 return {
 statusCode : 200,
 headers: {
 'Content-Type': 'application/json; charset=utf-8'
 },
 body: response

 };
}

The javascript file have a single function , which reads the stubbed response kept in template file . Then it calls another Javascript function to called “extractrequest”. We will see the details of it soon. For now, it actually returns the customer number from the request . For eg, if request is “http://localhost:4547/Blog.Api/3123/CustomerView ” then it return 3123 as customer ID. Once we extract the customer ID, then it will replace the customer ID in our template response with the value coming from request and return the response.

Let us take a close look at the extractrequest function.

extractrequest.js
1
2
3
4
5
6
7
8
9
10
11
12

module.exports = {extractor:function extractCIFAndPackageID (request) {

if(request && request.path) {
var req = request.path.split('/');
if(req.length >2 && req[1]) {
return { CustomerID: req[2] }
}
}

return null;
}}

This method will take the input parameter as the request and split it at “/” to get a an array . Then we will return the array[2] which is the customer ID from the request

Finally , the template response

CustomerFoundView.json

CustomerFoundView.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

{
"status": "success",
"code": 0,
"message": "",
"data":
{

"customerID": "123",
"firstName": "John",
"lastName": "Citizen",
"email": "John.Citizen@abcabacas.com"

}

}

Now let us fire up mountebank

mountebank

Make few request using postman, which have different request parameter

customerFound1

Another request

CustomerFound2

In above two examples,we  can see the CustomerID field is response is updated with number extracted from request.

Now let us try another example , where request is http://localhost:4547/Blog.Api/1234542323/CustomerView

CustomerNotFound2

As you can see, we are getting a customer Not found response. This is due to the order of predicates we use. In our 4547.json, the order of response are as below.

  1.  Customer Not found which has a predicate of “/Blog.Api/1[0-9]+/CustomerView”

  2. Customer found which has a predicate of “/Blog.Api/[0-9]+/CustomerView”

As you can see from above order, when a request comes through , mountebank will first match with predicate of first response and if it matches, it returns the response. If not, mountebank will keep trying with next one followed by all others. In this particular example, since our request have a customer ID of 1234542323, it matches with regular expression of first one ( 1[0-9]+)  and hence it return customer not found response.

In next blog post, I will provide more insights about how to extract request from different type of requests.

In the previous two blog post, I have explained about how to setup mountebank (here) and how to create a virtualised respone(here) . Now coming to more detailed use cases which we might encounter in daily life. In this blog post, I will explain how we can use mountebank to create a virtualised response based on a template response stored in a file and modifying certain fields in response based on the request coming through.

In below Step by Step example , I will have two mock responses for searching for a customer details. First response is when customer is not available in back end systems and second response is when customer details are found.

Before we start, below is folder structure which I have and in this blog post we are discussing about only one stubbed response, which is the NOT FOUND scenario.

folderstructure

Let us first create the imposter.ejs file

Imposter.ejs
1
2
3
4
5
6
7
8

{
"imposters": [

<% include 4547.json %>

]
}

Now let us create the file which specifies the port number where it should run and order of responses. Below code tells mountebank that port which it needs to listen for incoming request is 4547 and protocol is http. There are two set of mock responses planned.

4545
1
2
3
4
5
6
7
8
9
10
11
12
13
{
"port": 4547,
"protocol": "http",
"stubs": [

{
<% include CustomerNotFound.json %>
},
{
<% include CustomerFound.json %>
}
]
} 

In this example, let us look at first mock response.

CustomerNotFOund.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
"responses": [
{
"inject": "<%- stringify(filename, 'ResponseInjection\\GetCustomerNotFound.js') %>"
}

],
"predicates": [
{
"matches": {
"method" : "GET",
"path" : "/Blog.Api/1[0-9]+/CustomerView"
}
}
]

From above response, we can infer below. When ever an http GET request come to port 4547 , with a path matching “/Blog.Api/1[0-9]+/CustomerView', then we will call the Javascript function "GetCustomerNotFound.js” which is kept inside a directory “ResponseInjection” in same location. It is also good to notice that , predicate is a regular expression ( hence use matches) and all request where 1 followed by any number of numeric will be returned with this response

The javascript function listed here is responsible for reading the sample template response and sending it back .

GetCustomerNotFound.js
1
2
3
4
5
6
7
8
9
10
11
12
13
function GetTemplateResponse (request, state, logger) {

response = JSON.parse("<%- stringify(filename, 'StubTemplate\\CustomerNotFoundView.json') %>");

return {
statusCode : 404,
headers: {
'Content-Type': 'application/json; charset=utf-8'
},
body: response

};
}

Above function reads a json response kept inside directory “StubTemplate” and convert it to json and return to mountebank. Since this is for a scenario where customer records are not found,we set the status code as 404. We can also set the headers if needed

The stub template is as below

CustomerNotFoundView.json
1
2
3
4
5
6
{
"status": "fail",
"code": "CUSTOMER_NOT_FOUND",
"message": "Customer details not found."

}

Now let us run mountebank

mountebank

Request through postman

notfound.png

As you can see , the GET request matching with predicate is returning the stubbed response with status 404.

For real time usage for testing any web application which needs to get a 404 message from back end API calls, just point the end point to this local host end point and fire a request which matches the predicate.

Details of second response will be shared in next blog post

How to create HTML report with details of test execution

Very often , we will be required to create a report with details of test execution , so that it can be presented to various stakeholders. Specflow provides a feature to create HTML reports. Let us look into more details about how is this done

  • Read through and understand details of reporting from specflow.
  • Ensure packages for Specflow, Nunit, Nunit console runner are already installed.
  • If you are using Nunit 3, install NUnit.Extension.NUnitV2ResultWriter package via nuget package manager. If this is not installed, we will get an error “Unknown result format: nunit2”.
  • Follow setups required for running specflow test cases from command line. Details can be found here.
  • Modify the bat file to create nunit2 reports.
1
PathToNunitConsolerunner\nunit3-console.exe --labels=All --out=TestResult.txt "--result=TestResult.xml;format=nunit2" PathTo\AcceptanceTests.dll
  • Add below command into Bat file. This will create HTML Report called “MyResult.html”
1
PathToSpecfloPackage\specflow.exe nunitexecutionreport PathTo\AcceptanceTests.csproj /out:MyResult.html
  • Final bat file will look like below.
1
2
3
4
5
REM bat file to run test cases from console and create xml result file
.\..\..\..\packages\NUnit.ConsoleRunner.3.8.0\tools\nunit3-console.exe  --labels=All --out=TestResult.txt "--result=TestResult.xml;format=nunit2" .\AcceptanceTest.dll

REM Generate html report from test output
.\..\..\..\packages\SpecFlow.2.1.0\tools\specflow.exe nunitexecutionreport .\..\..\AcceptanceTest.csproj /out:MyResult.html

EDITED: If it throws below error in newer version of Visual Studio then ensure MS Build tool 2013 is installed. It can be downloaded from https://www.microsoft.com/en-US/download/details.aspx?id=40760

Error : “The tools version "12.0” is unrecognized. Available tools versions are “2.0”, “3.5”, “4.0”. “,

How to run specflow test cases from command line

We can use nunit console runner for running specflow test cases from command line. Running specflow test cases through nunit console runner will help to create test results in xml file, which can then be used for creating html reports.

Procedure for command line test execution are

  1. Define Nunit as the test runner. This is done in config file
1
2
3
<specFlow>
      <unitTestProvider name="NUnit"/>
</specFlow>
  1. Include Nunit.Console.Runner package to solution via nuget package manager
  2. Run specflow test cases using below command. We can create a bat file with below command and execute them as required.
1
2
3
4
pathToNunitConsoleRunner\nunit3-console.exe  PathToProject.dll

 example will be
 .\..\..\..\packages\NUnit.ConsoleRunner.3.8.0\tools\nunit3-console.exe   .\BDDFramework.dll

In current development world, there will be scenarios were both API and its consumers are developed in parallel. Inorder to decouple their dependencies, we can mock an api response using mountebank. In this example, I will explain how to get started with your first service virtualisation using mountebank. After installing mountebank as mentioned in here (Install Mountebank), we will proceed with configuring mountebank. It can be done in few ways. The method which I explain below is by using file based configuration. This involve setting up an imposter file and a stub response

How to Create a Stub

  1. Navigate to mountebank installation path

  2. Create a folder and name it as “StubResponse”. ( You can name it whatever you want)

  3. Create two json file using notepad and save it as “MockResponeForApiOne.json” and “MockResponeForApiTwo.json”( Or what ever you want).

  4. Copy paste below code to “MockResponeForApiOne.json” . Sample example only. Update the response and predicates to suite your need ( if required)

MockResponseForApiOne.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

"responses": [
 {
 "is": {
 "statusCode": 200,
 "body": {
 "Text":"Response ONE ","token":"username","expires_in":90
 }
 }
 }
 ],
 "predicates": [
 {
 "exists": {
 "body" :
 {
 "username": true,"password" : true
 },
 "method" : "POST",
 "path" : "/Apitesting/v1/test?type=ResponseOne"
 }
 }
 ]

-

  1. Copy paste below code to “MockResponeForApiTwo.json” . Sample example only. Update the response and predicates to suite your need ( if required)
MockResponseForApiTwo.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
 "responses": [
 {
 "is": {
 "statusCode": 200,
 "body": {
 "Text":"Response TWO ","token":"emailAddress","expires_in":90
 }
 }
 }
 ],
 "predicates": [
 {
 "exists": {
 "body" :
 {
 "email": true,"password" : true
 },
 "method" : "POST",
 "path" : "/Apitesting/v1/test?type=ResponseTwo"
 }
 }
 ]

How to create an Imposter

  1. Create another file called test.json in same path as above

  2. copy and paste below contents to it

test.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

{
"imposters": [
{
"port": 4547,
"protocol": "http",
"stubs": [
{
<% include MockResponseForApiOne.json %>
},
{
<% include MockResponseForApiTwo.json %>
}
]
}
]
}

Let us have a close look into Imposter and stubs

Responses – Contains an array of responses expected to return for the defined stub. In the above scenario the response will include status code as 200 and response body. For more info, http://www.mbtest.org/docs/api/contracts Predicates – is an array of predicates which will be used during matching process. Predicate object can be quite complex, it supports lots of different matching techniques. For more info, http://www.mbtest.org/docs/api/predicates

Let’s Mock it

Once all required files are created and saved, mountebank can be started by following command in command prompt , after navigating to installation folder of mountebank

test.json
1
2

mb --configfile StubResponse/test.json

cmd.jpg

Once mountebank is started, we can verify it by navigating to path http://localhost:2525/imposters

It will list out all active ports and a list of stubs available

imposter

Test It

Once we complete above steps, mountebank is ready with stubs. Now comes the part to test it and use. You can use any api testing tool ( Postman, soapUi etc ) for testing this. Just send the request matching the predicates and look for the responses

Below are the screenshot of Postman request

Requesting for First API.

Predicate of response One says that , request has to be of type POST, body of request should have “username” and “password” . Path of the request should have /Apitesting/v1/test?type=ResponseOne"

Now construct a postman request matching above and fire it

bgone

Request for second API

Predicate of response One says that , request has to be of type POST, body of request should have “email” and “password” . Path of the request should have /Apitesting/v1/test?type=ResponseTwo"

Now construct a postman request matching above and fire it

bgtwo

As you can see, both request has succesfully received expected response message

For actual development usage, just point your application to this localhost URL and start consuming virtualised API

What is Mountebank?

As per mbtest.org "_mountebank is the first open source tool to provide cross-platform, multi-protocol test doubles over the wire. Simply point your application under test to mountebank instead of the real dependency, and test like you would with traditional stubs and mocks_"

In short mountebank is a open source service virtualisation tool . Mountebank uses imposters to act as on demand test doubles. Hence our test cases communicate to Mountebank and mountebank responds back with relevant stubs as defined.

How to Setup Mountebank ?

Installation can be done via two methods

npm

Mountebank can be installed as a npm package. Node.js should be installed for this option to work

<code>npm install -g mountebank</code>

Self contained Installation file

OS Specific installation file can be downloaded from Download

Note: Please read through the windows path limitation mentioned in above link

The traditional approach for automating UI test cases is to create selenium web driver based ( or any UI testing tools) scripts for exercising complete end to end flow. However, it comes with its own challenges. It will have multiple steps as pre-requiste for reaching required UI page and hence it behaves as an E2E integration test rather than UI test.

A typical web application architecture will have one or more front-end application, which will talk to multiple back-end services, API’s etc. They will, in turn, talk to other back-end services or to different databases. On High level , architecture looks like below

On an enterprise world, all these will be developed and maintained by different teams. All of them will be working in parallel and will push in their code changes ( including occasional broken code) frequently. This will result in breakages since test automation scripts heavily depending on UI and its integration. Even if there is no broken code, a test can still fail due to multiple environmental issues for any of the backend services and other components. Hence it will become increasingly difficult for achieving a green build.

Hence UI based test cases are less robust due different reasons like

  1. Test depends on external factors which are outside of our control and not part of scope of testing

  2. Failing test may not pin point exact location of failure since it is trying to test too many things.

  3. There are chances that all components will not be ready when we want to test UI. Hence testing it pushed to the end , which will increase cost of fixing defects.

  4. Re - running of test cases may pass (if failure is caused by environmental issues)

  5. UI test are brittle by nature since they will even fail due to timing issues because it is depending on data from back end services.

The solution for above is to adopt more unit test like structure for UI testing. We should be testing UI in isolation to other back-end services and their dependency. This allows testing as much as possible early in lifecycle without any dependency on other streams. We should replace all backend service calls with stubs

Mountebank is a tool which we can use for mocking the service calls. As per mbtest.org, mountebank is the first open source tool to provide cross-platform, multi-protocol test doubles over the wire. We can use mountebank for stubbing the back-end service calls and there by use it for decoupling UI from unpredictable back end.