How to capture response body data in Newman result report


#1

I want to capture response body data in Newman result report so that I will get to know for which reason my APIs are failing.

I need help.
Please let me know if anyone knows.


Improve Newman Documention for Custom Reports
#2

Hi. On newman you could use --reporter-html-template to indicate the path of the handlebars template that will be used to generate the html report. The default html reporter uses an generic template ‘template-default.hbs’ in ‘\node_modules\newman\lib\reporters\html’ you could copy it and make your own template.

To get the reponse body data in the report, edit your handlebars template to iterate over ‘response’ object… the ‘body’ property contains the response data.

example:

  • To run newman with custom template

NEWMAN_PATH run YOUR_COLLECTION.json --reporters html,cli --reporter-html-export PATH_TO_REPORT.html --reporter-html-template PATH_TO_YOUR_TEMPLATE.hbs

  • On your custom template (.hbs) file

inside block ‘{{#*inline “aggregations”}} {{#each executions}}’ add a ‘div’ with ‘{{response.body}}’ this will make handlebars iterate over each executions in json result and print the response.body data inside a div.

This handlebars (.hbs) file is just an html file with entries that will be populated by the handlebars. For more info: http://handlebarsjs.com/


#3

I installed handlebars but I didn’t understand how to execute/use this. Can you help me?


#4

Hi @rmlira

i had added the a div as you explained (PFA :postmanReport).
i dont see respone in the report (PFA: ActualReport), can you tell where i went wrong ?


Improper newman report
#5

@rmlira
This is the report generated,


#6

Hi @gopikrishna4595 … you must use {{response.body}} outside the {{#with request}} block… 'cause the response.body is a child from executions directly… and not from request. I think this will solve your problem. Try it, let me now if it works.


#7

Hi, newman already comes with handlebars installed, you don’t need to install it… to get response.body in html report you just need to use a template that reads the response body.

In order to do this you must run Newman with the parameter ‘–reporter-html-template’ informing your template file. To create a template file you could copy the default template that comes with Newman (\node_modules\newman\lib\reporters\html) and create your own in any directory, just pass the fullpath for the file to --reporter-html-template. Do you have any other specific questions?


#8

Thank you @rmlira :clap:t4:,

Using {{response.body}} outside the {{#with request}} helped to get a response in the report.

But i see a different problem
I executed a collection through newman and generated a report with the modified HTML template.
The problem is when collection is run with iteration, in the report i’m unable to open all the requests, only first two requests are able to open in the report generated.
Below is the command used.

newman run Sample.postman_collection.json -d CSVFILE.csv -n 6 -r html --reporters cli,html --reporter-html-template C:\Users\gopikrishna\Documents\colouredDefault.hbs --reporter-html-export C:\Users\gopikrishna\Desktop\Doubts\Report201803.html

Below link is of Report file generated, Postman Collection.
https://drive.google.com/open?id=1a1OaZA7Gs7EnSq2vLZeQLgI1Wi8MwcHU

Postman Files

Plz have a look at the Report File Generated.

i think, the problem lies in the HTML code of Template.

can you help to get over this ?


#9

Yes… this problem occurs because in template it creates a div.panel-group for each execution, and the id attribute of this div is ‘collapse-request-{{item.id}}’. This way, for each iteration it will create 2 new div’s with same id, because item.id is the same in each iteration… this causes javascript to not work properly. When you click in a div it navigates throw DOM to find the first panel that must expand/collapse with the target id, as div’s has same id it always will use only first div. (https://getbootstrap.com/docs/3.3/javascript/#collapse)

To fix this, we should increment the id attribute with index of iteration… handlebars has a builtin helper that brings the index to us.

On template in block:

{{#*inline "aggregations"}}
    {{#each executions}}

you must find each ‘{{item.id}}’ and replace then with ‘{{item.id}}-{{@index}}’, this way we guarantee that each div has a unique id, and javascript could work properly.


Try it, let me now if it works.


#10

Hi @rmlira,

with the addition {{@index}} in id, working perfectly.:ok_hand:t3:

Now, able to open all the requests in the generated report.
and when i went through all the responses in the report, i found that all the responses are same which was generated for the first request in the iteration.

The same issue is seen for URL field in the {{#with request}} loop in the generated report.

any inputs on this ?


#11

This problem occurs because newman original html reporter was made for simple scenarios, without iterations.

To repair this problem will be necessary to edit the newman html reporter source, or create your own html reporter.

To edit newman html reporter you will need to access the folder node_modules/newman/lib/reporters/html in this folder there’s an index.js file, you need to edit this file to make it works properly on iterations.

The problem occurs because it considers only ‘currentExecution.id’ when aggregating data, we need it to consider currentExecution.id and currentExecution.response.id… cause our execution.id is the same but response is not. In order to do this you will need to replace multiple lines in file as follows:

line ~65:
executionId = currentExecution.id;
replace with:
executionId = currentExecution.id + currentExecution.response.id;

line ~92:
items[reducedExecution.item.id] = reducedExecution;
replace with:
items[reducedExecution.item.id + reducedExecution.response.id] = reducedExecution;

line ~126 and ~127:
current = _.merge(items[execution.item.id], {
assertions: _.values(assertions[execution.item.id]),
replace with:
current = _.merge(items[execution.item.id + execution.response.id], {
assertions: _.values(assertions[execution.item.id + execution.response.id]),

line ~132:
cumulativeTests: netTestCounts[execution.item.id]
replace with:
cumulativeTests: netTestCounts[execution.item.id + execution.response.id]

this will make it works with iterations, but I didn’t fully analyze the source, in time I’ll consolidate this and others improvements in a new html reporter, you can try this modifications in your own risk, in any case just restore the original index.js file.


#12

working perfect.

i had also added the request body to the report just like response body.

Thank you.:clap:t4:


#13

Hi @rmlira,

How are you doing ?

with the modifications made in index.js file, let us say in the executed collection has 3 requests and iteration is set for 30. a total of 90 requests is seen in the report.

instead of that can we have like request tab panel consisting of sub tabs(iterations).

To make the report look good
as a noob, i created collapsible tab for request and response, changed the colors. :stuck_out_tongue:

As of now, i created a tab for iteration1.
iteration2 and iteration3 are dummy tabs with no data.

how can we create “n” no.of tabs upon the no.of iterations and pass the data into it.

Following is the drive link of the latest hbs file and index.js file.:point_down:t3:

PostmanFiles-2

https://drive.google.com/drive/folders/1WNsjQOVMhHgG8bGQsIQM7ApDZf13Gxj2?usp=sharing

and i’m able to print the response body in Pretty Json String for REST requests. :blush:
(by default, it is in string format which doesn’t look good)

I’m still working on soap/Xml requests.

With the files shared to you, as of now ,you cannot get pretty json with the files shared i screwed up the code :exploding_head: while trying for Pretty Xml format.
you can un-comment the code and Feel free to check


#14

Hi @gopikrishna4595, sorry for the delay.

To make this modifications we’ll need to edit newman html reporter index.js file again. So again access the folder: node_modules/newman/lib/reporters/html and open index.js file.

line ~35:
AGGREGATED_FIELDS = ['item', 'request', 'response'],
replace with:
AGGREGATED_FIELDS = ['cursor', 'item', 'request', 'response'],

image

line ~124:
(after parent = execution.item.parent(),) add a new line with :
iteration = execution.cursor.iteration,

line ~141
(after id: parent.id,) add a new line with:
iteration: iteration,

Once you make these modifications we can now edit the template to create the tabs. To create the tabs dinamically according to the number of iterations we will need to add some javascript code. So in <head> block add:

line ~23:

<script type="text/javascript">
    "use strict";

window.onload = function () {

  // hides all blocks of response
  var allItems = document.querySelectorAll('[class*=iteration-]');
  allItems.forEach(function(elem){
    elem.style.display = 'none';
  });

  {{#with summary}} {{#with stats}}
  let totalIterations = {{iterations.total}};
  {{/with}} {{/with}}

  let menu = document.querySelector('#execution-menu .nav');

  for(var i = 0; i < totalIterations; i++)
  {
    let li = document.createElement('li');
    li.appendChild(document.createTextNode('Iteration - ' + i));
    li.setAttribute('id', 'iteration-' + i);

    li.addEventListener('click', function() {

      let allItems = document.querySelectorAll('[class*=iteration-]');
      allItems.forEach(function(elem) {
        elem.style.display = 'none';
      })

      let items = document.querySelectorAll("." + this.id);
      items.forEach(function(elem) {
        elem.style.display = elem.style.display == 'block' ? 'none' : 'block';
      })
    });

    menu.appendChild(li);
  }
}</script>

This block of code only sets css property ‘display’ to ‘none’ in all blocks of response data and populate an menu list with iteration-* menu according to number of iterations provided by handlebars in ‘totalIterations’ variable. I use pure javascript but feel free to improve it with jQuery…

Now after the ‘Requests’ title (line ~97 <h4>Requests</h4>) we will create our tab menu so add a new line with this block:

<div id="execution-menu">
        <ul class="nav nav-tabs">
        </ul>
    </div>

Then put the {{#each aggregations}}....{{/each}} block inside a div <div class="tab-content" id="execution-data">.

The first div inside {{#each aggregations}} block is: <div class="panel-group" id=.... we need to add a class in this div. So it will be: <div class="panel-group iteration-{{parent.iteration}}" id=

Once you make these modifications we need to replace every {{parent.id}} with {{parent.id}}-{{parent.iteration}}

Then in {{#each executions}} block (line ~129) we need to replace every {{item.id}} with {{item.id}}-{{cursor.iteration}}

And it’s done. These modifications will create a tab menu, when you click in each menu it will display only the requests corresponding to that iteration.

Try it, let me now if it works.


#15

I’m still working on a better approach. As soon as possible I return with the result.


#16

Thank you @rmlira for succour,

with these changes tabs are not observed. checked multiple times.

can these latest changes can be added to the same file where we made changes earlier (which let us to open each request separately.) ?


#17

Yes, you must use the index.js (reporter html file) with previous changes and just add these new ones. If error persists can you show me your index.js file and your hbs template?


#18

Yes Renan Lira, added these changes to the previous changes. Tabs are not seen.

Following are the files with Google Drive link
Postman Reports with Tabs


#19

I make a test using your template and the reporter… and it works. I’m testing on chrome. I’ll make more tests, do you know if any error appears in console?


#20

i had checked multiple times Renan Lira even in chrome,
executed the script several times, but tabs are not seen, just their text is seen on the top.

if possible can you drop your files in the

Postman Report Tabs

i’ll try with those files.

and just for reference, can you attach a image how you are observing .