Dynamic Data In CI/CD With Newman

Hey everyone.

So I have been meaning to share this for awhile now but just never got around to it. Thanks to @danny-dainton for the motivation to actually do it. :slight_smile:

An issue that I had run across that did not have an easy solution (or not that I could find).
Is how to dynamically pass in data when using a CI/CD tool like Jenkins.
Of course as part of my jobs I am using Newman to execute my tests.

A real world example of a solution I needed was to validate a micro service that dispatches emails. To do this I needed to use a valid email address.
To make this as flexible as possible I broke the email addresses down into its individual parts. These are passed in as String Parameters, when the Jenkins job is parameterized.

For example it would look something like this:

Now that I have the fields setup to pass these values in when I run the job. I need a way to get these from the passed in parameters into Newman.

So how do I put these values into a CSV or JSON data file to be passed in? To accomplish this I simply create a CVS file via command line using the following command:

echo emailPrefix,emailDomain,emailDelimiter > email.csv

This will create the base CSV file with one line in it, we will use this first line as the column headers.
*** Note the use of the > character in this line. This tells the command that it is to take these values and create the file with them.

Then we next need to create the actual data we are going to use for our test. In my scenarios I will only need one line for each run I am doing, however you can add as many lines as you please using the following technique.

echo $emailPrefix,$emailDomain,$emailDelimiter >> email.csv

This will now take the parameters from the Jenkins job and put them in the CSV file as well.
*** Note here how we use the >> characters. This tells the command line to append to the CSV file vs creating the file and inserting to it. If these are not used correctly you could end up wondering why your file never has the correct data in it.

In the end this section of your Jenkins job configuration will look something like this:

Now, to bring it all together. You simply need to add your Newman run command to the job and pass in the newly created CSV file as your data file.

Hopefully this may help others that face a similar issue. I know there may be Jenkins Plugins that can help but they can sometimes be a bit of a headache.
This is a very straight forward and simple way to accomplish the task of passing dynamic data into a Newman run without having to modify the data file each time when using a CI/CD job.

If anyone has any questions or feedback on this, send it my way as I would be happy to help or expand on this topic where I can.

– Trent

6 Likes

Awesome @tmccann! Glad to see you sharing this with the community!

1 Like

Thanks Trent, so if the .csv contains say a current value you want to use inplace of the existing default value in the environment.json file how would you do that in the scenario?

@rlsmall So you are looking to edit the values in the environment.json file on the fly?