I have a lot of experience writing automated tests suites for APIs in Postman. I have always used data-driven testing and as such have always been a little frustrated with Postmans limitations in this area. When I say data-driven I am trying to achieve two goals. First I want to decouple the test from the test data but more importantly, I want to be able to parameterise my tests so that I iterate over each with different data which represent different scenarios/cases. I have some API endpoints which I hit tens of times but with different data. I am looking for advice on a better way to structure and perform my testing. I will outline the issues below.
In Postman you can run a collection against a selected environment and/or a data file. The problem here is you can pass just one data file for a complete collection. While you can technically have all your required data in the single file it might become bloated that for brevity it is actually better to hard code your data in the test within the Postman UI.
What you can do is organise your collection tests into sub-directories and then select individual sub-directories to run, each using their own data. Likewise, you could split the tests up into individual collections. The big problem here is to run the whole suite of tests is no longer a click of a button but instead, you would have to go through each sub-collection of tests and run them. Linking sub-collections to test data might also become a frustration. Just as importantly, you no longer get a single concise report which is a big problem.
My workaround to date has been to use Newman to hit specific tests or groups of tests with a specific data file. This allows me to separate out my test data to match collections and also means I can kick off all my tests in one go (by firing all the commands in one go). The problem is I am still just running sub-collections when I do this and would like to end up with a single report as if all the tests were run from one collection at the same time.
My second workaround is just to just write the tests in code but this is not the ideal solution as it makes the test suite a lot more technical. Our less technical testers would not be able to contribute to the suite and someone technical would need to always be available to maintain the API tests.
I have seen pm.iterationData and can imagine creating a pre-request script before each test to load iteration data for the test but it feels a bit hacky. I don’t know if this is even possible.
Thanks
