I apologize if this was asked before, I couldnāt quite find the search term that would return similar questions.
Anyway, Iām thinking in terms of CI/CD. As issues are completed and their respective code checkins are pushed to the repo, tests will run and upon them passing, they will be promoted and deployed to other environments. E.g.
- Change code
- Push code
- Build code
- Run unit tests
- Deploy to isolated test environment
- Run API functional tests (postman)
- Break down isolated environment
- Deploy to QA/UAT environment and send out notifications
- QA runs runs manual tests and select users beta test
- Deploy to production and send notifications
The problem I see is that if we add tests to the āglobalā collection as weāre developing, theyāre going to start failing in every environment that doesnāt yet have those changes. The only workaround I can see right now is that the functional tests are written and added to the collection(s) AFTER the deployment which is problematic because a) which deployment and b) that breaks the CI/CD flow and detracts from TDD practices.
Does Postman have a solution for this? For example, can the tests simply be part of the source code so that it gets merged and travels along with source control? If so, how do I smoothly get those changes into the āMonitorā that runs on a schedule?
I feel like this would be best achieved if the following criteria were met (if theyāre not already and I just havenāt found it)
- The postman application (in windows, for instance) must have the ability to load a postman configuration that resides in my local repo. That way I can develop the tests in the Postman app and have those changes be part of my source branch. Also, if I switch branches, the Postman app can be updated to use that configuration.
- I may be working in a multi-project architecture (e.g. microservices) where I have many projects that will included in the same test suite, so Iāll need a way to manage this in the Postman app. A good example for this would be the environment manager, but instead, for applications. Iād basically save a path to the configuration file on a per application basis, that way I can switch the active configuration from project A to project B and have Postman already know which configuration file to load (since itāll be a file in that applications local repo on the file system)
- The configuration files that travel with the branch need to be developer-readable so that we can manage any merge issues that arise.
- Newman needs to be able to read this file in order to execute tests within the build pipeline
- As changes are merged to master branches (multiple projects/repos), weāll need some way to ensure that they are properly pushed/merged to a central repo or configuration service and eventually to whatever remote instance of the Postman API Monitor is running so that it can receive those test updates. The convergence here could be a challenge.
So is this currently achievable in any degree?
Maybe something like this:
Here, Iām illustrating that several branches of several projects will be using the same postman configuration file, but at some point, they need to converge. The configuration contained within the applicationās source control will be used to run tests via newman within the isolated test environment. I placed a hypothetical service to take care of that. it would need to support merging and versioning to some degree. When the config is conflict free (which may need interaction), then it gets pushed to the monitoring service. By the end of the line, it should everything it needs to run all updated tests.
I can see that the compromise is that each project gets itās own collection and it could help reduce or eliminate the need for the configuration file convergence, but I feel like in a microservice architecture, that could become problematic because of any inter-dependencies.