Testing Properties and Arrays correctly in GET

Hi, I just want to make sure that this is correctly testing that An Object and an Object within an array is present.

Example api:

{
  "items": [
    {
      "oID": "0000001211",
      "oInvoiceNo": "1234567",
      "OrderBlocks": [
        {
          "olLine": 1,
          "olProductCode": "001"   
        }
      ]
    }
  ]
}

Example test:

pm.test("Order Details are Present", () => {
    pm.response.to.have.jsonBody("oID");
    pm.response.to.have.jsonBody("oInvoiceNo");
    pm.response.to.have.jsonBody("OrderBlocks[0].olLine");
    pm.response.to.have.jsonBody("OrderBlocks[0].olProductCode");

});

It’s returning as a Pass, but I’ve been experimenting with syntax all day with loads of different tests(I can’t remember .js and pm.* so I just refactor constantly until it works), and I’m sure I had one earlier that brought back a False Pass, just wanted to make sure that this is a correct way for checking that the correct properties, and properties within arrays are present in a response.

I also don’t completely understand why this seems to work, shouldn’t “oID” also require me to have an array like (“items[0].oID”) ? Yet when I try this, it fails to find oID.

Was trying this initially and it failed, so the above is my new “fix”:

pm.test("Order Details are Present", () => {
     pm.response.to.have.jsonBody("items.OrderBlocks.olLine");
});

Any help/advice on testing this kind of stuff correctly is really appreciated, thanks!

Hey @LiamC,

I’m not sure what your overall goal is or what you would like to test but you could try something like this to check the types and keys of that response.

pm.test("Order Details are Present", () => {
    let jsonData = pm.response.json()
    pm.expect(jsonData.items).to.be.an("array");
    pm.expect(jsonData.items[0]).to.have.keys('oID','oInvoiceNo','OrderBlocks').and.be.an("object");
    pm.expect(jsonData.items[0].OrderBlocks).to.be.an("array");
    pm.expect(jsonData.items[0].OrderBlocks[0]).to.have.keys('olLine','olProductCode').and.be.an("object");
});

It’s a bit messy and it will only ever check the first object in those arrays. Lots of different ways that you could check/test that response just all depends on what you want to test for I guess.

Have you considered using some form of schema validation as a different option?

2 Likes

@dannydainton ! Oh wow, I’ve had your “All Things Postman” stuff open for the past 2 days trying to wrap my head around Postman. Crazy.

I think what I’m testing is probably made redundant by using:

const jsonData = pm.response.json();
const schema = JSON.parse(environment.testSchema)

pm.test("Schema is Valid", () => {
    pm.expect(tv4.validate(jsonData, schema)).to.be.true;
});

And having the API as ‘testSchema’ in Environment Variables.

I’m just trying to check that every response has the expected fields returned that I can see on the Swagger documentation of the API.

I was going to try and get something like forEach to work next, if I could, with pm.expects inside the function? For multiple ‘items’ arrays being returned.

jsonData.items.forEach(item => {
   pm.expect(jsonData.items[0]).to.have.keys('oID','oInvoiceNo','OrderBlocks')
});

But, maybe Schema validation is already checking all of this.
If I wanted to test if an object beyond the 1st one, was present in an array, how would that look?

I need to keep reading up more.
Your write-ups have been a huge help actually! I’m just greener than green at the moment so I’m probably getting confused!

Awesome! Thanks for checking that out - It’s very old now though and is falling behind the current functionality that we have introduced since then :cry:

You should be able to just use this syntax to get the schema from the variable:

const schema = pm.environment.get("testSchema")

What does your schema look like?

In terms of looping through the data, you could modify the test you wrote to check each object in the items array. I’ve taken the same response data and added another object with one of the keys different from what you’re checking against, just to see this fail.

If you add this code to your Tests tab, you should be able to see the failed test:

var sampleData = {
  "items": [
    {
      "oID": "0000001211",
      "oInvoiceNo": "1234567",
      "OrderBlocks": [
        {
          "olLine": 1,
          "olProductCode": "001"   
        }
      ]
    },
    {
      "wrongKey": "0000001212",
      "oInvoiceNo": "1234568",
      "OrderBlocks": [
        {
          "olLine": 2,
          "olProductCode": "002"   
        }
      ]
    }
  ]
}

pm.test("Debugging Test - Object Key are Correct", () => {
    sampleData.items.forEach(item => {
       pm.expect(item).to.have.keys('oID','oInvoiceNo','OrderBlocks')
    });
})

1 Like

Oh, I think I get the syntax now!

That Debug test example is pretty much exactly what I needed to understand, I think I was close-ish.

That’s just an example schema above, the one I’m actually working from belongs to the company I work for unfortunately, as much as I’d love to share it to get your insight on how to approach testing a brand new in-development API and what type of tests an experienced tester would do. In fact, this is all I’m doing at the moment - just trying to create a test case as robust as possible for the current proof of concept API I’ve been given, beyond 200 status, objects present and values match correct types/regex.

Regarding the debug test example, how would you go about automating these kinds of tests in a Collection Run? I feel like it would need engineering to fail but retrieve a “pass” test status so you can still integrate it with something like Jenkins, right?

Otherwise, a failed test that you intend to fail, would flag the API as not correct, when it’s functioning exactly how it should.

Thanks again!

EDIT: Can’t get tv4.validate to work either. I can set a variable to anything and it still returns as Pass. I don’t understand how to get tv4.validate to actually use the Schema. Example that should fail but doesn’t:

const = jsonData = pm.response.json();
var schema = "not real schema";
pm.test("Schema is Valid", () => {
    let validation = tv4.validate(jsonData, schema);
    if (validation !== true) {
        console.log(tv4.error);
    }
    pm.expect(validation).to.be.true;
});

I’ve tried reading through the tv4.validate documentation and I’m just not getting it, should I be copying the Schema into the Tests Tab to reference as you did in your above Debug Test example?. :confused:

1 Like

I would actually recommend using the ajv validation module instead - You can see an example of this in working in this other thread:

1 Like

I’ve copied the exact code in that link to, and changed the “id” to “string” instead of “integer” and no matter what I try, it will always pass.

I’m stumped.

You’ve copied the code and run it against your response?

That’s checking a different schema though, that user has a response like this:

{
    "access": [
        {
            "id": 1,
            "client": "127.0.0.1",
            "apikey": "apikey1",
            "description": "Local call"
        }
    ]
}

That wouldn’t work for your schema as the structure is different, it was more of an example of the structure.

You could use something like this:

var Ajv = require('ajv'),
    ajv = new Ajv({ logger: console, allErrors: true }),
    schema =  {
    "type": "object",
    "properties": {
      "items": {
        "type": "array",
       "items": {
         "type": "object",
         "required": [
             "oID",
             "oInvoiceNo",
             "OrderBlocks"
         ],
         "properties": {
           "oID": {
             "type": "string"
           },
           "oInvoiceNo": {
             "type": "string"
           },
           "OrderBlocks": {
             "type": "array",
             "items": {
               "type": "object",
               "required": [
                 "olLine",
                 "olProductCode"
               ],
               "properties": {
                 "olLine": {
                   "type": "integer"
                 },
                 "olProductCode": {
                   "type": "string"
                 }
               }
             }
           }
         }
       }
     }
   }
}

pm.test('Schema is valid', function() {
    pm.expect(ajv.validate(schema, pm.response.json()), JSON.stringify(ajv.errors)).to.be.true;
});

This should also log out all the schema errors so you’re not just getting a generic error message from the test result.

I suck at creating schemas, this is a site that I use to help generate them from a JSON response.

Hi All,

I also need some help to create some test cases where my url is fetching data in such form:

Group:[
{
“element”:{
“id”: “ABC”,
“name: :ABC”;
“ref”: 1
},
“rateplan”:[
{
“element”:{
“id”: “SAM”,
“name”: “XYZ”,
“ref”: 1000
},
“duration”:{
“type”: “Month”,
“amount”: 36
};
“category”: “Standard”,
“type”: “Postpaid”,
"price: [
{
“type”: “Month”,
“amount”: 1000,
},

there are various layers of data within elements.

any help is appreciated. I tried the examples provided above but i am getting syntax error.

thanks!!

Hey @Manoj_Mittal

Welcome to the community! :wave:

Would you be able to create a separate topic for this question please. I don’t want it to get lost within this topic thread. :slight_smile:

started a new topic, thanks!!

@LiamC writing schemas that work is not only an advanced topic but it is not really well supported in Postman.

My advice would be to stay away from schema validation, unless you REALLY know what you are doing.

Here are a few reasons:

  • writing and maintaining proper schemas is time consuming
  • there is no dedicated way to edit / save them and they tend to be quite big
  • just following a “paste here your response and I will generate something for you” tool rarely works
  • you can easily be fooled by “never failing schemas” (as you have already noticed
  • if later a schema validation does fail, you will have a hard time figuring out what has changed (at least with the tv4 lib)

Maybe the new ajv lib in Postman does a better job than the tv4 lib.

So consider if schema validation is something that you want to invest some time or if you can simply live with testing properties one-by-one.

Good luck!

1 Like

I managed to get something to work, after banging my head against a wall for hours.
I do agree this schema validation stuff may be a lot more than I can handle at the moment, so I may go back to what I had originally.

It’s been a learning experience though!

If you’re interested by the way, this is what I found would actually test Passes & fails correctly:

const jsonData = pm.response.json();

pm.test('Schema is valid', function() {
    var Ajv = require('ajv');
    ajv = new Ajv(),
    schema = { //Schema should contain an Array
        "type": "array",
        "items": {//In this Array there should be a Properties Object
                    "type": "object", //These are the properties within this object. This will test TYPE of Property.
                    "properties": {
                        "gameId": { "type": "integer" },
                        "name": { "type": "string" },
                        "description": { "type": "string" },
                        "minPlayers": { "type": "integer", "maximum": 1},
                        "maxPlayers": { "type": "integer", "maximum": 1},
                        "rating": {"type": "string" },
                        "online": {"type": "boolean" }
                        
            },"required" : ["gameId", "name", "description", "minPlayers", "maxPlayers", "rating", "online"]//Required will make sure that the Properties listed are present
        }
    };
    
    var validate = ajv.compile(schema);
    
    var valid = validate(jsonData)
    if(valid === false){
        console.log('Invalid: ' + ajv.errorsText(validate.errors));
    }
    
    pm.expect(valid).to.be.true;
  
});

Thank you

Glad you got a solution that works for you and learnt some new skill along the way.

It feels like the outcome changed slightly from the originally posted response data - Looks like a different endpoint is being checked now :grin:

Yeah, another example api I had set up for Crud testing haha.

1 Like