APIDog Client Tests complete but CLI Client Tests fail

[oborchers]

I am having problems with the API client. I have a set of tests and if I run them in the client (Europe) they work nicely and all of them pass.
If I am now running the CI/CD pipeline from my terminal, using the following command

apidog run https://api.eu.apidog.com/api/v1/projects/346712/api-test/ci-config/345383/detail?token=xxxxx -r html,cli

I am getting:

┌─────────────────────────┬───────────────────────────────────┬─────────────────────────────────┐
│ │ Executed │ Failed │
├─────────────────────────┼───────────────────────────────────┼─────────────────────────────────┤
│ Iterations │ 1 │ 1 │
├─────────────────────────┼───────────────────────────────────┼─────────────────────────────────┤
│ Http Requests │ 74 │ 14 │
├─────────────────────────┼───────────────────────────────────┼─────────────────────────────────┤
│ Assertions │ 23 │ 0 │
├─────────────────────────┴───────────────────────────────────┴─────────────────────────────────┤
│ Total Time Consuming: 18.8s │
├───────────────────────────────────────────────────────────────────────────────────────────────┤
│ Total Response Size: 396.68KB (approx) │
├───────────────────────────────────────────────────────────────────────────────────────────────┤
│ Average Request Time: 201.39189189189193ms [min: 43ms, max: 820ms, s.d.: 159.2098579020077ms] │
└───────────────────────────────────────────────────────────────────────────────────────────────┘

Therefore the CLI client does not produce the same result as the Europe client.

[frank_williamss (Frank)]

Thanks for the feedback. It would be helpful if you could provide specific error messages and screenshots from the CLI.

For detailed information about CLI FAQs, please check here: Apidog CLI Options - Apidog Docs

[oborchers]

Hi there! I have been checking this out and:
the error says: “$should not have additional properties pagination”

I have attached the full report to this. If I check the original endpoint definition this is clearly available and also well defined. Executing the identical (referenced) case on the Client works as expected

[frank_williamss (Frank)]

Could you please provide your cli version number? Are there specific reproduction steps? For detailed information about Validate Additional Fields, please check here: Validate responses - Apidog Docs.

[oborchers]

Frank : Thanks for the fast reply:

apidog --version
1.3.12

Regardling additional field validations: As said: All cases / tests run perfectly inside of the Client (EU) but not inside the CLI. At this point I am very confident that I configured the additional output properties correctly. If you need further details, my account is [email protected] (Oxolo API / Production).
Most recent test run on the client: Apidog Europe
Most recent test run on the cli: Apidog Europe

Command used to run: apidog run https://api.eu.apidog.com/api/v1/projects/346712/api-test/ci-config/345383/detail\?token\=xxxx -r json,cli --upload-report --env-var “IMAGE_FILE_PATH=/Users/oliverborchers/Desktop/image.png”

[frank_williamss (Frank)]

Could you please tell me your version of Apidog? The absence of the “$should not have additional properties” error message in the client might be due to an older version.

[oborchers]

Sure! I am using the API Dog Europe client version 2.5.32. During our conversation here I also upgraded to the most recent version. The error happened before and after then update.

[oborchers]

Additionally, I have enabled strict validation for that specific project.

[frank_williamss (Frank)]

Are the test results the same for both the Apidog Client and CLI Client now? Yeah, you enabled strict validation for that specific project, so it had error prompts.

[oborchers]

No. They are still different as in the beginning.
Client: Passed
CLI: Failed

CLI run via apidog run https://api.eu.apidog.com/api/v1/projects/346712/api-test/ci-config/345383/detail\?token\=xxxx -r html,cli --upload-report --env-var “IMAGE_FILE_PATH=/Users/oliverborchers/Desktop/image.png”

[oborchers]

Will be ooo for 2 weeks now. Would love if we could find a solution for this, since this is the only thing blocking us right now from onboarding our team to as our default provider