Botium returns test successfully and in ChatBot the answer is wrong

Hey, guys.

I runing test with botium and directLine (microsoft bot framework). And I verify that using botium shows that the bot answers are correct, I write the same questions in ChatBot manually and the bot answered wrong. What can it be?
can you help me?

attached as proof of tests

hi @ujja and @Florian

any help or idea what it could be?

Could you post the botium script please? (if you are using txt format, then its *.convo.txt)

where can i find the botium script?

I have separated the questions and answers in a separate folder. and creating the conversation flow in the convo folder.



this bot is using a QnA to fetch the Q&A, could this be interference?

ok, then which conversation flow covers the case, where your manual test fails?

For example if I run a question by botium more than 20 times it doesn’t display an error, if I run the same question multiple times manually the bot displays an error.

i ran by postman and it actually displays error, when asked bot multiple times. and by botium at no time presented the error.

for example, I ran this question several times through botium: “Who owns Blackline” - no error. See in bot, I did the same thing and it displayed error, in postman it also displays error

In Botium you have to set your expectations somehow.

most basic sample:



It means botium will send “Hello!”, and expect “Hi!” as answer.

See Technical Reference — Botium documentation and Botium in a Nutshell, Part 1: What Is Botium ? | by Florian Treml | Medium

Yes I know.

But botium does not display when the bot responds differently than expected.

And can I see what is your expectation?