Dialogflow API level testing

[nabsofken]

I am confused about the API level testing mentioned in the botium in a nutshell wiki (https://floriantreml.medium.com/botium-in-a-nutshell-part-9-selecting-a-botium-connector-f27a56d55bf4). The chatbot I am using uses dialogflow as its NLU engine and interactions are made using the dialogflow api. I have the following questions:

1. What exactly is the API level test for? How is this different than Backend Level / Dialogue and NLU/NLP Engine Testing mentioned?
2. To test the conversations should I test using the dialogflow connector and botiumScript or should I do it at the API level through the dialogflow endpoint and the http/json connector?
3. How can the botium dialogflow connector be used to import, using dialogflowimport, utterances for intents where the entities have synonyms? Can this be done automatically? If not, whats the best way to handle entity synonyms and variables?

[Florian]

@nabsofken

What exactly is the API level test for? How is this different than Backend Level / Dialogue and NLU/NLP Engine Testing mentioned?

From an architecture point of view, the Orchestration API is the single-point-of-access into the chatbot process - so it has a “text in / text out” API to be used for the channels hosting your chatbot (website widget, Facebook webhook, …).

In the article you are refering to there is also this statement:

Note that in most cases the NLU/NLP engine and the Dialogue engine are combined, for example with IBM Watson Assistant, or Dialogflow. For Microsoft LUIS and Azure Bot Service, those components are spearated though.

While NLU engine, dialogue engine and Orchestration API are separate components with separate responsibilities, it is depending on the technology often component-wise not separated at all.

• Dialogflow for example combines NLU engine, dialogue engine and Orchestration API (fulfillment webhooks) into one component
• Rasa on the other hand allows very fine-grained configuration of what components should be used in the processing pipeline.
• wit.ai and spacy are NLU engines only (extract computer-readable information out of plain text)

Typically, the Orchestration API is the right point to start conversation flow testing, and this is one of the most-used entry points for Botium.

@nabsofken

To test the conversations should I test using the dialogflow connector and botiumScript or should I do it at the API level through the dialogflow endpoint and the http/json connector?

This is not an either/or question. You clearly should use the Dialogflow connector (which connects to the Dialogflow endpoint) together with BotiumScript for describing your conversation flows.

@nabsofken

How can the botium dialogflow connector be used to import, using dialogflowimport, utterances for intents where the entities have synonyms? Can this be done automatically? If not, whats the best way to handle entity synonyms and variables?

The Botium Dialogflow connector has some helper functions getting you started more quickly - the importer functions for extracing information from the Dialogflow agent itself and build basic test case files out of it. This is really just a starting point, for making the most out of it you have to get your hands dirty and write/copy/extend the BotiumScript that make up the test cases.

There are plenty of NLP asserters available (see Botium Docs), and the Botium Dialogflow connector has some additional functionality that might come in handy (for example, for handling Dialogflow context, see Github Repository).

[nabsofken]

thanks so much @Florian