Do I need to use the generic http connector?

Botium Core
1

I have a chatbot that connects to a dialogflow agent, however, not directly, there is a middleware in between. The request is sent to a custom endpoint first, the request is then pushed to dialogflow, the response returned from dialogflow is then formatted, fallbacks go through another custom built model, and finally the transformed response is sent to the chatbot frontend. When running botium using the dialogflow connector, I don’t get the text response. I get intent information back but that about it. What’s the best approach to testing this bot?
The custom response returned looks something like this The fulfillmentMessages array represents a collection of response bubbles:

{
	"responseId": "{{ID}",
	"queryResult": {
		"fulfillmentMessages": [
			{
				"text": {
					"text": {
						"responseJP": "",
						"responseES": "",
						"responseFR": "",
						"response": "OK. I have information on topics."
					}
				},
				"platform": "CUSTOMv1"
			},
			{
				"text": {
					"text": {
						"responseES": "",
						"responseJP": "",
						"response": "Please go ahead and ask your question.",
						"responseFR": ""
					}
				},
				"platform": "CUSTOMv1"
			}
		],
		"outputContexts": [

		],
		"queryText": "Answer a question",
		"speechRecognitionConfidence": 0,
		"action": "",
		"parameters": {
			"fields": {
			}
		},
		"allRequiredParamsPresent": true,
		"fulfillmentText": "",
		"webhookSource": "",
		"webhookPayload": null,
		"intent": {
			"inputContextNames": [

			],
			"events": [

			],
			"trainingPhrases": [

			],
			"outputContexts": [

			],
			"parameters": [

			],
			"messages": [

			],
			"defaultResponsePlatforms": [

			],
			"followupIntentInfo": [

			],
			"name": "projects/test-proj/agent/intents/{{SessionID}}",
			"displayName": "My_CustomIntent",
			"priority": 0,
			"isFallback": false,
			"webhookState": "WEBHOOK_STATE_UNSPECIFIED",
			"action": "",
			"resetContexts": false,
			"rootFollowupIntentName": "",
			"parentFollowupIntentName": "",
			"mlDisabled": false
		},
		"intentDetectionConfidence": 1,
		"diagnosticInfo": null,
		"languageCode": "en",
		"sentimentAnalysisResult": null
	},
	"webhookStatus": null,
	"outputAudio": {
		"type": "Buffer",
		"data": [

		]
	},
	"outputAudioConfig": null,
	"adverseExists": false,
	"adverseQuery": false,
	"adverseEventCaught": false
}
2

The JSON/HTTP generic connector is hard to understand, are you able to provide a full working example or maybe guide me on how I could use it for the above response.
This is my url format (POST method): https://my-site/dialogflow/{{df-project-id}}/{{session-id}}/en
sample request body

{"query": {
		"text": {
			"text": "how are you?",
			"languageCode": ""
		},
		"button": false
	},
	"source": "",
	"queryParams": {
		"contexts": [
			{
				"name": "projects/test-proj/agent/sessions/{{sesion_id}}/contexts/{{df_intent_name_1}}",
				"lifespanCount": 1,
				"parameters": null
			},
			{
				"name": ""projects/test-proj/agent/sessions/{{sesion_id}}/contexts/{{df_intent_name_2}}",
				"lifespanCount": 1,
				"parameters": null
			},
			{
				"name": "projects/test-proj/agent/sessions/{{sesion_id}}/contexts/{{df_intent_name_3}}",
				"lifespanCount": 1,
				"parameters": null
			}
		]
	}
}
3

Documentation is available in the Botium Wiki and Docs:

4

Thanks, I was able to configure it for my project. When the bot returns multiple responses I have to assert them individually i.e. every response needs to be preceded with #bot. Is there a way to assert them all at once
e.g.

Desired
#me
How are you?

#bot
I’m fine
BUTTONS button1|button2

Actual
#me
How are you?

#bot
I’m fine
#bot
BUTTONS button1|button2

5

Can you post an example JSON response with multiple sections, for me to show you the JSONPath selectors.

6

hi @Florian here is a sample response with multiple responses and response types (text, carousel, quick reply)

"queryResult": {
	"fulfillmentMessages": [
		{
			"platform": "CMPv1",
			"text": {
				"text": {
					"response": "<p>When traveling with Basaglar, it’s important to store it at the right temperature.</p>"
				}
			}
		},
		{
			"text": {
				"text": {
					"response": "<p>Product must be used within 1 month</p>"
				}
			},
			"platform": "CMPv1"
		},
		{
			"text": {
				"text": {
					"response": "<p>If you’re traveling by airplane, here are some tips:</p>"
				}
			},
			"platform": "CMPv1"
		},
		{
			"carousel": {
				"carousel": {
					"response": [
						{
							"imageUri": "https://storage.googleapis.com/travel-Proof.jpg",
							"button": "",
							"postback": "",
							"id": 3,
							"body": "",
							"subtitle": "",
							"title": ""
						},
						{
							"subtitle": "",
							"body": "",
							"button": "",
							"postback": "",
							"imageUri": "https://storage.googleapis.com/travel-Travel.jpg",
							"id": 3,
							"title": ""
						}
					]
				}
			},
			"platform": "CMPv1"
		},
		{
			"platform": "CMPv1",
			"text": {
				"text": {
					"response": "<p>Please prepare in advance.</p>"
				}
			}
		},
		{
			"platform": "CMPv1",
			"carousel": {
				"carousel": {
					"response": [
						{
							"body": "",
							"id": 5,
							"postback": "https://www.tsa.gov",
							"subtitle": "fsdfasdf",
							"title": "Title with one line sample, test test test ",
							"button": "<p>Open page</p>",
							"imageUri": ""
						},
						{
							"button": "<p>Open page</p>",
							"id": 5,
							"title": "Title with one line sample, test test test ",
							"postback": "https://www.tsa.gov",
							"subtitle": "fsdfasdf",
							"body": "",
							"imageUri": ""
						},
						{
							"postback": "https://www.tsa.gov",
							"id": 5,
							"button": "<p>Open page</p>",
							"imageUri": "",
							"subtitle": "fsdfasdf",
							"title": "Title with one line sample, test test test ",
							"body": ""
						}
					]
				}
			}
		},
		{
			"quickReplies": {
				"quickReplies": [
					{
						"x": -236,
						"button": "<p>Got it, thanks!</p>",
						"inslot": "",
						"intent": ""
					}
				],
				"title": ""
			},
			"platform": "CMPv1"
		}
	]

This is my current delegateCaps object

  this.delegateCaps = {
    [CoreCapabilities.SIMPLEREST_URL]: this.caps[Capabilities.MYAPI_URL],
    [CoreCapabilities.SIMPLEREST_METHOD]: 'POST',
    [CoreCapabilities.SIMPLEREST_BODY_JSONPATH]: '$.queryResult.fulfillmentMessages[*]',
    [CoreCapabilities.SIMPLEREST_RESPONSE_JSONPATH]: '$.text.text.response',
    [CoreCapabilities.SIMPLEREST_BUTTONS_JSONPATH]: '$.quickReplies.quickReplies[*].button',
    [CoreCapabilities.SIMPLEREST_BODY_TEMPLATE]: require('./df_req.json')
  }
1 Like
7

I found out that this is something currently not possible to configure via the JSONPath-capabilities, it would require Javascript-Coding to process the JSON response.

Is there a special reason why you do not want to assert on the individual response components ? It looks like in a chat window those components would be shown in separate chat bubbles each, so assertions could follow this logic.

8

hi @Florian the only reason was for convenience. They are separate bubbles so I can keep individual assertions.

9

hi @Florian for the json connector, how can I extract and assert custom fields such as carousel or forms? In the docs I only see SIMPLEREST_BUTTONS_JSONPATH and SIMPLEREST_MEDIA_JSONPATH. Sample request is in my previous comment.

10

Two options.

  1. Use the generic JSONPath-Asserter to assert on custom payload content
  2. Write our own Response Hook to extract custom payload content, map it to Botium payload and use the included asserters
11

I tried that but I am getting a wrapBotiumError

#bot:
      at BotiumBindings.wrapBotiumError (node_modules/botium-bindings/src/BotiumBindings.js:77:14)
      at /Users/nabeelibrahim/Desktop/botium-conversation-testing/node_modules/botium-bindings/src/BotiumBindings.js:134:33
      at runNextTicks (internal/process/task_queues.js:58:5)
      at listOnTimeout (internal/timers.js:523:9)
      at processTimers (internal/timers.js:497:7)

My response hook looks like this:

this.delegateCaps[CoreCapabilities.SIMPLEREST_RESPONSE_HOOK] = ({ botMsg, botMsgRoot }) => {
        if (botMsgRoot.status === 'error') throw new Error(`MyAPI Error: ${botMsgRoot.message}`)

        if (botMsgRoot.quickReplies && botMsgRoot.quickReplies.quickReplies.length > 0) {
          botMsg.buttons = botMsgRoot.quickReplies.quickReplies.map(b => ({ text: b.button }))
        }
        if (botMsgRoot.carousel && botMsgRoot.carousel.carousel.response.length > 0) {
          botMsg.carousel = botMsgRoot.carousel.carousel.response.map(c => ({ url: c.imageUri, title: c.title, button: c.button }))
          console.log("bot message:\n"+JSON.stringify(botMsg)+"\n");
        }
        
      }
12

figured out the above issue, I used media instead of carousel. However the MEDIA asserter seems to just do partial matching
the bot returns

#bot:   MEDIA(https://storage.googleapis.com/test-images/ST02-Proof.jpg)
  MEDIA(https://storage.googleapis.com/test-images/ST02-Travel.jpg)

My test case has this assertion:

#bot
MEDIA https://storage.googleapis.com/test-images/ST02-Trav

This still passes the test even though the url is not complete and the other url is not present.

Another question:
The MEDIA type only seems to accept mediaUri, my carousel object contains the uri, text, and a button. How can I extract the text and button and assert against that?

13

yes, media assertion is substring matching

you can map the carousel to a sequence of card objects in your response hook, and you can then use the buttons/media/card asserters

https://botium.atlassian.net/wiki/spaces/BOTIUM/pages/38502401/Howto+develop+your+own+Botium+Connector

14

@Florian thanks I got it to work with cards. However, I have a few questions:

  1. how do i verify that i have the right number of cards? in my above example there are two carousels. How do i assert only 2 appear? My test case only asserts two CARDS items and technically the testcase would still pass even if there were more than 2.
  2. Also how do i assert that the last bot message in my testcase is the last response sent by the chatbot?
15

I added those things to our backlog for Botium Core. Those are reasonable requirements, so I gave them a high priority.

1 Like
16

@Florian when running tests through botium cli i encounter this issue quite a bit

########################################
TC01_HELLO_BOT/Cell B2: error sending to bot - rest request failed: ESOCKETTIMEDOUT
------------ TRANSCRIPT ----------------------------
#me: travel
      at BotiumBindings.wrapBotiumError (node_modules/botium-bindings/src/BotiumBindings.js:77:14)
      at /Users/user/Desktop/botium-conversation-testing/node_modules/botium-bindings/src/BotiumBindings.js:134:33

Do you know why this is or how i can avoid it?

17

this means that the http endpoint is not available

18

@Florian whats the recommended process of validating responses that contain formatted text such as bullet points, bolded text

19

By default, for doing assertions, the text is cleaned from any markup and formatting instructions (like line breaks, HTML tags etc). You can read about the “normalization” in our Wiki.
You can disable it and then you can apply exact string matching, including formatted text.

1 Like
20

@Florian I was having trouble asserting quickreply buttons that contain a link/url. i tried BUTTONS button_text | url but this was considering the url to be another button. Is there a way for me to validate this scenario?