Hi,
Appreciate to share the standard MS Excel template. The standard MS Excel Template in which data taken from IBM Watson Assistant bot can be uploaded directly in BOTIUM for testing.
e.g. Say we have 1000+ utterances along with 1000+ Intents. Then in such scenario taking data in standard MS Excel format from IBM Watson Assistant and then directly uploading it in BOTIUM will save lot of time. It will be highly efficient way of testing too.
Hi,
It is already possible with our “Test Case Wizard” to download the Watson workspace, including intents + utterances. Also, Botium supports using Excel or Google Sheets for writing test cases. Can you please share what standard MS Excel template for IBM Watson Assistant you have in mind ?
Hi Florian,
Please find the standard MS Excel template for IBM Watson Assistant. Please do refer the sheet named “Conversation” from the attachment.
Thank You,
Vivek Chauhan
INDIA
that looks nice. can you please tell me where it is possible to download such an excel template ? Is there an export function from Watson Assistant ?
Hi Florian,
From IBM Watson we can download the code in CSV file. It will be challenging to download and then upload it in BOTIUM being customer data and IP related issues involved.
Current approach is:-
- We are getting data in MS Excel templates for testing from SME’.
- We are manually entering the utterances in the LIVE chat of BOTIUM box. Hence evaluating the test results.
Expected and efficient approach for large number of utterances and Intents say more than 1000 utterances and intents.
- Uploading all such Intents and Utterances in MS Excel format in BOTIUM. Then evaluating the test results.
Remarks: There are MS Template for same in BOTIUM Enterprise version. My concern is that in that template there is only two column ie. for Intents and Utterances. In such scenario:-
i. How the sub-intents will be handled by BOTIUM.
ii. For conversational BOTs where BOT reply back with options say Yes | NO, or with some options say A,B or C. In such scenario how to put such data in BOTIUM 's MS Excel template?
Finally it is observed that for BOTIUM Enterprise version, when we tried to bind our IBM Watson based chatbot with BOTIUM using API Key. Then loading workspace failed. Can you please help to find the root cause of this issue? Please refer the attached screenshot.
When i checked the same on trail/mini version of BOTIUM. It is running perfectly.
(I have not find the option to upload data here yet. e.g. Uploading MS Excel tempate )
I don’t really see the benefit of doing the data exchange by CSV instead of using the Botium Test Case Wizard - see for example this blog post. Botium supports CSV, Excel file and Google Sheet import, but you will have to transform the data from the file you received from SME.
Doing it in the Botium Box Live Chat is obviously not the way to go.
With BotiumScript you can assert for the intents returned from Watson (with the INTENT asserter) as well as quick responses (with the BUTTONS asserter). BotiumScript can be used in text files, Excel files, JSON files and more - see the BotiumScript documentation.
The issue with binding Botium to Watson is most likely related to a firewall blocking the Watson API access.
Hi Florian,
While we are in touch with concerned team to resolve the issue as mentioned by you ,“The issue with binding Botium to Watson is most likely related to a firewall blocking the Watson API access.”
Hope it is not due to," bug with Watson and will be resolved with Botium Box 1.3"
As mentioned in some old post, which we found on internet.
Refer
Thank You,
Vivek Chauhan
INDIA
no, this is not related.
HI Florian,
We had consulted the expert in our organization. Following observations were made :-
1.Port 80 is opened for connectivity as mentioned in BOTIUM tutorial too.
2.Still there is challenge to connect the BOTIUM installed at Azure Linux VM server with internet
3.It is not initiating any traffic to place where IBM Watson Assistant is hosted ie. at Frakfurt (Legacy)
4. Please refer the logs and provide you inputs:-
Say Hello (Check Connectivity)
Error WATSON_WORKSPACE_ID capability required for V1
Hide Logs
2020-12-22T08:24:12.318Z botium-core-BotDriver Loaded Botium configuration file ./botium.json
2020-12-22T08:24:12.324Z botium-core-BotDriver Changed capability TEMPDIR to “./botiumwork” using environment variables.
2020-12-22T08:24:12.325Z botium-core-BotDriver Build - Botium Core Version: 1.10.2
2020-12-22T08:24:12.329Z botium-core-BotDriver Build - Capabilites: { PROJECTNAME: ‘Botium Project’, TESTSESSIONNAME: ‘Botium Test Session’, TESTCASENAME: ‘Botium Test Case’, TEMPDIR: ‘./botiumwork’, CLEANUPTEMPDIR: true, WAITFORBOTTIMEOUT: 10000, SIMULATE_WRITING_SPEED: false, SIMPLEREST_PING_RETRIES: 6, SIMPLEREST_PING_TIMEOUT: 10000, SIMPLEREST_PING_VERB: ‘GET’, SIMPLEREST_PING_UPDATE_CONTEXT: true, SIMPLEREST_STOP_TIMEOUT: 10000, SIMPLEREST_STOP_VERB: ‘GET’, SIMPLEREST_POLL_VERB: ‘GET’, SIMPLEREST_POLL_INTERVAL: 1000, SIMPLEREST_POLL_UPDATE_CONTEXT: true, SIMPLEREST_METHOD: ‘GET’, SIMPLEREST_IGNORE_EMPTY: true, SIMPLEREST_TIMEOUT: 10000, SIMPLEREST_EXTRA_OPTIONS: {}, SIMPLEREST_STRICT_SSL: true, SIMPLEREST_INBOUND_UPDATE_CONTEXT: true, SIMPLEREST_CONTEXT_MERGE_OR_REPLACE: ‘MERGE’, SCRIPTING_TXT_EOL: ‘\n’, SCRIPTING_XLSX_EOL_WRITE: ‘\r\n’, SCRIPTING_XLSX_HASHEADERS: true, SCRIPTING_CSV_SKIP_HEADER: true, SCRIPTING_CSV_QUOTE: ‘"’, SCRIPTING_CSV_ESCAPE: ‘"’, SCRIPTING_NORMALIZE_TEXT: true, SCRIPTING_ENABLE_MEMORY: false, SCRIPTING_ENABLE_MULTIPLE_ASSERT_ERRORS: true, SCRIPTING_MATCHING_MODE: ‘wildcardIgnoreCase’, SCRIPTING_UTTEXPANSION_MODE: ‘all’, SCRIPTING_UTTEXPANSION_RANDOM_COUNT: 1, SCRIPTING_UTTEXPANSION_NAMING_MODE: ‘justLineTag’, SCRIPTING_UTTEXPANSION_NAMING_UTTERANCE_MAX: ‘16’, SCRIPTING_MEMORYEXPANSION_KEEP_ORIG: false, ASSERTERS: [], LOGIC_HOOKS: [], USER_INPUTS: [], SECURITY_ALLOW_UNSAFE: true, SIMPLEREST_INBOUND_REDISURL: ‘redis://redis:6379’, SIMPLEREST_REDIS_TOPIC: ‘qprod_SIMPLEREST_INBOUND_SUBSCRIPTION_PROD_SIMPLEREST_INBOUND_SUBSCRIPTION’, TWILIO_IVR_REDISURL: ‘redis://redis:6379’, TWILIO_IVR_REDIS_TOPICBASE: ‘qprod_SIMPLEREST_INBOUND_SUBSCRIPTION_PROD_TWILIO_IVR’, CONTAINERMODE: ‘watson’, WATSON_ASSISTANT_VERSION: ‘V1’, WATSON_URL: ‘https://gateway-fra.watsonplatform.net/assistant/api’, WATSON_APIKE1Y: ‘****************’, WATSON_USER: null, WATSON_PASSWORD: null, WATSON_ASSISTANT_ID: ‘’, WATSON_WORKSPACE_ID: ‘’, WATSON_COPY_WORKSPACE: false }
2020-12-22T08:24:12.329Z botium-core-BotDriver Build - Sources : { LOCALPATH: ‘.’, GITPATH: ‘git’, GITBRANCH: ‘master’, GITDIR: ‘.’ }
2020-12-22T08:24:12.330Z botium-core-BotDriver Build - Envs : { IS_BOTIUM_CONTAINER: true }
2020-12-22T08:24:12.601Z botium-connector-PluginConnectorContainer-helper Botium plugin botium-connector-watson loaded. Plugin version is 0.0.21
2020-12-22T08:24:12.601Z botium-connector-watson Validate called
2020-12-22T08:24:12.601Z botium-core-BotDriver BotDriver Build error: Error: WATSON_WORKSPACE_ID capability required for V1
WATSON_WORKSPACE_ID capability required for V1
Thank You,
Vivek Chauhan
INDIA
First, port 80 is not sufficient. For Watson you will need port 443 outbound opened.
Botium is loading the list of available Watson workspaces into the dropdown list, as soon as network connectivity is available and the credentials are fine.
Hi Florian,
Appreciate you quick replies. As per expert for outbound we have no restrictions.
Any inputs what might be causing this challenge.
It is clarified that on BPTIUM’s mini/trial version we were easily able to connect it with IBM Watson bot. Hence list of workspaces were being displayed.
Thank You,
Vivek Chauhan
INDIA
The EAI_AGAIN error that is displayed in the screenshot has something to do with DNS hostname resolution (see https://www.codingdefined.com/2015/06/nodejs-error-errno-eaiagain.html). Ask the experts to check if DNS resolution is working on the server.
Hi Florian,
As an alternate temporary approach we installed the BOTIUM Enterprise version on AWS Linux VM. Because Issue still remain with Azure Linux VM.
There in AWS Linux VM we are able to connect it with IBM Watson Workspace using API key.
The challenge here is :- While running the test case it is being shown as pending from past around 20 minutes.
Upon contacting our infrastructure team , they confirmed that there is sufficient memory space. The details are:-
- Memory available is 31 GB
- CPUs available : 8
- Report shared by infra team is:-
top - 10:45:32 up 33 days, 17:07, 2 users, load average: 0.13, 0.14, 0.14
Tasks: 217 total, 1 running, 216 sleeping, 0 stopped, 0 zombie
%Cpu(s): 2.6 us, 0.6 sy, 0.0 ni, 96.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.1 st
KiB Mem : 32779004 total, 15139756 free, 5130692 used, 12508556 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 27211900 avail Mem
So looking for your inputs to find out the root cause for this pending period of test case execution.
Refer attached screenshot.
Thank You,
Vivek Chauhan
Looks like the Botium Box Agents are not connected to the Botium Box Server. You can have a look how many agents are there by going to the System Settings, settings Agents
Hi Florian,
We are currently using admin as agent and same has been registered as per your suggestions. Still we are facing some challenge.
Appreciate to describe the steps to connect Botium Box Agents to the Botium Box Server in detail.
Refer screenshot too.
Thank You,
Vivek Chauhan
INDIA
Hi Florian,
We desire to connect with you. Will share the screen and help us fix this issue. We have demo for tomorrow.
Thank You,
Vivek Chauhan
This is a misunderstanding - a Botium Box Agent has nothing to do with user logins. A Botium Box Agent is a separate component running in the background for executing the test sessions. In the docker-compose-file, this is the agent-service, and there should be a docker container running for it (named botiumbox_agent_1)
But this is actually the default configuration, that an agent is started and connected, so usually there is nothing to verify or configure.
Appreciate to join us the call. I have sent you the invite
Hi Florian,
Attached is the screenshot for your reference. We have sent it to your email too.
Please help.
Thank You,
Vivek Chauhan
INDIA
Hi Florian,
Can you please share the right syntax/file name to be put in convo file for Utterance list in excel format?
We have uploaded the excel file utterance in desired format. Now facing challenge to put its name in convo file.
e.g For normal utterance file we easily put say UTT_Utterances in convo file.
How to do same for excel file?
Consider Utterances file name as “Utterances” in excel format.
We referred your tutorial. But still facing challenge.
Thank You,
Vivek Chauhan