Performance testing on botium box

Botium Box
#1

Hello, I ran few load tests today and I have few questions.

  1. Some of my load tests failed. Is there an option to see why they failed?
  2. I am running a load test for 60 mins with multiplicator 10 and 25 parallel users. It is past 60 mins and is still running. Can you help me understanding this?
  3. It looks like load test results doesn’t appear under TEST RESULTS COLUMN. How can I see my previous results?
  4. When I try to download test results, I get a an error which says download failed with response code 404.
  5. I am looking at this graph and I don’t understand these colors and why does it shows colors for few and not for all?

bb
bb1521×443 64.5 KB

#2
  1. Yes, you can download the list of failures in the “Downloads and Analytics” tab of the performance test view
  2. The performance test itself is maybe ready, but processing and aggregating the result can take longer, depending on the computing power - you can see some hardware requirements in the Botium Wiki
  3. Either in the Test Project view, tab “Performance Tests”, or in the Test Results view, tab “Performance Tests”
    image
    image1267×722 100 KB
  4. What version of Botium Box is installed ? This is a problem that has been resolved with Botium Box 2.8.1
  5. This is a technical limitation of this chart type. For performance testing, usually the size of the test set is rather small (1-10 different test cases), but it is repeated often. It is not recommended to run performance tests with a full-blown regression or NLP test set.
#3

Hi Florian, thanks for response.

  1. I didn’t set any failure criteria. My test status in general says FAILED and I don’t see an option to see why it failed not the failed test cases.
  2. Makes sense. I will look into your wiki for performance test processing times.
  3. I am on 2.8.0 and it is under TEST PROJECTS, project name and under PERFORMANCE TESTING tab.
  4. I can’t seem to download test results but I can download failed convos.
  5. Sure this makes sense.
#4

We had some major improvements with our Performance Testing module in Botium Box 2.8.2, so I strongly suggest to upgrade.

1 Like