Bumblebee Jenkins plugin allows seamless integration of Jenkins jobs and build test results with Micro Focus ALM (former HP ALM) 11, 12, 14, 15. The plugin processes jenkins build data and sends it to the Bumblebee server for processing. Bumblebee automatically creates TestPlan, TestLab, TestSet, and TestRuns in Micro Focus ALM.

Bumblebee plugin is ideal for organizations who want to reflect the data from Jenkins into their Micro Focus ALM project. Organization use Bumblebee to show unit tests, Selenium test, Visual Studio tests, and various othertest frameworks that run in Jenkins into some Micro Focus ALM project. Bumblebee is a commercial product. For more details, please go to http://www.agiletestware.com/bumblebee

Changelog

Older versions of this plugin may not be safe to use. Please review the following warnings before using an older version:

• Unconditionally disabled SSL/TLS certificate validation
• Credentials stored in plain text

The Bumblebee Jenkins plugin provides the ability to

• Easily configure your Jenkins jobs to export their test results directly into HP ALM using the Bumblebee web service
• No need to do anything on HP ALM. The plugin will create Test Plan, Test Labs, Test Sets, Test Run on the fly based on job configuration.
• Allows unit tests, Selenium tests, or any other tests that you run in Jenkins to be reflected in HP ALM. Having all results in one place allows project managers to always get a clear picture of the project risks withing having to asks everyone for status.
• Support JUnit, TestNG, TRX, Cucumber, Serenity, JBehave result formats. Can add any custom result formats as needed.
• Trigger tests in HP ALM and report results back to Jenkins
• Run local HP UFT tests
• Start and monitor HP Performance Center tests execution

Install the Bumblebee plugin for Jenkins using the Plugin manager. Please restart Jenkins after installing or upgrade the plugin.

The Bumblebee Jenkins plugin communicates with the Bumblebee web service which in turn communicates with HP ALM. You can download the Bumblebee server from the Agiletestware website

Detailed user guide for installing Bumblebee server.

Configure the Bumblebee URL, HP ALM URL, username, and password.

Add the Bumblebee post build step in your job configuration. Specify ALM TestPlan and TestLab details. Bumblebee will automatically collect, parse, and insert test results in HP ALM.

Please refer to the documentation for details

Bumblebee's Jenkins build step allows you to run HP ALM TestSets directly from Jenkins and view the results in both Jenkins and HP ALM.

 Pre-Requisites for running HP ALM tests from Jenkins

• Job must run on the Jenkins agent installed on Windows machine
• Jenkins agent must have launch method: Launch agent agents via Java Web Start
• Jenkins agent must NOT run as windows service
• Appropriate version of HP ALM Connectivity Tool must be installed on Jenkins agent machine. Tool is available at http://your_alm_server_and_port/qcbin/PlugIns/TDConnectivity/TDConnect.exe
• Appropriate version of HP ALM Client must be installed on Jenkins agent machine. Available at http://your_alm_server_and_port/qcbin/start_a.jsp?common=true

To run test set from Jenkins you need to add Bumblebee HP ALM Test Set Runner build step to your build configuration

When Jenkins runs Bumblebee HP ALM Test Set Runner step it connects to HP ALM server and triggers specified test sets.

Here is an example of build log produced by Bumblebee:

Bumblebee HP ALM Test Set Runner puts results of test execution as a simple JUnit report which can be then published using Publish JUnit test result report step

Bumblebee allows you to run local Micro Focus Unified Functional Testing tests directly from Jenkins and reports results back to Jenkins.

• Bumblebee server v4.1.0 or higher
• Jenkins agent runs on Windows machine and have appropriate Micro Focus Unified Functional Testing version installed. Please see UFT requirements for a particular version of OS and other software.
• Jenkins agent must run as a console application (not as a windows service)
• UFT Batch Runner property of Global Configuration or UFT_RUNNER environment variable on agent must be set

To override path to UFT Batch Runner, defined in the Global Configuration, you need to set a UFT_RUNNER environment variable on a Jenkins agent.

To set a value to UFT_RUNNER environment variable of Jenkins agent:

• Open Jenkins agent configuration page
• Check "Environment variables" checkbox
• Add a new variable and type "UFT_RUNNER" as "Name" and path to UFT Batch Runner on that agent as a "Value"
• Click on "Save" button

To add a new "Bumblebee Local UFT Test Runner" build step, just add a new build step in Jenkins build configuration with name "Bumblebee: Run local UFT tests".

"Bumblebee Local UFT Test Runner" build step has the following configuration parameters:

• Test Path - the path to a test folder or test batch file (.mtb) which shall be executed
• Results Directory - directory inside your project where Bumblebee put JUnit-like execution reports. If it does not exist, Bumblebee will create it automatically.

Note: If you use .mtb file from GIT repository, you need to make sure paths to tests are correct and point to tests in build directory. You can use windows batch script for this.

Bumblebee UFT step puts results of test execution as a simple JUnit report into folder defined by "Results Directory".
These reports can be then published using Publish JUnit test result report Post-build step.
Please note that it shall be configured to scan the output directory of Bumblebee UFT task.

When Jenkins runs Bumblebee UFT step, it will trigger local HP UFT Batch runner and record its output:

 Bumblebee UFT step captures results of test execution and produces a simple JUnit report which are then attached to the build report and can be seen on "Test Results" page:

For failed tests, report contains an error message reported by UFT:

UFT also produces detailed reports with description of all steps, screenshots, etc...
You can setup Jenkins to capture those results as build artifacts and attach them to a build results.
To do that, you just need to add "Archive artifacts" Post-build step to your build configuration and define appropriate value for "Files to archive".
e.g.: 

After build has finished, artifacts are displayed on "Artifacts" tab:

Micro Focus Performance Center is a powerful set of tools for composing and running performance tests which is used by many companies.

Bumblebee offers Jenkins users ability to easy trigger Performance Center tests and report results back to Jenkins.

When the task starts it triggers a new test run in PC and then polls run status from time to time. When run reaches some terminal state or timeout is reached then task is finished.

Terminal states are:

• Finished
• Before Collating Results (if Post Run Action = Do Not Collate)
• Before Creating Analysis Data (if Post Run Action = Collate Results)
• Canceled
• Run Failure
• Aborted
• Failed Collating Results
• Failed Creating Analysis Data

If run finished successfully, all test results are downloaded into specified folder in the build working directory.

• Bumblebee server version 4.1.0 or higher

To start working with the new task the following Bumblebee Global Configuration should be set:

• Bumblebee URL - URL of Bumblebee server
• HP ALM URL - URL of HP ALM
• PC URL - URL of a Performance Center
• HP ALM user name - user name to connect to HP ALM and Performance Center
• HP ALM password - password for HP ALM and Performance Center
• PC timeout (optional) - the number of minutes to wait for the PC test to finish. 0 means wait indefinitely.

To add a new "Bumblebee HP PC Test Runner" build step, just add a new build step in Jenkins build configuration with name "Bumblebee HP PC Test Runner".

"Bumblebee HP PC Test Runner" build step has the following configuration parameters:

Since, Performance Center produces some test reports, Bumblebee task downloads them from the PC server and stores into Results Directory, defined in a build configuration.

To see those reports on Jenkins build page, they need to be attached as build artifact, so before running the build, Jenkins should be configured to capture and archive required artifacts.

To do that, you just need to add "Archive artifacts" Post-build step to your build configuration and define appropriate value for "Files to archive".

When task is triggered it starts a new run in HP Performance Center for a test specified by "Path To Test" and "Test Set" properties of the task. The following parameters affect test execution in PC:

• Post Run Action
• Time Slot Duration
• Use VUD Licenses

Please refer to HP Performance Center documentation for detailed description of these parameters.

After test is started, task waits for it to finish and polls run state from time to time ("Polling Interval"). If test reaches one of the following states, Bumblebee assumes that test has passed:

• Finished
• Before Collating Results (if Post Run Action = Do Not Collate)
• Before Creating Analysis Data (if Post Run Action = Collate Results)

If test reaches one of the following states or timeout has occurred, Bumblebee treats test as failed:

• Canceled
• Run Failure
• Aborted
• Failed Collating Results
• Failed Creating Analysis Data

If test has failed Bumblebee makes a decision on whether build shall be failed or not based of value of "Fail Build If Task Fails" property. If it is true, then the whole build is failed. If it is false, then Jenkins does not fail and proceeds with the next task.

If an error occurs during fetching runs status from PC, Bumblebee will try to retry failed action according to the retry settings defined for a task.

Here is an example of the execution log:

If you want to pull test results from Jenkins and display them as JUnit report of your build, you can use "Bumblebee: Import HP ALM Test Results" step.

•  Bumblebee server version 4.1.5 and higher

During the execution of Bumblebee: Import HP ALM Test Results test step, Bumblebee searches for a Test Set by path given in Test Set Path parameter, creates JUnit XML report file and puts it into Results Directory folder. This folder can be used by JUnit publisher to build test trends.