Integrating External Test Results into HPE ALM

Most HPE ALM users are aware of the numerous benefits of executing both manual and automated tests within HPE ALM. Some of these benefits include the ability to analyze test coverage against requirements, understand the status of the overall testing progress, and create drill-down analysis charts based on criteria important to your organization.

However, many customers today have automated tests being executed outside of HPE ALM for various reasons. This could include tests being executed via Jenkins to support a continuous testing process or tests executed from other testing tools such as Selenium. Each of these additional tools may help solve a specific testing challenge faced by the organization; but, this also introduces new challenges. How do you get a complete picture of requirements to test coverage? How do you provide one holistic view of all the testing efforts within an organization?

HPE ALM 12.21 (released June 2015) introduced a feature which allows us to incorporate these additional testing activities into the HPE ALM process. Since version 12.21 wasn’t considered a major new release, this new capability may have gone largely unnoticed by the HPE ALM community.

The new feature is referred to as “Integrating External Tests” in the HPE help files and is available in both HPE Quality Center Enterprise and HPE ALM editions. Let’s take a closer look at how this works.

At a high level, HPE ALM provides two methods for synchronizing external results with your HPE ALM project. External results can be automatically synchronized from Jenkins or manually imported into HPE ALM by using a results file from the external testing tool. With both methods, the following entities are created in HPE ALM:

• A test in the test plan module is created for each test specified in the imported results file.
• A test set in the test lab module is created for each test suite either defined in the imported results file or as defined in the Jenkins synchronization.
• Tests instances within a test set are created for each test case defined in the test suite of the imported results file or as defined by the Jenkins synchronization.
• Test runs are created for each test case run specified in the imported results file or synchronized from Jenkins.

HPE ALM supports the following XML formats for the imported results file: JUnit/XUnit, Nunit, and TestNG. If you want to test out this functionality, you will need to have an appropriate file available.

The overall process for bringing in results from an external testing tool versus Jenkins is similar; although, the mechanics are obviously going to be different since Jenkins can be configured to sync the results automatically. Let’s first look at the steps involved with integrating results from other automated testing tools into HPE ALM. Then, we will look at the differences when integrating with Jenkins.

1. Import results file from your external tool into the Test Plan module. Each test created will have a type of “EXTERNAL-TEST.”
   • Right-click a folder in the Test Plan Tree and select Generate from Test Report.
   • Select the appropriate results file and specify a Test Plan folder for the imported tests. This can be a folder that already exists or you can choose to create a new folder.
   • Make the appropriate selections for the Test Framework and Testing Tool used to create the external results file. The selections used for this step will be used to populate the details pane of the test.
   • The file will be imported and tests will be created in the HPE ALM Test Plan module.

2. Create coverage between your imported tests and the requirements in your HPE ALM project. This step is of course optional depending on your usage of the HPE ALM requirements module. However, documenting which requirements are covered by the tests in the external tool will provide the organization with a more complete picture of test coverage.

3. Execute tests using your external tool.
   

4. Synchronize external results with HPE ALM.
   • Right-click a folder in the Test Lab Tree and select Import External Test Report.
   • Select the appropriate results file, specify the Test Plan folder used above and a Test Lab folder for the imported test results. This can be a folder that already exists, or you can choose to create a new folder.
   • There are also optional settings for the External Results Network Path and the External Build/Version Number. Providing the path for the external results will allow you to open the file from within HPE ALM.

5. View external test results in HPE ALM.
   • Results can be viewed in the Execution Grid tab of the HPE ALM Test Lab module or in the HPE ALM Test Runs module. If a network path was provided for the results, the HPE ALM Test Runs module will give you the option to open the external file from within HPE ALM. This can be found in the Report tab after selecting a test run in in the HPE ALM Test Runs module.
     

The process described above does change when integrating with Jenkins. The test plan entities are still created by importing a results file from Jenkins. However, the test lab, test instances, and test runs are created by configuring Jenkins to synchronize automatically with HPE ALM utilizing the Jenkins plug-in HP Application Automation Tools. Complete information on how to configure Jenkins to synchronize with HPE ALM can be found in the HP knowledge base article KM01698877. While the Jenkins plug-in to HPE ALM has existed for some time; it has been recently updated to support synchronizing results from other testing tools.

Even though various groups within your organization may be utilizing other testing tools for specific purposes, that doesn’t mean you need to lose sight of those efforts. HPE ALM can continue to be the “go to” solution for answering questions and making decisions based on the overall status of the testing activities.