Skip to main content
  • Log in
  • Manage Cookies
EclipseCon - Eclipse Foundation
  • Conference
    • News
    • Registration
    • Program Schedule
    • Program List
    • Community Day
    • Keynotes
  • Sponsors
    • Our Sponsors
    • Sponsor Testimonials
    • Be a Sponsor
    • Sponsor Guidelines
  • Community
    • 2021 Speakers
    • Speaker Guidelines
    • Code of Conduct
    • Community Resources
    • Conference Guide Documents
  • About Us
    • EclipseCon 2021
    • Program Committee
    • The Eclipse Foundation
    • Past Conferences
    • Other Events
  • Code of Conduct
  1. Home
  2. EclipseCon
  3. EclipseCon 2021
  4. Sessions
  5. Dealing with Verification Data Overload

Dealing with Verification Data Overload

Session details
Status: 
Accepted
Speaker(s): 
Shelley Lambert (Red Hat, Inc.)
Lan Xia (IBM)
Renfei Wang (IBM)
Experience level: 
Beginner
Tags: 
test automation
jenkins
continuous testing
Session Track: 
All Things Quality
Session Type: 
Standard

Massive amounts of test logs and console output are generated each day at the Eclipse Adoptium and Eclipse OpenJ9 projects because of the huge number of tests multiplied by the number of versions, platforms and implementations tested. This 'noise' from daily, personal, and pull request builds requires processing in order to consume and make sense of it (87 million+ tests are executed per release and 20G+ of test output is produced daily).  Not only we have the challenge of completing all tests in a timely fashion with limited machine resources, but also we must take this raw data and refine it to more easily and effectively understand what next action to take.

  This presentation covers our approaches to using parallelization and handling copious amounts of verification data. The goal is to optimize for the best test builds throughput, make the verification data meaningful, and guide next actions. From a quality assurance perspective, there is value in results summary and aggregation.  We need best practices in the application of data visualization techniques, filtering, and categorization.  We need to continuously evolve and improve, employing relevant technologies such as deep learning, Github Actions, vm cloud agents, etc.  Ultimately, the goal of our data refinery efforts needs to be to display the results efficiently, allow users to quickly narrow down the problems and better monitor the farm to support the complex continuous delivery pipelines.
Objective of the presentation: 
Sharing our approaches to using parallelization and handling the copious amounts of verification data from multiple CI servers, our challenges, the lesson learned, and our continuous improvements. Due to the diminishing borders between industries, we have lots of type of tests from various sources. We need a common and effective way to deal with verification data from all types of tests. Also, our application monitors several open projects including Eclipse Adoptium and Eclipse OpenJ9, but the tool itself is project-agnostic and can be generally applied to other projects. We hope our stories could inspire the audience and help them to find better strategies to solve similar issues.
Attendee pre-requisites - If none, enter "N/A": 
N/A
Schedule info
Time: 
26 Oct 2021 - 16:10 to 26 Oct 2021 - 16:40
Room: 
Room 2
  • Sign in to post comments.

Elite Sponsors

Premium Sponsors

Basic Sponsors

Eclipse Foundation

  • About Us
  • Contact Us
  • Donate
  • Members
  • Governance
  • Code of Conduct
  • Logo and Artwork
  • Board of Directors

Legal

  • Privacy Policy
  • Terms of Use
  • Copyright Agent
  • Eclipse Public License
  • Legal Resources

Useful Links

  • Report a Bug
  • Documentation
  • How to Contribute
  • Mailing Lists
  • Forums
  • Marketplace
EclipseCon is brought to you by The Eclipse Foundation with the support of our sponsors.
Powered by Drupal and built on COD.

Copyright © Eclipse Foundation. All Rights Reserved.

Back to the top