Extending Jobs to speed up Eclipse

Status:
Accepted

This session explains the newly added Job Groups API, its uses and sample implementations to parallelize lengthy operations in Eclipse.

Job Groups provide a simple way to manage a set of Eclipse Jobs that are responsible for pieces of the same large task. The API supports throttling, join, cancel, combined progress and error reporting for all of the jobs in the group. The job grouping functionality can be used to rewrite performance critical algorithms to use parallel execution of cooperating jobs.

Many Eclipse operations like Java project classpath resolution, JavaModel cache initialization, Workspace refresh, File Search and many other search operations process a large amount of data in a single-threaded fashion. Eclipse performance can be improved significantly by parallelizing these and many other lengthy operations.

Eclipse provides a simple Jobs API to perform different tasks in parallel and in asynchronous fashion. Up through Eclipse 4.4, one limitation of the Eclipse Jobs is that there is no easy way to limit the number of worker threads being used to execute jobs. This may lead to a thread pool explosion when many jobs are scheduled in quick succession. Due to that it’s easy to use Jobs to perform different unrelated tasks in parallel, but hard to implement thousands of Jobs cooperating to complete a single large task. The newly added Job Groups API is extending the Jobs functionality to overcome that limitation.

Note: More details about the design of the new API and its uses can be found at https://bugs.eclipse.org/432049.

Slides

Session details
Schedule info
Session Time Slot(s):
Grand Peninsula EFG - Tuesday, March 10, 2015 - 15:00 to 15:35

Comments

I really like this, but do you really think it's a beginner talk? Of course everything is relative, but in my experience, beginners are not usually thinking about thread pools and job grouping yet ;).

Public comment

Yes, From the Eclipse plugin developers point of view, I think anyone who know how to implement an Eclipse Job should be able to understand and use the new API. But, I agree, when we see it from total Eclipse community point of view, it might not be of beginner level. I changed it to "Intermediate". Thanks for pointing it out Ian.

Public comment