We interviewed Achim Loerke, Managing Director at Bredex and this year's Program Chair to find out more about how your submissions will be reviewed and how the PC reaches a consensus on the final selections for the program.
Now that we've reached the submission deadline, what is the next step for the PC?
It all starts with a spreadsheet ;) In the first pass, a pair of PC members will do triage on the talks and assign a -1, 0 or +1. The goal is to sort out the ones aimed at the wrong conference, obvious sales pitches and some gems that we don't want to miss. As part of this process we contact the speakers where we think more work or clarification is needed.
How do you make the selections?
A few days later we meet to review all the talks that received two -1's to confirm that they will be eliminated. This is usually around 10%. With the remaining 90% each PC member reviews the abstract and assigns it a score between 1-9, with 9 as, "Won't want to miss it for the world". After two (long) days we have a list of proposals voted upon by every PC member and this is where the spreadsheet comes in again. We evaluate the average scores and the standard deviation of each proposal and discuss the proposals with a standard deviation that shows we don't agree. Those with the highest and lowest scores explain their choices and everyone can then adjust their vote if their convinced.
The final check is to see if we think that all topics are reasonably well represented. This is where emotions can run high as we're eliminating and adding talks that are slightly above or below our 'we agree' level. No blood shed yet though.
How much flexibility does the PC have?
You can see that the process is designed to get us to a consensus, based on everyone's expertise. Members are not allowed to vote on proposals from their own companies or on talks where they have any other conflict of interest. Aside from this, the selection is also influenced by scores, votes and having an advocate on the PC. However, even with an advocate, a talk may not make the program.
What is the hardest part of the process?
We established a rule several years ago that every rejected submission will receive a rejection note where we will explain why we had to reject the proposal. Sometimes it is only because we don't have enough slots - which makes it hard to reject.
You've served on the PC and now in your second year as PC Chair, what keeps you coming back ;)?
And there is getting to know people, both the members of the PC and a lot of speakers. One also gets a lot of insight what is going on in the Eclipse eco system. And there is always the satisfaction once one finally has put together another great conference.
Oh, and did I already mention bribes?