Delivering GeoScience with Microservices and ML

So we have recently started moving algorithms to separate microservices from a large Eclipse RCP application (which by the way is tens of millions of lines of code - so we have our work cutout and it will take a long time). It would be great to drop by the conference and show you some of the design of software which we are using to deploy scalable scientific and engineering algorithms using cloud technologies like DCOS and Kubernetes backed by AWS and Azure. One case study in the talk is a Python-based machine learning algorithm which we wanted to scale, so we linked it to a Java EE4J and Jetty microservice which splits the data and queues using Kafka. Then using Marathon autoscaler, we spin up multiple docker nodes with the image for the Python ML code and they consume the data from the queue. This approach turns out to be inefficient in memory but scales as far as nodes are available on the cluster you are using in the cloud.

Landmark creates software products for scientists and engineers working in the energy industry, it is a subsidiary of Halliburton an oil and gas company. This talk was presented at Eclipse Meetup London 23rd February 2018.

 

Schedule info
Session Time Slot(s): 
Thursday, June 14, 2018 - 09:45 to 10:20

Our Sponsors

For information about becoming a sponsor, please visit the EclipseCon France2018 sponsor prospectus page.

Premium

Basic

Media

JUG Partners

Regional Supporter

EclipseCon Support Other Events

Our Other Events

Eclipse events are hosted all over the world!

  • Foss4G NA 2018
  • Eclipsecon France 2018