This is the story of the lessons learned working on Eclipse Che to provision development environments on Kubernetes.
Before the rise of Docker, Eclipse Che hosted development environments in Virtual Machines. This is a common practice to provision repeatable and isolated environments and tools like Vagrant are still popular today. But when Linux containers became a thing the Eclipse Che project pivoted to replace VMs with containers. Thanks to this choice the project aroused community interest but very soon it became clear that, to attract enterprise customers, Docker support was not enough and that Kubernetes was the path to follow.
The choice to run Che workspaces as Kubernetes Pods rather than Virtual Machines is still controversial. Is a platform designed to run unprivileged, containerized and scalable payloads suited to run development environments that are notoriously single-user monolithic applications? Are the benefits of lightweight, ephemeral and scalability development environments worth the cost of altering the nature of Kubernetes to host developer tools?
In this talk we are going to review the challenges that we faced at the very beginning of our journey (permissions, storage and networking), the architectural changes to make development environments first class citizens in Kubernetes (the DevWorkspace operator) and the new challenges to leverage ephemeral containers and vertical autoscalers.
