A microservice-oriented science platform architecture
2020-11-10, 18:00–18:15, Times in UTC

A Science Platform (SP) is an environment designed to offer users a smoother experience when interacting with data and computing resources. Usually when referring to a SP, we assume a web based environment based on Jupyter notebooks and software containerisation, however also other approaches or definitions of SP may be feasible (e.g. full interactive desktop access).

In current SP architectures, the software available to the user is usually pre-defined and with little or not control by the users. This means that is almost impossible for a user to run a computing task using a specific analysis software (i.e. a specific version, a specific set of dependencies, or just a software not supported by the SP), and that from an administrative prospective there is a strong overhead in supporting the various (yet limited) software versions.

We tested several approaches to overcome these limitations and we propose a solution based on changing the prospective and framing user task of the SP as microservices - independent and self-contained units. This enables great flexibility, security and sustainability, and empowers the users to choose, build and run their own software environments, which also improves reproducibility.

We successfully used this architecture in the context of the ESCAPE project, at INAF, where we implemented the computing tasks microservices both as Docker and Singularity containers. The first impressions and user feedback are very promising, in particular for interactive desktop applications for data reduction and analysis.

Theme – Science Platforms and Data Lakes