heappeLogo vsbLogo

High-End Application Execution Middleware (HEAppE Middleware)

HPC-as-a-Service is a well known term in the area of high performance computing. It enables users to access an HPC infrastructure without a need to buy and manage their own physical servers or data centre infrastructure. Through this service small and medium enterprises (SMEs) can take advantage of the technology without an upfront investment in the hardware. This approach further lowers the entry barrier for users and SMEs who are interested in utilizing massive parallel computers but often do not have the necessary level of expertise in this area.

To provide this simple and intuitive access to the supercomputing infrastructure an in-house application framework called HEAppE has been developed. This framework is utilizing a mid-layer principle, in software terminology also known as middleware. Middleware manages and provides information about submitted and running jobs and their data between the client application and the HPC infrastructure. HEAppE is able to submit required computation or simulation on HPC infrastructure, monitor the progress and notify the user should the need arise. It provides necessary functions for job management, monitoring and reporting, user authentication and authorization, file transfer, encryption, and various notification mechanisms.

HEAppE identities mapping

In terms of security, HEAppE recognises two types of credentials: external user accounts and internal service/cluster accounts. External user accounts do not have direct access to the HPC infrastructure itself. They are used only to authenticate the external user via the HEAppE middleware to access only HEAppE’s provided functions. External user accounts can be managed directly via HEAppE, i.e., stored within HEAppE’s internal database, or handled externally by another identity and access management service such as KeyCloak, etc.

Additionally, for HEAppE to be able to submit jobs to the actual HPC cluster queue, a set (pool) of so-called cluster service accounts is required. These service accounts are usually non-personalised standard cluster accounts bound to a specific computational project that has been allocated with a specific computational resources. The service accounts are generated specifically to be used within the HEAppE middleware at the request of the Primary Investigator (PI) of a given computational project; the PI of the project agrees with the creation of service accounts and gives permission for these accounts to be used within HEAppE. HEAppE provides the mapping between external user accounts and internal service accounts. It keeps track of which service account was used for the submission of which compute job and which user made the submission. Using this mechanism HEAppE can be used in two submission modes: shared service accounts, where one service account can be used in parallel to submit multiple compute jobs even from different users; and the exclusive mode, in which the specific service account can be used only once at the time, and can be used again only when the current computational job using this account has finished and the account has returned back to the pool of available service accounts. The exclusive mode delivers extra security resilience, but limits the number of parallel job submissions to the number of available service accounts.

Architecture

HEAppE’s universally designed software architecture enables unified access to different HPC systems through a simple object-oriented client-server interface using standard REST API. Thus providing HPC capabilities to the users but without the necessity to manage the running jobs form the command-line interface of the HPC scheduler directly on the cluster.

_images/heappe_architecture.png

Acknowledgement

This work was supported by The Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPS II) project ”IT4Innovations excellence in science - LQ1602” and by the IT4Innovations infrastructure which is supported from the Large Infrastructures for Research, Experimental Development and Innovations project ”IT4Innovations National Supercomputing Center – LM2015070”.