High-End Application Execution Middleware (HEAppE Middleware)
HPC-as-a-Service is a well known term in the area of high performance computing. It enables users to access an HPC infrastructure without a need to buy and manage their own physical servers or data centre infrastructure. Through this service small and medium enterprises (SMEs) can take advantage of the technology without an upfront investment in the hardware. This approach further lowers the entry barrier for users and SMEs who are interested in utilizing massive parallel computers but often do not have the necessary level of expertise in this area.
To provide this simple and intuitive access to the supercomputing infrastructure an in-house application framework called HEAppE has been developed. This framework is utilizing a mid-layer principle, in software terminology also known as middleware. Middleware manages and provides information about submitted and running jobs and their data between the client application and the HPC infrastructure. HEAppE is able to submit required computation or simulation on HPC infrastructure, monitor the progress and notify the user should the need arise. It provides necessary functions for job management, monitoring and reporting, user authentication and authorization, file transfer, encryption, and various notification mechanisms.
Check the latest version here: https://github.com/It4innovations/HEAppE
HEAppE identities mapping
In terms of security, HEAppE recognises two types of credentials: external user accounts and internal service/cluster accounts. External user accounts do not have direct access to the HPC infrastructure itself. They are used only to authenticate the external user via the HEAppE middleware to access only HEAppE’s provided functions. External user accounts can be managed directly via HEAppE, i.e., stored within HEAppE’s internal database, or handled externally by another identity and access management service such as KeyCloak, etc.
Additionally, for HEAppE to be able to submit jobs to the actual HPC cluster queue, a set (pool) of so-called cluster service accounts is required. These service accounts are usually non-personalised standard cluster accounts bound to a specific computational project that has been allocated with a specific computational resources. The service accounts are generated specifically to be used within the HEAppE middleware at the request of the Primary Investigator (PI) of a given computational project; the PI of the project agrees with the creation of service accounts and gives permission for these accounts to be used within HEAppE. HEAppE provides the mapping between external user accounts and internal service accounts. It keeps track of which service account was used for the submission of which compute job and which user made the submission. Using this mechanism HEAppE can be used in two submission modes: shared service accounts, where one service account can be used in parallel to submit multiple compute jobs even from different users; and the exclusive mode, in which the specific service account can be used only once at the time, and can be used again only when the current computational job using this account has finished and the account has returned back to the pool of available service accounts. The exclusive mode delivers extra security resilience, but limits the number of parallel job submissions to the number of available service accounts.
Acknowledgement
This work was supported by The Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPS II) project IT4Innovations excellence in science - LQ1602 and by the IT4Innovations infrastructure which is supported from the Large Infrastructures for Research, Experimental Development and Innovations project IT4Innovations National Supercomputing Center – LM2015070. This work was also supported by LEXIS project which receives funding from the EU’s Horizon 2020 research and innovation programme (2014-2020) under grant agreement no. 825532. This work was supported by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90254). This work was also supported by the EVEREST project - the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 957269. This work was also supported by the LIGATE project. This project has received funding from the European High- Performance Computing Joint Undertaking (JU) under grant agreement No 956137. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Italy, Sweden, Austria, the Czech Republic, Switzerland. This project has received funding from the Ministry of Education, Youth and Sports of the Czech Republic (ID: MC2102).
Main Features
Deployment
Advanced configuration
Usage
Other