High-performance computing (HPC) research thrives on collaboration, yet institutional data silos often hinder progress. This extends beyond the storage hardware itself to where data is being siloed in departments and institutions alike. Accelerating scientific progress and innovation requires enabling secure data discovery and sharing across not only institutions, but scientific disciplines as well. Federated access models, controlled permissions, and distributed compute environments enable seamless yet secure collaboration. By facilitating data discovery across disciplines and optimizing shared infrastructure, organizations can break down barriers, enhance research efficiency, and drive cross-disciplinary insights that push the boundaries of scientific advancement.
Modern scientific computing demands flexible and scalable solutions that bring computing power closer to data while maintaining security and ease of use. We propose to present our solution which leverages Kubernetes to provide a platform to our employees, university members, and partner organizations that meets those demands and complements our existing HPC system. This presentation will cover how we utilize Continuous Integration and Continuous Delivery (CI/CD), coupled with GitOps and DevOps practices, to provide a robust and secure platform for hosting container-based workloads. These workloads include interactive web visualizations, JupyterHub instances, science gateways, data assimilation tools, data analysis tools, and those that require access to GPUs, including, but not limited to, AI/ML. The presentation will also cover how Cilium network and Kyverno access policies are implemented to secure the platform. It will also discuss how GitHub Actions are utilized to test and build codebases into containers that can run on the platform. Attendees will learn why we chose Kubernetes as our platform as well as practical strategies to implement similar solutions and common pitfalls to avoid.
