As research data volumes grow and mandates for data publication become more pervasive, automated means for managing these complex workflows to ensure data integrity have a growing role in modern science. In this session we will introduce Globus Flows, a foundational service for orchestrating secure and reliable data management tasks at scale, and Globus Compute, a service which enables you to execute functions on diverse remote systems. We will describe how Globus Flows and Compute fit into the Globus ecosystem of data and compute management services, and how flows can feed into downstream data portals, science gateways, and data commons, enabling search and discovery of data by the broader community. We will demonstrate how to run various Globus provided flows and discuss initiating flows with triggers and inserting compute tasks into your flows. We will conclude with an interactive tutorial detailing how to build custom flows using Jupyter notebooks and the Globus web app.
Modern scientific computing demands flexible and scalable solutions that bring computing power closer to data while maintaining security and ease of use. We propose to present our solution which leverages Kubernetes to provide a platform to our employees, university members, and partner organizations that meets those demands and complements our existing HPC system. This presentation will cover how we utilize Continuous Integration and Continuous Delivery (CI/CD), coupled with GitOps and DevOps practices, to provide a robust and secure platform for hosting container-based workloads. These workloads include interactive web visualizations, JupyterHub instances, science gateways, data assimilation tools, data analysis tools, and those that require access to GPUs, including, but not limited to, AI/ML. The presentation will also cover how Cilium network and Kyverno access policies are implemented to secure the platform. It will also discuss how GitHub Actions are utilized to test and build codebases into containers that can run on the platform. Attendees will learn why we chose Kubernetes as our platform as well as practical strategies to implement similar solutions and common pitfalls to avoid.
