November Community Conversation

This month's meeting was held Wednesday, November 12, 2025 at 12:00pm ET.

Quick recap

The meeting began with administrative updates regarding calendar notices and conference preparations, including repository reorganization and promotional materials for an upcoming supercomputing conference. The team discussed technical aspects of ColdFront and Slurm integration, including user synchronization processes and potential improvements to the Slurm plugin for better REST API integration. The conversation concluded with discussions about user provisioning automation, data archiving practices, and user status management changes, with particular focus on how these systems handle user accounts and storage allocations.

Summary

ColdFront Updates and Conference Preparations

Dori announced that the ColdFront repository had been moved to a separate organization and provided updates on the upcoming supercomputing conference, including the availability of ColdFront stickers and a talk by Anton at the VAST booth. Dori also mentioned that ColdFront flyers and posters with QR codes would be available at the Indiana University and Massachusetts Green High Performance Computing Consortium booths. Anton mentioned that he would review recent pull requests before the conference, and Chris inquired about implementing Slurm associations on a cluster using ColdFront, prompting a discussion about potential methods for updating Slurm configurations.

ColdFront and Slurm Synchronization

The team discussed the synchronization between ColdFront and Slurm, where Dori explained that their system uses a daily cron script to sync user accounts and allocations between the two systems. Chris inquired about handling user removals, and Dori confirmed that the sync process removes users from Slurm allocations when they are removed in ColdFront. Andrew mentioned that Claire from Harvard is leading efforts to develop a new Slurm plugin with improved REST API integration, which will allow for more robust and real-time control over user allocations. The team also discussed the need for automations with Slurm, GPFS, and LDAP, with Chris suggesting a 24-hour schedule for initial implementation.

User Provisioning and Storage Management

John explained the process of provisioning new users and storage allocations, which involves writing a text file to a staging directory that gets picked up by a service. The provisioning process takes 2-3 minutes and includes tasks like creating home directories and syncing accounts across campus AD and an ADLDS instance. John also described how they monitor Slurm accounts every 20 minutes using a Python script and how they send emails for various purposes, including warnings to users approaching their quotas.

Account Provisioning Automation Progress

John explained that their organization has implemented automation and triggers for account provisioning, which has significantly reduced manual work despite limited system administration staffing. They noted that while most provisioning processes are now automated, some tasks like data deletion remain manual due to concerns about accidental data loss and user expectations regarding data retention. The team has been using ColdFront for about a year and found it essential for maintaining HPC service during staffing challenges, though they still handle complex account status changes through manual review processes involving department chairs and deans.

Data Archiving and Storage Practices

The team discussed their data archiving practices, noting that while they don't delete data or charge for storage, they manually remove file permissions when users leave. John explained that while they don't have a formal policy for data retention, they handle requests through help tickets or direct communication with system admins. The team acknowledged they're unique in not having any chargebacks or financial transactions associated with data storage.

ColdFront Archive Functionality Discussion

Chris discussed implementing an archive button for staff-only use to track completed projects. John suggested renaming the functionality to avoid user confusion about data archiving, as many users mistakenly believe ColdFront is an HPC system. Dori inquired about changes to ColdFront's allocation user status system, particularly regarding how new users are added and approved.

User Status Management Updates

The team discussed changes to user status management, with Chris explaining he added a pending status for allocation users that becomes active once they are part of the Unix group and SRM account. Chris offered to share a detailed diagram of request statuses and workflows, and Dori mentioned ongoing discussions about status management in the GitHub repository that need review and refactoring. The team also briefly discussed the EULA attribute functionality, which Chris noted is currently used as a resource attribute for basic requests with an implied agreement through the submit form.