In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Dec 02, 2025 - 18:00 NZDT
Update - This maintenance has now been postponed to Dec 2nd at 1800hrs
Dec 2, 2025 18:00 - Dec 3, 2025 06:00 NZDT
Scheduled - The border switches will have an upgrade applied overnight on Nov 18th from 6pm. Any ssh and external connections to the cluster and OnDemand may get broken during this maintenance. Slurm jobs will be unaffected.
Dec 2, 2025 18:00 - Dec 3, 2025 06:00 NZDT
Update - There has been a period of IO stalls this morning as we've dealt with some storage hardware failures. That issue is now resolved, however space reclamation continues in the backend and is having a detrimental impact on read performance. We are working with WEKA support to look at mitigation options. Apologies for the performance impact - if your jobs are impacted and need a runtime limit extension then please reach out to support.
Nov 24, 2025 - 12:44 NZDT
Identified - Our storage system is currently very full and this is forcing the backend object storage to undertake some urgent administration by way of defragmentation. This increased load is having a detrimental affect on I/O performance, especially read I/O, and this is likely to continue for some days. We are urgently looking at ways to mitigate this. Researchers could help alleviate this in the short term by cleaning up any unwanted files and data as soon as possible please.
Nov 21, 2025 - 12:29 NZDT