All Systems Operational

About This Site

This page shares the system status of REANNZ's advanced computing platform and storage services, including impacts of any known outages or planned maintenance work.

Please note that these services are currently only supported within business hours, so there may be delays in communicating outages outside of these hours despite best efforts.

To view the status of REANNZ's network services, visit: https://reannz.status.io

Uptime over the past 90 days. View historical uptime.
Apply for Access Operational
Data Transfer Operational
Submit new HPC Jobs Operational
Jobs running on HPC Operational
NeSI OnDemand Operational
HPC Storage Operational
User Support System Operational
Support Documentation Operational
Flexible High Performance Cloud Operational
Long-term Storage (Freezer) Operational
Flexible High Performance Cloud Services Operational
Virtual Compute Service Operational
Bare Metal Compute Service Operational
FlexiHPC Dashboard (web interface) Operational
FlexiHPC CLI interface Operational
Public API of the FlexiHPC Service Operational
Login to Mahuika Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.

No incidents or maintenance related to this downtime.

Scheduled Maintenance

Upgrade OnDemand cluster from v1.33.6 to v1.35.9 Oct 19, 2026 08:30-11:30 NZDT

We will be performing maintenance to our OnDemand cluster on Monday 19/10 from 08:30. The maintenance is scheduled for 3 hours. During this time we will place OnDemand into maintenance mode and no new sessions will start. Any existing sessions can expect at least 2 disconnections as the cluster backend is upgraded. We recommend disconnecting to reduce the chance of any issues arising from the disconnections.
Posted on Oct 16, 2026 - 09:33 NZDT

Oct 16, 2026

No incidents reported today.

Oct 15, 2026

Slurm upgrade to 26.05.5
Completed - The scheduled maintenance has been completed.
Oct 15, 16:56 NZDT
Update - New job submissions are unavailable due to an issue on the compute nodes caused by the upgrade. Current jobs will continue to run but new jobs will queue until the issue is cleared.
Oct 15, 15:45 NZDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Oct 15, 14:30 NZDT
Update - We postponed this maintenance until 14:30 (thursday 15/10) so that it will not clash with our authentication services maintenance.
Oct 15, 09:33 NZDT
Scheduled - We will be upgrading the Slurm scheduler and Database to release 26.05.5
Running jobs will _not_ be affected. However, new job submissions and queries may result in temporary errors and delays.
Oct 14, 17:08 NZDT
Backend database maintenance
Completed - The scheduled maintenance has been completed.
Oct 15, 13:00 NZDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Oct 15, 11:00 NZDT
Scheduled - We will be undergoing scheduled maintenance affecting authentication services. During this window, new logins will be unavailable across impacted systems. Previously established sessions should remain connected.

Impacted systems: my.nesi.org.nz, ondemand.nesi.org.nz, and HPC lander / login node authentication.
We don't expect any impact on running Slurm jobs on the cluster.
Oct 14, 14:10 NZDT
Freezer storage restoration
Resolved - This incident has been resolved.
Oct 15, 09:26 NZDT
Update - We are continuing to monitor for any further issues.
Sep 4, 16:19 NZST
Monitoring - Thank you for your patience while we’ve worked to bring Freezer back online. As part of our recovery from the Tamaki Data Centre’s cooling failure incident on 29 July, we have now completed comprehensive testing of our tape library and tape media. Our results indicate that the tapes are healthy.

Freezer is now available for use.

We encourage you to contact support@nesi.org.nz if you notice anything out of the ordinary or have concerns about your data.

We will continue to monitor Freezer as usage levels return to normal.
Sep 4, 16:07 NZST
Update - We are working with our vendors to test the health of our tape library and tape media following the cooling incident at the TDC. We will provide updates to project leaders on the status of your data once known, but we expect this may take some time. Further updates may be provided as we work through this process. Our apologies for any concern or inconvenience this might cause.
Aug 11, 08:57 NZST
Investigating - Following the cooling incident at Tāmaki Data Centre (https://status.nesi.org.nz/incidents/yznlzzntjgpp) we are performing thorough health checks on the tape media used for Freezer storage.

As of now we do not have a timeline for the restoration of the Freezer storage service, but we will provide updates when we have sufficient information.
Aug 3, 12:47 NZST

Oct 14, 2026

No incidents reported.

Oct 13, 2026

No incidents reported.

Oct 12, 2026

No incidents reported.

Oct 11, 2026

No incidents reported.

Oct 10, 2026

No incidents reported.

Oct 9, 2026

No incidents reported.

Oct 8, 2026

No incidents reported.

Oct 7, 2026

No incidents reported.

Oct 6, 2026

HPC SSH access gateway, Thursday 6 October
Completed - The scheduled maintenance has been completed.
Oct 6, 10:30 NZDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Oct 6, 09:30 NZDT
Scheduled - We're updating the gateway servers that SSH connections to the HPC pass through on their way to the login nodes, to improve their security and reliability.

A dropped connection doesn't stop anything running on the login nodes. If you use tmux or screen, reattach to your session after reconnecting.

Your SSH connection will be interrupted. Twice during the window, new connections may fail for a few seconds while we update each of the nodes. If a connection fails, wait a moment and try again.

Please note that your running and queued jobs are not affected.
Sep 30, 16:19 NZDT

Oct 5, 2026

Ceph Storage Cluster Upgrade
Resolved - This incident has been resolved.
Oct 5, 11:57 NZDT
Monitoring - Issues with the upgrade have now been resolved, but it is expected to continue at a slow pace for another 36 hours.
Sep 23, 10:00 NZST
Identified - The overnight upgrade of the Ceph Storage cluster has taken longer that expected and resulted in longer than expected loss of access to parts of the storage system. RDC Object store and WEKA are affected. Suppliers are working on the issue.
Sep 23, 08:17 NZST
Openstack Infrastructure upgrade, Sep 29th and 30th
Completed - The scheduled maintenance has been completed.
Oct 5, 11:56 NZDT
Update - We are continuing to verify the maintenance items.
Oct 2, 09:12 NZDT
Verifying - Stage 2 of the Openstack upgrade was successfully completed overnight
Oct 2, 09:11 NZDT
Update - Due to some minor issues during preparation the second stage of the Openstack upgrade did not proceed last night. It will continue tonight instead. The Openstack dashboard, APIs, and external network connections may suffer brief (minutes) interruptions. Apologies for any inconvenience this delay may've caused.
Oct 1, 09:08 NZDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Sep 29, 19:00 NZDT
Update - Noting the Flexi RDC will also be subject to the same impacts as listed above.
Sep 23, 11:05 NZST
Scheduled - Our Openstack infrastructure, which backs Mahuika and associated services, is being upgraded by our third party support vendors, StackHPC. The upgrades will be implemented overnight commencing on the evening of Tuesday, Sep 29th, and continuing again overnight on Wed Sep 30th.
The Openstack dashboard, APIs, external network connections, and access to object storage may suffer brief (minutes) interruptions on both nights. Running Slurm jobs should be unaffected, but ssh connections to Mahuika may fail for brief periods.
Sep 21, 10:57 NZST

Oct 4, 2026

No incidents reported.

Oct 3, 2026

No incidents reported.

Oct 2, 2026

Slow Performance on Mahuika
Resolved - This incident has been resolved.
Oct 2, 09:10 NZDT
Update - We are continuing to monitor for any further issues.
Sep 30, 09:44 NZDT
Monitoring - A fix has been implemented and we are monitoring the results.
Sep 29, 19:47 NZDT
Update - We have identified the cause of this issue and have implemented a fix. Slurm jobs can now run as normal again. We will continue to monitor and provide a new status update tomorrow morning.
Sep 29, 19:42 NZDT
Update - We are continuing to investigate an issue with our filesystems which is causing this slow down. For the moment Slurm job submission remains available but we have suspended any new jobs from actually starting. We will reinstated jobs starting as soon as we are able to
Sep 29, 14:46 NZDT
Update - New jobs can still be submitted to Mahuika. However, no new jobs will begin to run on Mahuika at this time.
Sep 29, 14:44 NZDT
Update - We are continuing to investigate this issue.
Sep 29, 14:43 NZDT
Update - We are continuing to investigate this issue.
Sep 29, 14:26 NZDT
Investigating - We are currently experiencing filesystem performance issues on Mahuika, including the login node which users may find is running slowly at this time. Users may have trouble accessing Mahuika at all, or find other tools that use Mahuika not working. We are currently investigating the cause.
Sep 29, 14:18 NZDT
Incident History Powered by Atlassian Statuspage