Update - We are continuing to investigate this issue. To clarify, nearline put activity should not be impacted, but nearline retrievals will be significantly delayed.
Jan 8, 09:56 NZDT
Investigating - We are currently experiencing some issues with the underlying tape infrastructure of the Nearline storage system. All nearline activities will be experiencing delays.
Jan 8, 09:41 NZDT
Update - We are continuing to work on a fix for this issue.
Dec 23, 09:47 NZDT
Identified - We are aware of issues affecting many retrievals of files from Nearline using nlget. Users of the Nearline service are welcome to attempt to retrieve data, however please be aware that this may not work, and support for Nearline problems will not be available during the holiday season. Thank you for your understanding.
Dec 23, 09:47 NZDT
Resolved -
The underlying filesystem issue has been resolved and the Mahuika cluster is once again stable
Jan 21, 09:43 NZDT
Monitoring -
The underlying filesystem issue has been resolved. All compute nodes are now available. We will continue monitoring the cluster for stability.
Jan 20, 14:06 NZDT
Investigating -
Yesterday afternoon the Mahuika cluster had issues with numerous compute nodes failing with job related errors. Some of these nodes were rebooted whilst user jobs were still running. Apologies for the inconvenience but we had no choice. These symptoms have occurred again overnight and we will be taking similar actions to recover the compute nodes. Meanwhile we are working on identifying the underlying issue.
Jan 20, 09:06 NZDT
Resolved -
This incident has been resolved.
Jan 19, 13:46 NZDT
Identified -
Chrome users on MacOS may encounter a NET::ERR_CERT_REVOKED error when attempting to load JupyterHub or some of nesi.org.nz pages. The cause of the problem has been identified and the work is under way to have it fixed. As a workaround, please use an alternative browser.
Expected ETA for fix is midday 19th.
Jan 18, 15:29 NZDT