The XRP Ledger stopped processing transactions for about 15 minutes earlier today, according to developer Wietse Wind.
Update: it lasted not 2 minutes (my original Tweet) but 15 minutes. At the time of Tweeting (because I have decent monitoring and was warned straight away) the counter was at 2 minutes missing forward progress.
— WietseWind - ? XUMM @ XRPL Labs (@WietseWind) November 3, 2021
Also: 4 validators down: this was caused by a lot more being down.
Every server in the protocol relies on its own unique node list (UNL) for validating new ledgers.
The quorum requirement is set at 80%. If more than 20% of validators cannot reach consensus or remain unavailable, the network is not able to confirm new transactions.
As explained by Wind, four validators were down, which caused the quorum to drop below the required threshold.
This is what the ledger close times look like (courtesy of @Silkjaer). From UTC 12:25 » 12:48 validators couldn't agree on what should have been included in the next ledger. pic.twitter.com/MJMD4Bb2lh
— WietseWind - ? XUMM @ XRPL Labs (@WietseWind) November 3, 2021
The XRP Ledger Foundation claims that the accident did not result in a loss of any transactions.
As we are aware the XRPL choses safety over liveliness. There was a temporary halt in ledgers closing. There were no ledgers dropped, or holes created. Validators are proposing now, nodes are active and the network is back on track with no intervention.
— XRP Ledger Foundation (Official) (@XRPLF) November 3, 2021
Wind says that the network is now working properly after a sudden halt:
Finally: your XRP is safe. No history is lost, the XRPL is working. I believe we need more proactive infrastructure operators.
Ripple CTO David Schwarz supported implementing the Negative UNL feature, which ensures that the network can continue making forward progress even when several validators are down. In this way, transactions would continue being confirmed during a temporary outage:
We need to get the negative UNL feature deployed. More validators is also good, but has some of its own tradeoffs.