And we're back!

Posts: 80 · Views: 2501
  • 35739

    It's quite weird. a few days ago I saw a post saying that he was using Ubuntu. one night he left his computer on and when he woke up in the morning he found that NVMe had inexplicably read and written almost 40T of data. He didn't know which shell was doing the reading and writing.

  • 35740

    The TL;DR is: NVMe drives that serve as our boot and database storage went read only after ~4.5PB of data read/written to them. A better admin would have caught that way sooner. Whoops...

    Good work getting the site back up and running AksumkA. I too would love to read the post-mortem. Were the old SSDs actually toast from wear and tear or were the filesystems just completely full (because alerting wasn't properly configured) resulting in a hard lock? If the latter, will you risk putting them back into service after reformatting them?

    Out of curiosity, if anybody else wants to know what the last wallpaper uploaded to the site before it crashed was and the first one after it, then here they are:

    Well done WinterKitten, for triggering an outage 7 days 6 hours 10 minutes 6 seconds long. 👏👏👏😜

    Last updated
  • 35743

    Good to have the site back, and thanks for all the work getting it back.

  • 35746

    Welcome back! Thank you for your work! We love you and this site =)

  • 35747

    My heartbeat slowed when I heard about the hardware failure. I'm relieved that you had a backup and restored everything seamlessly. Thank you for running this website for so many years, without any ads or personal profit.

  • 35748

    Thank you for your hard work, glad that it's back without any issues!

  • 35749

    Nice, Welcome back! Could you share the cost of running this website?

  • 35754

    Thank you for your time and work verted in getting all back up, there's many sites for wallpapers but none like wallhaven!

  • 35757

    Phew! Scared me for a bit there. Really don't want to lose this gem of a website. Welcome back. :)

  • 35758

    This last week made me realize how much I depend on wallhaven. I also recognize the pattern of my procrastinating, I started choosing, changing wallpaper as I try to start working.

  • 35759

    Well 4,5 TB written is not much for a data center SSD depending capacity, but quite a lot for a consumer one.

    Thank you very much for your work, yeah SSD failures happens ,even with data center grade ones, admin did his job very well at least since backup was done and full restore was really quick ;)

  • 35760

    Thank you for your hard work!

  • 35763

    thank you for this amazing site

  • 35764

    vfgx said:

    Well 4,5 TB written is not much for a data center SSD depending capacity, but quite a lot for a consumer one.

    Thank you very much for your work, yeah SSD failures happens ,even with data center grade ones, admin did his job very well at least since backup was done and full restore was really quick ;)

    That's why it said PB in the initial post, not TB ;)

  • 35765

    Thanks Site owner and mods for your resurrection! You.ve been missed. Thanks.

  • 35766

    I basically never post in the forums, but wanted to say thank you for your dedication to the site!

Message