General MSB announcements


  • Admin

    We might as well have a sticky here for site-wide stuff that probably don't interest most people.


    The first item: Users reported they weren't receiving e-mails. This was verified, and as of now we upgraded to postfix over sendmail. I tested just now and it works, but if it doesn't please let me know.


  • Admin

    ... Second post! Already!

    So when I uninstalled sendmail since we no longer needed it, it didn't clean up after itself... but left a little present behind in /etc/network/interfaces.d/ , which was causing ifup -v eth0 to fail. In a layman's terms, the machine hosting MSB was up after that but it had no functional network until I fired up a console (thanks Linode for that) and figured out wtf had just happened.

    Sorry about the disruption.


  • Admin

    We had about a small unannounced downtime today. It's pretty rare for Linode to have those, but this is all I can tell you from their web interface:

    "We are aware of an issue affecting service to this Linode.

    If you are experiencing issues with this Linode, there is no need to issue any jobs or open a support ticket at this time. Please monitor our status blog for further information. Thank you for your patience and understanding."

    We seem to be okay now. I couldn't even access the server through the console before.


  • Admin

    FYI:

    Your Linode(s) have been assigned maintenance windows during which they will be rebooted. This maintenance is necessary to patch for variants 1 and 2 of the Spectre vulnerability[1].

    ...

    Your reboot schedule in UTC is as follows:

    2018-05-11 3:00:00 AM UTC - linode3343044

    The above listed maintenance windows cannot be altered, postponed, or rescheduled.

    A two-hour window is allocated for each reboot maintenance window, however, the actual downtime should be much less.


  • Coder

    @arkandel

    I pity anyone who tries a Spectre attack and gets our memory space.

    (By pity I mean laugh a lot.)


  • Admin

    During what I thought was going to be typical maintainance nodebb decided to go completely belly-up.

    ... The good news is that, a couple of hours afterwards, I can verify our database backups work as intended! Also that we're now on the latest and greatest branch.

    The bad news is that any new posts, chats... anything at all created since this morning (4/14/2018) at 07:00 when the latest backup had been taken are lost in the aether. I do apologize about that.


  • Admin

    Our backups filled up the partition they were being saved on since apparently 30 days' worth of them got larger over time. Who knew.

    We'll be storing them on a much, much roomier partition now. Of course nodebb shit its pants at all this, so I took the chance to upgrade it as well.

    Sorry about the unscheduled downtime on a Friday night. Please let me know if anything I'm not looking at is on fire.

    Edit: In case anyone wondered what super duper new features this latest version brought to your screens, read and be amazed! https://blog.nodebb.org/whats-new-in-v1-9-0/


  • Admin

    I am looking into the root causes but at some point last night - sadly before the daily backup hit - our redis database got corrupted.

    I restored us to the last day's snapshot but some posts will have been lost. Until I know more I'll switch to a bi-daily backup schedule.


  • Pitcrew

    @arkandel
    I lost an upvote as well, mah meaningless pretend numbers are important, they are easily the equal to the points in Who's Line Is It Anyway.



  • @thatguythere Have one now. I think I upvoted a few of yours yesterday, anyway, so it might have been me as it is.


  • Coder

    @arkandel said in General MSB announcements:

    Please let me know if anything I'm not looking at is on fire.

    I lost a level, 1300GP, and a +2 magic sword. Please restore.


  • Pitcrew

    @tyche said in General MSB announcements:

    @arkandel said in General MSB announcements:

    Please let me know if anything I'm not looking at is on fire.

    I lost a level, 1300GP, and a +2 magic sword. Please restore.

    Please provide screenshots,proof of purchase, and your firstborn child.


  • Pitcrew

    @auspice Hmm. It takes less time to farm for kids than gold.

    Deal!


  • Admin

    I'm taking MSB down in a few minutes for maintenance and an upgrade. Hopefully we won't be down for long.


  • Admin

    We're running the current nodeBB version (v1.10.1) now and Ubuntu is happier about its upgrade cycle. Sorry about any interruptions.