• some_guy@lemmy.sdf.org
    link
    fedilink
    arrow-up
    32
    ·
    2 months ago

    Meanwhile, friends at my old company run sites with CS and my current company doesn’t. I’m kicking back and having a great friday

    • aname@lemmy.one
      link
      fedilink
      arrow-up
      7
      arrow-down
      1
      ·
      2 months ago

      So the hindsight is always 20/20 but was there like warning signs or red flags which should have been obvious this is going to happen or are you just lucky in hindsight?

      • aeno@discuss.tchncs.de
        link
        fedilink
        arrow-up
        7
        arrow-down
        1
        ·
        2 months ago

        Red flags? Yeah don’t use “security Software” that just increases your attack surface. Why the fuck would you want to install a root kit on your critical infrastructure?

      • some_guy@lemmy.sdf.org
        link
        fedilink
        arrow-up
        5
        ·
        2 months ago

        The second one, as far as I can tell. But also, those calls are made above me and I have no insight into the decision-making. It could have been keen foresight by someone else.

  • ThrowawayOnLemmy@lemmy.world
    link
    fedilink
    arrow-up
    26
    ·
    2 months ago

    My office sent out this big message about people not being able to log in this morning. And I had absolutely no issues, and all of my tools are working. So I guess I’m stuck actually doing work.

        • magic_smoke@links.hackliberty.org
          link
          fedilink
          arrow-up
          15
          ·
          edit-2
          2 months ago

          Literally one of the largest enterprise grade endpoint protection packages. This isn’t an issue of a bad sysadmin, or even developer, so much an issue bigger than the industry itself. Up until now, as far as I knew, crowd strike has been recommended as a solid choice for endpoint protection.

          Who else are you going to trust? Fucking Symantec? Ask VMware how being owned by Broadcom is, then get back to me.

          No one gives a shit about their job anymore because they have no reason to. I hate to sit here and chalk everything in the world up to late stage capitalism, but jfc if it doesn’t seem like the recurring theme from hell. Something tells me the guys who work at Crowdstrike are no different.

    • Rentlar@lemmy.ca
      link
      fedilink
      arrow-up
      9
      arrow-down
      9
      ·
      2 months ago

      Probably, but the issue is in the interface between Windows and the CrowdStrike software causing Windows to go into a crashing bootloop.

      Closed source is great, I tell you. /s

      • Passerby6497@lemmy.world
        link
        fedilink
        English
        arrow-up
        21
        ·
        2 months ago

        It has nothing to do with closed source, this is entirely about a privileged application fucking around and not testing shit before pushing it globally. You can have the same issues with Linux too. My org has stated that our AV product is never to be installed on Linux machines because they hosed tons of machines years back doing something similar.

        High privilege security applications are always going to carry this risk due to how deeply they hook into the OS to do their job.

        • Rentlar@lemmy.ca
          link
          fedilink
          arrow-up
          3
          ·
          2 months ago

          That is true. An obvious failure is that the update that broke everything was pushed everywhere simultaneously.

          • J'Pol @lemmy.sdf.org
            link
            fedilink
            arrow-up
            4
            ·
            2 months ago

            That’s what has me confused. I haven’t even stepped into an office in 20 years so I have no modern experience there, but I would have thought that a company with such a massive user base would release things at different times based on region. Is it because with security based applications they don’t want to risk someone having time to exploit a new vulnerability?

            • Passerby6497@lemmy.world
              link
              fedilink
              English
              arrow-up
              2
              ·
              edit-2
              2 months ago

              Is it because with security based applications they don’t want to risk someone having time to exploit a new vulnerability?

              Pretty much. Given how fast the malware scene evolves and implements day 1 exploits, and how quickly they need to react to day 0 exploits, there’s kind of an unwritten assumption (and it might actually be advertised as a feature) that security software needs to react as fast as possible to malicious signatures. And given the state of malware and crypto shit, it’s hard to argue that it isn’t needed, considering how much damage you’ll suffer if they get through your defenses.

              That being said, this kind of a fuck up is damned near unacceptable, and these updates should have been put through multiple automated testing layers to catch something like this before this got to the end user devices. I could see the scale of it taking them out of business, but I also wouldn’t be surprised if they managed to scrape by if they handle it correctly (though I don’t see the path forward on this scale, but I’m not a c-suite for many reasons). Like I said above, we had an incident years back that hosed a bunch of our Linux boxes, but the vendor is still around (was a much smaller scale issue) and we even still use them because of how they worked with us to resolve the situation and prevent it from happening again.

    • sunzu@kbin.run
      link
      fedilink
      arrow-up
      1
      arrow-down
      15
      ·
      2 months ago

      Hard to tell, fake news running both of their names, looks like both?

  • variants@possumpat.io
    link
    fedilink
    English
    arrow-up
    14
    ·
    2 months ago

    I like how it’s the biggest IT issue and the best solution is to turn it off and on several times

  • Treczoks@lemmy.world
    link
    fedilink
    arrow-up
    20
    arrow-down
    7
    ·
    2 months ago

    Ftfy: ‘Largest IT Windows outage in history’

    I learned of the problems from the radio news on my way back home.

    • WindyRebel@lemmy.world
      link
      fedilink
      arrow-up
      27
      ·
      2 months ago

      CrowdStrike, not Microsoft, is responsible. Let’s put blame where blame is due.

      This could happen to any OS that has cybersecurity where permissions are needed at deeper levels to protect systems.