• Levsgetso@lemmy.zip
    link
    fedilink
    English
    arrow-up
    42
    arrow-down
    3
    ·
    11 months ago

    Lemmy has been feeling pretty empty the past couple of days. That just really goes to show how important federation really is.

    • ProdigalFrog@slrpnk.net
      link
      fedilink
      English
      arrow-up
      7
      ·
      edit-2
      11 months ago

      I noticed that as well, thought maybe things were just slowing down, but it’s good to know it was the software and not a sudden drop in community involvement. Since admins became aware of the bug and began restarting their servers as an ad-hoc fix, I’ve been seeing way more engagement on my instance.

    • RBG@discuss.tchncs.de
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      1
      ·
      11 months ago

      Yes and no, if you have enough users you do not need federation for a server to not feel empty. But overall many servers do not have that many users, so federation with larger ones is essential.

    • sir_reginald@lemmy.world
      link
      fedilink
      English
      arrow-up
      5
      arrow-down
      1
      ·
      11 months ago

      for the most part federation works just fine. there are occasional issues, yes, but it’s not like it can’t federate at all.

  • 1984@lemmy.today
    link
    fedilink
    English
    arrow-up
    13
    ·
    11 months ago

    I would comment on this but since federation is broken…this won’t be seen until the instance admin restarts Lemmy and federation works once again.

    • BlueÆther@no.lastname.nz
      link
      fedilink
      English
      arrow-up
      9
      ·
      edit-2
      11 months ago

      Federation seems to be working fine from where I stand?

      Edit:

      Edit 2:

      Even the edit federated out to your instance fine…

      • 1984@lemmy.today
        link
        fedilink
        English
        arrow-up
        6
        ·
        11 months ago

        I think it works for a while after restart of the server, and then stops working. Yesterday I was making comments that didn’t federate until 12 hours later when I assume the instance admin restarted the server.

        • BlueÆther@no.lastname.nz
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          1
          ·
          11 months ago

          My instance has been uo for over 3 days since 0.19.1 update with no issues - very little load though

          • 1984@lemmy.today
            link
            fedilink
            English
            arrow-up
            1
            ·
            11 months ago

            Yeah it seems to work for some instances with 0.19.1 but Lemmy.today has issues and feddit.de as well, just from what I’ve heard. Probably plenty more.

  • RandomVideos@programming.dev
    link
    fedilink
    English
    arrow-up
    14
    arrow-down
    1
    ·
    11 months ago

    A while ago, programming.dev had a problem where nothing from the instance federated for a week. Lemmy felt so empty

  • thisisawayoflife@lemmy.world
    link
    fedilink
    English
    arrow-up
    12
    arrow-down
    1
    ·
    edit-2
    11 months ago

    Am I little naive in wondering how this isn’t caught in unit, integration and E2E tests?

    Edit: looks more involved than that, but it forced me to dig into how some of the components are tested. Educational.

    • kattenluik@feddit.nl
      link
      fedilink
      English
      arrow-up
      20
      arrow-down
      18
      ·
      edit-2
      11 months ago

      The main Lemmy developers very often don’t have the biggest clue of what they’re doing, and they’ve proven that time and time again. A lot of times they forget normal web practices and rules, and a lot of side things like Jerboa are also just broken.

      They also don’t know how to interact with the community or how to be normal people. Let’s not forget they have been working on Lemmy full-time and this is the state we’ve come to, ignoring all the database and security and everything else issues they already had.

        • kattenluik@feddit.nl
          link
          fedilink
          English
          arrow-up
          24
          arrow-down
          4
          ·
          11 months ago

          Lemmy IS awesome, that doesn’t mean we can’t want it to be better or criticize it. No one said Lemmy isn’t great.

          • sir_reginald@lemmy.world
            link
            fedilink
            English
            arrow-up
            17
            arrow-down
            3
            ·
            11 months ago

            The main Lemmy developers very often don’t have the biggest clue of what they’re doing

            this doesn’t sound constructive criticism to me.

            that commenter has something personal against the devs. Lemmy is far from perfect but it’s functional and improving everyday.

        • linearchaos@lemmy.world
          link
          fedilink
          English
          arrow-up
          6
          ·
          11 months ago

          I love writing tests, It’s all the shit that comes after that that sucks.

          Those first few pushes that all come up green feel like magic. That first red that points out something you missed, you go back and make a quick change and it’s now green and it’s the best thing you’ve ever seen.

          It’s sooner or later, you throw a couple big red bois on a production build that don’t make any sense. You start digging through the code of some guy that only writes comments in haiku and has the impression he gets paid by the number of layers deep he can nest a ternary.

          Sooner or later you figure out it’s just an edge case there’s nothing actually wrong. You’ll need to refactor one of the systems but you still have production to push to fix a critical bug, so you hotwire the test and write it off as P1 tech debt.

          Eventually, you end up with unit tests that aren’t P1 and they fail. If you’re understaffed, or overscoped, sooner or later you just have a bunch of half-assed zombie test sitting around. Unless you can convince production to let you go back and clear up your tech debt it’s just a unit test graveyard. It still has the big bumpers in place so something serious can’t fail. But you never seem to be able to get back to make everything bright new and shiny again.

          • thisisawayoflife@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            ·
            11 months ago

            That does sound like a nightmare. I’m assuming you mean failed test when you say “red boy”, and that made me wonder about PR practices. I’m used to a very strict review environment and fairly quick review turnaround or requests to go over the code. I’ve heard horror stories about people not getting PRs reviewed for days or weeks or some people just plain refusing to review code. I work on microservices that are all usually less than 10,000 lines though, not something with over a million lines of legacy code.

        • fuckwit_mcbumcrumble@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          11 months ago

          I wish more developers would do QA. After working with QA my code improved so much because I was proactively thinking about how things might break or potential issues that I never would have thought of.

          • thisisawayoflife@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            11 months ago

            Yep. When I was still doing QA, I saw some pretty terrible practices and tested code that barely built. Now as a software engineer, I have no QA and rely heavily on my own testing practices, namely, unit testing first, integration testing and system/e2e testing. I can’t guarantee the code is bug free and there’s parts I know that could be refactored (tech debt), but I know each piece is tested and does what I expect it to. As corny as it sounds, I’m a big fan of TDD. Unit/IT/E2E don’t replace QA in my opinion, but better set QA up to focus on the bugs that matter and not basic stuff.

        • ExLisper@linux.community
          link
          fedilink
          English
          arrow-up
          1
          ·
          11 months ago

          Personally I dislike writing useless tests. I use test as a development tool (it’s easier to implement some DB operations for example by writing tests than performing some actions manually) and to test logic that can actually fail because of changes in other part of the code withouts me noticing. Testing thinks like “button calls click() method when clicked” is IMHO pointless. If someone can change this code and push to prod without testings manually or doing code review they can also disable the test without anyone noticing.

          • thisisawayoflife@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            11 months ago

            I agree, writing meaningless tests helps nobody and just creates extra work everyone. Unit tests should prove functionality and integration tests act as a vise. Much like you said, if a test breaks in that scenario, then you know something in another class has violated that contract. Good tests will have meaningful names and prove functionality, especially in the backend where it is especially important…

            You mention (what I would consider) a bad practice of allowing merges without review. While that should be possible on personal projects with only one dev, strict review guidelines should exist so that nobody can just “push to prod”. CICD is your friend - use it so that staging and prod never break. Again, I’m used to working on systems used by scores of millions of users so I appreciate forced automated validation. Nobody likes dumb breaks on a Friday before vacation.

  • walden@sub.wetshaving.social
    link
    fedilink
    English
    arrow-up
    10
    ·
    11 months ago

    I’m going to set up a cronjob to restart the Lemmy docker container every 6 hours. That seems to be around when stuff stops working usually.

  • thegiddystitcher@lemm.ee
    link
    fedilink
    English
    arrow-up
    6
    arrow-down
    1
    ·
    11 months ago

    Yup, absolutely nothing I posted from lemm.ee yesterday made it out into the wider world. In fact I’ll probably have to manually federate this comment too.

    Hopefully they can get to the bottom of it. Maybe next year we need a rule about releasing anything in December when everyone’s too busy to bugfix 😅

      • thegiddystitcher@lemm.ee
        link
        fedilink
        English
        arrow-up
        10
        ·
        edit-2
        11 months ago

        My other account is on .world so I can use search to forcibly pull things onto at least that server (search for the original URL of the unfederated post or comment and then wait a few seconds). Sometimes doing that will also encourage whatever it was to federate more widely to more servers, but that doesn’t seem to be happening as reliably either since the upgrade 🤷‍♀️

    • 1984@lemmy.today
      link
      fedilink
      English
      arrow-up
      3
      ·
      11 months ago

      A restart of Lemmy seems to fix it temporarily according to admin of Lemmy.today.

      But it’s inconsistent and stops working after a while it seems.

  • maegul (he/they)@lemmy.ml
    link
    fedilink
    English
    arrow-up
    2
    arrow-down
    5
    ·
    11 months ago

    I’d say the big seems to be slightly different from my observations, in that things are meant rather than broken. Could be wrong though.