https://github.com/LemmyNet/lemmy/issues/3245
I posted far more details on the issue then I am putting here-
But, just to bring some math in- with the current full-mesh federation model, assuming 10,000 instances-
That will require nearly 50 million connections.
Each comment. Each vote. Each post, will have to be sent 50 million seperate times.
In the purposed hub-spoke model, We can reduce that by over 99%, so that each post/vote/comment/etc, only has to be sent 10,000 times (plus n*(n-1)/2 times, where n = number of hub servers).
The current full mesh architecture will not scale. I predict, exponential growth will continue to occur.
Let’s work on a solution to this problem together.
I am probably missing something / being really oblivious (its been a long day…) but wouldn’t this same problem occur to the hub server in your model?
Although thinking about it a bit more, I thought I recalled seeing one of the Lemmy devs mention that the biggest issue is the SQL queries that are ran for various actions (such as loading the front page) - if that is the case, I don’t know if this idea would help with that.
The idea of a centralized hub server(s) also sounds like we’d be moving closer to the model of a centralized Reddit… But I guess in a way, the fact that larger instances exist in of itself poses the same issue?
… I’m probably just rambling to myself at this point, however, I do think a message queue type of system for federating events would be a good idea, for the sake of recovering from send failures.