• 124 Posts
  • 20 Comments
Joined 1 year ago
cake
Cake day: June 4th, 2023

help-circle
































  • The fediverse is a few thousand servers, from Mastodon, Lemmy, etc. Can’t say the amount of posts but there are a lot.

    So on the more technical side, I plan on using a light weight fast search engine called Sonic (It’s written in rust). I have already used it in other projects and it can handle billions of messages / posts. But it has a cost it doesn’t have faceted search, like for example if you want to exclude certain texts from the results. I think this is a fair trade off. The other solution would be to use something more mature like ElasticSearch but it’ll be expensive (I’m assuming not much money will be made from this and I’m talking about donations)

    For scanning sites there are premade lists to start with and it’ll be possible to scan new sites from other instances if found. So a bit of both.