noauthority.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
Long live NAS!

Administered by:

Server stats:

1.4K
active users

hyperscale is the new "big data"

@picofarad
big tech: :neocat_bottom: i must store petabytes
nature: :comfystoner: we do a little probablistic data store maxxing
picofarad

@icedquinn
you're the only person who replies to me where i consistently have to look up what you say to understand it. This is not a criticism.

And i knew i'd seen those words in close proximity, a bloom filter is a probabilistic data store

however i am not smart enough to get the reference to nature, probably (pun intended) because I'm quasi-innumerate

@picofarad basically a set of code that gives up absolute certainty for higher performance or simpler code. there is some geometric relationship between certainty and cost.

nature doesn't really try. a lot of stuff works on a "good enough" principle. like your immune system doesn't understand very much of diseases it just looks for cells that aren't giving the right protein flags or very partial images of known intruders.

this works for the most part, except for some rare diseases that cause it to learn patterns that also match things that are supposed to be there (some kinds of strep or something cause your immune system learn to attack heart valves)

however the sparse models are robust to data loss. the body is operating in a system that is highly lossy (radiation is always messing with your DNA and such) and the imprecise, uncertain models are robust to some corruption (enough of the rest of the model matches that it works) and mutations of disease still get flagged because enough of them matches the model that its a known bad.

absolute certainty is very expensive, and inflexible, and it turns out being inaccurate is a necessary part of how naturally grown systems are able to be flexible.

@icedquinn thanks for that. If i had just ruminated on bloom filters as applied to nature i may have gotten there sometime sunday