Split Large Keys/Values

[seancarroll]

From Foundationdb known limitations

Keys cannot exceed 10,000 bytes in size. Values cannot exceed 100,000 bytes in size. Errors will be raised by the client if these limits are exceeded.

I don't think we'll have issues with the key size, famous last words, given we can constrain size of stream name and keys however we'll need to add logic to split values so we can support larger values.

Other links

• https://apple.github.io/foundationdb/largeval.html
• https://apple.github.io/foundationdb/data-modeling.html#data-modeling-performance-guidelines

[seancarroll]

I like Hashicorp's Vault FoundationDB backend implementation using directory entry markers and directory path markers to determine paths entries or leaf entries