Namespace quota for multi-tenancy

[xh4n3]

Now etcd has a global storage quota, but for multi-tenant environment, it seems we cannot limit storage usage, key number usage for each user, or each role.

IMHO, if there is a quota for this scenario, it would satisfy these needs.

[xh4n3]

My first idea was adding KeyQuota and StorageQuota to the Role struct, Usage and Quota are created in the following ways:

• List all keys and calculate usage if none usage exists
• Using default quota when creating new roles
• Usage are calculated only when keys matched KeyPermission is being updated
• If Quota.Usage + Cost(Operation) > Quota.Quota, return quota exceeded error.

type User struct {
	Name     []byte
	Password []byte
	Roles    []string
}
type Role struct {
	Name          []byte
	KeyPermission []*Permission
	KeyQuota []*Quota
	StorageQuota []*Quota
}
type Quota struct {
	Usage     int64
	Quota      int64
}

But multiple users can be bound to a role. If userA, userB are in roleA, when userA exceeds role's quota, userB are impacted as well, this seems unfair.

So I'm thinking maybe creating a new bblot bucket for quota, the implementation can be just like leases, mapping key-space to quota.

[wenjiaswe]

cc @jpbetz

[xiang90]

@xh4n3

If we want to do multi-tenancy, probably we need to really have separate mvcc store. Or operations like compaction will affect other tenants. The role mechanism is not designed for isolation but simply for auth. The current isolation is pretty weak.

[texascloud]

I'm interested in this for using multi tenant etcd for kubernetes clusters. Would be nice to restrict certain users from taking up too much key space. Then again, I guess I could handle that at the kubernetes level with object quotas, hmmm

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[agargi]

I believe this would be an interesting feature. Solution would also need to `address number of requests that each tenant can fire into the cluster (and not only key/ storage quotas).

[tangcong]

yes, it also needs a QoS feature #8483. @agargi

[vivekpatani]

@tangcong @agargi @xh4n3 @xiang90 do you guys have any POCs for this? Just wanted to check in to see if you folks had something in mind. Thanks.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

[vivekpatani]

@tangcong @gyuho @jpbetz @xiang90 @wenjiaswe Have we investigated any type of sub buckets within bbolt to implement namespace quota?

Some features/issues related to this:
etcd-io/bbolt#120
boltdb/bolt#422
https://github.com/SkycoinProject/skycoin/issues/66
boltdb/bolt#293 (more tangential)

I'm just looking to gauge if the community would be interested in something like this? Is this a desirable feature and if it's fitting with the etcd offerings? Thanks.

[vivekpatani]

@tangcong @gyuho @jpbetz @xiang90 @wenjiaswe @agargi Here's a technical doc that solves the problem for multi tenancy. Would really appreciate a review and get feedback, would like to implement this once the review is complete and all the issues in the document are addressed. Thank you, appreciate your time and efforts.

Link: https://docs.google.com/document/d/1POdNdFBgpcGqLwurZjy8qQki6beOoVIcElTqHy4DnnA/edit

[wenjiaswe]

cc @jingyih @ptabor

[vivekpatani]

@tangcong @gyuho @jpbetz @xiang90 @wenjiaswe @agargi @jingyih @ptabor just a gentle reminder.

Link: https://docs.google.com/document/d/1POdNdFBgpcGqLwurZjy8qQki6beOoVIcElTqHy4DnnA/edit

[vivekpatani]

@tangcong @gyuho @jpbetz @xiang90 @wenjiaswe @agargi @jingyih @ptabor just a gentle reminder.

We spoke about this at the kube-sig-api-machinery. We'd like to get your feedback on this in order to move forward.

Thank you.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.