Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disk reservation on Teleport servers to make sure access is not lost due to lack of space #7388

Closed
Tracked by #8745
deusxanima opened this issue Jun 22, 2021 · 4 comments
Closed
Tracked by #8745

Disk reservation on Teleport servers to make sure access is not lost due to lack of space #7388

deusxanima opened this issue Jun 22, 2021 · 4 comments
Assignees
@gabrielcorado
Labels
c-af Internal Customer Reference c-at Internal Customer Reference c-bi Internal Customer Reference c-inm Internal Customer Reference c-ju Internal Customer Reference c-rs Internal Customer Reference c-td Internal Customer Reference feature-request Used for new features in Teleport, improvements to current should be #enhancements server-access

Comments

@deusxanima
Copy link
Contributor

deusxanima commented Jun 22, 2021 •
edited by programmerq
Loading

What

A number of customers have run into an issue where they're unable to access Teleport at times due to lack of disk space. Request is to have teleport reserve X amount of space on startup to ensure that tsh ssh access is not lost as the only remedy to this issue when it presents is ssh-ing into the server/node, clearing out space, and restarting teleport - something that's not feasible in certain environments where customers only have access to the node through Teleport tunnel and tsh ssh

How

Hard reservation to account for space needed by Teleport processes and tsh ssh

Why

Customers have run into issue and been locked out in the past.

Workaround

Customer have to manually guess and scope the amount of space required for Teleport, which may not always be accurate.

gz#4021
@deusxanima deusxanima added feature-request Used for new features in Teleport, improvements to current should be #enhancements c-bi Internal Customer Reference c-at Internal Customer Reference server-access labels Jun 22, 2021
@deusxanima deusxanima added the c-rs Internal Customer Reference label Jul 14, 2021
@pschisa pschisa added the c-td Internal Customer Reference label Aug 31, 2021
@corkrean
Copy link
Contributor

After testing, I have confirmed that node sync mode solves the issue of losing access when disk space is full.

@pschisa
Copy link
Contributor

pschisa commented Dec 3, 2021

Similar requests to issues #5424 and #2200

@pschisa pschisa added the c-af Internal Customer Reference label Dec 3, 2021
@russjones russjones added the c-ju Internal Customer Reference label Dec 7, 2021
@russjones russjones mentioned this issue Dec 7, 2021
Open
14 tasks
@nickbrennan1 nickbrennan1 added the c-inm Internal Customer Reference label Jan 10, 2022
@goakley
Copy link

goakley commented Feb 9, 2022

Since I didn't see it explicitly mentioned, I'd like to call out that we've seen similar behaviour when the filesystem is out of inodes as well, not just when it's out of space.

@gabrielcorado
Copy link
Contributor

Closed by #12916.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gabrielcorado gabrielcorado

Labels
c-af Internal Customer Reference c-at Internal Customer Reference c-bi Internal Customer Reference c-inm Internal Customer Reference c-ju Internal Customer Reference c-rs Internal Customer Reference c-td Internal Customer Reference feature-request Used for new features in Teleport, improvements to current should be #enhancements server-access
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@klizhentas @goakley @gabrielcorado @russjones @deusxanima @nickbrennan1 @corkrean @pschisa