Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't clone repo after commiting lfs files #21886

Closed
night-gold opened this issue Nov 21, 2022 · 10 comments
Closed

Can't clone repo after commiting lfs files #21886

night-gold opened this issue Nov 21, 2022 · 10 comments
Labels
issue/workaround it is or has a workaround type/bug

Comments

@night-gold
Copy link

Description

We are doing a test install of gitea and trying to use the gitea lfs to s3 feature. The configuration is straightforward and we did it like that:

[lfs]
#PATH = /opt/gitea/data/lfs
STORAGE_TYPE = minio
MINIO_ACCESS_KEY_ID = KEY
MINIO_SECRET_ACCESS_KEY = SECRET
MINIO_BUCKET = NAME
MINIO_LOCATION = us-east-1
MINIO_USE_SSL = true
SERVE_DIRECT = true
MINIO_ENDPOINT = s3.us-east-1.amazonaws.com

That works, I created a repo and cloned another I had at disposal with lfs config. I managed to commit and push all my lfs files and it uploaded everything on the s3. So that's great.

But now I have an issue when cloning the repo, the clone does get the git files but when trying to get the lfs files on the s3 I get this message:

Error downloading object: FILE (hash): Smudge error: Error downloading FILE (hash): LFS: Get "https://NAME.s3.dualstack.us-east-1.amazonaws.com/lfs/PATHTOFILE": dial tcp: lookup NAME.s3.dualstack.us-east-1.amazonaws.com on IP:53: dial udp IP:53: socket: too many open files
From what I get it's only trying to get one file but even that doesn't work.

Has anyone seen that before?

Gitea Version

1.17.3

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

Debian

How are you running Gitea?

Installed gitea using one of the downloads, it is for testing purpose launched manually on foreground.

Database

PostgreSQL

@lunny
Copy link
Member

lunny commented Nov 21, 2022

Looks like it's a DNS problem not Gitea related.

@night-gold
Copy link
Author

night-gold commented Nov 21, 2022

@lunny What makes you think it's DNS related?

@lunny
Copy link
Member

lunny commented Nov 21, 2022

From the log lookup NAME.s3.dualstack.us-east-1.amazonaws.com on IP:53: dial udp IP:53: socket: too many open files, I can see it failed when lookup the domain.

@night-gold
Copy link
Author

So I've been testing different things that did not works. I had to reinstall the gitea on an instance with public ip and no filter.

It changed the error but I still can't clone lfs files:
Error downloading object: PATH-TO-FILE (HASH): Smudge error: Error downloading PATH-TO-FILE (HASH): LFS: Get "https://NAME.s3.dualstack.us-east-1.amazonaws.com/lfs/HASH": dial tcp [2600:1fa0:8095:3948:34d8:fab0::]:443: connect: network is unreachable

The s3 is available I have access to it locally and the gitea has access, it could upload on it.
I have full access on the network of the instance, I opened every port, UDP and TCP on it and still nothing.

@lunny
Copy link
Member

lunny commented Nov 29, 2022

So I've been testing different things that did not works. I had to reinstall the gitea on an instance with public ip and no filter.

It changed the error but I still can't clone lfs files: Error downloading object: PATH-TO-FILE (HASH): Smudge error: Error downloading PATH-TO-FILE (HASH): LFS: Get "https://NAME.s3.dualstack.us-east-1.amazonaws.com/lfs/HASH": dial tcp [2600:1fa0:8095:3948:34d8:fab0::]:443: connect: network is unreachable

The s3 is available I have access to it locally and the gitea has access, it could upload on it. I have full access on the network of the instance, I opened every port, UDP and TCP on it and still nothing.

But the error is obvious that network is unreachable.

@night-gold
Copy link
Author

night-gold commented Nov 29, 2022

Totally, I've been asking for help because I can't read... Please don't post those type of answer... If you read the message you would have seen that:

  • I have full access on the network of the instance, I opened every port, UDP and TCP on it and still nothing.
  • I can access the s3 localy
  • The gitea can access the s3 as it could upload files.

The message doesn't tell me nothing and I've posted the error message as I have no idea where I should take a look at...

I didn't see those line in the log, but I've got them after the network error (During the same request for cloning):

error: le filtre externe 'git-lfs filter-process' a échoué
fatal: Assets/Plugins/SimpleFileBrowser/Sprites/Checkmark.psd : le filtre smudge 'lfs' a échoué

@night-gold
Copy link
Author

Soooo... I finally found what was wrong... And it had nothing to do with the network. The error had something to do with the LFS conf and more precisely the SERVE_DIRECT option that was at TRUE. When I switched it to FALSE everything worked!

I don't really understand how this option created a network error and I lost a lot of time on it.

Putting the solution here if someone has the same issue in the futur.

@lunny lunny added the issue/workaround it is or has a workaround label Nov 30, 2022
@arychj
Copy link

arychj commented Dec 12, 2022

@night-gold's workaround is the only solution for gitea v1.7.3, but it looks like there was a regression introduced at some point between 1.15.8 and now. It should be fixed in 1.18.0.

@artyomabgaryan
Copy link

artyomabgaryan commented Jan 5, 2023

solution form night-gold doesn't work for me.. getting this when running server in test or dev mode`
Gitea: Internal Server Error
Get "https://localhost:3000/api/internal/serv/command/14/artyom/repo?mode=1&verb=git-upload-pack": dial tcp: lookup localhost on 192.168.1.1:53: no such host
fatal: Could not read from remote repository.
versions main (1.19.0 267) and 1.18 too

SOLUTION`
#21528 (comment)

@jinyangcruise
Copy link

jinyangcruise commented Jan 8, 2023

SOLUTION`
https://discord.com/channels/322538954119184384/561007778139734027/1060668901656113283

first join gitea in discord then open this url, otherwise it shows nothing.

@go-gitea go-gitea locked and limited conversation to collaborators May 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
issue/workaround it is or has a workaround type/bug
Projects
None yet
Development

No branches or pull requests

5 participants