1.18.0 on Windows error: can not resolve localhost, ! [remote rejected] master -> master (pre-receive hook declined) #22370

jinyangcruise · 2023-01-08T13:51:54Z

Description

Save your time:

1.17.4 is good but 1.18.0 has a bug: 
dial tcp: lookup localhost on 114.114.114.114:53: no such host

Detailed:

I use 1.18.0 Gitea, when I push files, occur:

$ git push
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 20 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 305 bytes | 305.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote:
remote: Gitea: Internal Server Error
remote: Unable to contact gitea: Post "http://localhost:3000/api/internal/hook/pre-receive/jinyang/test": dial tcp: lookup localhost on 114.114.114.114:53: no such host
To http://localhost:3000/jinyang/test.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'http://localhost:3000/jinyang/test.git'

I searched some related issues but finally failed to fix this.

Then I installed 1.17.4 Gitea and this problem gone.

Then I re-installed 1.18.0 Gitea again and still can not push.

Gitea Version

1.18.0

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

NA

Git Version

git version 2.39.0.windows.2

Operating System

WINDOWS 11

How are you running Gitea?

download binary on https://dl.gitea.io/gitea
[gitea-1.18.0-windows-4.0-amd64.exe]

download it on my PC and move it to D:\Program Files\Gitea, then double click the gitea-1.18.0-windows-4.0-amd64.exe

Database

SQLite

The text was updated successfully, but these errors were encountered:

lunny · 2023-01-08T14:37:44Z

Could you ping localhost?

wxiaoguang · 2023-01-08T14:59:59Z

Workaround: set LOCAL_ROOT_URL with 127.0.0.1, do not use localhost

jinyangcruise · 2023-01-09T07:21:52Z

Could you ping localhost?

yes

jinyangcruise · 2023-01-09T07:22:39Z

Workaround: set LOCAL_ROOT_URL with 127.0.0.1, do not use localhost

I can't find a LOCAL_ROOT_URL in app.ini, should I add it at somewhere?

lunny · 2023-01-09T07:29:21Z

Workaround: set LOCAL_ROOT_URL with 127.0.0.1, do not use localhost

I can't find a LOCAL_ROOT_URL in app.ini, should I add it at somewhere?

Please check the documentation.

jinyangcruise · 2023-01-09T08:20:20Z

Workaround: set LOCAL_ROOT_URL with 127.0.0.1, do not use localhost

I can't find a LOCAL_ROOT_URL in app.ini, should I add it at somewhere?

Please check the documentation.

Thank you very much! Success!

By the way, english version documentation has this LOCAL_ROOT_URL but chinese version not.
https://docs.gitea.io/en-us/config-cheat-sheet/#server-server
https://docs.gitea.io/zh-cn/config-cheat-sheet/#server-server

emehmet · 2023-01-10T00:27:14Z

it does not work for me

starpeng · 2023-01-10T03:39:51Z

it work for me

Workaround: set LOCAL_ROOT_URL with 127.0.0.1, do not use localhost

jinyangcruise · 2023-01-10T04:09:49Z

it does not work for me

workaround:

LOCAL_ROOT_URL = http://127.0.0.1:3000/

lunny · 2023-01-10T08:19:15Z

it does not work for me

Is your problem the same as others?

zeripath · 2023-01-10T09:19:46Z

So leaving aside the workaround which I am sure will fix the issues people are describing - I don't understand why it is needed or what has changed to make it needed.

As far as I can see the determination of the value of setting.LocalURL (i.e. that which LOCAL_ROOT_URL maps to) has not changed and nor has the dialer for this code.

So I don't understand why this should suddenly be a problem...

zeripath · 2023-01-10T10:29:44Z

For others seeing this conversation the LOCAL_ROOT_URL is in the [server] section of the app.ini and it must be added there.

emehmet · 2023-01-10T10:44:18Z

Hey @jinyangcruise
It worked as you said.
Thank you! 👍

it does not work for me

workaround:

LOCAL_ROOT_URL = http://127.0.0.1:3000/

wxiaoguang · 2023-01-11T03:50:24Z

I think this issue should be re-opened and it needs a patch.

To make the workaround prominent, I didn't say too much about my guess before.

Here is my opinion of this bug:

How it happens?

Gitea (Golang net package) can not resolve host localhost.

Golang's net package need correct /etc/hosts and /etc/nsswitch.conf to resolve domains correctly. If anything is broken, it may not resolve some domains. On Windows, it also depends on C:\Windows\System32\Drivers\etc\hosts file

Why there are so many reports recently?

Possibility 1

Alpine / Go or Go for Windows changed some behaviors which conflict with user's environment, then Golang's net package doens't work well in some rare cases (no evidence yet). But since some users reported that 1.17.x works but 1.18.x fails, maybe there is something changed.

Possibility 2

Many users' systems were already broken, they messed up their hosts file, they might have deleted the 127.0.0.1 localhost line. I know a lot of such users.

Before, some public DNS servers can also resolve localhost, for example, there is still one now:

$ dig localhost @223.5.5.5
;; ANSWER SECTION:
localhost.		2495	IN	A	127.0.0.1

Recently some public DNS servers might have stopped such non-standard behavior, if they do not resolve localhost anymore, the users with broken localhost definition will encounter the problem.

So, some users with broken hosts files can get Gitea working before, but get errors now.

The patch

Approach 1

Use 127.0.0.1 instead of localhost for the default value of LOCAL_ROOT_URL

Pros
- 127.0.0.1 is the de-facto standard loopback IP, it should exist in every OS
- won't be affected by Host OS/Alpine/Golang anymore (in most cases)
- even if the user's hosts file is broken, it still works
Cons
- In a pure IPv6 network, the 127.0.0.1 may not exist, but the pure IPv6 network should be pretty rarer than broken hosts files, and a user with pure IPv6 must be an experienced user, they should be able to set LOCAL_ROOT_URL to a IPv6 loopback without any question.

Approach 2

Find the root case why 1.17.x works but 1.18.x doesn't.

Update: maybe it's a Go 1.19 bug: golang/go#53490

And: https://tip.golang.org/doc/go1.19#net

The net package now has initial support for the netgo build tag on Windows. When used, the package uses the Go DNS client (as used by Resolver.PreferGo) instead of asking Windows for DNS results. The upstream DNS server it discovers from Windows may not yet be correct with complex system network configurations, however.

Removing the netgo tag from Windows build may resolve the problem.

Approach 3

Try to resolve localhost at startup, report fatal errors if the resolution fails.

zeripath · 2023-01-11T05:41:04Z

We could use 127.0.0.1 instead of localhost only on windows then we get none of the cons.

zeripath · 2023-01-11T05:54:31Z

But yes I agree I think this was closed too soon and we need to add a patch - if not for the users, then just to save ourselves time.

DuaneDu · 2023-01-11T08:35:24Z

@zeripath
Config is

[server]
LOCAL_ROOT_URL = http://127.0.0.1:3000/
SSH_DOMAIN       = localhost
DOMAIN           = 192.168.2.50
HTTP_PORT        = 3000
DISABLE_SSH      = false
SSH_PORT         = 22
LFS_START_SERVER = true
LFS_CONTENT_PATH = D:/work/gitea/data/lfs
LFS_JWT_SECRET   = TGUsxz3aFCZEj8CnOPFzEPUXLhzRTAlOxvd8Rwvl9Q4
OFFLINE_MODE     = false

error

console log

gitea.log
gitea.log

KazzmanK · 2023-01-11T12:15:23Z

Same problem for me, windows install, for unknown reason problem raised today, yesterday it was ok.
in hosts we have 127.0.0.1 localhost record. It still calls for dns and fails.
workaround with LOCAL_ROOT_URL helped.

@DuaneDu , add DEV run mode, client will show bit more detailed error message

zeripath · 2023-01-11T12:20:21Z

@DuaneDu your logs now indicate that things are should now be working correctly for you.

Your pre-receive hook is being declined by the user does not have permission to that repo

wxiaoguang · 2023-01-11T12:21:19Z

I think it's related to this Go bug:

net: 1.19 DNS lookups broken on Windows if standard library built with CGO_ENABLED=0 golang/go#53490

And: https://tip.golang.org/doc/go1.19#net

The net package now has initial support for the netgo build tag on Windows. When used, the package uses the Go DNS client (as used by Resolver.PreferGo) instead of asking Windows for DNS results. The upstream DNS server it discovers from Windows may not yet be correct with complex system network configurations, however.

Removing the netgo tag from Windows build may resolve the problem.

natsutteatsuiyone · 2023-01-11T14:41:36Z

This webhook error may be related to this issue.

2023/01/11 23:20:28 ...s/webhook/webhook.go:122:handle() [E] [63bec58b-9] webhook.Deliver failed: Post "http://dev-738-11/gitea-webhook/post": dial tcp: lookup dev-738-11 on 192.168.10.3:53: server misbehaving

https://gist.github.com/natsutteatsuiyone/4c7b8e3b444a61080bb04ece20b316aa

DuaneDu · 2023-01-12T01:53:59Z

@zeripath

This account is administrator
this repo is public
this repo is created by me
I have all permission on this repo

RUN_MODE=dev
gitea.log
router.log

RUN_MODE=test

gitea.log
router.log

zeripath · 2023-01-16T11:51:41Z

The net package now has initial support for the netgo build tag on Windows. When used, the package uses the Go DNS client (as used by Resolver.PreferGo) instead of asking Windows for DNS results. The upstream DNS server it discovers from Windows may not yet be correct with complex system network configurations, however.

It certainly appears that the upstream DNS server it is discovering is doing the incorrect thing for at least localhost...

Removing the netgo tag from Windows build may resolve the problem.

I guess the question is what does the netgo tag do on Windows build and is it really necessary to make a "static" build with it for windows? Removing netgo here:

gitea/Makefile

Lines 759 to 764 in 16e9dec

    
           .PHONY: release-windows 
        
           release-windows: | $(DIST_DIRS) 
        
           	CGO_CFLAGS="$(CGO_CFLAGS)" $(GO) run $(XGO_PACKAGE) -go $(XGO_VERSION) -buildmode exe -dest $(DIST)/binaries -tags 'netgo osusergo $(TAGS)' -ldflags '-linkmode external -extldflags "-static" $(LDFLAGS)' -targets 'windows/*' -out gitea-$(VERSION) . 
        
           ifeq (,$(findstring gogit,$(TAGS))) 
        
           	CGO_CFLAGS="$(CGO_CFLAGS)" $(GO) run $(XGO_PACKAGE) -go $(XGO_VERSION) -buildmode exe -dest $(DIST)/binaries -tags 'netgo osusergo gogit $(TAGS)' -ldflags '-linkmode external -extldflags "-static" $(LDFLAGS)' -targets 'windows/*' -out gitea-$(VERSION)-gogit . 
        
           endif

would create "Static" builds for windows without the use of netgo.

zeripath · 2023-01-16T11:59:25Z

I guess an option is to ask users who are affected if they can run Gitea with the environment variable GODEBUG=netdns=cgo, e.g.:

GODEBUG=netdns=cgo gitea web

That should force the use of the win32 api.

wxiaoguang · 2023-01-16T12:05:35Z

I have reported the bug to Go team (it has been confirmed as NeedsFix there)

net: Windows build with netgo doesn't repsect hosts file, can not resolve localhost golang/go#57757

I guess the question is what does the netgo tag do on Windows build and is it really necessary to make a "static" build with it for windows? Removing netgo here:

I guess the netgo in Makefile was just a Copy&Paste and it's not necessary.

And, the netgo seemed to be a no-op on Windows before. But Go 1.19 rewrite the code for it.

So, IMO it's good to remove the netgo tag for Windows build at the moment. When Go team fix the bug, re-enable the netgo tag then (if it's really needed ...)

zeripath · 2023-01-16T12:06:57Z

Agreed I've just looked up the source code for golang and I've come to the same conclusion. netgo only affects lookup on windows and previously was a no-op. Therefore we can and should safely remove until/unless go can make it work correctly.

Are you able to send a PR @wxiaoguang ?

zeripath · 2023-01-16T12:09:45Z

As an aside I suspect the problem is here:

https://github.com/golang/go/blob/d52883f443e1d564b0300acdd382af1769bf0477/src/net/conf.go#L134-L145

	if c.netGo || r.preferGo() {
		switch c.goos {
		case "windows":
			// TODO(bradfitz): implement files-based
			// lookup on Windows too? I guess /etc/hosts
			// kinda exists on Windows. But for now, only
			// do DNS.
			fallbackOrder = hostLookupDNS
		default:
			fallbackOrder = hostLookupFilesDNS
		}
	}

That code cannot check the /etc/hosts file and thus I bet the dns lookup is going out to an external server and we're getting the external name for the host.

Backport #22467 Fix #22370 and more. Before Go 1.19, the `netgo` tag for Windows does nothing. But Go 1.19 rewrite the net package code for Windows DNS, and there is a bug: * golang/go#57757 This PR just removes the `netgo` tag for Windows build, then the Gitea for Windows can have the old DNS behavior.

Fix #22370 and more. Before Go 1.19, the `netgo` tag for Windows does nothing. But Go 1.19 rewrite the net package code for Windows DNS, and there is a bug: * golang/go#57757 This PR just removes the `netgo` tag for Windows build, then the Gitea for Windows can have the old DNS behavior.

zeripath · 2023-01-16T13:44:47Z

DESCRIPTION

This bug is due to a change in Go's handling of the netgo build tag for Windows see:

#22370 (comment)

We were bitten by this because we build our static precompiled binaries with netgo for all platforms. (see #22370 (comment))

#22467 and #22468 should fix this for 1.18-dev, 1.18.1 and 1.19-dev.

WORKAROUND

For 1.18.0 set the GODEBUG=netdns=cgo environment variable when running gitea.

#22370 (comment)

Builds of what will become 1.18.1 are available now on https://dl.gitea.io/gitea/1.18

jinyangcruise added the type/bug label Jan 8, 2023

lunny added type/question Issue needs no code to be fixed, only a description on how to fix it yourself. and removed type/bug labels Jan 10, 2023

This was referenced Jan 10, 2023

(pre-receive hook declined) error on windows #22381

Closed

git push.(pre-receive hook declined) error: failed to push some refs to "xxx"] #22293

Closed

lunny closed this as completed Jan 10, 2023

wxiaoguang changed the title ~~1.18.0 occur error: ! [remote rejected] master -> master (pre-receive hook declined)~~ 1.18.0 occur error: can not resolve localhost, ! [remote rejected] master -> master (pre-receive hook declined) Jan 11, 2023

wxiaoguang reopened this Jan 11, 2023

wxiaoguang added this to the 1.18.1 milestone Jan 11, 2023

lunny modified the milestones: 1.18.1, 1.18.2 Jan 16, 2023

This was referenced Jan 16, 2023

Remove the netgo tag for Windows build #22467

Merged

Remove the netgo tag for Windows build (#22467) #22468

Merged

zeripath closed this as completed in #22467 Jan 16, 2023

zeripath mentioned this issue Jan 16, 2023

Hook doesnt run on push #22466

Closed

zeripath added type/bug and removed type/question Issue needs no code to be fixed, only a description on how to fix it yourself. labels Jan 16, 2023

zeripath modified the milestones: 1.18.2, 1.18.1 Jan 16, 2023

zeripath changed the title ~~1.18.0 occur error: can not resolve localhost, ! [remote rejected] master -> master (pre-receive hook declined)~~ 1.18.0 on Windows error: can not resolve localhost, ! [remote rejected] master -> master (pre-receive hook declined) Jan 16, 2023

zeripath mentioned this issue Jan 16, 2023

[remote rejected] main -> main (pre-receive hook declined) #21528

Closed

go-gitea locked and limited conversation to collaborators May 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

1.18.0 on Windows error: can not resolve localhost, ! [remote rejected] master -> master (pre-receive hook declined) #22370

1.18.0 on Windows error: can not resolve localhost, ! [remote rejected] master -> master (pre-receive hook declined) #22370

jinyangcruise commented Jan 8, 2023 •

edited

Loading

lunny commented Jan 8, 2023

wxiaoguang commented Jan 8, 2023

jinyangcruise commented Jan 9, 2023

jinyangcruise commented Jan 9, 2023

lunny commented Jan 9, 2023

jinyangcruise commented Jan 9, 2023

emehmet commented Jan 10, 2023

starpeng commented Jan 10, 2023

jinyangcruise commented Jan 10, 2023 •

edited

Loading

lunny commented Jan 10, 2023

zeripath commented Jan 10, 2023

zeripath commented Jan 10, 2023

emehmet commented Jan 10, 2023

wxiaoguang commented Jan 11, 2023 •

edited

Loading

zeripath commented Jan 11, 2023

zeripath commented Jan 11, 2023

DuaneDu commented Jan 11, 2023 •

edited

Loading

KazzmanK commented Jan 11, 2023

zeripath commented Jan 11, 2023

wxiaoguang commented Jan 11, 2023 •

edited

Loading

natsutteatsuiyone commented Jan 11, 2023 •

edited

Loading

DuaneDu commented Jan 12, 2023 •

edited

Loading

zeripath commented Jan 16, 2023

zeripath commented Jan 16, 2023

wxiaoguang commented Jan 16, 2023 •

edited

Loading

zeripath commented Jan 16, 2023 •

edited

Loading

zeripath commented Jan 16, 2023

zeripath commented Jan 16, 2023 •

edited

Loading

1.18.0 on Windows error: can not resolve localhost, ! [remote rejected] master -> master (pre-receive hook declined) #22370

1.18.0 on Windows error: can not resolve localhost, ! [remote rejected] master -> master (pre-receive hook declined) #22370

Comments

jinyangcruise commented Jan 8, 2023 • edited Loading

Description

Gitea Version

Can you reproduce the bug on the Gitea demo site?

Log Gist

Screenshots

Git Version

Operating System

How are you running Gitea?

Database

lunny commented Jan 8, 2023

wxiaoguang commented Jan 8, 2023

jinyangcruise commented Jan 9, 2023

jinyangcruise commented Jan 9, 2023

lunny commented Jan 9, 2023

jinyangcruise commented Jan 9, 2023

emehmet commented Jan 10, 2023

starpeng commented Jan 10, 2023

jinyangcruise commented Jan 10, 2023 • edited Loading

lunny commented Jan 10, 2023

zeripath commented Jan 10, 2023

zeripath commented Jan 10, 2023

emehmet commented Jan 10, 2023

wxiaoguang commented Jan 11, 2023 • edited Loading

How it happens?

Why there are so many reports recently?

Possibility 1

Possibility 2

The patch

Approach 1

Approach 2

Approach 3

zeripath commented Jan 11, 2023

zeripath commented Jan 11, 2023

DuaneDu commented Jan 11, 2023 • edited Loading

KazzmanK commented Jan 11, 2023

zeripath commented Jan 11, 2023

wxiaoguang commented Jan 11, 2023 • edited Loading

natsutteatsuiyone commented Jan 11, 2023 • edited Loading

DuaneDu commented Jan 12, 2023 • edited Loading

zeripath commented Jan 16, 2023

zeripath commented Jan 16, 2023

wxiaoguang commented Jan 16, 2023 • edited Loading

zeripath commented Jan 16, 2023 • edited Loading

zeripath commented Jan 16, 2023

zeripath commented Jan 16, 2023 • edited Loading

DESCRIPTION

WORKAROUND

jinyangcruise commented Jan 8, 2023 •

edited

Loading

jinyangcruise commented Jan 10, 2023 •

edited

Loading

wxiaoguang commented Jan 11, 2023 •

edited

Loading

DuaneDu commented Jan 11, 2023 •

edited

Loading

wxiaoguang commented Jan 11, 2023 •

edited

Loading

natsutteatsuiyone commented Jan 11, 2023 •

edited

Loading

DuaneDu commented Jan 12, 2023 •

edited

Loading

wxiaoguang commented Jan 16, 2023 •

edited

Loading

zeripath commented Jan 16, 2023 •

edited

Loading

zeripath commented Jan 16, 2023 •

edited

Loading