Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

access_log to stdout and error_log to stderr #23

Closed

Conversation

oyvindio
Copy link
Contributor

@oyvindio oyvindio commented May 2, 2016

To easily get query logs into the kubernetes fluentd-elasticsearch setup, log access_log to stdout and error_log to stderr.

  • Move nginx.conf from the base image into the source tree so that we can modify it.
  • When we start nginx, attach the controller's stdout and stderr to the nginx process. For this to work we have to run nginx in the foreground (daemon off). Since we're running nginx in the foreground, use a goroutine to avoid blocking the controller process.
  • Set access_log to /dev/stdout and error_log to stderr.

I don't know if you want to merge this as it is a significant change to how logging happens, but we're using this and I think it's useful, so I thought I'd share.

Since we can now edit nginx.conf, I've also added $host, so we can more easily tell which backend we're hitting when using host based Ingress, and $request_time, since that is generally useful, to the main access_log format.

It should be possible to modify the default nginx configuration.
I borrowed this idea from
https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx:

* Symlink the access and error log files to respectively /dev/stdout and
  /dev/stderr.
* When we start the nginx master process, attach its stdout and stderr
  to those of the controller process. Since the controller process is
  the entrypoint for the container, this is the containers stdout and
  stderr.
* Run nginx in the foreground (daemon off), because in daemon mode it
  seems to detach its stdout. This also means that we have to start the
  nginx master process in a goroutine since it will no longer fork().
* nginx worker processes will inherit the master's stdout and stderr
  such that access_log will be written to the containers stdout and
  error_log will be written to the containers stderr.
Don't symlink /dev/std{out,err} to /var/log/nginx/{access,error}.log. I
think this makes it less convoluted.
@pleshakov
Copy link
Contributor

Hi,

Thanks for submitting a pull request. I think it's a good idea to make redirecting nginx logs to stdout and stderr by default.

Another way to do it is to insert the following lines in the Dockerfile:

RUN ln -sf /proc/1/fd/1 /var/log/nginx/access.log \
    && ln -sf /proc/1/fd/2 /var/log/nginx/error.log

What we do here is redirecting logs to the stdout and stderr of the process with PID 1, which is the nginx-ingress process. Its stdout and stderr are seen in docker logs. Changes to nginx conf files and ingress controller code are not required in this case.

@oyvindio
Copy link
Contributor Author

RUN ln -sf /proc/1/fd/1 /var/log/nginx/access.log
&& ln -sf /proc/1/fd/2 /var/log/nginx/error.log

That's a nice solution! 👍

Do you want to merge the access_log format changes, or would you rather keep the defaults? We'll still need nginx.conf in the source tree to do anything with log format I guess

@pleshakov
Copy link
Contributor

Do you already make or need any other customizations of NGINX configuration?

We want to add support for customization of NGINX configuration via Kubernetes resources--ConfigMap and Annotations in an Ingress Resource. So that the user can be able to set proxy_read_timeout, proxy_connect_timeout, etc. Log format is a good candidate for adding the ability to customize. Will it work for you if you configure log format via ConfigMap?

@oyvindio
Copy link
Contributor Author

oyvindio commented May 12, 2016

We don't have any other customizations besides access_log format right now, but I think we likely will need to do some sort of customization at some point.

ConfigMap would probably be suitable, yeah. That's what the contrib/kubernetes ingress controller is supporting also: https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx#custom-nginx-configuration

@pleshakov pleshakov closed this Aug 17, 2016
vepatel added a commit that referenced this pull request Apr 26, 2021
# This is the 1st commit message:

add ingress mtls test

# This is the commit message #2:

add std vs

# This is the commit message #3:

change vs host

# This is the commit message #4:

Update tls secret

# This is the commit message #5:

update certs with host

# This is the commit message #6:

modify get_cert

# This is the commit message #7:

Addind encoded cert

# This is the commit message #8:

Update secrets

# This is the commit message #9:

Add correct cert and SNI module

# This is the commit message #10:

Bump styfle/cancel-workflow-action from 0.8.0 to 0.9.0 (#1527)

Bumps [styfle/cancel-workflow-action](https://github.com/styfle/cancel-workflow-action) from 0.8.0 to 0.9.0.
- [Release notes](https://github.com/styfle/cancel-workflow-action/releases)
- [Commits](styfle/cancel-workflow-action@0.8.0...89f242e)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# This is the commit message #11:

Remove patch version from Docker image for tests (#1534)


# This is the commit message #12:

Add tests for Ingress TLS termination

# This is the commit message #13:

Improve assertion of TLS errors in tests

When NGINX terminates a TLS connection for a server
with a missing/invalid TLS secret, we expect NGINX
to reject the connection with the error
TLSV1_UNRECOGNIZED_NAME

In this commit we:
* ensure the specific error
* rename the assertion function to be more specific

# This is the commit message #14:

Bump k8s.io/client-go from 0.20.5 to 0.21.0 (#1530)

Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.20.5 to 0.21.0.
- [Release notes](https://github.com/kubernetes/client-go/releases)
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](kubernetes/client-go@v0.20.5...v0.21.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# This is the commit message #15:

Improve tests Dockerfile

* Reorganize layers so that changes to the tests do not cause a full
image rebuilt
* Use .dockerignore to ignore cache folders
* Convert spaces to tabs for consistency with the other Dockerfiles

# This is the commit message #16:

Upgrade kubernetes-python client to 12.0.1 (#1522)

* Upgrade kubernetes-python client to 12.0.1

Co-authored-by: Venktesh Patel <[email protected]>
# This is the commit message #17:

Bump k8s.io/code-generator from 0.20.5 to 0.21.0 (#1531)

Bumps [k8s.io/code-generator](https://github.com/kubernetes/code-generator) from 0.20.5 to 0.21.0.
- [Release notes](https://github.com/kubernetes/code-generator/releases)
- [Commits](kubernetes/code-generator@v0.20.5...v0.21.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# This is the commit message #18:

Test all images (#1533)

* Test on all images

* Update nightly to test all images

* Run all test markers on debian plus also

* Update .github/workflows/nightly.yml

# This is the commit message #19:

Add tests for default server

# This is the commit message #20:

Support running tests in kind

# This is the commit message #21:

Update badge for Fossa (#1546)


# This is the commit message #22:

Fix ensuring connection in tests

* Add timeout for establishing a connection to prevent potential
"hangs" of the test runs. The problem was noticeable when running
tests in kind.
* Increase the number of tries to make sure the Ingress Controller
pod has enough time to get ready. When running tests in kind
locally the number of tries sometimes was not enough.

# This is the commit message #23:

Ensure connection in Ingress TLS tests

Ensure connection to NGINX before running tests.
Without ensuring, sometimes the first connection to NGINX would
hang (timeout). The problem is noticable when running tests in
kind.

# This is the commit message #24:

Revert changes in nightly for now (#1547)


# This is the commit message #25:

Bump actions/cache from v2.1.4 to v2.1.5 (#1541)

Bumps [actions/cache](https://github.com/actions/cache) from v2.1.4 to v2.1.5.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](actions/cache@v2.1.4...1a9e213)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# This is the commit message #26:

Create release workflow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants