Better handling of tcp errors
If the tcp client connection is lost, or timeout during the response write (especially with /render), we get a panic that is dramatically reported in the logs, and a raw log message from http.ResponseWriter because of the extra call to WriteHeaders that is made by the panic middleware.
We need to specifically handle tcp errors in the panic middleware to show them as warnings. Also, the server logger should be customized so it sends messages to zerolog and provide a mean to intercept specific log messages that we want to silence.
The typical log trace we have is:
10:31PM ERR Caught a panic ip=10.0.0.163 req_id=c1ifftc5bl3q2r4n60og user={"id":"alpe-qualif","is-admin":false} user_agent=python-requests/2.20.0 exception=&net.OpError{Op:"write", Net:"tcp", Source:(*net.TCPAddr)(0xc00051e060), Addr:(*net.TCPAddr)(0xc00051e0f0), Err:(*poll.TimeoutError)(0x1b3b260)}(*net.OpError)
2021/03/31 22:31:58 http: superfluous response.WriteHeader call from orus.io/orus-io/rednerd/restapi.CatchPanics.func1.1 (panic_middleware.go:37)
/builds/orus-io/rednerd/restapi/operations/rendering/render_responses.go:59
orus.io/orus-io/rednerd/restapi/operations/rendering.(*RenderOK).WriteResponse
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/go-openapi/runtime@v0.19.19/middleware/context.go:506
github.com/go-openapi/runtime/middleware.(*Context).Respond
/builds/orus-io/rednerd/restapi/operations/rendering/render.go:81
orus.io/orus-io/rednerd/restapi/operations/rendering.(*Render).ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/go-openapi/runtime@v0.19.19/middleware/operation.go:28
github.com/go-openapi/runtime/middleware.NewOperationExecutor.func1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/go-openapi/runtime@v0.19.19/middleware/router.go:77
github.com/go-openapi/runtime/middleware.NewRouter.func1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/go-openapi/runtime@v0.19.19/middleware/redoc.go:72
github.com/go-openapi/runtime/middleware.Redoc.func1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/go-openapi/runtime@v0.19.19/middleware/spec.go:46
github.com/go-openapi/runtime/middleware.Spec.func1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/restapi/prometheus_middleware.go:28
orus.io/orus-io/rednerd/restapi.Prometheus.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/restapi/enginedoc.go:41
orus.io/orus-io/rednerd/restapi.EngineDoc.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/restapi/redoc_middleware.go:73
orus.io/orus-io/rednerd/restapi.Redoc.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/restapi/auth_middleware.go:84
orus.io/orus-io/rednerd/restapi.AuthMiddleware.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/rs/zerolog@v1.19.0/hlog/hlog.go:196
github.com/rs/zerolog/hlog.AccessHandler.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/rs/zerolog@v1.19.0/hlog/hlog.go:169
github.com/rs/zerolog/hlog.RequestIDHandler.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/rs/zerolog@v1.19.0/hlog/hlog.go:120
github.com/rs/zerolog/hlog.RefererHandler.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/rs/zerolog@v1.19.0/hlog/hlog.go:104
github.com/rs/zerolog/hlog.UserAgentHandler.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/rs/zerolog@v1.19.0/hlog/hlog.go:88
github.com/rs/zerolog/hlog.RemoteAddrHandler.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/restapi/panic_middleware.go:40
orus.io/orus-io/rednerd/restapi.CatchPanics.func1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/builds/orus-io/rednerd/.gopath/pkg/mod/github.com/rs/zerolog@v1.19.0/hlog/hlog.go:30
github.com/rs/zerolog/hlog.NewHandler.func1.1
/usr/local/go/src/net/http/server.go:2041
net/http.HandlerFunc.ServeHTTP
/usr/local/go/src/net/http/server.go:2836
net/http.serverHandler.ServeHTTP
/usr/local/go/src/net/http/server.go:1924
net/http.(*conn).serve