LogQL: Common Log Format Support for Server Logs

[jeschkies]

Is your feature request related to a problem? Please describe.
Most servers such as Apache HTTP Server and Nginx use the Common Log Format. This format is not easy to parse with regular expression.

Describe the solution you'd like
It would be nice to have a cmn_log_fmt or server_log parser along side logfmt and json to extract all values from a server log.

Describe alternatives you've considered
As mentioned in the problem statement regular expression can parse the server log. An alternative solution would be re-usability of parsers.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[RichardWright]

As someone who's currently searching for a regex to do this for loki, I do feel this should be built in.

[n2o]

Same here, I think this should be a built-in :-)

[agebhar1]

FWIW - a Go(lang) RE2 for the Common Log Format is:

^(?P<remote_host>\\S+) (?P<user_identifier>\\S+) (?P<user>\\S+) \\[(?P<ts>[^\\]]+)\\] \\"(?P<request>[^\\"]+)\\" (?P<status>\\d+) (?P<bytes_sent>\\d+|-)$

and with extracted request line:

^(?P<remote_host>\\S+) (?P<user_identifier>\\S+) (?P<user>\\S+) \\[(?P<ts>[^\\]]+)\\] \\"(?P<method>\\S+) (?P<path>\\S+) HTTP/(?P<http_version>\\d+\\.\\d+)\\" (?P<status>\\d+) (?P<bytes_sent>\\d+|-)$

[owen-d]

This is a reasonable request, but not one we intend to prioritize highly.

[colmaengus]

Is there planned support for custom named parsers? This would allow users to use custom regexp without having to copy and paste it into each query.

[cyriltovena]

I'm working on this

[dannykopping]

Satisfied by the new Pattern Parser: https://grafana.com/docs/loki/next/logql/#pattern

[timwsuqld]

@dannykopping Did the pattern parser get released? The link no longer references it, and it was listed as beta looking at the link in the way back machine

Edit: Looks like it was released, found the correct link for details to be https://grafana.com/docs/loki/latest/logql/log_queries/#pattern