I’ve setup Loggly today, really nice service but I found a few gotchas you need to keep in mind to get it working.
I’m doing this on a docker instance, I’m sure there are other and better ways of doing this (suggestions are very welcome), since I wanted it to be a scripted part of our current platform that’s the road I went down though.
We are using the scaled-down Debian Jessie AspNet vNext container as a base file when we’re deploying. Basically, I’ve copied their dockerfile and built on it. This means there were quite a few packages missing that are required to get Loggly working.
Required packages
Here is a docker run statement to install the required packages.
RUN apt-get -qq update && apt-get -fqqy install telnet sudo rsyslog wget
We need these packages in order to be able to run the configuration command to setup rsyslog to push the logs to the Loggly server. I found that I needed to use the -f flag to get wget installed properly, I can’t really vouch for why. It might have had to do with the pre-installed packages in the docker container.
Configure rolling logs
We are using Serilog and previously had the RollingFile sink configured. If you look at the Loggly guide about logging to standard file. They recommend having a single fixed file for the latest log output. This makes finding it and trailing it an easy task. Then you can configure a service to manage the file rolling instead of relying on the application to do it. We therefor changed the RollingFile to a simple File sink and configured logrotate to manage the files in the container.
This is done quite simply by adding a configuration file with the application name in /etc/logrotate.d/appname. The contents of which could look something like this:
/var/log/appname/*.log {
daily
missingok
rotate 7
compress
delaycompress
}
This would consume any file in the folder and rotate it daily keeping 7 copies.
Running rsyslog
I found that I had to add an extra command to my docker container to run the rsyslog before it kicked off, was simply not started by default:
CMD sudo service rsyslog restart && ... the rest of your command ...
Since the RUN only runs within the context of the image build and doesn’t actually start a process I put it into the CMD instead.
If you find issues with this, do have a look at the loggly troubleshooter for file watching.
Conclusion
That was about it actually, it took me a while to set it up but mostly because I had to rebuild the docker container several times. If you do this on a non-container it should be simple enough to have it up and running in a matter of minutes.
Also, sorry for the broken up format of this post, I ended up writing it a bit as I went along and kept appending/changing things.