Seq Documentation and Support

Welcome to the Seq documentation hub. You'll find comprehensive guides and documentation to help you start working with Seq as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    


Ask A Question

Troubleshooting Durable Log Shipping

We have 3 different azure cloud services, each with 2 vms (so total of 6 servers) that we are using with durable log shipping to log to a Seq server that is hosted on an azure vm. As of 1 day ago, some of the vms within these cloud services just stopped logging. Some other vms continued to work just fine. If I reboot the vm that stopped logging, then it will log fine for a few minutes and then stop again. In addition to that, once the logging stops, there seems to be a slow consumption of memory until the box uses all it's RAM and starts to become unstable - at which point we have to reboot it and restart the cycle. The first thing we've done is upgrade everything to the latest - client side is up to Serilog version 1.5.14 and all of the corresponding "Extras" and "Sinks" are at the most recent version compatible with 1.5.x. Also our Seq server is 2.4.2. We are monitoring to see if this change has any effect on the memory issue. So far, even after the upgrade and reducing the batchPostingLimit to 10, we are still seeing our logging stop after a while (seems to be taking longer than a few minutes now, but eventually it still seems to stop, at least on some servers). I enabled the SelfLog and the error we are seeing is this: 2016-01-26T02:59:41 Exception while emitting periodic batch from Serilog.Sinks.Seq.HttpLogShipper: System.AggregateException: One or more errors occurred. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. --- End of inner exception stack trace --- at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) at Serilog.Sinks.Seq.HttpLogShipper.OnTick() ---> (Inner Exception #0) System.Threading.Tasks.TaskCanceledException: A task was canceled.<--- So, the way I read that error is that the server is too busy to handle it? But that is an unvalidated guess because there is nothing to indicate any sort of problem on the server. Seems to be nothing notable in the Seq server logs. Certainly nothing to indicate that it is failing to handle requests. Also, once this error starts happening, it just keeps happening and the server never starts logging again until it is rebooted or the IIS app is recycled. So that is not very "durable" at all! I'm not quite sure how to figure out what our problem is at this point? It feels like maybe our only option is to stop using durable log shipping, but if we've got a server issue, is that going to help? Need some guidance on how to troubleshoot and understand what the real problem is so we can address it correctly. Thanks

Posted by James 5 years ago