AWS recently released it’s barely new CloudWatch feature for log analysis in eu-west-1 and us-west-1. It was started in us-east-1 first and got a rollout over to ireland yesterday. After installation and configuration of a small daemon called awslogs, instance logs appear in the Webinterface quite fast. They can get divided into log-groups and streams. The interface is quite basic and only allows simple listing of logentries with very basic filtering.
The really great stuff is defining filter patterns feeding CloudWatch Metrics. Implementing an alarm for a high rate of 5xx errors is absolutely easy this way. I’ve heard of all this being done by AWS Kinesis and hoped to get some logs out of CloudWatch by defining my own Kinesis consumers but log-streams aren’t displayed as streams in the Kinesis console. What a pity. All in all it feels very nice but seems to be an absolute MVP solution so far.
What’s missing for a great log-aggregation tool:
– real Log searchability like provided by Elasticsearch/Kibana
– definable outputs streams for logs
– a solution to get AWS services logs into it (ELB, RDS, …)
– customizable log patterns (only space-delimiters are acceptable right now)
I really hope for amazon to add some features for CloudWatch Logs. This could be a really big deal for aggregated log management in AWS.