A quick first look at AWS VPC Flow Logs
I woke up this morning to yet another new AWS feature, VPC Flow Logs, as described by Jeff Barr.
Jeff did a great job of providing an overview so make sure you read that before continuing.
Its really interesting to think what you can do with network flows logs. A lot of Enterprise customers ask for this so they can perform various security activities. Many of those security activities are really not needed in the new world of Cloud. However there are some valid ones that you may want to consider. There are also some good reasons to have flows available so you can perform some troubleshooting of your Security Groups or NACLs.
I suggest people turn them on, capture the data and set a retention period on the destination Cloud Watch Log Group, say 3 days up to 6 months. The data is then there if you need it. Just like Cloud Trail data. Its to late after the fact!
A great little use case would be some general visualization of network flows on a dashboard. Its not real time but its going to give you a general indication. You could analyze the amount of traffic by category, such as incoming, outgoing, cross AZ and within AZ (by reverse engineering the subnet ranges). You could even track it down to traffic to AWS regional based services such as S3. You may want to track these patterns over time, looking for trends. You could also look at top talker hosts internally or externally. I suspect it will be of interest to people at first, and then it will be a colorful screen to show visitors. After all, AWS handles all that heavy lifting of operating and scaling the networking.
Many will be interested in monitoring rejected traffic and if they see a lot if it starting, wonder if there is something else going on they should look at or take precessions on. Generally you probably don't care, nothing to see here, its just dropped traffic.
Be great to see what AWS Partners do in the visualization space, I sense some eye candy coming.
I quickly turned VPC Flow Logs on in my account this morning.
Here is my Cloud Watch console showing the Log Groups.
Notice I have set the expire at 6 months. You can see below that when I look at my Log Group each of my Elastic Network Interfaces (ENIs) is shown.
If someone asks you which hosts are communicating with the database at the moment you can quickly jump into the console and answer it by look at traffic to the right port.
The other nice thing you can do is create a metric on this filter to pull out the data. Here is one that creates a metic on the number of bytes accepted as SSH traffic into the ENI.