Media coverage of cloud security is so often full of generalizations, that an article about a specific, real-world problem gets by almost unnoticed. One story definitely worth reading this week comes from Network World, where Andreas Antonopoulos of Nemertes Research identifies log management as a missing piece of cloud security.
Say what you will about where log management capabilities really fit on the cloud security shopping list (I’ll agree with Andreas and put it near the top of my own), but its hard to argue with user experience. Here’s why: what Andreas doesn’t mention in his piece is that in this case he isn’t speaking strictly from the perspective of an industry analyst, but as a real world customer affected by this problem. You can’t argue with user pain.
A few months ago I spoke to Andreas about the work we’ve been doing at Alert Logic. As an analyst, he was impressed enough (I think?) with what I had to say about our work in log management and the unique user experience that can only be achieved with grid-enabled compute and storage resources operating at service provider scale, but towards the end of the call surprised me with a question as a potential customer – “why can’t I buy your log management service from Amazon?”.
I explained that we have enabled a large number of hosting and network providers to deliver log management services, but Amazon remains a black box to us and anyone else who believes that some security functions in cloud environments should be tightly coupled to service provider infrastructure. Yes, Amazon does a great job of enabling great companies like enStratus to fill the gaps in Amazon services through their APIs, but if you want to build embedded, network level solutions, you’re out in the cold. I started to mumble something about virtual appliances (I have trouble hiding my disdain for virtual appliances), but Andreas stopped me dead in my tracks. “I run my business on Amazon infrastructure and the last thing I want is yet another virtual machine to manage. Why can’t Amazon just give me an Elastic Log Management Service, allow me to deliver logs securely to a collector and scale it as transparently as they do everything else?”.
There is, of course, no good answer to this question. Log management is an essential capability and there is no reason why it should require the complexity and weight of traditional enterprise software, even if that complexity gets packaged in a virtual appliance. If Amazon is to seriously compete for enterprise IT budgets, they should deliver security services as native AWS functions as easily provisioned and scaled as their bread and butter services: compute and storage. Anything short of this fails to deliver the full promise of cloud computing, which leaves Amazon and other public cloud providers who faithfully emulate their model vulnerable to more nimble competitors who respond to enterprise customer concerns faster and at much greater level of depth.
Feel like arguing this point? Look me up on Twitter at @mgbits.
{ 6 comments… read them below or add one }
I don't think it has to be Amazon that is delivering the log management service. I think what Andreas needs (and, please correct me if I am wrong, Andreas), is a log management service that runs in the cloud and is easily integrated with the instances running in the cloud. That is exactly what you get from Loggly. We offer a logging as a service (LaaS) platform that can offers a very hands-off and simple approach to manage log files, just like it is simple and easy to manage your elastic instances on AWS.
Raffy – good to hear from you. I am looking forward to hearing more about Loggly when it’s available, but a couple of questions for you in context of your reply. How do you plan to 1) authenticate and secure syslog transport from AWS to Loggly and 2) provide agent-less Windows log collection?
If your answers include customers having to install certificates and agents on their cloud instances, I’d argue this isn’t exactly the definition of “very hands-off and simple”.
I agree with you that the log management service does not have to come from Amazon and at some point Amazon has to make it easier for service providers to embed their services in their infrastructure stack. This is exactly what’s missing from Amazon’s approach today. True simplicity and smallest possible footprint requires richer private networking options than available today and tighter coupling with Amazon infrastructure.
Well, it depends.
If we think that "logging is part of infrastructure" than Misha (and Andreas) are right.
If we think that "logging is layer on top of infrastructure" than Raffy is right.
Anton – don’t be such a consultant. We are in desperate need of guidance from a log management luminary, but I’d even settle for a security warrior in this case. Where do you stand on this?
He-he, thanks for calling my bluff on this
You are right of course and I need to write a long thoughtful blog post on this.
Here is what I think: we (you, Raffy, me, etc) might think that logging should be part of the infrastructure. In reality, I doubt that this will happen. Why?
Silly question first: is "cloud" more like an OS or more like an outsourcing arrangement? OS (Windows, Unix) have logging but you are on your own in regards to analysis, management, etc. If I "deploy on top of the cloud" (so to say), I can expect the same – i.e. logging APIs, etc. In some cases (real story!), a cloud provider will dump your logs in a web dir and "voila! cloud logging!"
If, on the other hand, you "outsource to the cloud", then you expect some logging service and not having it would mean tha you are not given what you paid for.
IMHO, cloud is MORE like OS than like outsourcing. Thus, no logging service from Google, Amazon, Azure, etc… we have to built it on top of what they have.
I would prefer not to use the phrase “outsource to the cloud” ; IMO, outsourcing is a high touch deployment model where as Cloud is all about automation, standardization and simplification with low touch (via API) integration model. Perhaps it should be phrased as when logging should be expected to be ‘bolt on’ and when it should be ‘baked in’ ? The answer lies in the cloud deployment models (SaaS, PaaS, IaaS) and the inherent abstraction model that encapsulates the capabilities and delegates the responsibilities based on the deployment model . With IaaS clouds where abstraction occurs at lower level (Anton’s reference to Cloud more like an OS), logging and other event management responsibilities are delegated to the customer. As you go up the stack into PaaS and SaaS, we “should” expect Logging-as-a-Service (LaaS) to be built into the platform and exposed via services.
Although AWS has its roots on IaaS deployment models, they are slowly moving into PaaS model driven by customer needs. Today, developers can choose to leverage NoSQL services such as SimpleDB to store logs but it does require considerable amount of plumbing work especially when transport and log storage integrity and confidentiality need to be maintained by the customer. This is where probably Loggly type services fit in (haven’t checked out your service Raffy on the security aspects)
OTOH, Azure which falls under PaaS model should be building LaaS into their fabric and offer APIs that support various functions including exporting logs out of cloud and to perform ETL functions for various business reasons.
Eventually, everything should be a service in cloud and log service should is no exception. I believe it is matter of time before the pieces of puzzle fall in place
{ 1 trackback }