• Terms of endearment – the ElasticSearch Query DSL explained – "all we have is a NoSQL document store which is fast, reliable, scalable & easy to use" @clintongormley not enough? http://t.co/Ah4jr80
• Track external service performance with New Relic | New Relic blog – RT @newrelic: Track external service performance with New Relic http://t.co/SvosXDU
• elasticsearch – videos – Road to a Distributed Search Engine – RT @kimchy: Road to a Distributed Search Engine http://t.co/hqZbTVV via @elasticsearch

• Gary's Blog – RT @gary_read: Systems/Network Mgmt products, scalability and big data: One of the challenges for vendors… http://goo.gl/fb/7t2Jr
• elasticsearch – blog – 0.17.0 Released – RT @kimchy: 0.17.0 Released http://t.co/1aHQZDY via @elasticsearch
• ‪Interaction with a Quadrotor via the Kinect, ETH Zurich‬‏ – YouTube – RT @dylanjfield: Happy Saturday! Here is a flying robot that can be controlled with magic: http://t.co/Eh8rnnZ

• A Decade of APM Acquisitions and Evolution – AppDynamics: The APM Blog – Good review of evolution of APM technology and vendors in the space by AppDynamics folks http://t.co/jXrnqoK
• Should VMware Administrators Care About Applications Performance | The Virtualization Practice – RT @Bernd_Harzog: Should #VMware and #vSphere Administrators Care About Applications Performance? http://bit.ly/m59gl5 Answer = Yes!
• Google – RT @marissamayer: Google logo has live imagery of today's total lunar eclipse. You can play it backward and forward with the slider. h …

• NYC cyclist vs. bike lanes – kamikaze law-abiding – Boing Boing – RT @abbashaiderali: NYC cyclist vs. bike lanes – kamikaze law-abiding http://j.mp/mzxvgQ
• Global Commission on Drug Policy | Report – RT @Werner: GCDP: "End the criminalization, marginalization and stigmatization of people who use drugs but who do no harm to others" htt …
• Why Perfomance Management is easier in public than onpremise clouds Application Performance, Scalability and Architecture – The dynaTrace Blog – relying on resource util to do perf and capacity mgmt? don't! Here is a good explanation by Michael Kopp of Dynatrace http://bit.ly/jgAPVy

• Lockerz.com .:. ostrykrzys's Photos – Never looked at it this way… Did you? #kfc – http://lockerz.com/s/10395192 may be it was raining bagels?
• How we provide real user monitoring: A quick technical review | New Relic blog – New Relic just got a lot more interesting by adding RUM to its arsenal. http://blog.newrelic.com/2011/05/17/how-rum-works/
• The Usability of Passwords (by @baekdal) #tips – someone making sense about password security for a change instead of fear mongering http://www.baekdal.com/tips/password-security-usability

• A Perfect Storm in Availability and Performance Monitoring | The Virtualization Practice – "A Franken Monitor" love it. Another great post by @Bernd_Harzog http://www.virtualizationpractice.com/blog/?p=9588
• Webinar: How to Avoid the Bankrupt Private Cloud – RT @Netuitive: Webinar today @ 11:00 EST: How to Avoid the Bankrupt Private Cloud – http://bit.ly/fNLKA2
• VMware hardware requirements make Hyper-V more appealing – RT @AndiMann: Interesting post by (VMware guy) @ericsiebert on why Hyper-V may be a better option than vSphere – http://bit.ly/hblaLC

• BI Becoming the Key Enabler for IT Performance Management – IT Management – TRAC Research – RT @tracresearch: TRAC's new article: #BI Becoming the Key Enabler for IT Performance Management http://ow.ly/2Sh4g
• ElasticSearch – Searchable CouchDB – ElasticSearch invasion in on the way, @kimchy has been on fire. Check out the rivers for RabbitMQ and CouchDB http://bit.ly/akRyAQ
• The Significance of the VMware Integrien Acquisition | The Virtualization Practice – another must read post from @Bernd_Harzog for IT management folks http://www.virtualizationpractice.com/blog/?p=7186

• berkay | Plixi – As long as it's genuine … http://plixi.com/p/45090793
• ElasticSearch – Geo Location and Search – RT @kimchy: Geo Location and Search http://t.co/iCg5HWB via @elasticsearch
• Fred Wilson Dot VC – RT @mcannonbrookes: "Our doors are open to everyone. Everyone with a dream and a willingness to work hard and play by the…" http://j.m …

• ElasticSearch – ElasticSearch 0.9.0 Released – RT @kimchy: RT @elasticsearch: ElasticSearch 0.9.0 Released http://bit.ly/cXkh07
• squid314: Stuff – RT @abbashaiderali: Hilarious rant about the implausibility of World War II (think if history channel was syfy) http://j.mp/c4fToZ
• Chronicles of a Wandering Mind » Impact of Infrastructure Automation Tools on Monitoring – What does proliferation of infrastructure automation tools like Chef mean for monitoring? http://bit.ly/b6MRyZ

IT management teams in charge of monitoring the IT infrastructure (network, servers, applications, etc.) mostly have little insight into what it is that they monitoring. (obvious or shocking?)

The tools we use clearly indicate this fact. Monitoring tools all have some sort of “discovery” functionality to figure out, what is out there to monitor. More often than not, when we set discovery loose on the network using (SNMP, ICMP, etc.), and it finds out devices and network connections customers did not know that existed.

Server/application monitoring tools start their cycle by scanning ports to see which ones would respond, or by sniffing the traffic to figure out which servers are out there, which applications may be running on which servers, etc.

The process would not be much different if you were attacking the infrastructure to find a way in. (How many of you triggered security alerts when performing discovery?) We’re outsiders. In enterprise environment, we often don’t even know the owners/developers of the applications.

In monitoring field, this has been the norm for so long that it no longer bother us. It should. Of course, monitoring teams & tools don’t do this for fun. It has to be done because in most cases, there is no truth teller; no place to get this kind of information. It is not uncommon for the monitoring tools to feed data they discover to inventory tools etc.

There are efforts like CMDB projects that attempt to create a repository that provides this information to all management tools but these projects often run into organizational as well as technical obstacles, and things are getting harder by the day with the dynamism introduced by virtualization and the cloud technologies.

What if we didn’t have to do all this crap to know what’s what? What if monitoring tools could be told which applications run on which server, where that server is in the network, etc. ?

There is indeed a better way, at least for some use cases. Proliferation of infrastructure automation tools (aka configuration management tools) such as Chef (and the management APIs exposed by VMWare, etc.) have the potential to change not only for how we deploy and maintain servers and applications but also how we monitor them.

Most obvious impact is that using these tools mean monitoring tools can have a reliable source to learn about the infrastructure that should be monitored. What the role of the servers are, how they are configured, which application components run on which server, what the change history is, etc. This is a huge step forward.

When you know how things should be, it’s much easier to detect the exceptions. A significant portion of the problems happen due to changes somewhere in the infrastructure. Ability to automate changes, see the change history and roll back when needed is an invaluable. And being able to correlate the configuration changes with the monitoring data can significantly reduce troubleshooting time and hence improve availability.

Another impact is that a safe framework that enables operations folks to take actions to troubleshoot and resolve problems (combined with run book automation, workflow, wiki, etc.) may finally mean that level 1/2 support folks can do more than record and route without giving them full access to the systems (which is not feasible), reducing number of problems escalated to higher levels and increasing overall productivity.

I should state just for the record that I don’t mean that infrastructure automation tools like Chef introduce brand new technology. Opsware (now HP), BladeLogic (now BMC), ConfigureSoft (now VMWare) for server configuration management, and TrueControl (now HP via Opsware), Voyence (now EMC), AlterPoint for network configuration management have been around for some time. But confluence of factors such as success of the (Apache licensed) open source model of Chef, and increasing acceptance of cloud economics, and patterns such as availability of open APIs move Chef into the center stage. It does not take great wisdom to infer that price point such as $50/month for 20 devices will make Chef very hard to ignore. Price is indeed a feature.

Looking forward to see how infrastructure tools like Chef will evolve as they move further into the enterprise world. Monitoring folks need to pay attention.