Chronicles of a Wandering Mind

So Vmware is acquiring SpringSource. Like most people, my first reaction was say what? I didn’t get it; what’s VMWare got to do with SpringSource and Java ?
I read the blog posts by Steve Herrod and Rod Johnson. They were explaining their vision at a high level as expected, and it did not help much. I was still drawing a blank. I hit twitter, read some immediate reactions and subsequent blog posts, hoping they do the hard work and spell it out for me. No such luck. Comments were so diagonally opposed to one another that I’ve concluded it wasn’t just me who was confused. @daveofdoom thought Hyperic was a key tech, Reuven Cohen concluded that, Hyperic was unneeded by VMWare and would be shut down. Confused yet?

A sneaking suspicion has come over me: We don’t understand what these guys are really up to just yet, and they are up to something groundbreaking, that may shape the future.

No help from usually insightful sources, (amazing how easily I got used to readily available quality analysis for free!) I had to do some thinking and speculation myself. I reread the announcements and started thinking.

First, why am I interested? We use many of the technologies provided by these companies in some form. iFountain is a java shop, and use VMWare for development and testing. Our product RapidInsight is built on Grails and heavily uses Groovy, and has integration with Hyperic (one of the best server/application management solutions in the market, open source or otherwise.)

What does this acquisition mean to us?
1. VMWare Springsource Java stack combination.
As mentioned in Steve Herrod’s post, it no longer makes sense for the VMs to be ignorant of the applications running on them. VMs and the applications need to be introduced to each other. If they become good friends, and start talking and sharing, all kinds of cool things can happen.
In my experience, actual hardware and operating system has lost its importance a while ago. If you develop pure java applications, JVM becomes your environment. Why do we need another layer of abstraction, especially if it is virtual? To run a java application like RapidInsight in the cloud (EC2, etc.) or on VMWare hosted image, one has to create an OS image, configure it, make it secure, maintain/administer it, etc. just to host the JVM so that java app can run.
What if instead, VMWare could run JVMs natively just like it’s running linux or windows?

JVM running natively without an OS would presumably have a lot smaller footprint, and save resources. However, more substantial savings would probably come from eliminating administrative overhead of managing the OS.
From an ISV standpoint, this combination may enable ISVs to deploy applications in VMWare JVM ready format, to be installed and running within minutes.

2. Instrumentation of the applications and VMware
Hyperic is different than other solutions in the market it’s competing. It has an agent that is installed on the host and can gather in depth information that is essential for the “conversation” between the applications and VMware.
Using Hyperic, applications can be instrumented, giving in-depth information on how the application is performing, etc. Imagine VMWare assigning another CPU and more memory to the JVM on the fly automatically based on various performance parameters! (afaik neither is currently possible without a restart of the JVM). Now that would be something!

3. Development tools – cloud integration
We use various VMWare images for development. Having an IDE that is integrated with the cloud (VMWare) resources and enable running apps directly on the cloud by push of a button from IDE sounds good to me. If VMWare/Springsource continues to invest in having great support for Groovy and Grails in Eclipse, and integrate it with the cloud, it will be a great combination.

In short, I can see VMWare becoming THE platform for us to develop our products. All of a sudden, this is not such a crazy acquisition. Hats off to everyone involved, there is some great potential here!

From traditional IT management perspective, things are a bit blurry. Springsource acquisition of Hyperic hinted that Hyperic may become more of a specialized management vendor, focusing on java application management. With the acquisition of Springsource by VMWare, this has become a stronger possibility.

With this acquisition, Hyperic (product line) is moving into dominating the future of IT management with a vertically integrated solution. Does VMWare intend to compete in the traditional IT management market with the likes of Zenoss, Groundworks, Tivoli, HP, BMC, etc. as well, or will they gradually abandon this market? What is the future of Hyperic product line, if any, apart from being a key enabling technology for VMWare’s cloud platform?

A while ago Doug asked whether anyone knew a utility to read XML and send SNMP traps. I mentioned that it would be easy to do with Groovy but did not have time to give any details. I’ve been meaning to write a post about how we use Groovy in RapidInsight and thought this would be a good excercise.

First I have state once again that I’m not a developer nor play one on Youtube. I can however, put together scripts, especially if there are examples, but writing java code is not my cup of tea. Groovy makes this stuff easy enough to deal with for people like me (system integrators, admins, etc.)

In this example, we’ll read the RSS feed (which is XML) from Doug’s blog and take action if the feed title includes the word BSM in it Groovy includes powerful XML utility called xmlslurper that makes reading XML a breeze.

There it is, we already  have a Groovy script to retrieve the RSS feed, iterate through the posts and determine whether the title includes BSM keyword. One can take a look at how xmlSlurper works and use it to get the information she needs.

RapidInsight makes it even easier for the script developers to work with the external interfaces without having to learn the intricacies of each interface. In RapidInsight, we can create a class called say RssFeed, store data related to the RSS feed as object properties.Here is how our class may look like:

The class can be used to store the information related to the Rss feed as objects. We can create a UI for the user to enter this information, or simply use a script to create the instances. Creating an object in RapidInsight repository is straight forward, pass the properties as name value pairs to add operation:

By storing the url, etc. as object properties, we can now have a more generic script that would work with any RSS feed. Next we can add “operations” to this class to deal with the Rss feeds, hiding the complexities of the external interface from the script developer. An operation can read the feed and return name value pairs in a map to the user.

This means the script developer does not have to understand the structure of the RSS or how to use XmlSlurper. The operation code can be modified to add support for different RSS formats (v1, v2), use ROME library to parse RSS instead of XmlSlurper, etc., scripts would continue to work as before and script developers like me would not have to learn something different. All it matters is what is passed to us, which is a List of Map (name value pairs), and this is same for all external interfaces.  (Take a look a this post to see more on how RapidInsight uses name value pairs when working with external interfaces)

The operation may be something like this:

Now that we have removed both the data (like url) and the external interface interaction out, our script can be simpler:

Or we can extend it to do the same for each RssFeed object we may have defined:

Let’s move to the second part of the requirement: taking an action like sending an SNMP trap. Groovy can use any java library and it seems like there is an open source java library for pretty much anything you can think of!
We use SNMP4J in RapidInsight. The java library can be used directly in the groovy script, however SNMP is more complicated than RSS, hence figuring out how to use the library is difficult for a non java developer like myself. Here again, RapidInsight helps by providing infrastructure to make it easier to work with SNMP and hide the complexity even further.

SnmpTrap class can be used to store common data such where (ip/port) to send the traps, which snmp version, etc. SnmpTrap class has an operation called send() that takes name value pairs and sends the trap.  Usage is very easy:

If I only have a few different kind of SNMP traps to send, like up, down etc., I can move the trap definition into the operations to make the script itself even simpler.

The functionality described in this post is not difficult to implement in perl etc. for a good developer; there are libraries available and the whole thing can be implemented in a single script.  RapidInsight provides functionality at every step to make not only implementation but maintenance and operations support easier.  Scripts can be executed from a web based UI, scheduled to run periodically, only available to subset of authorized users, logging, notifications, etc.  instead of collection of scripts running as cron jobs that can get out of control rapidly.

I have been having problems with the Dell Inspiron 1505 laptop for a while. Hard drive has been giving signals of a failure, and OS had become so cluttered with numerous applications I have tried last 2 years. Inexplicable hard drive trashing despite 2GB RAM was really getting to nerves and slowing me down.

A reinstall of OS and essential apps would have solved some of the problems but the initial install was painful due to number of drivers that had to be added etc. The laptop comes with loads of junk/trial software and a clean install was unexpectedly difficult. I couldn’t risk any downtime during reinstall so I kept postponing it.

At the end, I ordered a new, faster (7200rpm) hard drive and decided to try Windows7 beta with it, fully expecting to revert back to XP. Happy to say Windows 7 exceeded expectations and was much better at recognizing all the hardware. In addition, with XP I had to use VMWare images for number of applications that required Windows Server 2003, since they did not work with XP. With Windows 7, they Windows 2003 compatibility mode solved this problem.

My definition of a good OS have somewhat changed throught the years. I want an OS that handles the hardware seamlessly and won’t do much more than that. Windows 7 does handle the hardware nicely. The search interface to launch apps is a winner! within minutes I got used to doing everything from going into folders to running apps without touching the mouse using the search box.  It’s really well done.

Security is a pain in the ass! It’s going to take me a while to learn how to disable various automatic stuff, firewall configs, etc. I can’t imagine a non-technical user dealing with this stuff. It’s crazy complex. I can’t imagine helping someone over the phone, etc. to get software installed. There are so many things where things can go wrong in the name of security.  I am an administrator and I’m still getting “you don’t have sufficient access rights” messages. WTF? If I don’t have the rights, who does, NSA?

When I started installing the applications I use, I’ve noticed that majority of them are free applications, if not open source. I have mixed feelings about this. Does this mean that desktop software market has essentially disappeared and not possible to charge for software anymore? Apparently I’m not paying for them. The least I can do is to list and thank the creators. Here is the list in no specific order:

• Firefox Despite continuous annoyance with memory usage, browser of choice. I use it with delicious, firebug, greasemonkey, read it later, add-ons. Eliminated a lot of add-ons, some of them will likely make it back in time as I need them. Haven’t decided what to use for tabs yet for example.
• Sun JDK So easy to take it granted. Java VM is such an invaluable technology. Thanks Sun!
• OpenOffice Encore, kudos to Sun. It’s amazing that we can get this for free.
• Skype I live in Skype, use it 24/7 both in professional and personal life. Who would have thought that we can have video conferencing for free.
• Putty much more than an SSH client. What would I do without tunneling getting me though security hoops?
  
• VLC Media Player It just works, doesn’t install bunch of crap on your PC.
• WinSCP scp and ftp. Like putty, it’s an essential tool I would be hard pressed without it.
• Digsby my current/recent multi IM/social networking tool.
  
• Crimson Editor My text editor for quite some time. works very well. thanks!
• JungleDisk I’m a paying customer using it to backup files into Amazon S3.
  
• Mozy Backup solution for my PC, works well enough.
• Synergy What a gem! Use it to control other PCs in the office from a single keyboard/mouse.
• DynDns another free service/software, essential to work with others when you don’t have a static IP.
  
• iReasoning Mib Browser these guys have great SNMP libraries. They make the mib browser available as a free tool. Works well.
• Groovy my favority scripting language
• Active Perl Perl is still must have in my work. Active Perl distribution made installing Perl on Windows a pleasant experience.
• Picasa I still prefer desktop photo organizers. Picasa from Google is pleasant to use.
• RealVNC VNC is part of the essential tool in the arsenal of a remote worker.
• 7zip works with different compression file formats nicely. another essential tool.
• Adobe AIR, Acrobat Reader & Flash Player Adobe is yet another great tech company providing number of great technologies over the years that have become defacto must have tools.
• Jing Project After using the free version for a while, I’ve become a paying customer when they offered the Pro version. Use it all the time for screenshots and screencasts.
• VMware Player. Another essential tool for development, support, testing, etc. In addition, it has become a key enabler for me to create clean environments to connect to different client networks.

So there it is.  Overall, it feels like I got a new laptop since everything changed and works faster. Now I need to figure out how I can save this state of the laptop so that I can come back to this clean state if necessary.  Hoping that Retrospect, the backup software that came with the external USB drive will help with that.

Have you ever spend hours/days jumping through the hoops,  painstakingly collecting data on a problem and find out that the others had the problem and done all this before? If only you could have had access to this hugely valuable information base, you’d save hours/days and minimize down time?

Quality of support has long been used against open source software, suggesting that enterprises need an entity to deal with that is accountable to them. Fair enough, even though the mailing lists can be more responsive than some support organizations, enterprises require more assurance. This issue is largely resolved since there are now open source companies providing support for most open source products. The fact that this is still being tossed around as a concern is annoying to say the least but c’est la vie.

What irritates me more is that how inferior proprietary vendor support systems can be due to specifically their closed nature. Most bugs/problems are experienced by multiple users/customers and problems that are difficult to isolate can take a lot of time and effort.

Support information (generated by customers) is mostly locked inside enterprise systems, and not visible/searchable by the public or even the customers. Correlation of the problems reported by the users, their troubleshooting journey, the problems encountered on the way, gotchas, etc. all stay hidden from rest of the people.

In my experience, most people prefer using self help tools and research themselves before calling support unless there is an emergency.  If the customers could search/access to this information, not only the customers would save time and may be reduce down time, but the vendor would also benefit from reduced support costs since fewer support calls would come in. Even further, customers may even be able to avoid problems altogether by learning from experiences of others.  So why is this information not accessible, available at least by the customers?

• tools may be inadequate. The support systems may not naturally facilitate this type of sharing and collaboration
• vendors may not want this information out in the open for “competitive reasons”, keep dirty laundry hidden so to speak
• support information may include sensitive information that customer may not want to be public
• others.. ?

Open source products by their nature are more transparent, the problems are reported and discussed out in the open in mailing lists, bug tracking systems, forums, etc.  hence Google is ready to help you find the answers.  Vendors -open source or otherwise- should make the effort to make the support process as transparent as possible, the cost of keeping this information locked inside isolated systems is just too high. Open source companies have an opportunity here to differentitate themselves in yet another way that directly impacts the customer experience.

I recently had an interesting discussion with a colleague on pros and cons of proprietary vs open source software and access to code vs APIs. We’ve done a lot of work on integrating our products with proprietary products in the past and recently have been working on integrating RapidInsight with open source products like OpenNMS and Hyperic, and have been comparing pros and cons of both worlds.

Integration in proprietary world is painful experience in most cases. It is hard to get a hold of the software, often not even possible. There are all kinds of restrictions on what you can and cannot do with the software that prevents high quality integration, contracts need to be signed, and small companies like us are often ignored.
It does not need to be this way. Proprietary software can have open APIs and vendors can establish a process that can be used by anyone, much like it’s been done by the web companies (Amazon, Google, etc.), but it’s rarely done this way.

8 years ago, working for a consulting company, I had given a presentation in the company about the developments in the web world, web services APIs, its impact on IT management, and systems integration. The company had significant revenue stream implementing IT management solutions by integration a set of IT management software products, usual suspects like Netcool, Remedy, Smarts, Nervecenter, Infovista, Patrol etc. and most of us were specialists on one or more of these products. I had suggested that with the spread of common/standard integration APIs (SOAP, XMLRPC), nature of integration projects would shift from point to point integration into use of the standard APIs, and generic enterprise integration tools. If so, the demand for the skillsets would also shift, requiring more software development skills than systems integration skills, in depth expertise and understanding of inner workings of the IT management products.

Needless to say, it’s an understatement to say that I was wrong! In the last 8 years, very little has changed in the IT management world. Ad-hoc, point to point integration is still the mainstream, and often the only option. Consultants do just fine with the same skills they had a decade ago and IT management projects still have very high costs, implementation (integration) costs often well exceeding the licensing costs. I’ve seen may RFP, product comparision matrixes, etc. over the years and support for standard APIs is often just a check box. Massive cost impact of not having standard APIs are mostly missed, and as long as there is no strong demand from the customers, vendors will not move.

Some of the proprietary software products do have APIs and if you can get access to the software (for us typically on a client site who wants us to integrate with) you can use the API to develop descent integration. But there are still risks for the customer if the API is not available publicly available. If the API is not embraced and widely used, the vendor has no qualms about changing it liberally from version to version, so upgrades become a much riskier proposition. So how widely the API is used is crucial for the customers. When an API reaches high level of use, organizations become more conservative in changing them and think twice before breaking more compatibility.

In the open source world, it is assumed that integration is easier since the code is available. And it is indeed easier. To start with, you don’t have to deal with the commercial/legal contract nonsense to implement the the integration. I can tell you this phase takes longer and is more frustrating than writing the code.
Since the code is open, you can look at the code, learn from it, modify it if necessary, etc. You don’t necessarily need the help from a company, etc.  We’ve recently integrated RapidInsight with OpenNMS and Hyperic with only using publicly available resources, forums, mailing lists, etc.

Yet source code being open is not a replacement for a well documented API. There is no guarantee that the data structures, methods, etc. will not change from version to version. After all, an API is a software contract, a commitment to the outside world and implies stability, but internal code and data structures do not. So integrating open source products without standard APIs is carrying some of the same risks of integrating proprietary products.

The good news is that since open source products are developed transparently, one can get a heads up, adjust the integration for the new version, provide feedback during development to ensure backwards compatibility if it is possible, and even better participate in development of the standard APIs.

Rapidinsight OpenNMS integration gets data directly from the underlying postgres database and uses java methods for graphs. Fortunately, OpenNMS has a large community and evolves slowly, hence integration should be safe for some time. In the long term, implementing an API on the OpenNMS side to expose the information would be the better solution. If OpenNMS community sees value in this integration, we would certainly look at contributing at that end.

For Hyperic integration, we’ve used some groovy classes (we love groovy! ), implemented a Hyperic plugin to expose the information we need, and used some UI methods to access to the graphs. We’ve already noticed that something has changed with v4.0 and we’ll need to adjust the integration accordingly to make the integration work with the new version. The good news is they seem to be working on implementing a web services API which would make our lives so much easier!

Wouldn’t it be great if there was a site like programmable web for IT management software

I’ve been meaning to write this post for a while. Christmas time seems to be appropriate for the sentiment. I’d like to say/write thank you to the folks at ReadWriteWeb. The web is evolving incredibly fast, blogs have become a phenomena and according to some it’s already being replaced with microblogging.

Web offers many choices to people and to me RWW model offers the best return on my (time) investment. What makes RWW different?

Content with substance.
You can tell that it’s not just blurted out. I was surprised to read that the blogs play “breaking the news” game like the traditional media.  Erick Schonfeld had written:

“There is not much time for story-telling (except for weekend posts like this one). It is mostly breaking news, reporting facts and providing analysis.”

Breaking news is clearly important to a lot of people. Techcrunch provides is very useful to many, as it has millions of readers (including myself). However from my perspective, “breaking blog post” is not as valuable as well throughout, researched one. It won’t kill me if I read about a hot new startup a day late. I rather read about it with supporting information that provides context. That requires research and takes time.
Many RWW posts are like references, often contain in-depth analysis, compare & contrast different players, philosophies etc. I can imagine that it takes time to write them. But as a result, they don’t become stale as fast as”hot news” posts. I wonder how much traffic RWW gets, through google hits for older posts. I suspect it is significant.

Comments rock
I prefer reading blogs like RWW, Techcrunch, A VC a day later. Significant part of the value provided by these blogs is the comments added by the readers. Reading the posts later give me a chance to read the comments on the posts, which are as valuable(if not more) as the post itself.

I suspect the next innovation for the conversations on the web will come in the comments space. I think we desparately need better tools to manage comments (for the readers). An obvious idea is digg like system for comments, to get good comments bubble up. Amazon book reviews may be a model for this.

Practicioners beat reporters any day
RWW has number of contributors who are not primarily reporters but practitioners in the field. Personally, I think hands-on experiences of someone in the field is many times more valuable than a (no offense) reporter. It helps that RWW writers are using heavily the cutting edge web tools themselves and share their experiences.
When the subject steers outside the expertise of the regular contributers, it is best to get a guest writer to contribute rather than fumble a shallow opinion based post that does not add any value.I hope RWW will continue to have subject matter experts contribute. It is not easy to find these people and blogs like RWW play a critical role in bringing talented people to daylight for all to enjoy.

A new year resolution for me: I have to get better at being concise. In summary, a big thanks to RWW. Hope their business rewards them sufficiently for the excellent service they provide.

Build vs buy has long been a critical decision point for large organizations. Service providers in particular have often chose to build in house solution to manage the infrastructure for variety of reasons:

• package solutions were not able to cope with the size (scale and performance)
• unique requirements that not met by packaged solutions
• maintaining control over how the solution evolves
• strategic differentiation from competitors

Obvious disadvantages of the in-house build solutions is the potential high cost of ongoing develoment and maintenance, dependency on small number of people, etc. It is understandable that developing every solution in-house for a single organization is a costly proposition regardless of how big the organization is.

Like it or not, most service providers have been driving towards using packaged solutions whenever there is a viable solution in the market. Problem with the packaged solutions is the proprietary nature of them. Hard to find skill sets in proprietary products, lack of innovation by the vendors, feature set not meeting the requirements, scalability challenges to meet the needs of large organizations, and integration hurdles significantly diminish the perceived cost savings and strategic value of these solutions.

Open source solutions offer an alternative that may be best of both worlds. Organizations can can collaborate in developing solutions that meet their common requirements, giving organizations the opportunity to share the cost of development and still have control over the solution, etc. Open nature of the open source solutions make integration much easier as well.

Yet there are no significant open source projects in IT management driven by these large organizations. Most open source solutions is product of companies or handful of individuals, and typically target the SMB market. Service providers invest massive resources for in-house development, often have multiple teams with dozens of developers. Moving the development of these solutions into open source projects would offer significant benefit to these organizations. I’ve seen projects in “maintenance mode”, meaning none of the original developers are around anymore and no one knows how the solution works, and no enhancement is possible.

There is clearly a failure here, inability to collaborate, corporate barriers to share, etc. Whatever it is, it’s preventing these organizations to take advantage of what open source development model would have to offer.

So what does all this have to do with the clouds? There are significant parallels between the needs of cloud providers and traditional service providers in terms of IT management solutions. Scale, multi-tenancy, strategic role of the tools, need for flexibility, open interfaces etc. mean that it is not likely that cloud providers can use packaged solutions offered by Big 4 and others.

The good news is that these companies have open source in their DNA, and already use it heavily, typically using open source components to build internal solutions.They have not been so far collaborating to build solutions specific to clouds as open source, but it is more likely to happen than traditional service providers.

It is already safe to say open source software is the de facto choice in the public clouds. What remains to be seen is whether this meme will transfer through the firewalls into the private clouds. My guess is that it will. Simply because open source tools will be ahead in meeting the management needs in the cloud from scalability to integration. Thought leadership is important and open source tools like Hyperic is paving the way for the open source tools to play a leading role in mangement in the cloud. Will the cloud providers play a more active role in development of open source tools for the management in the clouds? It would serve them to learn from the troubles faced by traditional service providers.

Image via Wikipedia

If you’ve never read Seth Godin’s blog (or books), you should. It’s full of gems. One of his recent postings was on a subject that is very relevant for us, hence the title of this post is borrowed from his.

“That’s the choice most of us make when we launch a product or service. We can make a market or we can take share from a market.”

Software products are particularly hard to describe by their nature and it does not help that IT terminology is polluted. Even commonly used terms like network management, event management, etc. mean very different things to different people. As a result, we often face using existing products to describe new ones:

“This is just like the Gillette razor, but cheaper.”

“It is just like HP OpenView, but better/cheaper/works…”

As Seth indicates, this strategy takes advantage of the recognition established product has and makes it easier for the potential customer/user to compare & contrast.

I think this explains why HP OV (NNM) is still the yardstick new monitoring tool vendors use to compare/define their products even though HP OV has been stagnant for over a decade and not even close to be the best tool from a technical perspective.

RapidInsight is different than the alternatives in the IT management market. It is meant to be. We’ve designed it with the hindsight of having lived with the the shortcomings of traditional solutions currently in the market. But articulating this difference in the market is challenging.

So far, RapidInsight customers have been direct contacts. We knew the customers’ well, hence we could articulate what RapidInsight is, why it is better than alternatives, etc. within the context of the needs of these customers. But when you go to the broader market you don’t have the opportunity to talk to each potential customer at length, if at all. Direct access to customers is where smaller companies run into difficulties. We need a way to articulate what RapidInsight is, and why it is worth fir anyone to invest their time to learn more about it.

So we face with the choice Seth states eloquently in his post. Should we try to take market share by identifying RapidInsight with existing solution or should we try to make the market? The answer should not be that difficult. We’re way too small to make a market. We can easily say “RapidInsight is like Netcool, only better . An open source IT event management solution” It’s probably better marketing position for us. From a technical perspective, this is an inadequate description of our product. I want to talk about why focusing only on consolidating events, restricted proprietary languages, storing data in relational databases are flawed approaches and why one needs a built-in CMDB, object based data store, use of standard based dynamic scripting language , etc. Yet I have to accept that “making the market” is out of our reach, however “right” it feels.

So it helps when others join in describing why consolidated event management is not sufficient and we need more It turns out, HP has just announced a new product/release, HP Operations Manager,OMi.

The blog author (shouldn’t a blog have info on who the author is?) describes HP OMi as a “next-generation consolidated event and performance management product“. What make OMi different? It sits on top our HP’s CMDB, which means it has access to service dependency information and have access to availability and performance data. It sounds like folks in HP understand that consolidated event management alone is a positive but insufficient, and why a solution that models and consolidates all IT operations information is needed. That will be great if they can educate the market to why. May be then, we can say RapidInsight is like OMi, but only better

It’s official! If there is anything I suck at, it’s predicting how long it’ll take to develop software. Granted I’m not alone in this flaw, but it’s not even funny how far off I was.

Almost a year ago, we’ve made some major decisions in the company. One of them was the move to open source. Not only developing our software as open source but also replacing existing components with open source ones wherever possible. As you can imagine, this required major (colossal?) changes to existing software. We found excellent open source solutions for web development, data storage, etc. that were similar to our in-house developed solutions, but we had to invest significant time to migrate to these technologies.  I thought it would take us 3 months, doubled the amount to give some room and thought 6 months would be a realistic estimate. Well let’s just say it took longer but we’re there. Today, we’ve released RapidInsight v3, an open source automation, integration and presentation solution for IT operations management.  Although it took about a year of hard work, I think it was worth the effort. It was not easy to shed what we developed specifically to fit our needs with external more generic solutions, but going forward the payoff is already clear.

I’ve written a (too) long post on iFountain blog about the history of IT Event Management (as I see it), the need for a different approach, what drove us to develop RapidInsight and why I think it’s different (and better) than alternatives currently available, open source or otherwise. But I think there is a significant gap in open source management tools in this area.  RapidInsight does not compete directly with any of the open source IT management tools currently available (that I’m aware of).

There are number of open source management tools such as Nagios, OpenNMS, Zenoss, Hyperic, etc., for monitoring, Puppet, ControlTier, ZipTie for configuration/deployment automation, etc.RapidInsight does not directly compete with any of these tools. I think there is significant potential value in integrating RapidInsight with these tools, using RapidInsight built-in CMDB, to integrate IT management information, and to provide  single interface (both for users and programmatically). Potential uses of RapidInsight is listed over here.

We’ve already started working on integrating RapidInsight with open source monitoring tools. Provided that these communities share our belief that there is value in the integration, we will continue improving the quality of the integration.

I’d love to hear your thoughts! Any feedback would be much appreciated. Please feel free to contact me directly or go over to our development site and participate in shaping RapidInsight.

Watch this space for the news on integration with your favorite tools. Better yet, tell us what you’d like to see integrated first!

Tarus has a very interesting post where he reflects on the book predictable irrational and what it may mean for IT management field and the business models. A lot to digest in the post, and I’ll have to read that book for sure.

In the post Tarus says:
“I’m often surprised by the success of open core software, especially in the US. To me it is a bit irrational. Why give up per-node priced proprietary software with the attendant vendor lock-in for cheaper per-node priced proprietary software with vendor lock-in, even if there is an open component? ”

Good question. Why indeed?

Borrowing from Tarus’s food analogy, let’s analyze different options available to people in the IT management field.

(Big 4) proprietary management software vendors are like expensive restaurants. They use loads of resources to convince people that they have a great, luxurious restaurant with exquisite food. They promise that they use the best ingredients, but don’t let you in the kitchen or tell you what the ingredients are. There is no menu with the prices; you have to pay upfront for the meal, will likely have to pay more later and there are no refunds. You may even be asked to sign a paper that says you won’t tell bad things about them to others.

In short, you take a big risk as you have no idea what is in what you’re eating but hey,  most people eat in these places, so how bad can it be?

Smaller proprietary vendors are like more economical restaurants. They are similar to expensive restaurants described above, but more accessible and attentive to your needs since they are smaller. They often (not always) do have a menu and lower prices. As the costs are lower, it is less of a financial risk, but you may instead be worried about their survival or associate cheap with low quality.

Open source projects are like communal cookouts. People invite you to come cook and eat with them. You can not only see the “kitchen” but also get in it and cook yourself. If you’re not a good cook or into cooking, no worries. You can still help out other ways: wash the dishes, clean up etc. Recipes are not secret, but may not be written down clearly either. Nonetheless, you can observe others cook, ask questions (nicely), and learn the recipes.
You don’t have to contribute but no one likes a free loader

Open source services vendors (like OpenNMS group) are cookout organizers. They typically volunteer to do most of the work, organize the cookout, publicize it, help people join in the action etc. They may also offer services for a fee. If you just want to eat, don’t have time, don’t really like cooking yourself, don’t want to bring your own silverware, or don’t like cleaning up, you can pay them to these things for you.

Open core vendors are restaurants that also organize cookouts. Like the open source services vendors, they organize and participate in the cookouts, and do a lot of the heavy lifting. Their cookouts are often more polished as they typically have (human and material) resources dedicated to make the cookouts success.
For them, cookouts are a way to showcase some of their dishes and cultivate a community familiar with their offerings. If some people prefer to have a sit down dinner or eat some of the dishes like “Broiled lobster with crab stuffing, vegetable, rice and mango salsa” that may not be available in the cookouts, they can always go to the restaurant and have a more traditional dining experience.

I may have pushed the analogy little too far, but it does make sense to me

So going back to Tarus’us question: Why do people choose open core solutions?

In my experience, most people don’t care whether the software is open source or open core or even proprietary. I can anticipate that Tarus may not agree with this statement. In contrast to Tarus, majority of the people I meet in IT management field are quite far from the open source world. As a result, I have a different experience.

People I meet often prefer eating in a restaurant, have good service, and willing to pay a fair price for it.  They don’t necessarily care whether they are paying for the meal or the waiters time. Having tried the free dishes and seen the ingredients and how the food is cooked; they have a certain level of trust. In short, they may feel they have the best of both worlds in open core companies.

The difference between the open core and open source models may be vast if you’re inside the open source world, but it’s not significant if you’re outside it. At the end, it comes down to how good the food tastes and how much does it cost to eat it (not just to buy it).