Infrastructure Management: Part #2, mCollective

In the second part of our series on great tools to help with infrastructure management, we are going to talk about mCollective (Marionette-Collective). This project was recently acquired by Puppet Labs and helps to sensibly manage day-to-day system administrator tasks with an RPC framework. With so many people trying to write complex SSH for-loops to handle tasks that should be treated more elegantly, mCollective can offer what a system administrator has been missing: consistency!Pulling Strings with EaseBy using an RPC framework, mCollective stands apart from 'mass-host-execution' programs or scripts. Plugin creation for mCollective is written in Ruby, leveraging base frameworks to handle the interaction between servers and clients. All machines in your infrastructure run a server, but only one or more need to run a client to interact with the machines. In this model, a queue server is utilized to handle messages between the two, in a publisher/subscriber capacity. Commonly this will be accomplished with ActiveMQ but RabbitMQ could also be used.A client can use either an included rpc tool to execute plugins or, more likely, a front-end client for each plugin will be available. Some common examples of what plugins would control are: packages, services, host discovery, process information, files (touch/remove/status), pings, and Facter facts. Additionally, I recently created an mCollective plugin for APT functionality.Administrate BetterSeeing is believing, after all. Here's an example of utilizing mCollective to find only x86 hosts and then providing the status of the 'puppet' package on those systems. You'll notice the progress bar looking for known-hosts to execute this check for:[code][root@puppet puppet]# mc-package -W "architecture=x86" status puppet * [ ============================================================> ] 10 / 10host01.example.com version = puppet-0.25.5-1.el5host02.example.com version = puppet-0.25.5-1.el5host03.example.com version = puppet-0.25.5-1.el5host04.example.com version = puppet-0.25.5-1.el5host05.example.com version = puppet-0.25.5-1.el5host06.example.com version = puppet-0.25.5-1.el5host07.example.com version = puppet-0.25.5-1.el5host08.example.com version = puppet-0.25.5-1.el5host09.example.com version = puppet-0.25.5-1.el5host10.example.com version = puppet-0.25.5-1.el5---- package agent summary ---- Nodes: 10 / 10 Versions: 10 * 0.25.5-1.el5 Elapsed Time: 1.03 s[/code]Not only is the output sane and readable, but you can filter hosts based on their Facter facts as desired to help you execute plugins only on the desired subset of your total host structure. Rather than having that 'all-hosts.txt' file living on your servers, let mCollective do the work by letting you know what hosts are available and the metadata that make them unique or similar to each other.A Use-Case For MostThe larger an infrastructure grows, the more complexity is added to ensure consistency. Puppet can certainly help with your configuration management if configured properly, but even a Puppet manifest is only as good as your insight into how a server should look.Take for example a situation where a module for Puppet intends to keep a certain core package synchronized. Due to competing yum repositories, most servers have a package installed from the correct internal repository, but a few have the package from EPEL. In your Puppet manifest it checks to ensure the package is installed... but not where from! Using mCollective it's trivial to quickly check a package's status and version if installed, and quickly note disparities. The same goes for the other plugins mentioned above and many more.Don't Just Hope for the Best!By utilizing the RPC framework of mCollective, you will know when a command fails, which hosts are available real-time, and be able to orchestrate your systems with greater ease. Through parallelization, mCollective will add efficiency to your tasks and provide you with answers rather than a hung connection. The next time you think "for i in `cat all-hosts.txt`; do ssh $i command; done" perhaps think, "How can I create a reliable plugin to abstract guesswork from my execution of this command and ensure that the proper sub-section of hosts I want this to run on is accurate and up-to-date?"Much like Puppet, mCollective is not just a tool, it's another step in the DevOps direction of effective, reliable, and trustworthy system administration technique!

Looking to automate your infrastructure?

We are infrastructure management experts. Give us a call at 888-877-7118 or click here to request a proposal.