The Cost of Adopting Hadoop

Apache Hadoop is a popular open source software framework for developing distributed computing applications. It offers a real potential for creating a new business edge, and companies from many domains are starting to integrate it into their existing infrastructures.

However, because Hadoop has strong roots in the open source community, there is a general perception that the only cost associated with the framework is the time to learn how to use it. While this assumption is true for just getting started with Hadoop, the question about cost becomes more complex as you get further into implementing the software into a product.

During my recent consultations with product development companies around incorporating Hadoop as part of their product offerings, I noticed that the majority of the discussion revolved around workforce costs, support costs, integration overheads, etc. Although Hadoop is a free software framework, there are various “hidden costs” behind adopting it. Below is a brief checklist to help you identify if integrating Hadoop in a sustainable product offering is really cost-effective:

• Cost of Search: A significant cost is incurred during the upfront evaluation study and proof-of-concept implementation for Hadoop. These tasks need to be done by expert resources (which can be costly) since you are basing your decision of whether or not to use Hadoop based on the results.

• Cost of Acquisition: Although Hadoop itself is free, most businesses prefer to partner with a Hadoop-focused development partner to integrate the software into their existing solution. Having a partnership with a Hadoop vendor also ensures that if there is an issue with the software, you don’t have to wait for the Hadoop open source community to come up with a fix (which could take months or even years). There are also associated support and customization costs for specific business requirements, along with the cost of integrating Hadoop with the company’s existing platform.

• Cost of Workforce: Regardless of whether you develop the solution in-house or outsource it, you will need to identify people with the right skill sets at a reasonable cost. You must also account for the costs of training your team, migrating your existing application and users, and developing processes and best practices.

• Cost of Maintenance: In addition to general product support costs, companies must also continuously upgrade and maintain their product in order to keep up with the many disruptive innovations in the market.

So instead of looking at Hadoop as just free software, you should view it as a technology investment, weighing the immediate costs against the long-term benefits for overcoming the limitations imposed by your existing infrastructure.

About the Author

Navneet Kumar is a Technical Architect at GlobalLogic with over eight years of experience. He specializes in architecting and building custom Big Data Hadoop solutions for data ingestion, transformation, and analysis.