The journey started several years ago. Cisco wanted to find new ways of creating and unearthing value from the information scattered across the company and its various technology systems. Not just structured data about customers, products, and network activity, but also unstructured data found in web logs, video, email, documents, and images.
At the time, the big data movement was in its infancy. There were no answers or roadmaps. Only possibilities and hypothetical outcomes.
“We needed to come up with a use case that marries IT opportunity with business opportunity,” says Piyush Bhargava, a Cisco IT distinguished engineer. “At the same time, we wanted the platform to support any number of use cases, so it needed to be broad, horizontal, and enterprise ready.”
Building the platform
To unlock the business intelligence hidden in globally distributed big data, Cisco IT turned to Hadoop, an open-source software framework that supports data-intensive, distributed applications.
“Hadoop behaves like an affordable supercomputing platform,” Bhargava explains. “It moves compute resources to where the data is stored, which mitigates the disk I/O bottleneck and provides almost linear scalability. Hadoop enabled us to consolidate the islands of data scattered throughout the enterprise.”
If data consolidation is the first step, analytics are the second. Before it could be achieved, however, Cisco IT needed to design and implement an enterprise platform that could support appropriate service level agreements (SLAs) for availability and performance.
“Our challenge,” says Bhargava, “was adapting the open-source Hadoop platform for the enterprise.”
- Cisco IT built a Hadoop big data analytics platform using the Cisco® Common Platform Architecture (CPA) for Big Data, which is based on the Cisco Unified Computing System™ and Intel® Xeon® processors
- The platform provides high performance in a multitenant environment, anticipating that internal users will continually find more use cases for big data analytics
- It also takes advantage of the Cisco Tidal Enterprise Scheduler (TES) to facilitate job scheduling and workload automation; with built-in connectors to Hadoop, TES minimizes programming and debugging tasks and saves hours on each job
Putting the first use case into production
The first big data analytics program in production at Cisco helped increase revenues by identifying hidden opportunities for partners to sell services.
“Previously, we used traditional data warehousing techniques to analyze the installed base and identify opportunities for the next four quarters,” says Srini Nagapuri, Cisco IT project manager. “But analysis took 50 hours, so we could only generate reports once a week.”
The other limitation of the old architecture was the lack of a “single source of truth” for opportunity data. Instead, service opportunity information was spread out across multiple data stores, causing confusion for partners and the Cisco partner support organization.
The new Cisco big data platform has removed such limitations. Not only does it bring disparate datasets together for analytical purposes, but it also processes 25 percent more data in 10 percent of the time.
- Analyses are now completed in six hours instead of 50
- Opportunities are identified for the next five calendar quarters instead of four
- Partners and Cisco employees can dynamically change the criteria for identifying opportunities
And the business outcome?
“The solution processes 1.5 billion records daily, and we identified new service opportunities the same day we placed the system in production,” says Nagapuri. These opportunities, he adds, represented $40 million in incremental revenue from partners in the first year alone.
Advice for others
While initial results like Cisco received are certainly desirable, Bhargava acknowledges others are still wrestling with the newness, uncertainty, and investment required for big data.
“There is always resistance to change, and big data architectures are still very new, very different,” he says. “Getting executive sponsorship, having a strong change management strategy, and building momentum are essential. It’s also important to get a handle on internal data before incorporating external data from partners, industry sources, and consumer channels like social media.”
Being successful also requires:
- The right technology infrastructure
- Proactive education so stakeholders and users have a full understanding of the capabilities
- A clear definition of timeline, costs, and benefits
“The challenge is often justifying the upfront cost. Start with a small investment focused on a defined use case that brings together technology opportunity with business opportunity,” Bhargava recommends. “But make sure you have a big vision and a platform to support it, or there’s a good chance you will end up with small data silos. Once you exhibit the value of big data analytics, other business units will invariably start thinking of additional use cases. That’s when the value of big data gets bigger.”