Growing demand for consumer packaged goods (CPG) intelligence. A niche industry going mainstream. And progressive, more affordable technologies for big data processing, storage, and analytics.
SPINS is capitalizing on all of them.
“Our business revolves around CPG insights in the natural and organic products market,” says Jim Scott, senior vice president of technology at SPINS. “With the maturation of big data technologies like Hadoop, we had a golden opportunity to process more data, and do it faster.”
Speed is everything for SPINS’ clients, he explains. The faster natural products manufacturers and retailers can turn CPG data into actionable intelligence, the faster they can:
- Adjust their marketing and promotional efforts
- Fill distribution gaps
- Refine their sales forecasts and activities
“Shrinking the window between gathering data, processing and analyzing it, and delivering it in a usable format is the key,” Scott says.
A perfect combination
The cycle of data acquisition to client delivery has traditionally required nine days, but Scott knew SPINS could shrink the timeline with the latest big data technologies. The company set an aggressive goal of 24 hours or less from data in to data out.
- SPINS’ pre-existing analytics environment was built with relational databases, but as it grew, it became increasingly difficult—and costly—to scale and administer.
- To accommodate greater customer demand, storage requirements, and analytical investigations, the company is in the process of deploying a MapR Hadoop Distribution supported by the Intel® Xeon® processor-based Cisco Unified Computing System™ (Cisco UCS®).
“We chose MapR because it delivers exceptional speed and there is no single point of failure. And the network infrastructure plays a huge part in performance. You need big, capable pipes to handle all of the traffic. Cisco UCS has a superior design for data transfer between servers, so it was the clear choice for our needs,” Scott explains. “In my opinion, the combination of Cisco UCS and MapR is the best possible platform for big data analytics.”
- The new system will have 4-5 times the storage capacity of the company’s relational databases.
- SPINS will soon be able to store and process upwards of 200 terabytes of data.
“We are arming our data analysts with more data and more tools,” Scott says. “There are countless ways to slice and dice the data, and that means we can deliver deeper analyses to all of our customers. And we can also solve specific problems or answer specific questions for our clients that weren’t previously possible or economically feasible.”
- Using Cisco UCS Manager, the environment can be administered as a single system instead of a collection of servers, no matter how large it is scaled.
- It will be tightly aligned with a network and security infrastructure based on new Cisco technologies.
- The time that was previously spent managing technology can now be devoted to better, faster analytics.
“The system will simplify the work process for our IT department as well as our data analysts,” says Scott. “Everyone will be able to work smarter instead of harder. That leads to better employee productivity and satisfaction, and helps us grow both internally and externally.”
Near real-time analytics
In SPINS’ pre-existing analytics environment, a single job would sometimes take 45-50 minutes to run. In a recent test environment, the MapR Hadoop software processed the same job in 5-6 minutes.
“The results have been fantastic, and we’re not even on the new hardware yet,” Scott adds. “We stand to go substantially faster when the production environment goes live.”
And that means a near real-time analytics package for SPINS’ customers, with better depth and quality of CPG insights than ever before.
“More companies are seeing that knowledge is power,” says Scott. “The faster we can deliver this knowledge to our clients, the more valuable our services become.”