Gartner’s Hype Cycle offers something of a barometer for the technology trends adopted by business.
In the case of data warehouse appliances, acceptance of the technology is moving well beyond the hype, says Roxane Edjlali, Gartner research director in information management.
In the appliance model, the supplier commits to preloading hardware with the necessary operating system and database, all tuned to data warehouse-type workloads in advance.
Although further refinement will be necessary to optimise to the business’s specific demands, the appliance comes out of the box more attuned to the needs of a data warehouse than a generic transactional database.
"The reason they have been well received is they are tuned and balanced for this job," says Edjlali.
"They have become quite common as a data warehouse solution to the point that, as of 2014, Gartner moved them out of the Hype Cycle and into mainstream. Customers understand the advantages and are adopting them as part of data warehouse modernisation."
Fine-tune performance
The other advantage for users is that, as opposed to buying database, operating systems, management software and hardware from different suppliers, buyers of the appliance get one supplier to deal with to help fine-tune performance or iron out problems, says Edjlali.
Key suppliers
IBM - In 2009, IBM acquired Netezza, a company credited with
kick-starting the data warehouse appliance market. But now it is trying to augment this hardware offer with cloud services.
Heidi O’Mahoney, IBM UK analytics director, says this approach is designed to allow customers to experiment with new approaches to analytics without capital investment, providing an early assessment of return on investment.
"You can load data onto the cloud and quantify the value of those analytics," she says. This approach, supported by IBM’s DashDB offer, applies to traditional data stores as well as the distributed file system Hadoop. Businesses can then decide whether to take the system back in-house or continue in the cloud.
Oracle - With its Exadata database and Exalytics discovery platform, Oracle can help users scrutinise structured data and unstructured data held in Hadoop in the same system, both using SQL, says John Abel, engineered systems and public technology cloud leader for the UK, Ireland and Israel.
Abel also argues that because Oracle’s public cloud and private cloud use the same management tools and service categories as the appliances, it lowers the barrier to adopting data warehouse cloud service.
SAP - Although SAP appears in the data warehouse market with its SybaseIQ database, historically it has mainly been a supplier of enterprise business applications. The company argues that appliances exploiting its Hana in-memory technology can bring data analytics closer to live business data.
Mark Darbyshire, chief technology adviser at SAP UK and Ireland, says: "People are talking about using data as an asset, not just for analysis. They have started to realise that it should take part in your business processes."
Teradata - Teradata’s Astor discovery appliances is another technology that offers access to Hadoop unstructured data and data in relational databases in the same system, both queried in SQL.
But Teradata also helps users to fine-tune their appliance to the demand profile, says Mark Perrett, industry consultant: "Some users have limited, predictable access, such as a defined number of users accessing data for regulatory reporting. They can buy something that optimises storage rather than processing power. At the other end, we had an insurance customer that wanted to open the data warehouse to their customers to check their claims online. That could be 10,000 concurrent users. That is when single query response time becomes very important."
Apart from this benefit, suppliers will have different advantages depending on customer requirements, she says. Information on the market share of data warehouse appliance suppliers is scarce. Gartner has Oracle, Teradata, IBM, Microsoft, SAP and HP in the "leaders" segment of its Magic Quadrant research. ServerWatch has Oracle, SQL Server, IBM DB2 and SAP Sybase Adaptive Server Enterprise as the respective top four in the database management system markets, although this also includes transactional databases.
However the market share is measured, more recent drivers for the adoption of appliance technology in the data warehouse comes in the form of big data and demand for analytics performance more generally. Firms are looking to incorporate unstructured data, such as click-streams or social media comments, in an environment they understand.
Although distributed file storage system Hadoop can help store this data, it is open source and many users are unfamiliar with it. Using an appliance to incorporate Hadoop into existing approaches to management, query and analytics is appealing to users, says Edjlali.
"The addition of Hadoop into the appliance is interesting," she adds. "This is extending the data warehouse appliance to support big data scenarios."
Microsoft, Pivotal, Oracle and Teradata all allow data scientists to query unstructured data held in Hadoop using familiar SQL rather than less well-known MapReduce, the original approach to querying Hadoop, Edjlali says. IBM allows users to query Hadoop in SQL via its BigInsight console which integrates Netezza with Hadoop, while SAP also supports SQL on Hadoop.
The only restriction on the adoption of appliances for data warehouse may be budget and size of organisation, adds Edjlali. "There are smaller use-case solutions that may be better suited for hardware and software to be bought separately. There are also reasons within the organisation, depending on the way IT is managed. Sometimes there are different groups managing hardware and software, so having everything in one box does not fit with the IT approach."
Matthew Aslett, research director for data platforms and analytics at analyst firm 451, says a downside of the appliance approach is high cost and restrictions on technology choices, which will be dictated by the primary supplier. But these must be offset against the cost of doing it in-house.
"There might be some advantages having separate hardware and database suppliers, but I think the complexity of configuring and deploying all that yourself would be high," he says.
In any case, businesses can choose analytics tools to suit their needs away from the main data warehouse appliance supplier, says Aslett.
The real choice is not between an appliance and a mix-and-match approach to in-house hardware and software. It is between an on-premise appliance and an external cloud, he points out.
"You could argue that what Netezza did by introducing the appliance in the 1990s, Amazon is now doing with [cloud data warehouse] Redshift in the last couple of years. The ease of adoption and pre-configuration that was only available with the appliance is now in a cloud service," he says.
The main suppliers understand this dichotomy and avoid forcing customers to make a choice by providing seamless cloud services aligned to their approach to data structure and management in the appliance.
Using big data appliances
UK mobile phone retailer Carphone Warehouse was concerned about poor performance and availability of its data warehouse making it hard to measure retail business activities.
It worked with IBM to replace its Oracle Database environment with IBM’s PureData System for Analytics, powered by Netezza, the company says.
The system is designed to support decision-making in the company’s retail division – which supports store, supply chain and insurance operations.
Carphone Warehouse says it has achieved a reduction of more than 50% in the time to market for new business intelligence services; up to 1,200 times faster performance; increased profitability through new revenue streams; and a reduction in costs.
Windows manufacturer Velux is another company that uses a data appliance. This time, it chose SAP Hana in a bid to create a “single version of the truth”.
In a search for faster processing, the firm moved to SAP Business Warehouse and SAP Business Planning and Consolidation, both powered by SAP Hana’s in-memory technology, with business planning and rolling forecasting capabilities.
The result was query times reduced to four or five seconds, a 300% increase in usage of business intelligence tools and a transition from annual to monthly forecasting and planning.
Appliances have grown from a new product segment to the mainstream choice for data warehouse hardware and software. Only businesses whose requirements are too small, or that have extensive in-house skills should consider choosing hardware and software independently.
Meanwhile, the growth of cloud services for the data warehouse is adding another dimension to the decision-making about how to support the growing demands for data and analytics services to the business.
