The detonation of data, or THE BIG DATA, as it is colloquially referred, is already happening. What is sometimes referred to as the Internet’s first wave — say, the 1990s until around 2005 may be — brought completely new services like e-mail, the Web, online search and eventually high speed internet. For its next act, the industry has pinned its hopes, and its colossal public relations machine, on the power of Big Data itself.
If pencil-marks on some colossal doorjamb could measure the growth of the Internet, they would probably be tracking the amount of data sloshing through the public network that spans the planet. Christened by the World Economic Forum as “the new oil” , these vast loads of data have been compared to transformative innovations like the steam locomotive, electricity grids, radio, air-conditioning and the television.
Big analytical data storage systems are looking at ways of storing data outside the traditional database models because it inherently uses what is known as ‘a loosely coupled relationship’ for its flexibility. If we look at the relational databases when first introduced, flexibility and interactivity were at the forefront of the thought process. Now, there are new technologies like NOSQL where there is no relational model at all. With the advent of service oriented architecture (SOA) the external world is only going to interact to the services that, in the past, only spoke to the database. In today’s architectural context, one can refer to a service to get the data. That being the case, why do we need flexibility in both places, it’s only needed at the service layer that provides a much higher level of intelligence as an integration point thus eliminating the need for a database at all.
Considering the kind of data explosion that is now happening, it might be preferable to go with a file system or non-SQL data storage methodology (the likes of Hadoop and Cassandra). With today’s application servers deploying multiple applications we are now seeing platform or process containers, especially for large enterprise, high-performance transaction processing. As we move closer to a ‘real-time’ world we will move to in-memory data objects for speedy analytics and an appliance model with arrays of boxes in the cloud exclusively for applications.
Is the end of the relational database era in sight – it seems so!