Big Data has caught the fancy of each and everyone. Consumer and enterprise world is looking to adopt the Big Data. Analysts are talking about it, proclaiming it to the next big thing!

Below is my take on how the Big Data market space will evolve in 2012 and years to come.

• Data Analytics Service – We will start seeing SaaS vendors that will start offering data analytics as a service (e.g., 1010data). These vendors will be able to pull in your unstructured data and provide you standardized reports and data services. This is very similar to the Web analytics (like Omniture) but access lot more data to provide patterns. 70-80% of the data analysis pattern will be similar across industries. These vendors will be able to provide these reports as off the shelf capabilities. For the other 20%, the web interface will allow the user to create patterns to analyze the data
• Data Visualization Service – Analysis is only one part of the data, the other big thing will be able to visualize the data (more like infographics) that help make sense of the scale and data points. Business will need help to make sense of all this analysis and again 3rd party vendors (via products or Saas) will be fulfilling this space
• Crowdsourcing Model – enterprise will start using the crowdsourcing model to help analyze and find patterns in the data. Goldcorp was one the original pioneers that used the crowdsourcing model to analyze the geological data. This model will be seen more when dealing large geospatial data in the areas of mining, drilling, oil ad gas
• RDBMS vendors will make a comeback – Big Data has been scoring high on account of their ability to analyze and process unstructured data. The NoSQL vendors and products are all claiming to be the holy grail of data storage, fast processing and analytics. I believe, the Big Vendors –Oracle, IBM and to some extent Microsoft will release their version of relational databases that can store (using automated data sharding) and process large amount of data (with a SQL MapReduce layer). The vendors will incorporate techniques from the NoSQL world coupled with in-memory storage (think Coherence, eXtreme Scale) to provide sub second read write response times. The market is too big for the existing vendors to not claim supremacy
• NoSQL market will shake up – The current crop of NoSQL solutions has emerged as need to fulfill certain use cases. Their base features and functionality have been decided by the patronizing user client/project co-sponsors (e.g. Cassandra – Facebook, Hadoop-Yahoo, membase-Zynga). These NoSQL solutions work very well for the specific use case. Attempts to broad base the use case, mean a lot of effort needs to put in. Along the way, some products will merge, some will lose community support and other will get acquired by product vendor companies
• Data Analysis Libraries - To help the developers analyze data, the common patterns and designs will be available as off the shelf libraries. This means, analyzing data for most common patterns will be easier and will become no-brainer. The projected need for Data scientists will not materialize to the same extent with the proliferation of these libraries. E.g. MS Cloud Numerics

Big Data will follow the Consumerization model, where the core infrastructure either comes as a service or an appliance. The data analytics, data visualization will be provided as a standard set of services on top of the raw data with an ability to create your own model.

Do let me know, how do you see the Big Data market moving forward and do you agree or disagree with my predictions?

• Benjamins and Big Data
• Benjamins and Big Data