Welcome!

Cognitive Computing Authors: Pat Romanski, Yeshim Deniz, Liz McMillan, Elizabeth White, Jason Bloomberg

Related Topics: @CloudExpo, Microservices Expo

@CloudExpo: Blog Feed Post

Big Data Is All the Rage. Why?

We see Big Data solutions daily through tools such as Twitter and LinkedIn

On Monday, December 5, Bob Gourley went on the Enterprise CIO Forum to explain Big Data and why it matters. First, he defined Big Data simply as the data your organization cannot currently analyze. Though some technologists give more precise definitions, this sums up the challenge enterprises now face. If you can deal with all of your data now, you don’t have a Big Data problem, but as soon as you have more data than you can effectively manage to finding the answers you need fast enough to use them, you need a Big Data solution. Structured data and relational databases can also be Big Data but what we’re really talking about is the type and volume of information that exceeds traditional methods. New solutions include MapReduce, originally developed at Google to analyze and index the entire Internet, and Hadoop which grew to use those new methods.

We see Big Data solutions daily through tools such as Twitter and LinkedIn, which analyze massive amounts of information from user accounts and actions to perform searches and generate content in real time. Twitter, for example, looks at millions of tweets fast enough to determine what topics are currently trending, and LinkedIn can analyze your networks and profile to suggest people who you would want to connect with.

Big Data doesn’t have one tool or even a toolset, it has a framework. For example, there is a growing and evolving ecosystem around Hadoop, shepherded by Cloudera, including a variety of capabilities for structured and unstructured data. Hadoop itself allows the use of commodity hardware to efficiently store and process massive amounts of unstructured data.  According to Gourley, Hadoop is the essence of the current Big Data phenomenon, though there are other niche solutions out there. At the moment, government, finance, energy, and science are all turning to the Hadoop family for their Big Data solutions. Hadoop, formally known as Apache Hadoop, is an open standard managed by the Apache Foundation, and is combined with software such as HBASE, Hive, and Flume in distributions, such as the popular Cloudera’s Distribution including Apache Hadoop.

Big Data has created a “Cambrian Explosion” of capabilities and uses. For example, by analyzing social media and messages, organizations have distilled member’s “digital characters” to find criminals, rogue traders, and unusual behaviors. Other use cases include detecting cyber attacks and better internal search results and recommendations to clients, such as the federal government’s USASearch.

Initially, major IT companies were cautiously exploring Big Data solutions, but now many have jumped on the Hadoop bandwagon. Microsoft showed great agility when it recently abandoned its proprietary software Dryad to contribute to the open source Hadoop community, and many major companies have their own Hadoop distributions such as SGIIBM, EMC, and Dell. Users now have the choice to download Cloudera’s distribution and use its management tools to configure and run Hadoop, or purchase an incredibly powerful piece of hardware from SGI that already has Hadoop configured and guaranteed to run. That same firm will sell you training and services while providing patches, making it a good option for an enterprise CIO.

In the next year, expect to see the continuing evolution of management tools for Hadoop clusters. Currently, Cloudera’s configuration manager is the dominant tool and will continue to evolve as well. Dozens of firms are also beginning to provide applications on top of Hadoop, which will allow analysts to interact with Big Data themselves rapidly, without the help of the IT department. And as Hadoop grows more prevalent, now is the time to go out and get training.

Read the original blog entry...

More Stories By Bob Gourley

Bob Gourley writes on enterprise IT. He is a founder of Crucial Point and publisher of CTOvision.com

IoT & Smart Cities Stories
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
DSR is a supplier of project management, consultancy services and IT solutions that increase effectiveness of a company's operations in the production sector. The company combines in-depth knowledge of international companies with expert knowledge utilising IT tools that support manufacturing and distribution processes. DSR ensures optimization and integration of internal processes which is necessary for companies to grow rapidly. The rapid growth is possible thanks, to specialized services an...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...