Techaisle Blog

Techaisle's Big Data study of 3,360 businesses shows that mid-market businesses typically started their big data journey in one of four ways. However, the highest success rate (determined by reaching a successful implementation of a big data project within six months of initiation) was achieved when an external consultant or organization was brought in to develop proof of concept, advice on database architecture and ultimately develop the big data analytics solution.

Once a decision was made to embark on a big data deployment project, the mid-market organization tended to quickly align behind the initiative. They did realize that big data was not a typical cloud application deployment where independent department purchases could be made, nor was it infrastructure deployment where only IT could be involved. Big data required a new type of alignment between business heads, namely, Marketing, Finance, IT and a completely new set of players known as data scientists or data analysts.

Study shows that businesses are moving from “whack-a-mole” analytics to “business perspectives” to get newer insights into their operations and better knowledge about their customers as they rethink their marketing strategies because mobility, social media, and other transactional services have increased the number avenues for connections with their customers. There are many different tactical objectives for deploying big data projects but the top among them are sentiment monitoring, generating new revenue streams & improving predictive analytics. And businesses are expecting some clear cut benefits from big data analytics such as increased sales, more efficient operations, improved Customer service.

 

In a parody of Start Trek, Silicon Valley technology companies describe their business goal as “Scale, the final frontier…”.  Mid-market companies, defined as those having 100-2500 employees, may indeed provide an opportunity to emerging technology vendors to scale their business. According to Techaisle, a market research firm, these 800,000 global companies spend $300B on IT and are sought after by technology vendors big and small. In the last decade, technologies such as Cloud, SAAS and Virtualization have reached scale with a large number of mid-market companies as early adopters. Intuit, Salesforce.com, NetSuite and Amazon are just a few examples of companies who have relied upon mid-market companies as a key building block for their business.

What does this mean for Big Data? To find out, Carpe Datum Rx spoke to “SMB Guru”, Anurag Agrawal, CEO of Techaisle and the former Head of Worldwide Research Operations at the Gartner Group. Techaisle recently talked to 3,300 global businesses about their Big Data adoption plans. Here is an excerpt from our discussion.

The SMB Market is considered the Holy Grail for technology vendors because it is hard to penetrate. Does your research show that mid-market companies will adopt Big Data before large enterprises do? Are they the early adopters of this technology?

Yes, you are right the SMB Market is the Holy Grail as it is hard to penetrate but with the highest potential. To elaborate, there are slightly over 70 million small businesses and 800,000 mid-market businesses worldwide. They constitute over 97 percent of the business segment. And their collective IT spend is projected to grow by 6.5% between 2013 and 2016 which is quite a lot faster than the Enterprise segment. To really identify the SMB segments and their type of technology spend is a mind-numbing exercise due to the sheer volume of data points. This is compared to the enterprise segment where there are fewer companies and larger dollar amounts being spent.

To answer your second question about whether mid-market businesses will adopt big data before large enterprises, let us look at some facts. Cloud computing started as an enterprise play, however, it was quickly discovered that SMBs would be the more relevant target segment with a faster path to adoption. Similarly, as enterprises adopted Virtualization, vendors shifted their focus to the SMBs with some very creative solutions. Mid-market companies, defined as those with 100 to 2500 employees could certainly be the early adopters of Big Data. We recently did a study where we surveyed 3,360 mid-market businesses worldwide covering all regions – North America, Europe, Asia/Pacific and Latin America. What we found is that the promise of superior data-driven decision making is motivating 43 percent of global mid-market businesses to at least look at Big Data technology. And above all, 18 percent of mid-market businesses are now investing in big data related projects.

In the mid-market segment, there is also a competitive imperative to understand customers, create innovate products and improve operational efficiencies. They are not burdened with too many silos and large legacy systems deployments. The absence of large legacy systems is an important point to consider because it makes mid-market businesses more agile to implement new types of solutions that solve their business problems. It is expected that in year 2016, global SMBs would spend US$3.6 Billion on big data solutions exhibiting a growth rate that is faster than what was exhibited by cloud computing solutions.

We understand that you cast a very wide net to get your 43% number. Is there a consistency in the sentiment on big data across different parts of the world? 

Yes, we had to cast a wide net to really understand the adoption and trends within mid-market businesses. And yes, there is a difference across geographies and employee sizes. North America has both the largest market and the highest level of adoption in Big Data overall. In terms of actual deployment activity, the market grows in relation to the size of the companies. Additionally, mid-market business attitude towards Big Data transitions from “Over-Hype” to must-have technology with the increase in employee size. Let me give you some examples. A small-to-mid-sized bank is developing a Proof of Concept for fraud analytics. Another example is of a small advertising agency that is trying to deploy digital advertising analytics. So big data is not only within the radar of enterprises, the same problems exist across all sizes of business, only the volume of data, available budget and the required simplicity varies. The problem is that we all get caught up in technology which instills a sense of fear. We have to shift the conversation from technology to solving business problems.

Big Data adoption is often stalled by a lack of knowledge or understanding of the technology and its capabilities. Do mid-market companies have a better understanding of this technology than large enterprises? Do they have an advantage over large enterprises in implementing effective solutions?

You are right. Three things – Technology, Resources and Data are the biggest roadblocks for big data project implementations within mid-market businesses. In recent years technology and technology options have evolved extremely rapidly for an average business to understand, evaluate, purchase and implement. Big data is no different. Mid-market businesses consider big data as very complex resulting in very steep learning curves. The complexity gets further exacerbated with lack of experience, lack of skilled manpower and innate difficulty in identifying external consultants who would be the right fit for their big data business objectives and budget availability. In spite of challenges, the study shows that there have been some successes when business units, IT & data analysts exhibit extraordinary alignment.

Our study shows that mid-market businesses typically start their big data journey in one of four ways and the highest success rates have been achieved when IT and data analysts work with external consultants from project inception. It is still very early days for these businesses to fully embrace big data but the seeds are being planted. And we believe that these businesses may very well race ahead of enterprises with their deployments as technology becomes simpler and consultants become experienced. As we like to say it, SMBs could be the path to big data simplicity.

You talk about the linking of structured and unstructured data. Why is this problem so important compared to all the others? 

The issue of analyzing data from diverse sources leads a mid-market business to naturally consider linking structured and unstructured data. If we look back, CRM solutions had first established the need for analyzing customer data. However, the data was mostly two-way transactional structured data. This changed when customers began visiting business websites to explore, browse and perhaps make purchases thus leaving behind a trail of information. And everything changed with the onset of social media, blogs, forums, wikis and opinion platforms where the identification of false positives and negatives became difficult and knowledge about the customer and resulting segmentation became an inaccurate undertaking. Big data analytics presents the possibilities of connecting together a variety of data sets from disconnected sources to produce business insights for generating sales, improving products or detecting fraud. Thus the importance of linking structured and unstructured data to analyze social media data, web data, customer and sales data along with click-stream machine generated data and even communications data in the form of emails, chat, and voice mails. But extremely limited expertise creates a major challenge. If they can figure it out, one-fourth of mid-market businesses say that they will use big data as an integral part of their overall analytics efforts. The possibility of analyzing a variety of data producing action-driven business insights is too big to ignore for mid-market businesses.

How are big data projects getting started globally? Are they championed by LOB managers? Are they getting adequate support from executive management? Are customers demanding it?

The study reveals that the initiators are marketing, finance or operations and the ultimate user of the analytics is the business user. Big data requires a new type of alignment between business heads, namely, marketing and finance (main drivers of big data projects), IT and a completely new set of players known as data scientists or data analysts. As I mentioned before, once the decision is made mid-market businesses show an extraordinary alignment across departments. Our study shows that mid-market businesses typically started their big data journey in one of four ways. However, the highest success rate was achieved when an external consultant or organization was brought in to develop proof of concept, advise on database architecture and ultimately develop the big data analytics solution right from the moment of project inception.

What is one piece of advice or Carpe Datum prescription can you share for our members?

You have adopted cloud, you have adopted mobility, you have adopted social media so do not be afraid to develop Big Data analytics proof of concepts. Do not ignore big data just because of perceived complexity and big data solution providers’ inability to create bite-sized messaging that directly address pain-points. Do not forget that business intelligence has now become one of the fastest solutions to be adopted by SMBs and mid-market businesses. If done right, big data will address three key pain points: Increased sales, More Efficient operations, Improved Customer service.

Various organizations define Big Data differently. Some use “petabytes of data” as a benchmark to isolate big data from other normalized and structured data sets that exist within an organization. However, this measure quickly boxes big data analytics into the large enterprise market segment. Small and mid-market businesses certainly do not have this extent of data but Big Data still relevant for them. In fact Big Data solutions are more relevant for Small and Mid-Market businesses. However, it will take some creativity on the part of solution providers to make Big Data accessible, easy to use and comprehend for segment that constitutes 97 percent of global businesses.

Cloud computing started as an enterprise play, however, it was quickly discovered that SMBs will be the more relevant target segment with a faster path to adoption. Similarly, as Virtualization market started getting fully penetrated within the enterprises, vendors shifted their focus to the SMBs with some very creative solutions. As far as big data is concerned SMBs are starting to show interest and even adoption. However, there is a stark difference in approaches between mid-market businesses and small businesses. While mid-market businesses are experimenting with bespoke solutions, small businesses are gravitating towards a multi-tenant, aggregated and federated big data solution that has a mix of publicly available data and their own internal data.

It is expected that in year 2016, global SMBs would spend US$1.6 Billion on big data solutions exhibiting a growth rate that is faster than what was exhibited by cloud computing solutions. Cumulatively between now and end of 2016, SMBs itself would have shelled out US$3.9 billion on big data solutions. This spending includes hardware, software and services.

So why are many big data solution providers ignoring SMBs? Simply put, because of perceived complexity and inability to create bite-sized messaging that directly address SMBs pain-points. But they should not forget that business intelligence has now become one of the fastest solutions to be adopted by SMBs. If done right, Big data address three key pain points of SMBs: Increase sales, Efficient operations, Improve Customer service.

Promise of Superior Decision Making

Let us take Techaisle’s recent global mid-market businesses’ Big Data Adoption & Trends study which clearly shows that the promise of superior data-driven decision making is motivating 43 percent of global mid-market businesses to either invest in or investigate Big Data technology. Out of these, 18 percent of mid-market businesses are actively investing in big data related projects. The possibilities of analyzing a variety of data sources, producing action-driven business insights is too big to ignore for these businesses.

Similar to cloud, the attitude towards Big Data is transitioning from “Over-Hype” to “Must-Have” technology with the size of business. Even within the businesses that consider big data to be over-hyped, 29 percent think that it will be an important part of their business decision making process in the future.

Extracting Business Perspectives

Business intelligence by itself has provided enough business insights, however, mid-market businesses are now looking for extracting business perspectives to drive superior decisions and ultimately achieve superior results.  Extracting business perspectives has become important as they rethink their marketing strategies because mobility, social media, and other transactional services have increased the number avenues for connections with their customers and partners.

CRM solutions had first established the analytics for analyzing customer data. However, the data was mostly two-way transactional data. This changed when customers began visiting business websites to explore, browse and perhaps make purchases thus leaving behind a trail of information. IT vendors and mid-market businesses figured out the need to analyze the data and combine it with transactional information.

However, everything changed with the onset of social media, blogs, forums, wikis and opinion platforms where the identification of false positives and negatives became difficult and knowledge about the customer and resulting segmentation became an inaccurate undertaking.

Big data analytics presents the possibilities of connecting together a variety of data sets from disconnected sources to produce business insights whether be for generating sales, improving products or detecting fraud.

It is therefore not surprising that global mid-market businesses are turning towards big data analytics to analyze social media data, web data, customer and sales data along with click-stream machine generated data and even communications data in the form of emails, chat, voicemails.

Leap of Faith or Solution Readiness

Analyzing data from diverse sources leads a mid-market business to naturally consider linking structured and unstructured data. This also drives them to evaluate and select the technology that can be used for simplified implementation. Simplified implementation is important because mid-market businesses do not yet have in-house capabilities to analyze unstructured data and those that have them consider the capabilities at best rudimentary.

Big data therefore is a major leap of faith for mid-market businesses resulting in treating big data analytics projects usually as separate to the existing analytics within the business. More aggressive adopters are planning to use big data analytics along with other analytics in a coordinated manner so that one does not become an inhibitor for the other.

In recent years technology and technology options have evolved extremely rapidly for an average business to understand, evaluate, purchase and implement. The complexity gets further exacerbated with lack of experience, lack of skilled manpower and innate difficulty in identifying external consultants that would be the most right fit for their business objectives and budget availability.

In spite of challenges, the study shows that there have been some successes when business units, IT & data analysts exhibit extraordinary alignment. Our study shows that mid-market businesses typically started their big data journey in one of four ways. Highest success rates for project implementation and generating new insights have been achieved when IT and data analysts work with external consultants from project inceptions.

SMBs as the Path to Big Data Simplicity

The global SMB spend on big-data related deployments will cross US$1.0 billion in 2013 which is a 32 percent increase from 2012. SMBs are still experimenting to see if big data analytics can provide newer insights into their operations and better knowledge about their customers. It is still very early days for small and mid-market businesses to fully embrace big data but they are planting the seeds in terms of re-architecting their IT infrastructure to plan for the future. But we believe that SMBs may very well race ahead of enterprises with their deployments as technology becomes simpler and consultants become experienced.

 

Techaisle’s global mid-market businesses’ Big Data Adoption & Trends study shows that the promise of superior data-driven decision making is motivating 43 percent of global mid-market businesses to either invest in or investigate Big Data technology. Out of these, 18 percent of mid-market businesses are actively investing in big data related projects. The possibilities of analyzing a variety of data sources, producing action-driven business insights is too big to ignore for mid-market businesses.

Big Data requires a certain level of IT sophistication and a history in the linear investment in Information Technology enablers to be successfully. While these factors predispose larger accounts to Big Data, the competitive imperative to understand customers, innovate products and improve operational efficiencies has already started to reach down to the mid-Market, forcing a search for how to leverage primary and secondary data that is generated by the business.

The current and planned investment represents a sizable opportunity considering that the segment is relatively new and requires a certain level of IT sophistication and a history in linear investment in Information Technology enablers to be successful. North America has both the largest market and the highest level of investment in Big Data overall in SMB and mid-market segments. Mid-Market attitude towards Big Data transitions from “Over-Hype” to “Must-Have” technology with the increase in employee size. However, nearly one-fourth of lower mid-market businesses consider big data to be over-hyped and yet 29 percent think that it will be an important part of their business decision making process in the future.

Business intelligence by itself has provided enough business insights, however, mid-market businesses are now looking for extracting business perspectives to drive superior decisions and ultimately achieve superior results.  Extracting business perspectives has become important as they rethink their marketing strategies because mobility, social media, and other transactional services have increased the number avenues for connections with their customers and partners.

In addition to understanding customers, mid-market businesses are also considering big data analytics as an important initiative to help them improve operational efficiencies.

Techaisle’s study shows that there are many different tactical objectives for deploying big data projects but the top among them are sentiment monitoring, generating new revenue streams & improving predictive analytics. It must also be said that businesses have figured out that there is a lot of publicly available data which could also be analyzed to their advantage.

The mid-market businesses actively investing in big data technologies are expecting some clear cut benefits from big data analytics such as increased sales, more efficient operations and improved customer service. These objectives differ slightly by different geographic regions. As the growth rates continue to lag in mature economies, the pressure to increase revenue grows resulting in developing robust analysis and extracting insights from all sales and customer data including transactions.

When specifically asked about preferred deployment choice in terms of on-premise vs. cloud, mid-market businesses are unsure as they are still navigating through their technology options. However, Hadoop dominates as the preferred platform but confusion exists.

In terms of analytics skill-set and long-term vision, the potential of linking structured and unstructured data sources to create new business insights is being considered very useful but at the same time mid-market businesses are not really prepared for it. In fact one-third of mid-market businesses agree that linking structured and unstructured data would be very useful for big data analytics but over 70 percent mention that they have either none or very limited capabilities of analyzing unstructured data. This is where they are turning to external help for guidance.

Needless to say, survey reveals that big data deployment is posing tremendous challenges. Technology confusion, lack of skilled resources and potential unclean data are being considered as the biggest roadblocks for big data project implementations. Big data technology and its far-reaching capabilities are being viewed by mid-market businesses as very complex resulting in very steep learning curves.

In spite of challenges, the study shows that there have been some successes when business units, IT & data analysts exhibit extraordinary alignment. Highest success rates for project implementation and generating new insights have been achieved when IT and data analysts work with external consultants from project inceptions.

Detailed Global Mid-Market Big Data Adoption and Trends report is available for purchase. Details are given here.