• SIMPLIFY. EXPAND. GROW.

    SIMPLIFY. EXPAND. GROW.

    #SMB #MIDMARKET #UPPER MID-MARKET #CHANNEL
    LEARN MORE
  • HYBRID WORK IS HERE TO STAY?

    HYBRID WORK IS HERE TO STAY?

    NOT SO FAST SAYS THE DATA
    ANALYSIS
  • NEXT CHANNEL - THE FUTURE OF PARTNER ECOSYSTEM

    NEXT CHANNEL - THE FUTURE OF PARTNER ECOSYSTEM

    Networked, Engaged, Extended, Hybrid
    DOWNLOAD NOW
  • FEATURED INFOGRAPHIC

    FEATURED INFOGRAPHIC

    2021 Top 10 SMB Business Issues, IT Priorities, IT Challenges
    GET IT NOW
  • BUYERS JOURNEY

    BUYERS JOURNEY

    Influence map & care-abouts
    LEARN MORE
  • DIGITAL TRANSFORMATION

    DIGITAL TRANSFORMATION

    Delivering Connected Business
    LEARN MORE
  • SECURITY RESEARCH

    SECURITY RESEARCH

    SMB & Midmarket Security Adoption Trends
    LATEST RESEARCH
  • MANAGED SERVICES RESEARCH

    MANAGED SERVICES RESEARCH

    US SMB & Midmarket Managed Services Adoption
    LEARN MORE
  • CLOUD RESEARCH

    CLOUD RESEARCH

    SMB & Midmarket Cloud Adoption
    LATEST RESEARCH
  • CHANNEL PARTNERS

    CHANNEL PARTNERS

    Transformation or Consolidation
    LATEST RESEARCH
  • ANALYTICS & ARTIFICIAL INTELLIGENCE

    ANALYTICS & ARTIFICIAL INTELLIGENCE

    SMB & Midmarket Analytics & Artificial Intelligence Adoption
    LEARN MORE
  • 2021 SMB & MIDMARKET PREDICTIONS

    2021 SMB & MIDMARKET PREDICTIONS

    Top 10 SMB & Midmarket Predictions for 2021
    READ NOW
  • WHITE PAPER

    WHITE PAPER

    SMB Path to Digitalization - Prologue and Epilogue
    DOWNLOAD
  • COVID-19 IMPACT

    COVID-19 IMPACT

    on SMB IT Spend Growth Rates
    FULL ANALYSIS
  • SAAS RESEARCH

    SAAS RESEARCH

    US SMB & Midmarket SaaS Adoption
    LEARN MORE
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
6 minutes reading time (1166 words)

Meet the New Boss: Big Data (WSJ) – Techaisle Take

Wall Street Journal Article

This is an interesting article from the WSJ concerning how we are slowly allowing decision-making processes to move away from people and be handled by algorithms instead. It caught our attention at a time when we are completing survey work for Business Intelligence report. As discussed in an earlier post, one of the key trends in BI is how deeply it is being embedded into all kinds of applications , and this article is a good example. Please let us know what you think: comment, like, tweet or forward.

Laying the Foundation


Analytic software has evolved through several generations over the last 70 years from around WWII, when a series of unprecedented number-crunching challenges gave rise to Decision Support Systems (DSS) designed to solve problems such as best equipment production mix given material constraints, how to logistically support the Allied invasion of Europe, split the atom, or break the Japanese code. These kinds of problems tended to be monolithic, using stochastic models to find the best answer – and were a major catalyst to development of the mainframe computer.

Business Intelligence (BI) followed this linear and iterative approach with one that supported solving business problems, mostly operational, within divisions and departments of large commercial organizations, using more distributed equipment within a wider audience, i.e., Finance, Operations and Distribution. In the late 1990s there was an explosion of data resulting from widespread adoption of CRM, the killer app of the Client/Server era, adding mountains of Sales and Marketing Data to the volumes of operational information. There was a growing need to get a top down view of how performance in one area of the organization was impacting the others, to begin taking a more structured approach at understanding cause and effect, setting objectives and consistently measuring performance to improve results. BI was evolving into Enterprise Performance Management (EPM) - which is where market leaders are today.

EPM is characterized by using Business Intelligence software to understand the best performance scenarios, measure actual performance indicators (KPIs) and determine how to close the gaps, using exception reporting for most front office functions (CRM/SFA) and rules-based processing for the back office (Process Manufacturing/Real Time Bidding, SCM/Advanced Web Analytics).

Optimization Nation


Equally important as the individual BI technology advances are some of the underlying rules that have accompanied the evolution: Moore’s Law, Metcalfe’s Law, the Law of Accelerating Returns all drove exponential growth in production, adoption and utility. Over a 20 year period, these have resulted in a slow-motion Black Swan event based on the cumulative effect of technology investments, and having huge impacts on our society, including but not limited to the following optimization activities:

Law of DisruptionEconomy – development of consumer mortgage products designed to optimize sales volume regardless of risk, bundling them into bonds to optimize profit on the debt, creation of derivatives to optimize transactions and create demand for increasingly suspect debt, development of new financial instruments that have no underlying value such as  synthetic derivatives that truly have nothing but conceptual paper profits behind them, etc. By 2008 these financial instruments had optimized leverage to create risk greater than the combined GDP of industrialized world.

Employment – The WSJ article goes into depth about how algorithms have already replaced a hiring manager's decisions based on probabilities of how the employee might behave under certain circumstances. Employer choices have also been optimized by a flattening of the market caused by oceans of virtually unlimited supply from sites like Monster.com, 100K Jobs, Dice, etc. Middle management has been optimized out of the organizational chart and replaced with productivity tools, more efficient communications and a lower ratio of managers to workers. And the actual number of staff required to hit the bottom line has been optimized while CEO salaries have been optimized. If we look a little further down the line, Andrew McAfee's POV is deep on this subject, and more technical than mine.

Industry – We all know that manufacturing was moved offshore en masse over the past three decades to optimize production costs, but several other industry segments have been optimized as well, including Retail which has optimized through consolidation and healthcare which has optimized revenue per patient. Retail has been optimized at a structural level, to provide one-stop shopping for almost everything you need in a single location while volume has been optimized to produce the absolute lowest price and any cost, including optimizing the number of worker hours to keep an optimal ratio of full time to part time employees and save the resulting benefit costs. And it has also optimized the number and variety of retail outlets and small businesses required to service an optimized population in square miles. Healthcare prices have been optimized to take advantage of tax structure, potential law suits, healthcare insurance gaps, maximizing federal matching funds, Board and C-Suite compensation, pharma industry profits, and many more.

Government – Automation has also enabled a profitable business model that optimizes the use of Federal Government funds and ensures that every available dollar is spent, whether it is to make sure everybody gets a state-of-the-art mobile wheelchair, their 120 monthly catheters, a speaking glucose meter, maximum disability benefits, etc.  “Don’t worry - we’ll handle all the paperwork.”

Set it and Forget it


Complex SystemsThe imminent next generation of analytics involves truly “optimized” complex systems with human intervention largely removed from the process. Not to single out Wall Street, but they offer one of the best examples of unbridled application of technology in the singular pursuit of optimization, in their case, profit for themselves and their shareholders. The Financial Services industry has invested billions into technology and employed thousands of physicists and Ph.D.-level mathematicians to achieve a couple-millisecond transaction advantage, and programmed algorithms to use the advantage and change the rules (i.e., share price represents perfect information is no longer true). This has not proved to always produce predictable results, and the ghost in the machine has put us back on the precipice more than once, as seen in this TED video by Kevin Slavin. As we move into a brave new world that combines optimization software with networks that operate too fast for human intervention, more of our lives will be controlled by how rules are programmed into the system than what we do as individuals to impact the results. One of the best examples of where this is heading is the IBM’s Smarter Cities Initiative, which combines intelligent networks that manage Traffic, Water and Electric Utilities, Transportation, Healthcare, Public Safety, Education and others into an overall “Intelligent City”. Everyone hates traffic, so the video example from the IBM innovation site does more to explain this than I can by writing more on the subject.

Whether you agree with it or not, we are on a direct course to this future that is almost impossible to divert. This is a philosophical question and everyone will have their own opinion about the cost/benefit of chasing optimization. Comments and Opinions are welcome, please let us know what you think.

SMB Marketing Automation User Snapshot
Pick of the Week - Exxova
 

By accepting you will be accessing a service provided by a third-party external to https://techaisle.com/

Search Blogs

Find Research

Blog Archive

Research You Can Rely On | Analysis You Can Act Upon

Techaisle - TA