Business at the Speed of Now

Customers of all types have ever-increasing expectations for personalization and immediate response.

Modern business operations of all kinds are becoming digital and smart. They can be highly agile and adaptive in a dynamic world – one that is enabled and fueled by data, AI, cloud computing, digital networks, and relationships. Accordingly, business leaders are looking for better results and competitiveness in today’s constantly changing market. With an increasing number of connected devices, the adoption of 5G, and the explosion of big data and technology-enabled ways to effectively leverage data real-time to power AI-driven models, workflows, and rule engines, now more than ever, we can run data-driven organizations in real-time. Therefore, it is time to adopt the new NOW management thinking, the state-of-the-art capabilities, and AI that will get your organization doing business at the speed of now (or maybe even at the speed of tomorrow!)

Let’s start with some basics and then look at some examples and Myridius framework to identify and execute real-time streaming intelligent analytics.

Streaming data processing is big data technology. The data is generated continuously by thousands of data sources, usually transmitting data records simultaneously in small sizes (think kilobytes). There are many types of streaming data, including files generated from mobile apps, web applications, eCommerce purchases, in-game player activity, information from social networks, financial trading floors, or geospatial services, and telemetry from connected devices (IoT) or instrumentation in data centers.

Streaming data analytics is called by many other names: real-time analytics, streaming analytics, complex event processing, real-time streaming analytics, and event processing. At Myridius, we call it intelligent streaming analytics. This name reflects the fact that we see the vital pairing of streaming data with machine learning and other rules engines to help run the business at the speed of now by enabling a continuous data stream to feed machine learning models and rules engines, detect conditions and anomalies from the time of receiving the data enabling decision models and rules engines.

For example, with intelligent streaming analytics, you can feed a credit card fraud detection machine learning model, receive an alert, initiate a bot to create a case and a workflow, and make a decision on what to do immediately without human intervention or hand off relevant data to a human for a specific action.

This type of real-time streaming analytics works with data flows, feeding decisions and analytics models and tasks that include correlations, aggregations, filtering, and sampling which provide a line of sight into many aspects of business operations and customer activity.

This can enable timely decision-making and enable intervention in a proactive, or at least highly responsive, way in emerging scenarios. A typical goal is to allow up-to-date information and keep the state of data updated for users and automation.

That said, it is now possible to use even more complex AI and machine learning models that can run in real-time or near-real-time. From live video processing to real-time prediction using machine learning techniques suited to the task (like gradient boosted trees) we can create and train the mathematical models in ways that reduce them to abstract vectors. The importance of a speedy decision (such as using computer vision to identify a bad guy at the airport) will then drive decisions around the business case of how much computing power (and cost) can be thrown at executing a real-time performant solution. This approach then, effectively, creates the possibility for analytics as a service with real-time responsive intelligence.

One simple example is where a firm can continuously analyze social media streams and track real-time changes in customer sentiment about their experience, products, services, and brands and products to respond in a timely fashion in a way that makes sense for the business.

Intelligence in our view comes both in terms of the “smarter decisions and actions” a firm will take and the potential for the execution of artificial intelligence to make the use case execution “smart.”

Many organizations are building a hybrid model by combining both streaming data processing and batch processing approaches.

Stream processing is useful in different use cases

In general, stream processing is useful in use cases where

• a problem or trend can be detected,
• there is an excellent opportunity to improve the outcome or make a decision in real-time
• speed of intervention and decision-making is essential for operations happening in milliseconds.

Business outcomes from smart operations include lowering operating costs, lowering inventory levels, improving product quality or service, improving customer service, decreasing time to market, lowering capital costs, reducing operational risk, and increasing innovation like the development of usage-based business models.

Additionally, better customer management can yield better insight for decision-making, reduced churn, optimized marketing investment, better targeting and segmentation, and delivering better overall customer experience.

While the opportunities indeed are endless, here are some examples:

• Financial
  • Transaction processing, market/currency state monitoring
  • Algorithmic Trading, Stock Market Surveillance,
    A financial institution tracks changes in the stock market in real-time, computes value-at-risk, and automatically rebalances portfolios based on stock price movements.
• Healthcare
  • Real-time monitoring of health-conditions, clinical risk-assessment, patient-state analysis, and alerts;
  • Smart Patient Care
• Insurance
  • Real-time claims processing
  • Real-time underwriting
• Consumer, Retail, and Marketing
  • Customer location intelligence
  • Context-aware promotions and advertising
  • Proximity marketing
  • Advertising optimization
  • Real-time customer churn prediction and intervention
  • Retail/customer service: customer behavior analysis and operations improvement
• Operations, Manufacturing, and Industrial
  • Production line monitoring
  • Intrusion and Fraud Detection (e.g., Uber, Credit Card, Healthcare, etc.)
  • Operating heavy machinery and transportation fleets (sourcing data streams from sensors and IoT devices)
  • Predictive maintenance
  • Sensors in vehicles, industrial equipment, and farm machinery send data to a streaming application. The application monitors performance and detects any potential defects in advance.
  • Manufacturing/supply chain: real-time monitoring, predictive maintenance, disruption/risk assessment
• Information Technology/Systems
  • Computer system and network monitoring
  • Geospatial data processing
  • Any kind of real-time data analysis like fraud detection or system maintenance.
• Supply Chain
  • Predictive inventory a spare part order automatically prevents equipment downtime
  • Merchandising optimization
• Other
  • Social distance capacity alert and management
  • Traffic Monitoring, Geofencing, Vehicle, and Wildlife tracking
  • Anomaly detection (e.g., power theft, medical care, etc.)

With an understanding of the opportunity and some types of use cases, let’s explore how to build a portfolio of opportunities for the use of streaming data with machine learning.

High performing organizations will utilize streaming data, analytics, and AI to enable a data-driven learning organization.