Implementing Machine Learning Projects
Machine learning provides tactical business insights for marketing teams who want to maximize impact on their spend. In particular when teams require deep analysis of their user's behaviors. For example, an online advertising machine learning model that allocates the ad spend to users who have a higher probability of conversion.
According to the Gartner Data Science Team survey (2017), less than half of Artificial Intelligence (A.I.) projects end up being fully deployed. Not surprisingly, the greatest barrier is the difficulty in deploying A.I. products into existing marketing operations.
It's critical to embed these models inside appropriate marketing campaigns that deliver value. For organizations that have a full stack data science teams, the return on investment has been significant.
However, most marketing teams lack the technical skills or access to their data science teams to unveil the opportunities within their own datasets. The focus of data science teams has traditionally been on developing analytical assets, while dealing with the implementing A.I. has been merely a science lab. Many reasons underline that oversight:
Implementing Machine learning projects is a critical step in aligning analytics investments with strategic marketing objectives to achieve organizational goals and values.
The traditional data science model is a niche experiment, conducted in vitro. Data science teams must ensure Artificial Intelligence models can be traced, monitored, and secured, which requires discipline and management capabilities. Deploying analytical assets within operational processes in a repeatable, manageable, secure, and traceable manner requires more than a set of Application Programming Interface (APIs) and a cloud service.
The current analytical open-source movement is producing a wide range of analytical assets that will eventually have to be consumed. However, current open-source deployment techniques provide "dissemination means" not "managed production means." From that perspective, containerization techniques are efficient and effective in pushing models into production, but they do not relieve the organization from adopting a formal implementation process.
But establishing best practices within the marketing department requires a commitment from key stakeholders. It starts with clearly identifying the organization's mission, objectives, and goals. Then systematically applying a model life cycle discipline. This takes a commitment from upper management to invest the necessary time, talent, and resources necessary to stay competitive.
Bonding the Organization.
No machine learning project should be started without understanding its mission, objectives, and goals. Key stakeholders (e.g. namely, the CEO, CTO, and CMO) should be able to clearly articulate what the tangible business benefits they are expecting from the cross polination of teams. Questions to be considered include:
Gartner developed a Business Value Model that can be used to establish the appropriate set of metrics. This Business Value Model is a structured framework and definition of nonaccounting metrics that can be applied generically to help organizations identify how their marketing activities will impact financial performance. Ultimately, the goals should be specific and measurable. For example, increasing online conversions by 10%, or reducing churn by 7%, or improving cross-selling on item X and item Y by 15%.
In addition to defining the marketing goals of Artificial Intelligence projects, the exercise (not experiment) has a symbiotic function: to bond the data science team with the marketing department, which align with the organization's mission, objectives, and goals mandated by the executive suite. This alignment is critical during the implementing phase and increase the probability of repeat success.
It is important to note that A.I. has often baffled data scientists with seemingly irrational decisions. In many cases, these irrational decisions often demonstrate A.I.'s ability to think intelligently (or unintelligently). In both circumstances, model interpretability is gaining importance as more organizations implement machine learning into marketing campaigns. Model interpretability is critical to understand why a particular A.I. model succeeded, or failed. It is essential to explain conclusions when results deviate from expectations. More simply, A.I. trainers need to be able to explain how an insight was derived. However, attempting to understand why a conclusion occurred, is as complex as the human brain and its impact on our behavior (outside the scope of this piece).
Recommendations:
Create a Systematic Operationalization Process
The model development cycle is a variation on the Cross-Industry Standard Process for Data Mining (CRISP-DM) methodology that has been used by data science practitioners for two decades. This methodology is simple, powerful, and imposes a development system that promotes robustness. The development cycle has always been the glamorous part of the full analytical process, which developers tend to gravitate towards.
The model operationalization cycle, on the other hand, is less glamorous but it's where business value is realized. Neglecting the operationalization cycle has often prevented organizations from fully realizing the business value of their A.I. projects, while generating distrustful attitudes toward data analytics.
Operationalization Cycle Functionality
Successful marketing teams have adopted various technologies to provide the foundation to manage operational decision services in a reliable, repeatable, scalable and secured way. That foundation should provide:
Operationalization Cycle Process
Once a model has been released from the development cycle, it enters into the operationalization cycle. This takes place in two main phases: the release phase and the activation phase. The goal of the release phase is to run a model in a live sandbox to test the model with real world data. Once the model has been tested and proven in real world samples, it is ready for full deployment in the activation phase. The goal of the activation phase is to launch the model into existing business operations identified (and validated) in the release phase of the production process.
Depending on their project scope and complexity, not all organizations undergo two phases. It is possible to integrate the release phase as part of the development cycle.
It is possible to launch a live single phase model to provide insights within a isolated campaign. However, with less testing and with greater variables, the possibilities of errors will increase exponentially.
Release Phase: Live Testing
Operationalization Phase
The operational cycle will continue to revolve as long as the performing, meeting attribution thresholds, and aligning with the campaigns objectives. As long as models are performing, they keep their place in the production cycle. Should the models require re-evaluation, the models are then recycled back into the development cycle process.
Machine learning systems carry a significant technical debt that has to be addressed upfront. To confront the problem early, the data scientists should:
- Monitor the KPIs that determine the success of analytical models in production
- Organize and manage the performance of machine learning models deployed across the marketing campaigns
- Maintain data governance to ensure regulatory compliance and data custodianship
Teamwork
Typically, data science teams work in silos. When it comes to operationalizing Machine Learning into marketing campaigns, orchestration is required between cross functional teams. These include: System Experts At the application level, system experts have intimate knowledge of the data flows and sources. System Architects Architects have intimate knowledge of the infrastructure supporting the processes, systems, and applications that will integrate machine learning models. System Administrators Maintain IT operations and enforce coherence to regulations. Application Developers The builders of product applications along with their User Interface/Experience (UIUX) counterparts. Process Engineers Process Engineers understand the complexity and middleware relationships of the various business processes leveraging analytical assets.Beyond these roles, a business translator orchestrates and communicates the overall goals of machine learning projects. The business translator could be a business savvy data scientist or inversely, a technical marketer. Business translators have a strong understanding of the technical workings and requirements as well as the high level business structure. This individual typically emerges as the common denominators to the executive stakeholders.
Recommendations
Monitor and Modify Machine Learning Models in Production
The Key Performance Indicators (KPIs) should to be revalidated while the model is in production. Market conditions can sift and consumer preferences may change, shifting the mission, goals and objectives of the organization. Although these variables can degradate machine learning models, others factors to consider are:
KPIs should also be subjected to a higher level of scrutiny across the business processes that are impacted by the machine learning decisions or recommendations. This will avoid the local minimum problem where a few decisions are optimized for marginal issues, ignoring more critical problems. Through continuous machine learning audits, refinements can be made of the model to glean insights and actionable tactics.
Recommendations
Summary of Challenges:
Lessons to learn:
To launch your organization's machine learning projects, marketing teams should:
Ask your doctor about Morfene.
We can help you discover deep insights by integrating Machine Learning processes into your marketing campaigns.