User:Suburbadad/sandbox

= AutoAI = AutoAI uses artificial intelligence and machine learning to automate data preparation, model development, feature engineering, and hyper-parameter optimization to build model pipelines best suited for specified use cases.[1] In September 2019, it won the won the Best Innovation in Intelligent Automation Award at the AIconics AI Summit in San Francisco.[2]  AIconics defines intelligent automation (IA) as the combination of artificial intelligence and automation, enabling companies to achieve unparalleled productivity and efficiency by dissecting and synthesizing vast swarms of data to automate entire workflows and processes.[3]

The Automated Machine Learning and Data Science Team (AMLDS),[4] a small team within IBM Research, which was formed to “apply techniques from Artificial Intelligence (AI), Machine Learning (ML), and data management to accelerate and optimize the creation of machine learning and data science workflows,” is crediting with developing AutoAI.

History
In August 2017, AMLDS announced that they were researching the use of automated feature engineering to eliminate guesswork in data science.[5] AMDL members Udayan Khurana, Horst Samulowitz, Gregory Bramble, Deepak Toraga, and Peter Kirchner, along with Fatemeh Nargesian of the University of Toronto and Elias Khalil of Georgia Tech, presented their preliminary research at BYU that same year.[6]

Called “Learning-based Feature Engineering,” their method learned the correlations between feature distributions, target distributions, and transformations, built meta-models that used past observations to predict viable transformations, and generalized thousands of data sets spanning different domains. To address feature vectors of different sizes, it used Quantile Sketch Array to capture the essential character of a feature.[7]

In 2018, IBM Research announced Deep Learning as a Service, which opened popular deep learning libraries such as Caffe, Torch and TensorFlow, to developers in the cloud.[8] AMLDS continued their work and used it in a top Kaggle competition.[9] It finished in the top 10 percent.[10] Jean-Francois Puget, PhD, a distinguished engineer for machine learning and optimization at IBM, who entered the competition, decided it was ready to be a capability for IBM AI and data science platforms like Watson Machine Learning.[11] In December of 2018, IBM Research announced NeuNetS, a new capability that automated neural network model synthesis as part of automated AI model development and deployment.[12]

In “A Formal Method for AutoML via ADMM,” a May 2019 Cornell research paper (updated in June 2019), authors Sijia Liu, Parikshit Ram, Djallel Bouneffouf, Deepak Vijaykeerthy, Gregory Bramble, Horst Samulowitz, Dakuo Wang, Andrew R Conn, and Alexander Gray proposed a method for AutoML that used  the alternating direction method of multipliers (ADMM) to configure multiple stages of a ML pipeline, such as transformations, feature engineering and selection, and predictive modeling.[13] This was the first recorded time that IBM Research publicly applied the term “Auto” to machine-learning.

AutoAI: The evolution of AutoML
2019 was the year that AutoML became more widely discussed as a concept. “The Forrester New Wave™: Automation-Focused Machine Learning Solutions, Q2 2019,” evaluated AutoML solutions and found that the more powerful versions offered feature engineering.[14] A Gartner Technical Professional Advice report from August 2019 reported that, based on their research, AutoML could augment data science and machine learning. They described AutoML as the automation of data preparation, feature engineering and model engineering tasks.[15]

AutoAI is the evolution of AutoML. One of AutoAI’s principal inventors, Jean-Francois Puget, PhD, describes it as automatically performing data preparation, feature engineering, machine learning algorithm selection, and hyper-parameter optimization to find the best possible machine learning model.[16] The hyper-parameter optimization algorithm used in AutoAI differs from the hyper-parameter tuning of AutoML. The novel algorithm is optimized for costly function evaluations such as model training and scoring that are typical in machine learning, enabling rapid convergence to a good solution despite long evaluation times required in each iteration.[17]

[1] “AutoAI Overview,” IBM Watson Studio Documentation, 11 Oct. 2019.

[2] Max Smolaks, “AIconics Awards San Francisco 2019: Winners Announced,” aibusiness.com 24 September 2019.

[3] “Best Innovation in Intelligent Automation,” AIconics, 15 Oct. 2019

[4] Automated Machine Learning and Data Science [AMLDS Team ], IBM Research.

[5] IBM Research, Thomas J Watson Research Center, “Removing the hunch in data science with AI-based automated feature engineering.” 23 Aug. 2017.

[6] Udayan Khurana, Horst Samulowitz, Fatemeh Nargesian (University of Toronto), Tejaswini Pedapati, Elias Khalil (Georgia Tech), Gregory Bramble, Deepak Turaga, Peter Kirchner “Automated Feature Engineering for Predictive Modeling.” 2017, p. 1

[7] Ibid, p. 24

[8] Bishwaranjan Bhattacharjee, Scott Boag, Chandani Doshi, Parijat Dube, Ben Herta, Vatche Ishakian, K. R. Jayaram, Rania Khalaf, Avesh Krishna, Yu Bo Li, Vinod Muthusamy, Ruchir Puri, Yufei Ren, Florian Rosenberg, Seetharami R. Seelam, Yandong Wang, Jian Ming Zhang, Li Zhang, “IBM Deep Learning Service,” Cornell University, September 18, 2018.

[9] Sanyam Bhutani, Interview with Twice Kaggle Grandmaster: Dr. Jean-Francois Puget (CPMP), Hackermoon, 25 September 2018.

[10] TrackML Particle Tracking Challenge, High Energy Physics particle tracking in CERN detectors, Leaderboard, 2018.

[11] Julianna Delua, “AutoAI wins AIconics Intelligent Automation Award: Meet a key inventor.” IBM Big Data & Analytics Hub, 25 September 2019.

[12] Cristiano Malossi, “NeuNetS: Automating Neural Network Model Synthesis for Broader Adoption of AI,” IBM Research, 18 December 2018.

[13] Sijia Liu, Parikshit Ram, Djallel Bouneffouf, Deepak Vijaykeerthy, Gregory Bramble, Horst Samulowitz, Dakuo Wang Andrew R Conn, and Alexander Gray, “A Formal Method for AutoML via ADMM,” Cornell University, (Submitted on 1 May 2019 (v1), last revised 10 Jun 2019 (this version, v2)), 1.

[14] Kjell Carlsson, Ph.D. and Mike Gualtieri, The Forrester New Wave™: Automation-Focused Machine Learning Solutions, Q2 2019.” Forrester, 28 May 2019, 1, 8

[15]  Carlton Sapp, “Augment Data Science Initiatives With AutoML.” Gartner Technical Professional Advice, 30 Aug. 2019, 1

[16] Julianna Delua, “AutoAI wins AIconics Intelligent Automation Award: Meet a key inventor.” IBM Big Data & Analytics Hub, 25 September 2019.

[17] “AutoAI Overview,” IBM Watson Studio Documentation, 11 Oct. 2019.