We're pleased to introduce Version 2014-R2 of Frontline Solvers for Excel, including our new XLMiner Platform, a completely rewritten and far more powerful data mining tool. If you Google "XLMiner" you'll find that it was the first-ever serious data mining tool for Excel, and that it's been around for a long time. XLMiner Platform is completely new from a software technology perspective, but it is faithful to the design of "traditional" XLMiner (and we'll extend its user interface from there). Here's the XLMiner "back story":
Ten Years of Use in Teaching and Learning Analytics
XLMiner was originally developed more than ten years ago, to bring data mining to a broad audience, by Cytel Software, a Massachusetts based firm whose primary business is advanced statistical software for pharmaceutical clinical trials. It was marketed by Statistics.com, a Virginia based firm whose primary business is offering online educational courses in statistics and data science. They've both been friends of ours for a number of years. XLMiner inspired a very popular textbook Data Mining for Business Intelligence, by Professor Galit Shmueli, Nitin Patel of Cytel and Peter Bruce of Statistics.com, published by John Wiley.
In August 2011, Frontline Systems acquired rights to XLMiner and took over development, marketing and support of the software. In September 2012, we released a much-improved XLMiner version that featured new data visualization capabilities and integration with Microsoft’s Power Pivot add-in for Excel, extending XLMiner’s data-handling capabilities. In January 2013, we released Analytic Solver Platform, a deep integration of XLMiner with its very popular Risk Solver Platform software, yielding a comprehensive toolkit for advanced analytics.
Analytic Solver Platform for Education, a special version with problem size limits suitable for teaching, has become the most popular software used to teach analytics in MBA education. We estimate that about half of all enrolled MBA students used this software in 2013 alone.
Completely Rewritten Algorithms and User Interface
Given XLMiner’s growth, we decided in early 2013 to invest in a major technology upgrade of the underlying software. As an Excel add-in, XLMiner’s user interface was written in VBA, and its statistical and data mining algorithms were written in C, based on methods current ten years earlier. Over the last 18 months, we have redesigned all of the algorithms, and rewritten hundreds of thousands of lines of code to produce XLMiner Platform, and its subset XLMiner Pro (the same software, but with dataset size limits).
XLMiner’s data mining algorithms, now written in C++, are based on studies of the latest published papers, doctoral theses and conference proceedings in data mining and machine learning, and are written to take maximum advantage of multi-core processors and modern vector instruction sets. The result is a dramatic improvement in performance (often 100 times faster or more) and capacity to handle large, complex datasets. We've benchmarked the performance of the new XLMiner against software packages such as SAS JMP , IBM SPSS and Minitab, to ensure that XLMiner could handle datasets as large as these well-known statistical packages, with similar or better performance.
XLMiner’s user interface code, now written in JavaScript, works in desktop Excel, but is designed to be ported to the Web. Because XLMiner and Analytic Solver Platform are featured in so many textbooks, with screen shots and step-by-step instructions, we have taken great care to ensure that the new XLMiner would be "upward compatible" from the old one. XLMiner’s Ribbon interface and "wizard-style" dialogs are familiar to virtually all Excel users, so its learning curve is far less steep than current "enterprise data mining" software.
XLMiner Use with Excel 2013, Power Pivot and Power Query
But can an Excel add-in really be a serious alternative to far more expensive data mining tools from SAS, IBM and others, for larger data mining applications? The ability to access and work with large amounts of data, from a variety of sources, is crucial for these applications.
To achieve this, we're leveraging Microsoft’s recent investment in data access and "self-service business intelligence,” which has made Excel much more than a spreadsheet. With Power Pivot – an Excel add-in based on SQL Server Analysis Services’ xVelocity database engine – Excel has a modern in-memory multidimensional database, easily capable of holding 100 million rows of data. The Power Query add-in, based on SQL Server Integration Services, provides an exceptionally powerful and easy to use ETL (Extract, Transform and Load) capability for this in-memory database. It can draw data from a huge range of sources, from enterprise databases on-premises to cloud-based public data stores on Windows Azure Data Marketplace.
You can easily summarize and "slice and dice" data from this in-memory database into Pivot Tables on the spreadsheet, but XLMiner can draw representative samples of the data directly from Power Pivot. As taught for many years by SAS with its SEMMA methodology and SPSS with its CRISP methodology, a data mining model can, and usually should be "trained" by using a representative sample of a larger dataset, then "validated" against a different sample from the larger dataset. This methodology is built-in and easy to use in XLMiner.
Predictive Analytics Plus Powerful Prescriptive Analytics
The world of predictive analytics is beginning to move beyond classification and prediction, seeking to make better decisions based on data and models. This is where our 20 years of development of advanced analytics software, using the latest methods for mathematical optimization, Monte Carlo simulation and risk analysis, and stochastic optimization, pays off heavily for you.
XLMiner is deeply integrated into Analytic Solver Platform, which combines all of Frontline’s industrial-strength tools for advanced analytics. For example, you can quickly apply k-means clustering to Monte Carlo simulation trial data, or use a time series forecasting model created in XLMiner for "time series simulation" -- automatically forecasting ‘sample paths’ during Monte Carlo trials.
And as you probably know, our software handles the largest and most challenging decision problems. We've featured multi-core parallel algorithms for optimization and simulation since 2009, robust optimization since 2007, and algebraic model analysis of Excel formulas since 2005 – capabilities that some vendors are just now beginning to offer in 2014. We believe that with Analytic Solver Platform, you can build, train, simulate or solve virtually any data mining, simulation or optimization model that you could create using ‘enterprise’ software tools costing 10 to 20 times more, and do it faster and more easily in Excel.