Discussing Analytics Model Tournaments at SAS Innovate Singapore 2023

At SAS Innovate Singapore 2023, we were told that SAS used to be a data company for expert users. However, today, it has fundamentally shifted to become a company for non-expert users.

Whilst expert users may already understand certain concepts such as a “model tournament,” non-experts likely hear that phrase and think of fashion models competing on some reality TV show.

But given that SAS is now “talking to” non-experts who want to increasingly leverage data and analytics, they took the time at the event to break down important concepts that are important for “us non-experts” to grasp.

One such example is how they broke down the concept of a model tournament, and for those non-experts out there, we decided to share how SAS described it so you too can understand.

Let’s start with the idea of a predictive model. Essentially, this is an algorithm that uses large amounts of historical and current data to predict outcomes.

Predictive models can be used to assist in fraud detection, supply chain forecasting, sales forecasting, credit scoring, and staff retention, among many others.

There are numerous types of predictive modelling techniques, such as decision trees, neural networks, logistic regressions, and isolation forests; the list goes on. Even within the modelling types, the specific construction of the model can be altered with different variables using the same model, resulting in different outcomes or predictions.

So the question looms large: Which model should you use?

The whole point of these models is to accurately identify instances such as fraudulent transactions or to predict the level of risk that a particular applicant will default if granted credit. So, it’s really important that the model you select is the most accurate one.

According to SAS, the answer is to run a “model tournament.”

You take historical data, choose a number of well-suited models, withhold some data which will be used to check the accuracy of the models, and then put them all to work.

The result will be that one model performs better than the others, consistently delivering more accurate results. This model becomes your “Champion Model,” and with the automation that SAS has built, it enables you to take your Champion Model and swiftly roll it out into your live environment.

But on speaking with Udo Sglavo, Vice President of Advanced Analytics, he warned that every model will deteriorate over time. If you take the example of fraud, he explained that the fraudsters get smart to your model and develop new ways to bypass or fool the model.

In this way, he explains that predictive models eventually affect reality. So the process is iterative and continuous, always having to build and adapt models based on the analysis of the most current data.

So, as you advance through your analytics journey, let your tournament begin!