Deep Learning Experiments

While learning about deep learning, I completed the online course Deep Learning A-Z on Udemy.

It included several case studies designed to progressively introduce different types of neural networks with Keras: tabular data classification, image recognition, time series and anomaly detection.

The datasets and scenarios were provided with the course. My work mainly involved preparing the data, implementing the models, testing different configurations and interpreting the results.

Predicting customer churn for a bank

The first case study focused on a bank trying to identify which customers were most likely to leave for a competitor.

The dataset contained several anonymised attributes, including customer tenure, number of products, account activity, balance and credit score. A target column indicated whether the customer had left the bank.

After cleaning the data, encoding categorical variables and standardising the features, I built a dense neural network with Keras:

• 11 input variables
• two hidden layers with 6 neurons each
• one output neuron with a sigmoid activation function

The purpose of the exercise was not only to train the model, but also to explore several ways of evaluating and tuning it:

• training and test data split
• confusion matrix
• cross-validation
• dropout regularisation
• hyperparameter tuning with Grid Search
• decision threshold adjustment

The exercise required progressively improving the model’s accuracy until it exceeded the target of 90%. I reached that target by combining these techniques.

Accuracy alone is not enough to properly assess this type of model. In a churn prediction problem, it is also important to measure how well the model identifies customers who are genuinely likely to leave, using metrics such as recall, per-class precision and the confusion matrix.

This first case study mainly taught me that evaluating a model cannot be reduced to the score displayed at the end of training.

Recognising cats and dogs with a CNN

The second case study used a dataset of 10,000 images, split approximately evenly between cats and dogs.

I built a convolutional neural network with Keras:

• one convolutional layer
• one max-pooling layer
• one flattening step
• one dense layer
• one binary output

The images were resized before being passed through the network. To reduce overfitting, I used Keras image augmentation with small rotations, zooms, translations and horizontal flips.

This first architecture performed significantly better on the training data than on the test images. I therefore experimented with several adjustments, including adding more convolution and pooling layers, and managed to improve the model’s ability to generalise.

This project gave me practical experience with several core computer vision concepts:

• convolution and feature extraction
• dimensionality reduction through pooling
• image dataset preparation
• artificial data augmentation
• overfitting diagnosis
• the influence of architecture and image resolution

It also taught me that adding layers is not automatically the right solution. Data diversity, data quality and the evaluation strategy remain essential.

Experimenting with LSTMs on time series

The third part of the course focused on recurrent neural networks, and more specifically on LSTMs.

The case study used five years of Google stock opening-price history, from 2012 to 2016. The objective was to estimate values for January 2017.

The values were normalised and transformed into sequences: for each prediction, the model received the previous 60 observations, representing roughly three months of trading data.

The network contained four LSTM layers with 64 units each, dropout regularisation between the layers and a single output neuron. It was trained using mean squared error as the loss function.

The predictions broadly follow the direction of the real curve, but two characteristics are clearly visible:

• they are smoother
• they react with a slight delay

This chart is useful for understanding the model’s behaviour, but it is not enough to demonstrate any real ability to forecast financial markets. The network only used past opening prices, without trading volume, economic indicators, news or other variables likely to affect the stock price.

The main purpose of the exercise was to work with time sequences and understand how to prepare inputs for an LSTM, not to build a trading system.

Flagging unusual profiles with a SOM

The final case study used an anonymised banking application dataset containing 690 records and 14 descriptive variables, plus an identifier.

The bank already performed a manual review of applications. There were therefore not enough confirmed fraud cases to directly train a reliable supervised model.

The chosen approach used a Self-Organising Map.

A SOM projects observations described by many variables onto a two-dimensional map. Records with similar characteristics are grouped around the same neurons, while more unusual profiles appear in areas that are further away from the rest of the map.

In the visualisation:

• the lightest areas correspond to neurons with a high average distance from their neighbours
• green squares represent manually accepted applications
• red circles represent rejected applications

The map could therefore highlight interesting cases for further review, such as an accepted application located in a highly unusual area, or a rejected application surrounded by otherwise common profiles.

However, these anomalies should not be interpreted directly as fraud. The SOM was only used to identify and flag suspicious applications for further investigation by the bank. That analysis could then be used to refine the criteria used to select anomalous neurons.

The course also suggested using these results as input for a supervised neural network that could rank applications by their estimated probability of fraud. This was outside the scope of the case study, as the limited data did not allow for sufficiently reliable training and evaluation of such a model.

What I learned from these exercises

These four case studies introduced me to several neural network architectures, but their main value was methodological.

They taught me how to:

• prepare different types of data
• build and train models with Keras
• select a loss function suited to the problem
• use cross-validation and hyperparameter search
• apply regularisation techniques
• identify overfitting
• distinguish a statistical anomaly from a business decision
• question a result even when its visualisation looks convincing

They also showed me that network complexity is only one part of the problem. Data quality, the validation strategy and error interpretation are often more important than the number of layers.

These experiments provided a first foundation before my later specialisation in data science and more applied projects involving NLP, OCR and time series.