A paper, a post and a paragraph - 2nd August 2019

I had an idea.

There’s so much to learn, to read, to do and not enough time. I’m having to become a lot better at filtering the content I consume and that why I’ve started this series of posts: A Paper, a Post and a Paragraph.

The idea is that I aim to post about a paper I’ve read (Academic or White Paper), a post I liked and a paragraph related to some of my thoughts on current happenings in the topics I’m interested in. While my profession as a data scientist will guide a lot of the content, there may be a few odd new things here and there, like my recent interest in bread baking.

But without further ado, lets get into it.

A Paper - Deep Generative Models for Reject Inference in Credit Scoring

tldr: Authors trained a neural network that learns wheteher uncreditworthy individuals would default on loans, should they have been given credit. They obtain state-of-the-art results but the solution is unstable and not implementable.

Technical Summary

I often find myself wondering how we can bring banking to the unbanked. One of the issues that I see is around the bias in credit scoring where, over time, these models create a selection bias within the data you’re obtaining and using this data to create new models has even more inherent bias. A way around this problem is to look into reject inference, including information about individuals who didn’t receive loans.

This paper is interesting. It’s not the norm to use neural networks for credit scoring because model explainability is incredibly important. It also shows that generative models, often not used in industry, do have applications outside of altering and creating images.

This paper is mathematically heavy but the principles are simple. I’m going to focus more on the second model they built (it’s more interesting).

The authors use a variational autoencoder to learn the latent space for customer data, an autoencoder (shown above) tries to reconstruct the input data, while going through a bottleneck. What this does is force the network to learn a compressed representation of the data. In fact, the model makes use of two latent spaces, one that represents the customer’s data and another which represents the probability of default. There is an interaction between these two latent spaces which improves the performance of the model.

What this jargon means

I understand this is a lot of jargon. But what the authors have done is create a model that learns how people, that were denied credit, will behave once they’ve been given credit. This is interesting because we don’t have this information, yet we are able to infer whether they will default on loans or not. While this isn’t new, it does have state-of-the-art results.

I got very excited about this when I read it, until I reached the section on Model Implementation and Training where the authors mentioned this model is unstable and sensitive to initial weight randomisation. So no production for this type of model!

Sorry my fellow banking and insurance friends, this isn’t going to be trialed anytime soon. But this is a step forward in the practical applications of generative models in industry.

A Post - The Building Blocks of Interpretability

tldr: Researchers at google wrote a paper, then published an interactive article on image classification interpretability. PLEASE go have a look, just at the interactivity. This is one of the coolest things I’ve seen in a while.

Oh man. I don’t even know where to start with this. Researchers at google looked at how to interpret image models using the activations in the neural layers (or convolutional filters) and then visualise them.

No explanation I give here is ever going to do justice to this post. I suggest you go have a look and play around with the visualisations and prepare your brain for some seriously trippy images.

A Paragraph - Why I Started “A paper, a post and a paragraph” (and how you can too)

You’re reading this blog and you’ve read others before. But how much of this knowledge do you share with your community and how much of this knowledge do you understand and deeply retain? These two questions are also two reasons why I’ve started this. I want to engage with my community and share knowledge, my success is largely influenced by the data community (shoutout to StackOverflow) and I want to give back. At the same time, I want to stay up to date with research and retain the knowledge and one of the best ways to do this, is to try explain to someone else.

Hence the structure follows my specific goals of staying up to date with research, A paper. Staying up to date with the community, A post. A paragraph is a little different, it’s purely a space to write about whatever I feel like at the time, whether it be thoughts on bread baking, what I’m reading at this point in time or maybe even a quote that’s resonated with me.

If you decide to jump on board the train of posting even the smallest information around A paper, a post and a paragraph on social media. Tag me. Lets engage and lets create a community.

– Dan