Sunday, 7 February 2016

Will the blockchain liberate the analyst?

With the advent of the blockchain, it opens up efficiencies for customers and banks alike, not to mention a vast array of opportunities, many of which are currently unrealised!

What excites me most about this technology is that the general ledger would be by definition open for anyone to see and thus analyse, generating with it an incredibly powerful dataset available in the public domain where typically banks would have shielded for a variety of reasons.

This would then create an opportunity for analysts no longer sitting within the firewall of the big banks the possibility of generating a variety of insights for purchases of the highest bidder. It would open up also a bigger dataset given that it would be both cross-sectional and international. The possibilities seemingly endless!

It would not become the end of the in-house analyst but would certainly create a compelling extra dimension for analyst sitting outside the big 4 to sell their wares.

Thursday, 13 February 2014

Reverse engineering customer communications for analytics and model building

The average top tier Australian company has several business rules in place to prevent customers being bombarded with numerous communications leaving them confused and annoyed. Whilst generally speaking the average communication that gets sent out by these companies are generally well thought out and of a good quality, it makes sense that they have these business rules in place to prevent unnecessary customer dissatisfaction, churn, and/or unsubscribing from future communications.

I have spoken previously about conducting customer communications audits for the benefit of understanding what communications each customer is receiving in a given time frame. The reason for raising this topic again in a similar vein is the increase in technological capability of the average top tier company to store and utilise this data quickly.

Rolling up the following information into a flat file with a row for each customer counting the number and recency of each contact channel in the last 90 days can have great benefit for modeling out the effectiveness of the competing interests of the business rules versus the customer models and campaign selection criteria against key business outcomes such as:

  • churn
  • revenue per customer
  • sales
  • customer satisfaction
  • net promoter score
  • service utilisation amongst other things
Information to roll up in to the flat file are:
  • Service messages sent out to customers by channel
  • Billing
  • Inbound call centre contacts by skill group
  • Outbound call centre contacts by skill group
  • eDM's sent out with related information such as number of opens, clicks, and opt-out
  • Direct mail sent out
  • Missed payments
  • Returned goods
  • Mobile App interactions such as number of logins and length of usage
This flat file in conjunction with the customer models already in place as well as the key business outcomes you wish to improve or maintain can form a very rich source for further model development and some incredible data driven revenue implications in a very positive way.

I've seen a number of companies let this fantastic information lie by the wayside and all that does is leave great opportunities to improve your customer and business outcomes in neglect whilst your competitors take full advantage.

Thursday, 16 January 2014

Privacy and the ethical uses of customer data

Having worked as an analyst in customer insights and analytics for 8 years it may seem odd to write a blog on privacy and the ethical uses of customer data. You might say it is in the best interest of an analyst to make use of as much data as possible as there is huge monetary and customer value in doing just that. However, with market and consumer forces so strong these days it pays not to disenchant too many of the exact ones that are keeping the business alive.

Do no evil
Googles motto of doing no evil really is at the best interest of the modern analyst. The whole not to bite the hand of the one that feeds us is really at the core of the issue. And we can see this clearly with the increasing popularity of social apps with limited data collected and promises of increased privacy. In Europe, privacy is becoming more and more a concern as we can see with the NSA debacle and European leaders hitting back.

A mindset shift
So this is where we as analysts need to become more aware of community concerns and become responsive and open as to exactly how we go about using customer data. It originates in the purpose of the company; the culture at the core of it. Companies can no longer only strive to be number one or to make as much money as humanly possible. These companies unfortunately lose their appeal in the long run - it's like a one-sided relationship that is destined for derailment.

Importance of company culture and strong leadership
Strong leadership is not all about getting your employees to do exactly what you want when you want. It is about setting an example, creating a team atmosphere, and really challenging your employees to do their best. In customer analytics that is about getting the team of analysts to focus on how they can make the consumer experience better. If in doing that, more data needs to be collected, then tell the consumers how much and why and how this will benefit them. There's a lot of people out there willing to pay a bit extra for a better service. There's even more out there willing to work for a company that looks to do good.

So my motto is look to do good and provide a really great customer experience and the customers will come flocking back! Do the opposite and it may be the worst mistake you every made.

Wednesday, 15 January 2014

App usage analytics to become the new gold mines of business intelligence

As more and more Australian companies are bringing out quality mobile apps that add real value to the consumer experience, the app usage data that is collected is becoming more and more valuable both for customer experience and retention gains as well as insights and segmentation.

The likes of Myer and co who have intelligently created smartphone apps whereby they collect customer and purchase data at Point-Of-Sale (POS) and have communications triggered off the back of it are closing the business intelligence loop between what they are selling and who is it that is buying. This is really important as they can use marketing automation to keep the flame alive in 'hot leads' whilst provide some fantastic reporting on who really is buying what.

A bit about App data
App data is great because there is a lot you can capture that can tell you a great deal about the person that is using it such as the device type, the operating system, the carrier, their telephone number, what they've spent with you, usage logs and much more provided you have their permission of course. However, app data is a bit like a bank transaction account, it gets used a lot and so there's a lot of data - huge amounts of it.

Getting bogged down in storage
Many companies with this in mind spend quite a bit of time integrating this data into their data warehouse and coping with the huge volumes coming in. So when it comes to use cases, generally speaking it may be that the business will chose something simple such as - "Okay, let's just send out a trigger communication (an email) to anyone who has used the app in the last day or so with generic messaging about the latest offers." In that way they can achieve a use case quickly and efficiently (without spending too much time devising complicated models whilst at the same time not using too much server load running the procedure).

What more could you be doing?
Generally speaking, the app usage data is like every single one of your customers having a conversation with you. When they exit out of the app in less than 5 seconds they are saying maybe the app is buggy or perhaps they've logged in and realised there wasn't really much interesting in there and so they will move on to something else. However, it might only be with certain segments of your customers that they do this such as a young age group or gender. Being able to drill down and build models on this data could mean the success or failure of your app. It could mean the success or failure of your customer retention strategy. Or it could mean the success or failure of your businesses attempt to stay ahead of the curve in the digital age.

Either way, my advice is that if you have app data, delve a bit more deeply into it. And if you don't have the skills in-house, get them! Private message me if you would like to get someone in to your business today.

Saturday, 16 November 2013

What's the point of bringing data together?

So why is that we bring data together? What is the benefit in the end?
I'm asking these questions because with the advent of the digital age, and rapid development cycles, much of many companies data sources are left stranded, on their own, floating off into the ether and never to be seen again - unless you are directly concerned with that particular data source.

So what are these data sources and why should I care?
Well they could be your website and product page traffic data locked up in a third party analytics tool. It could be your app user data again locked up in a third party app solutions company. Maybe it's your media spend data? Is it your e-commerce data? Email marketing data? Facebook app data?
Whatever it is, centralising it into your data warehouse can add value and visibility across the business!


So what could I get out of centralising my data?
Firstly, it will provide a one stop shop for all your company data. This means whatever you are looking for you can find it in the Data Warehouse. Ask many an executive, business analyst, marketing manager, etc., of the times they have wanted visibility over specific activities and their cross-functional influence and the answer has been Too Hard? It's usually many, many times! It's also quite frustrating for those involved; especially when KPI's are riding hard on them.
Secondly, it gives a vast array of data for your analysts to mine and extrapolate insights from all at their fingertips. It is literally the equivalent of laying down water pipes to your house to save you from fetching your drinking water from the local well! It is of course absolutely doable to go and fetch water from the well but it is hardly the ideal situation and it will be much quicker in the long term to have everything at hand.
Thirdly, and most importantly, it's all about measurement! It's being able to know how many of your highest paying customers are using the app you created. Or whether the advertising you are conducting in certain media markets are producing a greater increase in sales versus those that are not receiving the activity.

So how does one actually bring it all together?
Well that's where you pick up the phone and bring in a Business Intelligence and Data Warehousing team in to create a data strategy and then begin the process of creating the connections between these far distant islands of data and your data warehouse.

Thursday, 7 November 2013

The treasure within

I recently watched Todd Sampson's redesign my brain series on ABC1 and I found it really cool how we all use so little of our brain and that with a little training we can increase our performance dramatically. It is fair to say that it was the opportunity and the gains to be made through some training that could result in a real impact that was most impressive to me. I guess the reason it struck a chord with me was my long held view of a similar ilk that we use only a little of the vast array of data within our companies' datamarts and warehouses and that with a little exploration and perhaps a bit of training a lot of value can be gained.

So where is it?
Largely this untapped data is contained in the transactional data that is collected every time a customer has an interaction with your company. This might be logging in online, updating information, it might be a late payment, a call into your service centre, it could be their payment timings; it could be a range of things. The key is that they are stored in transactional tables and by definition measure a behaviour of sorts.

So how can I use it?
Typically the way data modellers use the data in this sense is to link it back to the account table and rationalise the data into a flat file. This is done by aggregating the transactional data into a measure. For instance it could be the number of logins in the last 90 days, or calls into to the service centre in the last month or a yes/no flag as to whether they have made a late payment in a given time period.

From there it is about what you are looking to achieve. Is it to reduce churn? Is it about increasing sales? Is it about targeting people who are most likely to purchase? All of this data is typically held in the average data warehouse and it's about getting in touch with the right people who can give access to these tables.

I'm interested to hear about peoples experience with using data from strange sources to achieve a measurable outcome - please share if you can and any challenges faced!

Thursday, 24 October 2013

Product recommedation engine built in sql server

Hi there, yes the topic of today's discussion is a product recommendation engine built in sql server.

It seems a strange concept to most I'd imagine. Most people would only talk about such a thing in programs such as R, python or hadoop because they imagine that only these programs hold the processing power to do such things. However, the reality is that unless you hold the data of google and you are looking to build a search engine from scratch, you probably don't need hadoop. R and Python are brilliantly powerful tools and I don't discount them for a minute, however, if you come from the humble background of a sql developer/analyst, you can achieve some great results without needing to learn a whole new language even though they may be quite attainable.

Warning
So I'll warn that matrix factorization in sql requires a lot of repetitive coding. The reality is that if you also have a good grasp of excel, you can probably do all the leg work there. The key is to have your list of products that you wish to build your recommendations off as well as your customer table and your transaction table handy. With those at hand it is simply a coding issue and if you're handy with sql this shouldn't be an issue.

Use case
I've built a product recommendation engine based on 3.3 million customers and 20 products. The end result is a product recommendation on what a customer would like to purchase based on what the customer would like to purchase next. This is generated from the cohort effect (ie what other customers similar to them have purchased). It also removes the purchases they have already made so as to remain relevant.

Benefits
The benefit of the way this was developed was that it is easy to run and re-run on a as desired basis so that campaigns can be generated off the back of them. A lot of recommendations within CRM's are generated from data outside of an organisation such as census data, credit bureau data, as well as other intel collated via list brokers. This method however sources the richest form of data available to an organisation which is its own transaction data linked to its customers. Many will discount this as hard to decifer but the reality is that if you compare the r-squared values of transactional data to any other form of purchased data to the likelihood of sales, you'll find that transactional data wins out hands down.

Relevancy is key
The fact is a lot of communications coming out of organisations are promotion or product based. This means that either an offer or sale is being promoted or a new product is. So the centre of the communication is either the promotion or the product. Unfortunately, in these examples the individual consumer is not the first though, and in many examples, the individual consumer is not the thought at all.

Consumer centric marketing
Imagine a world where Google only served you the search results they wanted to show you based on what their advertisers wanted to pay for. Or imagine a world where Amazon would show you only what the highest bidder would pay to promote their latest book. This is what product centric marketing looks like and is a polar opposite to what either company does.
The reality is consumer centric marketing is key. This can be achieved in a variety of ways but I guess the most successful companies in this space have achieved this via product recommendations tailored to each and every individual customer. The reason is that this can be achieved in a fairly automated fashion.

How to make this work in your environment
The simple answer is run matrix factorisation over the products you wish to. Then summate this across the products your customers own. Next remove the products your customers own from the recommendations. Then viola, you have built your own product recommendation engine.

Get in touch for more information on how to make this work in your organisation.