Well, this morning was rather interesting, lol. I see a post on VentureBeat saying that Mixpanel has launched a new Predict feature. Of course I go to Mixpanel to see this rad new feature, but it is nowhere to be found.
I was bummed; WTF (where) is it?
Soon after all this, I get an email from Mixpanel announcing the feature. It had a sweet gif, so I gif’d it out for you too (blog continues under gif).
When I first read the email, I was pretty blown away. This is a feature we have talked about building internally here at McGaw.io. We tend to work with some smart SaaS and e-commerce companies that want to predict acquisition or even churn.
This lead us to think about adding it as a service offering, using people’s Mixpanel and KISSmetrics data to do so. Predicting shit is always really cool, and it can be super helpful when trying to push the needle forward.
When I worked at KISSmetrics, we used Infer for our predictive lead scoring; man did it help us get focused. They took our Salesforce (SF) data, scraped the interwebs for data about the people in SF, and then looked at what was in common between the won oppty’s and the loser oppty’s. Then they gave the leads with common attributes to the winning oppty’s an A, B, C or D grade.
This helped us a lot at KISSmetrics because we knew A scored leads would convert at a higher rate than D scored leads. This totally changed the game for our marketing and sales team :)
Back to the Mixpanel Predict. This seems like a rad feature and completely in competition with Infer’s product.
Mixpanel Predict Walk-Through:
First and foremost, to use the new Mixpanel Predict tool, you have to be using the People side of Mixpanel. For those folks who only use events, you are gonna want to upgrade. The predict tool makes you use the Engage side of the Mixpanel platform, but also makes you want to use the People side, which many people don’t use. Good upgrade tactic by Mixpanel :)
To find the Mixpanel Predict tool, you need to click a special link. I wasn’t sure if this was intentional at first. In the Mixpanel blog post, their directions said to go to the Notifications section of Mixpanel and then click on the Applications section. On our first visit to Mixpanel, the Predict tool was not there. When we clicked the link from the blog post, we saw it in the application center.
Once we saw the Predict tool, we followed the step-by-step instructions on how to use it. I selected an event I wanted to predict users getting to, “Reported Prices” an event in our app Fuelzee (our activation moment).
The walk-through Mixpanel made on youtube:
https://youtube.com/watch?v=X0gGkqO3ObM
So it is relatively simple to get up and get running. After I selected the event and walked through the whole process, it said it would email me when ready. I went back to work, expecting it to take an hour or something to get the email they promised me, but to my surprise it came in about 2 minutes.
This doesn’t make much sense though. Mixpanel predicted that these people will report anyways (based on their previous actions), so why do I even need to do anything? If they are predicted to report, they don’t need the nudge, right? They are supposed to report prices regardless.
This left me with wondering why should I even send them a message. Maybe that would mess up the prediction model, and now the outcome would be different because I interfered. It is kind of like time traveling; if you predict the future you are time traveling, but if you change an action between now and then, the future might not be the same.
Oh god, now I am looping in Back to the Future stuff!
All joking aside, this is a real thing to worry about when doing prediction. By changing the flow, you can alter the path and then change the prediction (since you changed the path it had been predicted from).
Even with that aside, I wondered what lead them to that prediction. Was it a combination of events, or a combination of people properties and the events, maybe even all of the above plus the gravitational spin of the earth today?
In the email they said it was based on “Using all the Enagement Data”, but was it the events and properties?
Anyway, back to what I am getting at here :) ******
This is one super great feature: being able to predict users before they churn is a huge deal. I can now reach out to people who we predict will churn and say something, but I am still wondering what am I supposed to say. Do I give them a high five and tell them we love them, so please don’t hit the cancel button?
In the short term, I will keep playing with this feature, which only allows me to predict and act on one thing at a time, and find out what is rad. Then advise you and my other clients how to use it to its fullest power.
May 2019 update: Amplitude has also worked on their predictive analytics capability – via their Personas feature. We compare Amplitude’s Personas to Mixpanel’s Predict in the Special Powers section of Mixpanel vs Amplitude.
Looking forward to your comments and feedback below.
Hi Dan, nice write-up.
I saw this also popup in my account and played around with it. I completely agree that it would be good to know how Mixpanel comes up with this prediction as that can inform what to do with this information.
Thanks Dennis. We are still playing with it trying to figure out what to do. Mainly looking at churn metrics.
Dan thanks for the review and insights. What if you changed your focus away from the “A” grades and instead looked at the “C” and “D”. It may seem counter intuitive, but you have a larger universe to target, you have little to loss (they probably wouldn’t convert anyway), and you can be more creative and test things you wouldn’t do to those more likely to convert.
I guess the question to me is, what do I send them as a test?
Without more context on why they are a B or C I dont know what to send them :(
Rad :) – it’s been a time since I heard this word.
I remember seeing the Mixpanel email and saying to myself… oh, that looks cool. (though next time I’ll use rad :) )
But then gave it a few minutes and tried to think of a specific scenario I would like to use this, like right now.
Not knowing exactly what happens behind the scenes, it makes it difficult.
Predict is probably something I would look into when I finished optimizing the present.
Keep us posted with any experiments you run.
We have run some small tests, but not been able to have much luck since we do not know the reasons why they have the grades. Our biggest hurdle is we cannot automate on multiple things at once.
You probably:
* Don’t want to intervene for users graded A
* Send a little nudget ot B’s and C’s
* Send an incentive (so they change their mind) or else to the D’s
My 2 cents.
These are the steps we intend, but at the end of the day we already have those powers. We have Key Churn Indicators which our data science team has created. Because we know the Key Indicators we can act upon those.
This is all kind of wizard of oz’ish right?
Dan, as usual a great write-up. Without a shadow of doubt, Predict is definitely useful – but then are the numbers for A resulting finally into those conversions. If A is converting well, then they will probably convert with or without Predict.
Since predictions for A are based on “Using all the Enagement Data”, well like some of the people here posted, could be way to get back to B, C and D’s and nudge them to convert. It seems MP did not share that information, am I correct?
Yeah, MP does not share the data yet. When speaking to a lead engineer there, they said it is in the works to become more revealing someday. Right now they are just testing to see what happens.
thanks Dan, lets wait to see.
Is there a sequel of this post? Is there a write-up/blog/post where we can see results and & see how effective Mixpanel Predict has been for some folks? I am exploring mixpanel predict for predicting funnel behavior in my work but before I bet on it, I need to know if this will be helpful.
Thanks in Advance for the response :)