George Xing leads the analytics team at Lyft, where he joined as the first analytics hire and scaled the team over the last 5 years. He’s worked on problems across a number of different business areas, including growth, marketplace, operations, and finance. He spoke at last year’s Codemotion Berlin, providing a fascinating insight into the challenges of scaling analytics a transport startup to become a household name.
George shared: “When I started at Lyft, we were doing about 100,000 rides. And most of this was in San Francisco and Los Angeles because Lyft didn’t have a really big presence and we were miles behind our competitor Uber. We had mostly grown through viral marketing.”
George was responsible for a new user coupon program: “From an analytical standpoint, and the work involved, figuring out how much money we were going to offer you. So how much of a coupon how much of a discount were we going to spend on the promotions? How long we’re going to run it for? When do we pull it back? How do we change the offers? And the goal was to figure out how to do this as efficiently as possible.”
Limited Runways increases the importance of scaling analytics
In 2014 the company was facing leadership challenges and limited runway. As George shares:
“We immediately stopped spending all marketing dollars. We tried different tactics but there was no unifying strategy. And there was no goal. When I saw this, my immediate reaction was, let’s find what the growth drivers of the business are, and then focus the company around that. And so we looked at the data and we saw a really strong relationship between wait times, and pretty much every other metric that we cared about whether it was cancellations, conversion, repeat, use, long term retention, driver satisfaction, whatever the metric was, we saw a relationship with wait time.
This was the first time we had actually quantify the impact that it was having on the business. And more importantly, we figured out what the inflexion points was for 30 cities. And on the back end, we built a model for every single city that told us how much additional demand and how much additional supply (additional drives) we would need on a by week basis in order to get to the thresholds that we need to get to.”
Test, analyze, repeat
By the end of 2015, analysts noticed that the more coupons run in a particular area, the less organic signups were getting.
“We ended up running a test. And we found that now 90% of the spend was actually cannibalistic. So we pulled back pretty quickly. And then really diverted the money that we were spending on new user coupons to the driver side. The lesson here is to always be testing. And so even if something worked for you before, the world might change.”
Hire analysts for people leadership potential
According to George Lyft was basically doubling the size of their team every single year. But in the context of scaling, one of the things that was most important was finding people early on with people leadership. As George notes:
“When you’re growing really fast, you’re going to need managers. it’s almost always easier at that point to find somebody on your team to promote. And in my case, the first person I hired ended up taking on the majority of the people that we hired between seven and 12. And then the third person that I hired ended up scaling the team even more, and both of them today are phenomenal leaders managing large teams, the first person is managing a 50 person team, and then the second person, she had to move to New York where she is now the general manager of New York, managing a 100 person team.”
So even if these people are not going to be helpful for your team, they’re going to be leaders elsewhere within the organisation. And if you can find these people early on, when you know your company, your team is going to scale.
Build a machine using scaling analytics
Recruitment is a challenge for any company: How do you keep the bar high while bringing in all these people but not letting your team drown in interviews and endless meetings? How do you scale the process?
George shares: “Of course, as analysts, we applied scaling analytics to the problem. So we started measuring everything. We started measuring the conversion rates of the different hiring funnels for different channels. We started backing out how many new leads we needed to bring in per channel in order to hit our hiring goals. We measured the consistency and the quality of not just the interviewees but the interviewers. So we found out which interviewers were really good.”
The team came across a problem: “we weren’t getting the people that we needed to convert at a high enough rate.” But they didn’t know why, and the problem could have been a result of a number of factors including:
- The leads that were bringing in weren’t as high quality as first thought.
- The screening interview questions were not right
- Introducing too many steps in the process.
The subsequent approach was to apply data analytics to test as many as these problems as possible. The team had one hypothesis that pre-interview phone screen was adding too much friction to the process. So they were literally A/B testing – “half did the phone screen and half went to the next step. What we found was very quickly that the phone screen was adding value because the people that were passing or skipping the phone screen, were just getting screened out at the next step. And so this was, this is our way of building a machine.”
As a result, the company standardised the interview questions. This made all the homework objective and easy to grade. The company also adopted blind grading – removing the name, gender and other details from applications, which helps to eliminate bias and improves diversity hiring.