How to Flail at Doing a Startup, Part 2: Product/Market Fit Update

There’s product market fit in theory, and product market fit in practice. In my experience, two different things.

I launched Quickapply once in November, with a run of flyers at UCSD. I launched again this month, February 2020.

I got only 5 customers with the flyers in November, which was discouraging. I doubled down on user interviews, because I realized I wasn’t learning from the launch. I did maybe 15, but many of them were a waste—I needed to interview a specific type of internship seeker, not just any internship seekers.

Overall, that first launch, not very handy.

In January, without any effort, the flyers that were still up, generated ~10 customers in January. This helped me want to pick up the project again. So I did.

I planned a different launch for February. I gave it more thought, and I decided to track the metric Superhuman used for product market fit: the number of users that would be “very disappointed” if your product disappeared.

As opposed to, “somewhat disappointed” or “not disappointed”.

This metric makes the state of product market fit cut and dry. This is fantastic. It might be because I need to work on user interviews, but I rarely felt certain about the state of product market fit through doing interviews. Perhaps I needed to bump up my interview count to 100 or something.

In any case, the minimum number of survey responses to have a usable Superhuman metric is 40. So I needed to get 40 customers to fill out a product market fit survey after using the launched product.

There were a few other things I did differently about this launch. I came to these conclusions mostly with help from my friend Justin.

1. I wanted to talk to customers over Facebook.
   
   Previously, I talked to them over email, which was terrible for response rates to my questions, and just left me feeling blind. I had to instead ask questions to “potential” users in my friend circles, which gave me that bunch of useless interviews.
   
   Over email I also found it difficult to establish rapport with the customers I got, which is necessary when you want to talk to them. Facebook, I thought (and found) would help with that.
2. I wanted to talk to customers earlier in the funnel.
   
   Some people were dropping off before filling out the form, I knew it. How would I figure out why they were dropping off? I solved this by having them message me on Facebook to get the form. I’d establish rapport with whoever messaged me (mutual friends, what year are you, what’s your experience), then if they didn’t end up finishing the form after a few days, I’d send a message to figure out why.
   
   I used the following script:
   
   Hey X, it seems like you’ve been busy and haven’t found time for the form yet, is that right?
   
   And they’d either confirm, or correct me. I might prod them a bit more too.
3. I needed to recruit users manually to get to 40 users.
   
   If a traction channel gave disappointing results, I should just ignore that. It’s a false negative. It might just be the traction channel sucks, it says nothing about the state of product market fit.

And so I launched with this plan. I posted on the UCSD CS Facebook group, my friend Justin on the Cal page, my friend Ayush on the UCLA page. Here are the stats for the last week:

To be clear, the above image is a bunch of vanity metrics. I got tons of pageviews, but most didn’t convert to anyone messaging me for the form.

Upcoming numbers are approximate, didn’t get a chance to update my spreadsheet:

As of today, I have ~30 potential user chats. 8 of the 30 completed the form, 3 finished the product market fit survey. The others dropped off, so I used the script up there to uncover their objections / nudge them. A good number procrastinated on filling out the form. I heard “too busy”, I heard “midterms”, I heard “lazy”. Some of these people seemed like they would proceed to fill out the form, the ones that said oh, I meant to, let me get on it rn. Others said, oh, maybe later. The way to improve this is to either make the form easier, or make the product’s value go up.

Price was an objection for a bunch, cuz for some reason it wasn’t clear it was a paid thing. At least four dropped off cuz price. $19 was too steep for them to pay, so they ended up starting to apply to the companies on their own.

Got a bunch of other small feedback: one question on form was confusing, a few people were worried their stuff was actually being applied to, was annoying to select all the companies, etc.

I’m giving trivial details though. What’s most noteworthy:

The 3 users that submitted the PMF survey so far all marked “somewhat disappointed” if Quickapply was gone.

Hmm. My intuition says the majority of the people that I get to fill out the form will be “somewhat disappointed”.

I haven’t had the time to analyze the supporting questions on the PMF survey to see how i can boost this number. But I’ll analyze that later, it’s getting late.

But this initial result is a bit discouraging…