The Hypest Validation Method of Them All🔥: Apple Testflight (#TT03)
Why Apple's beta environment is the best for product validation.
This post is part of an ongoing series about Startup Validation called Testing Tuesdays.
Gain the edge with the hacks, tools and smarts to build startups the right way by subscribing below (if you haven’t already):
Testflights are by far the hypest validation method right now. They combine user-feedback with pre-hype in a way that is hard to recreate through other channels.
Testflight, Apple's iOS Beta Environment, is nothing new, but the recent hype around it is, in what has become a weird example of accidental product-zeitgeist-fit.
It hits all the right notes of startup culture: early access, exclusivity, 'discovering the next it', and a feedback-channel that actively solicits your 'hot takes' on products.
If you're looking for proof of concept, just look at the buzz Jordan Singer was able to create with his 'App Store for Testflight Apps' (making them easier browse and discover) called Airport.
Since nearly all of them have waiting lists, it is a FOMO nightmare. But I am addicted in a way that reminds me of when I first got my hands on the original App Store. It’s just that much fun.
Almost as if by design, Testflight's second youth comes with a brand new logo. Perhaps it was meant to be. Either way, in this post we will explore what makes Testflight work so well as a validation tool for your startup.
Why Testflights work so well.
Early adopters and VCs naturally gravitate towards becoming test-pilots.
The people who spend their free time looking for the next big app, are the same people who will be interested in testing those apps early. If someone enjoys sites like Product Hunt and Indie Hackers, chances are they will enjoy test-piloting.
Where do you find sneakerheads? In line outside the sneaker store waiting to get their hands on the latest drop.
Where do you find product-hungry neo-maniacs? Wherever Testflight tickets to the latest beta are being handed out, like @AppAirport.
Beta-testers often become customers or even investors.
Scarcity is built in.
The limit on any Testflight app is 10k users. As a bootstrapping founder, that might sound like a lot, but for a massive audience of startup-loving Tweeps the FOMO is real.
To make matters worse (or better), developers can choose to make their beta 'invite-only', giving them full control over the supply-side and guest list. As with the line outside the club: The more people want to get in, the more people want to get in.
Access is the highest status-signal when you're a 40-something ex-founder on Tech Twitter.
Pre-Hype is built in.
Early adopters are vocal. They will share and promote apps on social media. They will post screenshots to signal being part of the in-crowd.
People who aren’t will want to be part of the conversation even more, and do whatever they can to move up the waiting list and gain access at launch.
Either way, people become genuinely excited about having their hands on your product. So when you do officially launch demand may already be there.
Feedback is built in.
Lastly, Testflight's is a testing tool first and foremost. Apple has added in features to make it easy for users to give feedback and send bug reports, f.i. by taking a screenshot.
Gathering feedback is an important part of building a product, and Testflight provides it out of the box.
The feedback on Testflight comes from a community of mostly builders. This makes it an extremely high-signal environment.
How to use Testflight for Validation.
If you want to use Testflight to validate your app idea, I'm assuming you already validated both problem and solution using other methods beforehand. If you haven’t, check out this Database of 52+ validation experiments and do so first.
If you have, let's dive in.
First and foremost, always provide clear instructions to your users in the 'Test Information' section of your submission, outlining the app, the purpose of the Testflight, and the type of feedback you ask from users.
Based on your idea's level of validation, your approach might change.
To Test Solution Fit.
Launch to a beta-group you control (n ≤100). Make sure they seem willing to provide feedback.
For technical feedback (feasibility), Testflight's built-in feedback system is your go-to spot.
For desirability feedback, monitor social media for mentions of your app. See if you can convince some of your test-pilots to do a 1-on-1 interview.
As with any Solution Fit experiment, your goal is to find out to what extent your solution solves the user's problem. Ask them to rank features. Ask what they couldn't live without. Ask what the alternative would be if they didn't have your solution.
To Test Commercial Fit.
To Validate Commercial Fit, you have two options.
Launch a public beta (limited to 10k users). A public beta is good for:
Testing paid go-to-market strategies
Platform building
One-to-many Interactions
Accessibility
Launch an invite-only beta. An invite-only beta is good for:
Testing organic go-to-market strategies
Community building
Close interactions
Exclusivity (Luxury software, memberships, ...)
Always incentivise people to move up the waiting list. The amount of hurdles a user will take to gain access is a good measure of the value your product has.
To Test Product-Market Fit.
Even though I would not advise a Testflight to test product-market-fit, sometimes it just happens.
The hype behind an app like Clubhouse is so substantial there is hardly any question how the market will respond after they officially go live on the App Store.
That being said: by allowing invitees only, you can control cohorts, and get small scale data on retention, MAU / DAU, etc, which could give you an early indication of P-M Fit.
However, a successful Testflight is by no means a sign you can lean back and relax. You're not making any revenue yet.
What if my app doesn't create a buzz?
If your app isn't the talk of the town yet, don't bail on your startup. In this series we focus on validating your ideas – Hype isn't a part of the equation.
Don't mistake a distribution problem for a product problem.
Even though organic growth is a welcome sign of desirability, the goal here is to talk to the users you do have, and find out what they like and dislike about the product.
Once your product is so good people start asking how to pay you for it, you can worry about distribution.
If you want to learn more about the specifics launching on Testflight or becoming a test-pilot, check out these resources:
How to become a Test-Pilot
How to submit your App to Testflight
Thanks for reading! 🙏🏻
Want to get the tools, hacks & smarts to build startups the right way?
Subscribe to get my newsletter straight in your inbox:
If you feel the need to reciprocate, feel free to share, leave a like, or buy me a coffee ❤️.