A Weekend with Lean Startup Machine

I love Eric Ries’s book, The Lean Startup. I love the concepts of the build-measure-learn feedback loop, the minimum viable product, the pivot and innovation accounting. But how do you apply Lean Startup in the real world? How do you make mistakes, learning as you go, without endangering your own carefully nurtured idea for a startup?

That’s where Lean Startup Machine comes in. Over a weekend, you get a chance to work with 3-5 other people on a team, generating a business hypothesis, testing your assumptions by “getting out of the building” and talking to prospective customers, and pivoting (changing your hypothesis), and perhaps generating some real revenue. Most important of all, you learn a repeatable process, which you can then take back and apply to your own startup.

I attended Lean Startup Machine weekend in Seattle, June 29th to July 1st.

In just a little over 48 hours, my team and I interviewed about 25 prospective customers, pivoted 4 times, signed up 11 attendees and 1 paying sponsor for our event (with 2 more sponsors in the wings) and collected our first $2,000 in revenue. Here’s the story of my Lean Startup Machine weekend.

Friday

The evening started with an ice breaker. Everyone was asked to write their own name tag, adding an adjective before their name, and introduce themselves to as many people as possible. After 10 minutes or so, we had a contest to see who could name the most people. I met Dangerous Dan, Pretentious Paul, Kool Ken, Reticent Rebecca, Confused Cindy and a bunch of other people. About 50 people in all attended. Reticent Rebecca turned out to be anything but reticent, and she went first, naming about half the people in the room. She won by acclamation.

Next, anyone who wanted to give a 50-second pitch explaining their Startup idea joined a line. We heard at least 25 pitches, everything from an app to track lost children to a fishing application, and a number of things in between. We narrowed the ideas down to 6 through two rounds of voting, and anyone who didn’t come with an idea, or whose idea wasn’t chosen, joined a team. There were 4 teams that formed prior to the event, so we ended up with 10 teams.

After some instruction in the Lean Startup Machine Canvas and the process they’d like us to follow, the hard work began.

The Lean Startup Machine Canvas

If you’re familiar with Osterwalder and Pigneur’s Business Model Generation Canvas or Ash Maurya’s Lean Canvas, the Lean Startup Machine canvas looks quite different. It radically simplifies these other canvases to the 3 essential questions: Who is the customer? What is the problem? and What is the Solution?

You begin by documenting your answers for these three questions. On your first iteration, you may not even document the solution, because you don’t know enough to come up with a solution.

Then, you brainstorm about all the assumptions that you’ve made in your business model. These assumptions are usually around things like “Is the problem we’ve listed a real problem for our audience, one that is a migraine, not just a headache (as one of the judges described it)” or “have we targeted the right audience” or (later) “will they pay for it”. It can also include how do we reach these customers, do we have the right solution, do we have the right monetization model, etc.

Once you’ve brainstormed all your assumptions, you decide which is your riskiest assumption, the one that if it isn’t true, your whole business fails. Then, and I thought this was one of my most important learnings of the event, you set your pass/fail criteria before you begin the interviews. For example, if your riskiest assumption is that people have a certain problem, you may decide that unless 5 out of 10 people consider this to be their number one or two problem, you’re going to invalidate the assumption. This prevents retrospective justification.

If you validate your assumption, you then persevere, if not, you have to pivot.

Saturday

I thought that the toughest thing about the weekend would be to find people to interview over the weekend on short notice, especially for B to B businesses. And it was tough, but not as hard as I thought. Between the five of us on the team, we had a lot of contacts, and we also sent email to some lists, and we were able to find a sufficient number of people to interview.

Teams that had consumer facing products interviewed people at coffee shops, at Pike Place Market and in some cases, by just walking up to people on the street. This resulted in some funny stories, with at least one team talking to a couple of prostitutes about their idea for a service storing and streaming your personal DVD collection.

We started with the assumption that everyone had made a bad hire at some point in their career, and that the usual reason for bad hires was not whether someone had the technical skills to do the job, but because they were a bad cultural fit. After talking to 11 hiring managers, we confirmed our assumption, as 7 of 11 of them agreed with this. However, they were adamant that this was not their number one problem.

Their number one problem was sourcing quality developers. As one person put it, developers are the hot girl at the dance now, and they know it. Everyone wants to dance with them, and that makes it difficult to recruit the best developers, particularly if you’re a startup, and you can’t compete on salary or brand name with companies like Amazon, Facebook and Twitter.

So despite the fact that we technically passed our criteria, we pivoted, creating an event that we hoped would source quality candidates, and give both the hiring managers and the candidates a chance to evaluate each other in a non-interview setting.

Startup Curious Weekend

We had the idea of creating a weekend event, much like the one we were attending, where programmers with at least 5 years experience, who worked in big companies, could satisfy their curiosity about what it would be like to work in a startup.

The startups could send 2 or 3 people from their development team, and the programmers could work on a project or a problem for 2-3 hours with the startup devs, and they could get to know each other. Over the course of the weekend, each attending developer could work with 4-5 teams, and each team could check tens of candidates in a more in-depth fashion. Our tag line was “Try before you buy”, and it applied for both the developers, who were making a major career decision, and the hiring managers, who wanted to make a good hire.

We put up a landing page for both developers and hiring companies, and set our criteria. If we could get at least 20 developers signed up by 10 am on Sunday, we’d validate our assumption. We posted on hacker news, some local lists, LinkedIn, tweeted about it, called our friends, and generally marketed the hell out of it. Unfortunately, by 10 am, we only had 11 developers signed up. This clearly demonstrated the value of setting our criteria in advance, as we could have easily been tempted to revise our criteria. 10 developers signed up in a few hours on a Sunday morning? Sounds good to me.

Sunday

We pivoted twice more before the deadline to finish our presentations. We realized that we hadn’t spoken to developers (only to hiring managers), and so we interviewed a number of them to understand why they might not have signed up. We had assumed they wanted to work at startups. We wanted to work at startups, doesn’t everyone? Since this was a very quick iteration, we only talked to 6 developers, but 4 of them said they’d consider startups, some exclusively, some in addition to larger companies.

In addition to validating our assumption, we learned much more about what developers were looking for in an event. That caused us to tweak the event, and with that knowledge, we quickly went through one more iteration, where we tested our pricing.

If anything, our pricing is probably low ($2,000 to attend the event, and 10% of the first year salary if the company hires someone). As the customer who gave us his credit card for the $2,000 said, “If I pay $1,500 to post on job boards for a month and get nothing, why wouldn’t I pay $2,000 to attend an event where there are at least 11 qualified developers, looking for a job, where I can spend some in-depth time with them? This is a no-brainer”.

Conclusions

In the end, I learned a heck of a lot at Lean Startup Machine. I can’t compare it to other offerings, like Startup Weekend or various hackathons, since I haven’t attended these. What I can say is that no book, even one as good as The Lean Startup, can prepare you for the intense chaos of a startup. A weekend like this is the nearest preparation for that experience, and in many ways is better than many startup experiences, because it follows a process and distills the learning into one focused event.

Improvements

Lean Startup Machine isn’t perfect. Attention to detail isn’t their strong suit. From the first video, whose production values took away from the content, to the schedule (or lack thereof) to the uneven quality of the speakers, Lean Startup Machine felt more like a first or second iteration, and not an event that has been produced over a hundred times. Also, the quality of your experience will depend substantially on the quality of your team. My team was great, but a friend of mine, who attended the same event, had 50% attrition in his team, and he didn’t enjoy the weekend near as much as I did (although he still felt that he learned a lot). Even with these caveats, I’d strongly recommend Lean Startup Machine.