Our strategic review May 2014

NOTE: This piece is now out of date. More current information on our plans and impact can be found on our Evaluations page.

---

Introduction – where have we come from?

Less than two years ago, we were simply a group of student volunteers aiming to have the biggest possible impact. We thought we had some powerful ideas, which had caused some people to completely change their careers. But we didn’t know how to turn our ideas into a high impact organisation. We were doing a mixture of campaigning, community building, research and one-on-one coaching, and were unsure where to focus.

Since then, we have focused our model, tested several prototype programs (online content and coaching) and gained an initial proof of concept by showing these can change careers. we also think we’ve had enough impact by changing career plans to justify our costs and have overall been a high impact use of resources.

In light of these milestones, this document explains how our strategy has changed over time and where it stands now. It is part of our annual review.

Summary

We intend for the next year to be the final year of our ‘discovery phase’. The aim of the discovery phase is to identify the most high potential business model. (By ‘business model’ we mean the combination of programs, promotion, team structure and financing strategies we use to have a social impact).

Our key focus will be on improving the quality of our prototype programs, in order to test some of the most important remaining uncertainties in our model.

Within this, we have two aims. First we’ll focus on research to deepen our knowledge of how to choose the most high impact careers. We think further research is likely to be valuable, both because this body of knowledge is neglected but highly important, and because we think high quality research is crucial to the appeal and impact of our programs. High quality research is the most important way for us to become more credible (therefore able to persuade more people), it’s the key factor that sets us apart from our competition in attracting users, and it’ll ensure we highlight careers that are genuinely better than those our users would have taken otherwise (a key uncertainty in our impact evaluation). At the same time, we’re unsure how rapidly we’ll be able to make progress on research. Focusing on research for the rest of 2014, tracking our progress and submitting ourselves to an external research evaluation will reduce our uncertainty about the value of further research.

Second, we’ll focus on improving our online content. We’ll expand our research page into a series of five, covering (i) the key criteria we suggest for comparing between careers, (ii) ranked lists of promising careers and causes (iii) supporting career profiles (iv) advice on how to find a career that fits and (v) a step-by-step planning process. We’ll also add pages to guide users to the best of our old content.

We’re focusing on online content because we think our online content has more potential for impact than coaching. In our plan change analysis, we found over 30 of the 107 significant plan changes were attributable to online content alone. This would make the online content similarly important to the coaching in terms of changing careers. Given that online content is also far more scalable than coaching, it makes sense to prioritise it to test the hypothesis that it’s a better program. Moreover, developing the online content involves the least additional work over just doing research, and we think our current offering could be significantly improved relatively easily through better summaries of our existing content.

We’ll continue with our one-on-one coaching as part of our research process. We’ll deepen our knowledge of social impact careers by doing rounds of coaching talented, altruistic, young people, then writing up answers to their most pressing questions for the blog. We’ll also write career profiles covering the careers our coachees most want to know more about.

In 2015, we plan to start our ‘execution phase’, in which we aim to realise our model’s full potential for impact. We intend to initially focus on making our online content easier to use, most likely by developing it into an online careers guide, while continuing with research. The aim is to have the careers guide in place before the promotional campaign accompanying the launch of Will MacAskill’s book on effective altruism in August 2015. After that, we may increase our outreach work to connect with our entire target market.

In the longer term, we’ll aim to develop further programs to deepen engagement with our users, such as expanding the coaching service. Our aim is to become the best source of advice in the world for the most talented, altruistic young graduates, enabling them to best use their 80,000 hours to solve the world’s most pressing problems.

---

How has the strategy evolved?

July 2012 to November 2012 – from a campaign to a service

In July 2012, we started working full-time on 80,000 Hours. We thought there were many things we could do to help people have dramatically more impact with their careers, but we weren’t sure where to focus our efforts. In the first six months, among other achievements, we developed from presenting ourselves as a campaign to a service. We decided to focus on providing useful programs for our audience, like online content, coaching and a community of like-minded people. This was rather than focusing on campaigning for our basic ideas, like earning to give, counterfactual reasoning and cause selection.

Nevertheless, research, coaching and building a community are very different activities which require different financial models, skills and evaluation processes. We had sprawling research plans, including: career and cause profiles, profiles on the ‘five types’ of social impact career (earning to give, influencing, improving, research and innovating), guidance on strategic questions like what discount rate to use in career decisions, decision making tools and guidance on how to evaluate which careers would fit you. This was far more than we could do effectively.

December 2012 – May 2013 – greater focus on research and coaching, and the introduction of the case study model

In the second six months, we reduced our focus on community building. we reasoned that the best way to build the community was to develop useful research and coaching. this would be aided by our already established online network.

We also worked on improving the focus of our research plans and building prototypes. We decided to produce a couple of pages introducing our core concepts and criteria for comparing careers, followed by career rankings. This would be supported by a one-on-one coaching service for the most high potential individuals. Our introductory content included three video lectures introducing our core concepts. We created a step-by-step coaching process, based on a standard career plan.

Despite this increased focus, we realised that we needed an even better process to prioritise research. Career decisions can be informed by many types of information, and we expected some types to be far more useful than others. To solve this problem, we developed the case study model, in which we work on the questions that are most relevant to the decisions arising in our one-on-one coaching. The overall structure of our programs would be several research pages summarising our key findings, and a one-on-one coaching service that would drive the research from the bottom up. An overview of our business model from this time is available here.

June 2013 to present – initial proof of concept

Throughout this process, our audience continued to grow, read our new online content and request coaching, showing interest in what we offered. strong membership growth was suggestive of high cost-effectiveness (as examined in our first impact evaluation), and we knew there were individual cases of significant career change. however, we didn’t have detailed information about how we were changing our users’ career plans. in july 2013, this became a key priority.

We started by carrying out a pilot of our coaching service with a new impact evaluation process. We recorded our coachees’ plans before and after, and if they shifted their credence in a career option by more than 20%, we counted it as a ‘significant plan change’. We [carried out 18 in-depth case studies, finding that 56% made significant plan changes. We carried out 18 short one-on-ones, which only involve a single meeting and no original research, finding that 39% made significant plan changes. You can see the full results in our coaching evaluation.

Our next step was to perform a complete historical impact evaluation in terms of significant plan changes (our review of program performance gives an overview, and is supported by an in-depth investigation of the impact of the significant plan changes caused). In total, we collected 107 cases of significant plan changes since our founding. Over 30 said that using our online content was the only cause of their changes. Moreover, the evaluation showed the value of these plan changes probably justifies our historical costs.

Taken together, we think these results are proof of concept that our body of knowledge about social impact careers is novel and compelling enough that it can cause our users to make valuable career changes. Moreover, we think they provide proof of concept that our prototype online content and coaching programs can change careers. Overall, we think there’s sufficient evidence to warrant further investment in developing our programs.

With initial proof of concept established, the next question was where to focus in the future. Two key decisions were:

1. Should we spend more time deepening our knowledge, or improve how we communicate what we already know through our programs?
2. Should we spend more time on our online content or coaching?

These questions are important partly because a coaching-focused organisation would more naturally be funded by users than a research-focused organisation. These two types of organisation would also require different skills and evaluation processes.

We decided to focus on deepening our knowledge of careers and developing online content, although we’ll continue with the coaching as a key part of our research process. Our reasons for this decision are given in the next section.

Turning to the online content, we improved our plans and developed well received prototypes of two of the most important parts: the cause list and the careers list.

Our strategy for the coming years is explained in the next section. You can see a complete overview of our current business model here.

Where are we going?

What will we provide our users with?

Our work has two components. First, we do research to build a body of knowledge about social impact career choice. Second, we create our programs, which deliver this knowledge to our users. The two are connected, because we use our programs to find out which research questions are most in-demand, and to gain feedback on the usefulness of our research by tracking the significant plan changes it causes through our programs.

Our programs

For the next year, we’re focused on our online content, though we will continue to deliver coaching as a core part of our research process.

Online content

Our online content is summarised on the research pages, and hosted on the blog. The next version of the research pages will have a summary page on each of the following five types of content:

• Six criteria for comparing career options – influence, cause effectiveness, career capital, fit, discovery value and keeping options open.
• A ranked list of careers , with supporting career profiles focusing on analysing the paths using the six criteria.
• Cause effectiveness, including a ranked list of causes in support.
• How to find a career that fits you as an individual, including our work on how to find a job you love.
• A step-by-step planning process, tying together the rest of our content, based on our standard career plan.

We’ll also add an archive page overviewing the remainder of our best research.

When we change our focus to making our programs more usable, it’s likely we’ll turn these pages into a in-depth careers guide, perhaps supported by online tools such as a careers test.

One-on-one coaching

Our process will continue to work as follows:

• We select the most altruistic, high-achieving and impact-minded individuals from those who apply for coaching on our website.
• The individual fills out a career plan, containing their expected cause, career path and next steps.
• We meet to clarify their plan and greatest uncertainties.
• We write a brief report giving our current best answer to their career questions, linking them to the most relevant additional resources.
• We upgrade a small proportion to full case studies, for which we do ten hours of novel research.
• We introduce them to other coaching alumni when relevant.
• We meet again to discuss the findings and create an action plan.
• We finish by asking for feedback and collecting information on how their plans have changed. We may follow up further by e-mail.

Community

By making introductions through coaching and having an online members directory, we’ll keep building a community of people who want to make the most difference through their careers.

What will our research process be?

We’ll deepen our knowledge of careers by coaching people, seeing which issues their decisions depend on, and then writing up answers to the most generally applicable, pressing questions we identify. We have a list of prioritised research questions, which we update after each round of coaching. We’re also developing a template careers profile, and will aim to complete profiles on the most asked about careers.

In answering the questions and writing the profiles, we’ll focus on collating existing materials to provide action-oriented recommendations. Most often, this involves identifying relevant experts and interviewing them. We vet our work by seeking feedback from our research advisors, and making all of our work public, with a clear description of our research process.

We’ll present our overall findings on how to choose the most high impact careers on our research pages, and then we’ll submit these for external evaluation. We’ll explain this process in an upcoming post. Individual research blog posts will also be evaluated using our blog quality rubric.

Who is our target market?

In the US alone, 1.8m students graduate every year.¹ At least 30% of these seem to be highly concerned with making a difference.² Our target market is the 10% most high-achieving and analytically-minded among this group in their twenties, which is about 500,000 people in total, with 50,000 new graduates added each year.

We’ll initially focus on satisfying the most engaged subset of this market: active members of the effective altruism community. By doing this, we can build a body of knowledge, credibility, a strong team and a strong base of donors before expanding into our full target market. It will also mean we’ll contribute to building the effective altruism community, both through helping the existing community and helping it to grow. We think that building great programs for effective altruists will be among the best way to attract new people to the community.

We think the majority of the research will still be relevant to people outside the effective altruism community, and careers advice is generally poor (as explained in our policy paper), so this initial focus will not prevent us from appealing to our full target market later. For similar reasons, in the long term we’ll also have the option to expand our focus beyond our target market (for instance, appealing to less altruistically motivated people or expanding the age range).

What will we focus on over the next year?

We hope the next year will conclude our ‘discovery phase’, during which the key aim is to identify the most high potential business model. From there, we’ll move into our ‘execution phase’ in which the aim is to deliver the model to its full potential.

So far, we have clarified our model, shown we can gain an audience, built several prototype programs, and shown that these can change careers. Now, we think it’s most important to focus on building a better prototype program. To that end, over the next year, we’ll focus on:

1. Deepening our knowledge of social impact careers through research.
2. Developing a better online content prototype.

We think it’s most important to deepen our knowledge of social impact careers because:

• We think this body of knowledge could be highly valuable. There are thousands of highly talented, altruistic new graduates each year, but no-one has systematically researched what career choices they should make to maximise their impact. Due to this neglect, it could also be easy to make progress, and given our skills and resources we think we’re well placed to succeed.
• Improving our knowledge has the potential to make our programs much more effective and attractive
  • Providing better information is the key source of value in our programs and what makes us different from other careers services.
  • It’ll help us better design our programs. We’re still uncertain what the crucial considerations will turn out to be, which makes it difficult to structure our programs. For instance, if individual considerations turn out to be much more important than general lessons, we’ll favour coaching over a careers guide.
  • We think high-quality research is the most important way for us to gain credibility. Credibility is one of our most important weaknesses, and gaining more can multiply our impact.
• At the same time, one of our key uncertainties is how rapidly we’ll be able to make progress on research. Focusing on research for a year, monitoring our progress and submitted our work to external evaluation, will help to reduce this uncertainty.
• A key uncertainty in understanding our impact is the value of the careers we recommend compared to what our users would have done otherwise. We can also reduce this uncertainty through more research.

We don’t want to deepen our knowledge without testing it in a program, and without putting it in the public domain for feedback, so we want to continue to develop our programs. The question is whether to focus on online content or coaching. We decided to prioritise the online content, though we will continue with the coaching as part of our research process. This is because:

• Publishing online content involves less additional work over just doing research to deepen our knowledge.
• We think the online content is a more high-potential program. It’s far more scalable, and so far it seems to have been as important as coaching for changing careers, with similar levels of investment.
• Online content driven by research is more naturally funded as a non-profit, and continuing as a non-profit better keeps our options open. It seems much harder to switch from for-profit to non-profit status than for a non-profit to develop a for-profit arm.

Alternatively, we could have decided to focus on outreach. We explain why we didn’t in the appendix. We could have also focused on fundraising, team building or organisation building. Fortunately, due to investments made over the last year, we don’t need to make these our focus, although they remain important priorities.

Execution phase, 2015 and beyond

In the execution phase, our options on where to focus will be:

• Research to deepen our knowledge of social impact careers.
• Making our programs easier to use.
• Outreach.
• Enabling goals – fundraising, team building, organisation building.

Our best guess is that the initial priority will be making our online content more usable, while continuing to deepen our knowledge. Our key goal is to have high quality online content in place by August 2015, in order to benefit from the potential interest generated by the promotion of Will MacAskill’s book on effective altruism. We envisage developing our online content into a careers guide, with supporting online tools like a careers test.

We’ll continue to monitor the extent to which we should focus on outreach. We suspect that like GiveWell, it may yield higher returns to focus on improving the quality of our programs for some time, and relying on word of mouth and promotion from other groups. Still, we can easily see ourselves increasing outreach and accelerating the extent to which our target market is aware of our programs.

In the longer term, having gained this audience and a strong body of research, we’ll aim to develop further programs to deepen involvement. For instance, we could train a career coach in every major university, who will foster a local community of people aiming to make the most difference.

Schematic summary of our progress

Discovery phase – work out what model to pursue

1. Decide which programs to provide. ? We decided on online research supported by coaching.

2. Test that our initial prototype changes careers. ? We just finished this.

3. Improve our prototype programs and test remaining key uncertainties ? We are here.

Execution phase – deliver the model

1. Increase the quality of our programs.
   • Make the programs easier to use (e.g. develop the online content into a careers guide).

   • Deepen our knowledge of social impact careers through research.

2. Engage in outreach with our entire target market (content marketing, networking and events).

After this, we could develop further programs, such as training a career coach to go to every major university.

---

Appendix – Key strategic decisions

In this section, we list some of the key strategic decisions we face, and outline our current stance on them.

To see more on the evidence behind our model, see our overview of our business model.

1. Should our focus be online content, community, coaching or campaigning?

We decided to focus on online content, as explained in the body of the document.

Why might we be wrong about this? Some plausible reasons include:

• Providing information alone won’t suffice to significantly change careers (e.g. because people struggle to apply information to themselves or are constrained by other factors, like motivation or accepting certain big ideas), which might favour coaching or campaigning.
• Coaching could be easier to turn into a paid-for product, which would enable us to reach a greater scale more quickly in the long-term. More generally, it could be highly important to get feedback on the value of what we provide, and it’s difficult to get this for research.
• It could be far more important to work with a small number of potentially very high impact people, which would favour coaching.
• It could be far more important to provide highly individual advice, which would favour coaching and community.

2. Should we focus on deepening our knowledge or making our programs more usable?

We explained why we think we should focus on deepening our knowledge of social impact careers above.

3. What online content should we focus on providing?

Our current answer is outlined above. each element is designed to perform a key function that our users have demanded:

• A broad list of key criteria, to help with comparing options.
• The ranked lists of causes and careers, to help people generate concrete, promising options.
• The information on fit, which helps users to narrow down these options.
• The step-by-step planning process, which helps people to act on the advice.
• The career profiles and case studies which arise from coaching.

Prototypes of each have been tested in our coaching, and proved useful in changing career plans. We think they will also be useful to most of our target market.

We think the main weakness of this plan is that it may still be too unfocused. It may also not be systematic enough, which will make it hard to delegate. So over the next year, we’ll consider dropping one of the types of content, and developing a standardised form of career profile that is easier to delegate.

4. How should we balance outreach and improving our programs?

We think we should focus on improving our programs, because:

• It’s likely that improving quality will help us spread by word of mouth, which has been our most important form of promotion so far. Improving the quality of our research is potentially a more effective method of outreach than doing outreach directly (this is what GiveWell found).
• We think that in a year we could make the online content far better, and this would greatly amplify the returns from future outreach efforts.
• We expect to gain enough promotion through word of mouth and from other groups to gain enough feedback about the quality of our programs. In particular, Will MacAskill, will be focusing on outreach through CEA’s outreach project in the lead up to his book launch.
• Developing further program prototypes and deepening our knowledge of social impact careers does more to reduce uncertainty about our business model. We’re already relatively confident we can get people interested in our programs given our successful outreach record.

5. How should we fund 80,000 Hours?

We’re currently philanthropically funded by two sources:

1. Two thirds by members of the effective giving community (impact-focused donors who follow groups like GiveWell and Giving What We Can).

* About half of these are former coaching alumni.
2. One third by other philanthropists.

One disadvantage of philanthropy is that it’s less scalable than charging users. We can partly overcome this by seeking funding from former users of our programs – a source which will grow as we scale.

Another potential disadvantage of philanthropy is a weaker connection between income and providing an effective program. The criticism is that the people who best understand whether we’re providing good programs are our users rather than our donors, thus funding the program by charging users enables a better connection between success and income. However, we think that this is less of a problem for us because our donors are relatively impact-driven. Indeed, we think it’s plausible that our donors offer better feedback than charging users, because they can use systematic evidence of impact, whereas users could pay us for providing a high-status or fun program, rather than a high impact one.

Overall, at our current level of scale, philanthropy suffices and has proven a good funding method. It hasn’t consumed too much time, we’re fairly financially secure, and it has let us focus on developing and testing our programs, which we think is the top priority.

6. Should we charge for our programs?

We think there are two main reasons to charge for our programs. First, charging could be an effective way to learn whether we’re valuable to our target market, which should correlate with helping our users have more impact. Second, a for-profit approach has the potential to make us more scalable and financially secure in the long-run. For these reasons, we may experiment with adding a suggested donation to our coaching service.

Still, charging for our programs is not a priority. We prioritise impact evaluations over market feedback, because we think it would be possible to produce a product that people will pay for, but which doesn’t cause them to make valuable plan changes. So, although market feedback would be valuable, it’s not the most valuable type of feedback. In addition, we think further research and improving our online content are our most important priorities, and these activities are relatively difficult to monetise.

7. What’s our model organisation?

We think the closest model is GiveWell applied to social impact career choice rather than charitable giving. But notable differences exist too, like the fact that career decisions depend on the individual’s attributes in a way the best giving opportunities don’t. This is one reason why we have a coaching component to our research process, which doesn’t resemble GiveWell.

An alternative model is Ycombinator, which incubates high potential startups (equivalent to our coaching), and uses what they learn to improve practice among startups in general.

8. Is our lack of job experience a problem?

We’re not much older than most of our target market, and don’t have much more career experience than them. But we don’t think this is a significant problem. That’s because we specialise in strategies for maximising and assessing options in terms of social impact. We don’t specialise in having in-depth knowledge of what certain careers are like.

Moreover, in-depth knowledge of what certain careers are like is less valuable than it first looks. This is because the labour market changes rapidly, and it’s easy to overgeneralise from your own important experiences. Youth can be an advantage too, because it means we have a deep understanding of our target market.

When in-depth knowledge of specific careers is required, we can interview experienced people and seek experienced advisors. We can also let users gain advice dependent on career experience from conventional advisers and mentors.

Most importantly, our evaluation shows that we’re already improving the impact of our users.

9. How quickly should we grow?

We’ve decided to prioritise building an outstanding team over immediate growth, because we think this maximises our chance of having impact in the future.

We also decided to focus first on conducting impact evaluations, so we can gain confidence that we’re having a real impact.

10. How broad should our target market be?

We’ve decided to focus on satisfying our early adopters, who are largely members of the effective altruism community. From there, we’ll expand into our full target market as explained before.

11. What’s our relationship to the effective altruism community?

We think it’s highly important to build the effective altruism community. Currently we do this by introducing people to the community through our content. We also help members of the community to make better career decisions through our coaching and online content, because we’re currently focusing our content on the community. Finally, we do networking within the community through our coaching service and members directory.

12. How will we monitor our impact?

In pursuit of our vision and mission, our immediate focus is identifying the most promising career options through research and enabling more people to take them through our programs. The key metrics we aim to track are ‘research quality’ and ‘significant plan changes’. These metrics don’t fully capture our impact, but if we produce high quality recommendations and lots of people act on them, this guarantees our cost-effectiveness and proves we’re having a substantial influence.

We define a significant plan change as follows:

An individual has made a significant plan change if they have changed their credence in pursuing a certain mission, cause or next step by 20% or more; they attribute this change to 80,000 Hours; and there’s a plausible story about how engaging with 80,000 Hours caused this change.

Our research quality evaluation process is not yet sufficiently developed. We’re intend to develop a process over the next six months, which we’ll write about on the blog.

Periodically, we’ll conduct an evaluation which asks and answers in detail: ‘how valuable is a significant plan change?’ You can see our most recent evaluation here.

13. How will we ensure our research is valuable to our users?

As explained above, we’ll track the extent to which it causes significant plan changes through our programs, and we’ll conduct evaluations into the quality of our research.

We’ll also:

• We’ll ensure the research questions are demand-driven through user surveys and coachee requests.
• We may test our users’ willingness to pay for our advice through our coaching.
• We’ll monitor engagement metrics like website visits.

14. Why is this a high impact project?

We’re addressing a highly important and neglected problem – millions of young people want to make a difference with their careers, but lack high quality information about which options are best.

In providing this information, we’re contributing to building the effective altruism community, which we think is an important cause.

Most importantly, our model has the potential to act as a multiplier on the best career opportunities. We’ll find the most high impact career paths and encourage thousands of people to take them, leveraging resources into the world’s most pressing talent gaps.

An analogy is GiveWell. Rather than GiveWell’s founders donating all their money to the best charity they know, they invested in building GiveWell, which now directs tens of millions of dollars each year to even better charities.

Notes and references