80,000 Hours annual review Dec 2016

NOTE: This piece is now out of date. More current information on our plans and impact can be found on our Evaluations page.

Summary

2016 was an excellent year for 80,000 Hours. Here are some highlights – full details follow.

Metrics

• In our last review in May 2015, we set the goal of 50 significant plan changes per month by October 2016. That month, we actually recorded over 200.
• To make it harder to grow by adding lots of small plan changes, in October 2015 we started “impact rating” the plan changes, and tracking the impact-weighted total. 31 Dec 2015, we set the target of tripling the monthly rate of impact-adjusted plan changes over the year, which we achieved in November 2016. We now track about 150 impact-adjusted significant plan changes (IASPC) per month.

Impact and cost-effectiveness

• Our costs in 2016 were £250,000, up 13% on 2015. Considering that our staff could have earned to give instead, the total opportunity cost is perhaps £350,000 – £500,000.
• Since our last review, the ratio of costs to IASPC fell almost 3-fold.
• In 2016, we caused 115 people to take the Giving What We Can (GWWC) 10% pledge. GWWC estimates this is worth about £5 million in donations to their recommended charities (counterfactually-adjusted, time-discounted, dropout adjusted). So this alone plausibly justifies our costs, although our aim is to solve talent gaps rather than funding gaps.
• In addition, the plan changes since our last review now include three people who each intend to donate over $100m over their lifetimes, and we’ve advised several other high net-worth donors who we met through 80,000 Hours. So, we expect the total amount of money influenced will be much higher in the long-term.
• In 2015, we thought one of the most pressing talent gaps in the community was technical researchers to work on artificial intelligence safety. So, in 2015, we created a career review about this option, which has been viewed over 8,000 times, and identified 46 people who (i) have studied a relevant technical subject (ii) are concerned by existential risks (iii) want to enter this path (iv) have made a plan change due to us. We found a machine learning researcher affiliated with the Future of Humanity Institute to mentor them. One has taken a job at a top AGI company due to us, and another took at job at the Centre for the Study of Existential Risk at Cambridge University.
• Another talent gap we identified was people to work in effective altruist organisations and on global priorities research. Over 2016, 9 people who’d made significant plan changes took these roles, and likely wouldn’t have the job otherwise (at Centre for Effective Altruism, Founder’s Pledge, Animal Charity Evaluators). The two most senior people in the Centre for Effective Altruism (CEA) besides the trustees now likely wouldn’t have the job without 80,000 Hours (Michael Page and Tara MacAulay).
• Another major aim of ours is to grow effective altruism community. Over 2016, about 43% of the (raw) plan changes say they want to get more involved in the community, which would be about 600 people. Our newsletter is now larger than all the other effective altruist organisations put together (88,000), and was the largest source of applicants for the Effective Altruism Global conference with the highest acceptance rate. Over the last 12 months, our online guide became the largest website in the community by traffic, whilst our our budget was about 22% of the rest of CEA and about 6% of GiveWell.¹
• Past plan changers have continued to succeed. For instance, in Dec 2015, £2.5bn of UK aid spending was reallocated in the direction of the Global Priority Project’s recommendations (though we don’t know to what extent GPP was counterfactually necessary). Animal Charity Evaluators (ACE), which was initially a project of 80,000 Hours, has moved £2.4m to its recommendations, up from £0.8m in 2015, and is being considered for a grant from Open Phil.

Progress on programs

• We wrote a new online career guide that’s over 20,000 words, and turned it into a book. The first article in the guide has been read over 155,000 times, the least read article has been viewed 10,400 times, and the ebook has been downloaded over 10,000 times. Overall since 2015, web traffic has grown 60%, the conversion rate to the newsletter has doubled to 5.7%, and the conversion rate to online-driven IASPC also doubled to 0.06%.
• We developed a new workshop, hired Peter McIntyre to give it, and delivered it to over 2,000 people. About 10% have already become an IASPC. Hiring 10 people to give this workshop full-time seems like a solid path to scale up 80,000 Hours, though we think improving the online guide is even more effective.
• We also created a new “advanced” workshop, for people who are already familiar with our career guide and ideas, which covers issues like giving now vs. giving later, how best to help the long-run future, and risk.
• In research, our main achievement was adding 9 new problem profiles, and rating each problem area on a newly developed quantitative framework. We also released novel research on coordination in the community.
• In our fall outreach campaign, besides the workshops, we signed up 10,000 students at our 12 target universities to our newsletter, and promoted the book, gaining a further 15,000 subscribers.
• In August, we hired Jesse Avshalomov to lead on growth. He was formerly Director of Growth and Product at Teespring, where he managed a team of about 20. He oversaw our book promotion campaign. In March, we hired Peter McIntyre to lead on in-person advice. He co-founded Effective Altruism Australia, which has already raised over $500,000 for GiveWell-recommended charities. Peter has already delivered about 40 workshops, and created the AI researcher pipeline.

Plans

• Over 2017, our target is to triple the rate of IASPC again, reaching 450 per month by the end of the year (though this will be challenging).
• The top priority will continue to be improving the online guide. In particular, we’d like to create better ways to enable people to enter the highest-impact options. For instance, we’d like to create a set of career reviews covering all the major policy options, create a list of interested people, and introduce them to mentors. The same is true of working in effective altruist organisations, bioengineering, founding new GiveWell non-profits, and many other paths.
• There’s also a lot we could do to improve our introductory content, such as creating better videos, or making a Coursera course, which could plausibly double readership.
• We’d also like to start scaling up marketing, with the aim of reaching most of our target audience at top universities.
• Due to recent success, we’d like to scale up faster than in previous years, so we have an especially large room for funding.

Our full review continues below. If you’d like to support our work over 2017, find out how to donate.

Recap: what does 80,000 Hours do?

80,000 Hours helps talented graduates in their twenties find satisfying careers with a big social impact.

80,000 Hours began in 2011 when we were trying to work out what to do with our own careers. Like a third of graduates, we wanted to find something we’d enjoy with a positive impact on the world.Net Impact survey</a>, 31% reported that making a difference was 'essential' in their choice of career. 45% also reported that they would take a 15% pay cut to make more of a difference. </li> <li>A <a href=https://80000hours.org/2016/12/annual-review-dec-2016/"http://www.webcitation.org/6TNAV7CSg">Guardian survey</a> found that over 70% reported that ethical considerations were "crucial" in choosing an employer.</li> <li>In a Bentley University survey, <i>Millennials in the Workplace</i>, 84% reported that "knowing I am helping to make a positive difference in the world is more important to me than professional recognition". </li> <li>A <a href=https://80000hours.org/2016/12/annual-review-dec-2016/"https://web.archive.org/web/20220731185031/https://www.theguardian.com/sustainable-business/2015/may/05/millennials-employment-employers-values-ethics-jobs">survey of millennials by Global Tolerance</a> found "42% of individuals...want to work for an organisation that has a positive impact on the world. Meaningful work that benefited others was more important than a high salary for 44%"</li> </ol> <p>Many of these surveys were performed by groups that have an interest in inflating the figure, which is why we expect the true value to be towards the lower end. Our overall estimate is that it's about one third.</p> <p>The percentage is likely to be higher among talented students, because they're less concerned by financial security. The University of Cambridge Careers Service told us that positive impact is the second most important career priority for Cambridge students, second only to intellectually interesting work.</p> <p>Our growth also provides direct evidence of interest. For instance, at Oxford and Cambridge and other UK universities, most students attend a fresher's fair when they first arrive. We've been able to sign up more than 1/3 of the undergraduates to our newsletter mutliple times at these fairs. Typically our opener is something like "do you want to make a difference with your career?".</p> " rel="footnote" class="footnote-link no-visited-styling" aria-label="Footnote">² But we couldn’t find any good advice on the most important questions: Should I work on global poverty or local poverty? Should I contribute through philanthropy or work directly at a non-profit? Which skills are most useful?

The aim of 80,000 Hours is to do research to identify the highest-impact career strategies and paths, and enable people to take them.

We do this by providing:

1. Online advice – our career guide and supporting articles. It’s based on five years of research carried out alongside academics at the University of Oxford. The guide covers novel concepts, such as our framework for comparing global problems, and recommends specific options people don’t usually consider, like earning to give, working on artificial intelligence, and becoming an academic research manager.
2. In-person advice – our workshop and one-on-one follow up with the most engaged people.
3. Community – we link people with the effective altruism community to help them take action of their new plans.

Our aim is to become the default source of career advice for graduates who want to make a difference. By doing this, we can enable a generation of young people to have far greater social impact with their careers, and make a major contribution to ending global poverty, factory farming, existential risks, and other global challenges.

Key metrics

The key way we track our efforts is measuring the number of significant plan changes we cause. We track these with surveys embedded in the website, workshops, and newsletter.

Each week, we review the plan changes to (i) check whether they meet our criteria and (ii) rate them based on “additional impact”, as 0.1, 1 or 10. We often follow up one-on-one to get more info. (Read more about the definition).

In 2016, we tracked over 900 impact-adjusted significant plan changes (up almost 3-fold on 2015), and 1,400 in total (up 4-fold).

Here’s a summary of other key figures:previous report</a> the number of plan changes for 2015 are reported as slightly higher (326.9 vs 320). This difference is due to people who made plan changes in 2015 making subsequent plan changes in 2016 with a higher impact adjusted value, meaning they are recorded in 2016 instead, and due to disconfirming 2015 plan changes in light of new information.<br /> </p> <p>Also note that our cost figures are preliminary estimates. Our UK accounts with CEA UK have been audited through the end of June 2015 and the US accounts with CEA USA through the end of Dec 2015. But costs after that time will be subject to revisions.<br /> <br /> We measure unique visitors to our website using the Google Analytics client side tracking library. We believe that a significant fraction (~20-50%) of our website visitors use browser extensions that block this tracking method, so our true visitor numbers are higher.</p> " rel="footnote" class="footnote-link no-visited-styling" aria-label="Footnote">³

Note that the figures for 2016 don’t include December, so our year-on-year growth rates will be somewhat higher once all of 2016 is included.

See more detail in our metrics report.

Plan change impact and cost-effectiveness

What did the plan changes consist of?

We categorised the plan changes which we scored as 10s in 2015 and 2016 by the paths they switched into:

We selected a random sample of 30 plan changes scored as 1s in 2016, and categorised them by what they now intend to do.

We did the same for a random sample of 30 plan changes scored as 0.1 in 2016:

For more detail, see these statistics on the plan changes.

Example plan changes

Here are some of the highest-rated examples of plan changes. Unfortunately we don’t have permission to publically share many of the best cases.

Michael Page

Background
Michael is one of the most senior people in CEA, heading up the research division, which is now advising several billionaires on where to give. He’s widely seen as one of the most able and hard-working people in the organisation. He’s focused on the long-run future and capacity building for the effective altruism movement.

Before that, he was a rising star within law on partner track, while giving most of his income to charity.

In his own words

Yes, I think you can count me as an official 80,000 Hours plan change. Counterfactuals are obviously tricky, and I had been considering this question for a while, but I definitely updated in a significant way after reading your post on talent gaps. In a world without 80,000 Hours, I think there’s a 75% chance I’d still be earning to give today and a 50% chance I’d be earning to give in five years.

Re other ways you might have changed my career, I’m sure there are. I’m a big 80,000 Hours fan, and I suspect it has significantly shaped my entire way of thinking about what I do with my time, to such an extent that it’s hard to pinpoint specific instances of its effect.

How 80,000 Hours helped
In the post on talent gaps, he realised how much money is already available and that direct work within effective altruism can be more valuable. This prompted Michael to leave earning to give sooner than otherwise.

While Michael was very enthusiastic about effective altruism he didn’t have many social connections due to being located in DC, so our online articles played an unusually important role for him. He was worried that if he waited too long he would find it hard to make the change, which gave extra impetus to the move.

Counterfactual impact
He would have donated hundreds of thousands of dollars per year working in law, so was already on a high-impact career path.

We think, however, that his current role is higher impact than that. He’s making it more likely that CEA can influence billions of dollars in the future by expanding its management capacity. He also helped with a major restructuring of the organisation.

It’s likely that Michael would have transitioned to another role in the next 1-10 years due to tiring of earning to give. So the benefit comes from speeding up this change and making it clearer that he should do direct work in effective altruism.

Nick Robinson

Background
Nick works at ASI Data Science, which is run by members of the effective altruism community who want to mitigate existential risks, and which has received investment from Jaan Tallinn (a major existential risk focused donor). ASI intends to start using some of its resources to train safety-concerned researchers and encourages awareness of risks posed by AI. Right now, Nick is most focused on AI safety. Before this job, he studied philosophy at Cambridge.

How 80,000 Hours helped
Nick’s first contact with effective altruism was via GWWC as a philosophy undergraduate. He then interned at GWWC and became exposed to broader ideas due to being around CEA/80,000 Hours/FHI.

He received one-on-one advice and became a regular reader. Exposure to the broader effective altruism community and its ideas persuaded him to switch focus from international development to concern for the long run future. He became concerned about AI among other x-risks, but didn’t initially see a path to contribute personally. He later met Sean O’H at CEA and helped him when setting up Centre for the Study of Existential Risks at Cambridge.

The point at which 80,000 Hours most unambiguously changed his career plans was by publishing a career review on Data Science and how to get into it. Nick had not considered this option before and thinks it’s very unlikely he would have pursued it without 80,000 Hours. But he was quickly convinced that the path was good on a number of dimensions, and so taught himself data science with online courses.

After studying data science he found his first job at ASI Data Science through Roman Duda, who put him in touch with Marek Duda (a past CEA staff member and plan change who was working at ASI at the time).

Nick now has multiple paths to impact ahead of him which he takes seriously:

• Supporting and expanding a training program within ASI data science to provide financing and industry training to potential AI safety researchers. ASI and Nick are building this now. Their goal is to find a way to take smart graduates and produce AI safety researchers in a faster and more scalable way.
• Using credibility from being a data scientist at ASI to “try to convince existing AI researchers to do safety research themselves”.
• Earning to give in data science (plans to donate 30% or so for now).
• Less probable options he’s open to include: (i) going back to do a ML PhD (ii) retraining e.g. work on biosecurity (iii) working at an effective altruism org that is finding it hard to hire (iv) founding his own company.

In his own words

Previous intentions:

Unclear as I was a 1st or 2nd year undergrad when I became involved with 80,000 Hours. Probably become a teacher or possible work at an international development NGO, donate 10% of my income and not study beyond my philosophy degree.

New intentions:

I intend to work on AI Safety, either directly or more likely through some kind of capacity building of the field.
Long-term I think it’s possible I work instead on scaling work on some other technological risk area (e.g. if there seems a lot of progress reducing AI risk compared to biorisk). Possibly I will try starting my own tech company in the future.
I self-studied Machine Learning and may do an AI PhD if that becomes the best option (probably I can get most of the same skills outside academia and others besides).
I work as an early technical employee at a tech start-up with founders involved in effective altruism.
I intend to donate significantly more than 10% of my salary in the next 5 years and to donate the majority of my income in the long-run (unless I take a much reduced salary to do direct work).
All of this is different from what I would have predicted 4 years ago and all due to 80,000 Hours and the connected community.

What were your main reasons for making the change?

Coming to believe I could do directly useful work that sufficiently many others were unlikely to do, and getting a lot of support and encouragement when trying to make a career change.

Counterfactual impact

Now Nick is committed to effective altruism, and well placed to help find and train more AI safety researchers. It’s highly unlikely to have pursued data science and be doing his current projects without 80,000 Hours.

If only GWWC existed, there’s a good chance he would be focussed on international development, or have only donated 10%.

Frances Ding

Background
Frances is an undergrad at Harvard studying biology and CS, contributing to the Harvard effective altruism group and aiming to work on the AI control problem.

How 80,000 Hours helped
Frances’ first contact with effective altruism was an event run by the Harvard Effective Altruism society in 2014. She then went to various events at which they discuss key concepts.

Having learned about the ideas she began doing more and more quantitative courses such as computational neuroscience or CS to build better career capital. She also came to believe that there was a real problem aligning AI with human interests but didn’t initially see a way to contribute to solving this problem personally.

Frances was then influenced by our various online materials about AI safety and the workshop and coaching we gave at EAGx Boston conference in April 2016. This was reinforced by reassurance from the local community at Harvard and her professors that working on aligning artificial intelligence with human interests is a practical career path.

The main reasons for the switch were:

• Now seeing her skills as being suitable for work on AI value alignment.
• Thinking clinical medicine has a lower direct impact and is quite ‘replaceable’.
• It’s a fairly lengthy and inflexible path, so doesn’t make much sense to commit to medical school now if she has significant doubts about it being the best option.
• Thinking AI value alignment is a more neglected problem.
• Being concerned that there is not enough diversity among people working on developing AI.

In her own words

If 80,000 Hours never existed, I think my career plans would be significantly different. Before learning about 80,000 Hours and getting further involved with effective altruism, I was mostly planning to go to med school. 80,000 Hours was pivotal for me because it made me really question how much impact I would have as one of many almost identically trained doctors, and pushed me to be less risk averse, so now I’m hoping to use my neurobio background on AI safety work. I’m not sure what exactly this is going to look like, but I think this field switch fits me better and also has a higher chance of impacting the world.
I haven’t decided/heard back from everything yet, but options I’m considering are: a masters in CS, interning at DeepMind or Google Brain, and being a research assistant at Harvard, MIT, or Berkeley.

Added Dec 2016: Frances has been accepted into a Cambridge Machine Learning program, and is waiting to find out if she will be funded. She has also applied to a neuroscience research team in London.

Counterfactual impact
In early 2015 Frances was fairly sure she was going to go into clinical medicine and had taken steps to advance down that path.

It’s unlikely she would have chosen to work on the AI control problem without 80,000 Hours’, but it’s possible she would have made another career change due to exposure to ideas in effective altruism.

Ben Garfinkel

Background
Ben did a Bachelor’s degree in physics, mathematics and philosophy at Yale, and then summer research internship on a topic relevant to quantum cryptography. He is now a visiting researcher at the Centre for Effective Altruism and a researcher at the Global Politics of AI Research Group at Yale.

How 80,000 Hours helped
Ben was about to graduate and had no firm next step. He was expecting to take a gap year or two to figure out his future plans, and considered pursuing further study in quantum cryptography.

Rob (a staff member at 80,000 Hours) met Ben at an event put on by 80,000 Hours at Yale. Rob spoke to him over dinner about the expected societal impacts of quantum cryptography (which seem of unclear sign) and other options.

Rob later followed up online and offered coaching. He introduced him to Owen (an employee at the Global Priorities Project) to discuss possible careers in global priorities research which seemed suitable. He also recommended he apply to Global Priorities Project
/GiveWell/OpenPhil, and suggested he visit the Bay or Oxford to meet more people in the community. Finally, he said he thought AI risks should be taken seriously, and recommended he read Superintelligence to understand the arguments, which Ben did.

Subsequently, Ben got the job at the Global Priorities Project and started living with Rob Wiblin. During this time, they discussed at some length whether he should pursue global priorities research or AI safety research above returning to cryptography. They decided that because it was unclear whether quantum cryptography was good, it was unlikely to beat those options.

Ben applied for and was accepted to work with Prof Allan Dafoe at Yale on the political implications of AI.

Since interning at the Global Priorities Project over the summer he has been promoted to a ‘visiting researcher’ at CEA. Both of these roles put him in a good position to take a useful permanent position later on.

In his own words

Previous intentions

I essentially didn’t have a plan. I was somewhat considering a philosophy masters program for the next year, and I thought I might want to eventually pursue a career in physics. I also had a vague interest in earning-to-give. The most likely thing I would have been doing for the next year is probably take time off to travel and study more. I had also already signed the Giving What We Can Pledge.

New intentions

I didn’t necessarily develop clear plans, but I was exposed to possibilities that I hadn’t otherwise thought of. After talking to Owen at the Global Priorities Project I applied for a summer internship there, and after talking to Claire at GiveWell I applied for a job. I also read Superintelligence at Rob’s suggestion, after previously being skeptical of AI risk.
Now I’m interning at the Global Priorities Project for the summer, and the odds look fairly high that for next year I’ll be a research assistant on an AI risk project with Allan Dafoe at Yale. [Which he now is.]

Counterfactual impact

It is very unlikely Ben would be in his current roles now without 80,000 Hours, as he had not considered those paths at all and was not convinced AI was a problem. Ben is now highly involved in the effective altruism community due to spending time in Oxford.

As Ben already knew other people involved in effective altruism at Yale and had taken the GWWC pledge, it is possible someone else would have convinced him to pursue these or equivalently high impact paths at a later time. We can’t say for sure. Ben thinks there’s a decent chance he would have drifted out of effective altruism had he not met Rob at that point:

Pretty unlikely that I would have been doing anything this year that would have had me significantly in-touch with effective altruism people. Also wasn’t heavily exposed to other effective altruist people even when at Yale – since the group there was really small, and I wasn’t heavily involved in it.

Chris Cundy

Background
Chris is doing a Masters in machine learning at Cambridge and hopes to do AI safety related research. Chris studied physics at Cambridge as an undergraduate, where he became involved with running the 80,000 Hours student group.

How 80,000 Hours helped
Chris was planning to do a physics PhD, but after getting heavily involved in 80,000 Hours Cambridge, he ended up spending a lot of time with Adam Gleave, who was head of 80,000 Hours:Camb and doing a CS masters at the time (and is also someone who was influenced by 80,000 Hours). This caused him to become “much more cause-neutral, started to think more about the impact of my intended career”.

First he became convinced that AI safety was a problem where he had not been convinced before. Then, through 80,000 Hours’ various articles on AI safety became convinced that his skills could be applied to solving the problem.

A key part of this was hearing us in workshops frame working on global catastrophic risks as an entirely reasonable thing to do, as these ideas are often presented in a needlessly contrarian manner.

Now he intends to study a machine learning PhD as a next step. He then plans to either do technical or political AI safety work. Global priorities research and promoting effective altruism are the next most likely options, also due to advice found through 80,000 Hours’. Quant finance and entrepreneurship are backup options if those fall through.

In his own words

80,000 Hours has definitely changed my career plans – I was fairly committed to a physics PhD up until a few years ago, now I’ve switched to doing Computer Science with a machine learning focus.

Counterfactual impact
In 80,000 Hours’ absence he would likely be aiming for a physics PhD in cosmology or a similar field. Before encountering us he had planned his career based on present interest rather than impact.

It’s now likely now that he’ll end up working within AI safety research or at an effective altruist organisation. He has also had a large impact already by leading 80,000 Hours Cambridge, organising our workshops for hundreds of students among other events to spread our ideas.

If only Giving What We Can existed he likely would have taken their pledge, but not otherwise changed his plans.

Chris noted that he might have been persuaded to worry about existential risks another way as the ideas are out there on the internet, but thinks 80,000 Hours and its presentation made it significantly more likely.

Historical impact and cost-effectiveness

See this analysis of our past impact, and whether it has justified our costs.

Since our last review in May 2015, due to the points mentioned in the summary of this post, we think our case for the impact of an IASPC improved. At the same time, the ratio of costs to IASPC has fallen almost three-fold. So, we expect our marginal cost-effectiveness has increased.

Progress on programs and capacity

This section outlines our key achievements as an organisation over 2016. See an overview of how we allocated team time in 2016 in the footnotes.⁴

Major update to the online guide

Our most effective work is probably spreading our key ideas – things like flexible career capital, problem selection and earning to give. This is because these ideas are relevant to everyone, and can have a major effect on your plans. For this reason, we made improving the core online content the top focus for 2016, and Ben worked on the online guide from February onwards, basing it on our career workshop.

The first version was finished in May 2016, containing 8 articles with a total of over 20,000 words, and we did several more updates over the summer. We think it’s by far the best explanation of our key ideas so far, and it covers most of the key ideas of effective altruism applied to careers, as well as lots of advice that’s relevant to everyone. We also added the option to receive the guide via one email a week.

Due to all this, our total web traffic increased over 50% in 2016, reaching 96,000 unique users per month in November, which means 80000hours.org has had more traffic than any other website in the effective altruism community over the last 12 months.¹ About 15,000 people read the first article each month, of which about 1,500 finish the entire guide. Of these, 5-10 take the Giving What We Can pledge having clicked through from our website.

The conversion rate to the newsletter has doubled to 5.7%, and the conversion rate to online-driven IASPC also doubled to 0.06%.

In the fall, we also turned the guide into a book, which is live on Amazon and has been downloaded over 10,000 times.

New site section on global problems

Another key weakness in our content (and in the effective altruism community in general) is lack of information about how to contribute to different problem areas other than through donating, and even donations aren’t well covered outside of global health. So, we wrote 9 problem profiles, which evaluate different problem areas and explain how to contribute to them most effectively.

In collaboration with the Global Priorities Project, we also published the first quantitative framework for comparing problem areas, and applied it.

We cracked the workshop

In Dec 2015, we created a new 4 hour workshop. It covers content similar to the online guide, plus a pitch for top problem areas, and exercises. We quickly realised it’s more effective than one-on-one advice, so we added it to our 2016 plans. It also let us rapidly test content for the online guide.

The workshop has a 10% conversion rate to IASPC, so a workshop of 30 people leads to about 3 plan changes right away. In contrast, this would take about 24 hours of one-on-one advice. We also expect the conversion rate to increase over time because it often takes several years to change career. Moreover, we discovered that we can increase the number of attendees to at least 100 without significantly harming the metrics. Over the year, the total cost per plan change was £215 (not including opportunity costs of time).

Why is it so effective? We think one reason is that each person engages much more deeply. A 4 hour workshop for 30 people is 120 hours of engagement, which would be weeks of one-on-one advice. Another reason is that people engage with the workshop at an earlier stage in their career decisions, while coaching tends to attract people who already have specific plans, so big shifts are more likely.

In March 2015, we hired Peter McIntyre to lead our in-person advice, and he focused on workshops. He’s the perfect person for the job, having trained as a doctor, founded Effective Altruism Australia (which has raised over $500,000 for GiveWell-recommended charities), and worked in finance, sales and at the Future of Humanity Institute.

Peter gave about one workshop a week from April. Combined with some help from Rob and Ben, we gave 48 total, reaching over 2,000 people, while maintaining the conversion rate. Most of these people were at global top 20 universities, such as Oxford and Harvard. We also toured to effective altruism groups in Germany, Norway and Hong Kong.

We also trained Brenton Mayer to give the workshop. He’s a doctor from Australia, and also co-founded Effective Altruism Australia. He has accepted an offer to join the team from February 2017.

The final piece is that we worked out how to market the workshops. Jesse made several improvements to the Facebook event and sign-up forms. This let us use Facebook adverts to get attendees for about £2.50 in London, and a little more in smaller cities.

All together, the workshops are a solid path to scaling up 80,000 Hours. We’ve shown they’re highly effective at creating plan changes, we can get people in the door, and we can hire people to give them. To pay for themselves, we just need each workshop to have a 10% chance of finding a medium-sized earning to give donor who donates back to us (or a 1% chance of large donor). In two years, we could hire a team of 10, and cover all of our target universities. This would mean that in 20 years, a significant fraction of future leaders would have been to a workshop.

Created the advanced workshop

To follow on from the main workshop and online guide, we created an advanced workshop and gave it 4 times. It’s focused on people who are into effective altruism and want to maximise their impact. It covers advanced content like how best to help the long-run future, giving now vs. giving later, and how to coordinate with other people who want to do good. This workshop generates an even higher rate of plan changes than the regular one, and in particular helps to identify people who might have an extraordinary impact.

Fall outreach campaign

The fall is the best time to do outreach to students, so this year, most of the team focused on outreach from Sept to Nov. To this end, we:

1. Signed up 10,000 students from our target universities through offline outreach (mainly tabling at fresher’s fairs)
2. Gave over 20 workshops, with over 1000 attendees.
3. Turned the guide into a book. By using a free giveaway on the 12th Dec and £10,000 of Facebook advertising, we gathered over 15,000 newsletter subs. We’ll be doing more book promotion over the coming months.

Technical AI research pipeline

We found a group of 50 who want to enter AI safety research, and helped them by creating a guide on how to enter, giving one-on-one advice, making a Google Group and finding an AI safety researcher to advise them (David Krueger, who was at FHI). So far, one has taken a job at a top general AI firm. Over the next year, we can directly match-make people with jobs. See more info.

We surprised ourselves at how quickly we were able to grow this group, and the ability of many of the people. Over the next year, we’d like to try something similar for other key areas, such as AI policy and working in effective altruist organisations.

Plan change tracking systems

We redesigned our surveys, which makes the plan changes about twice as easy to rate, and gives us much better data.

Note that since we’ve also increased the rate at which we’ve surveyed users, some of our growth comes from tracking a greater fraction of plan changes rather than causing more of them.

Hiring and team

We had several disruptions to the team in 2015, but we put this behind us in 2016.

Over 2016, the team has worked really well together, morale has been high, and everyone has been highly productive. New hires have all said the quality of the team is a major reason for joining.

Besides hiring Peter and Brenton, who are mentioned in the section on workshops above, we hired Jesse Avshalomov. He was previously the Director of Growth and Product at Teespring, one of the most successful Y Combinator startups, where he led a team of 20, conducted hundreds of marketing & product experiments, and oversaw the growth of the company to 19 million products sold. Before that he ran SEO for the North American Apple Online Store… and did professional opera.

Our main challenge has been finding a freelance web engineer. We did a recruitment round and a trial, but didn’t end up finding someone long-term. Fortunately Peter Hartree, our former developer, is still able to give us 1-2 days per week of support. We intend to try again in 2017, taking advantage of our larger audience, advertising a higher salary, and aiming for someone full-time rather than freelance. We also learned that if we hire a part-time engineer, we should (i) make sure they spend several days talking to our existing engineer to on-board (ii) have experience in remote work and WordPress.

It also took longer to on-board Jesse than we expected, in part due to working remotely and spending 20% time on other projects. This again suggests spending longer on-boarding right at the start, and making it a priority that new staff work in the same place as everyone else for the first month. We’ll also make further efforts to avoid part-time staff in the future.

Challenges and mistakes

Progress has been ahead of expectations, so we don’t feel like we’ve made any major mistakes. Here are some ways things could have been better.

1. Our main challenges relate to hiring, and are covered above.

2. The book launch was delayed over a month, and might have had a smaller reach than it could. The delay was due to it taking longer to on-board Jesse than we expected (as covered above), spending more time improving marketing for the workshops (which paid off, as covered above), and doing too many things at once (covered below).

3. Produced fewer high-value career reviews and problem profiles than planned. Roman spent more time on the plan change tracking systems than planned, while Rob spent more time on outreach. We also switched priorities several times (as covered below), which probably hurt research output.

4. Too many competing priorities. We make a lot of effort to create focused plans, and are much more focused than we used to be, but we still probably switched priorities too many times over 2016, and there were also times when we had too many priorities at once. For instance in spring, the research team switched from career reviews, to supporting articles for the guide, to problem profiles. In October, we were doing campus sign ups, workshops and book promotion at the same time. All this creates switching costs, is less motivating and leads to unfinished work, which contributed to many of the other problems. Some of this was made worse by being in different offices. We’ll continue to emphasise focus when creating our priorities. The plan is also for everyone full-time to be based in the Bay Area, in the same office.

5. Growth of high-value plan changes much slower (~50%) than growth of medium-value plan changes (360%). This wasn’t a mistake but could be a worrying trend. However, the fact that it’s happening isn’t too surprising since high-value plan changes take several years, so our growth over 2016 depends on efforts made in 2014 when we were much smaller. We expect that a significant fraction (perhaps 10%) of the medium-value plan changes will eventually become high-value plan changes. At the start of 2017, we also intend to especially focus on getting high-value plan changes.

6. Many people didn’t get responses from [email protected] for 6 months, affecting about 70 emails. This was due to an error with a new inbox client, which was hard to notice, though could have been found earlier with better testing.

Plan for the next year

Strategy

Broadly, our strategy is unchanged from the last review. Our key focus is to keep improving the online guide, with the aim of making it overwhelmingly the best career resource for our audience.

Here’s a summary of the key strategy decisions we face, and our current views on them. We have a few review points during the year, so might change course.

Targets and priorities for 2017

Our aim for 2017 is to triple the monthly rate of impact-adjusted significant plan changes, going from 150 per month to 450, reaching 2,000 over the year. We think this is achievable but aggressive, so wouldn’t be surprised if we miss it.

To achieve a tripling, we’d roughly need to double our conversion rate to plan changes, and increase the number of users by 50%.

Here are some concrete projects we’re considering that could make a major contribution to that goal.

1. Dramatically improve the career reviews and problem profiles, so we have in-depth profiles of all the best options, including advice on how to enter each area. When we do one-on-one advice, this is the main area where we feel weak. For instance, we’d like to have in-depth profiles covering the main policy options, working at effective non-profits, bioengineering, setting up new GiveWell non-profits, work in meat substitute startups, and many other paths. We expect improvements here could significantly increase the rate of plan changes from the guide, while also bringing in more traffic.
2. Create mentor networks. The most high-value plan changes require a combination of knowing the ideas and in-person support. It seems like the easiest, scalable way to get more high-value plan changes is to create ways to spot high-potential people and get them involved with our community. This year, we used a form on our website to find people interested in AI safety research careers, found a group of 50 with the relevant background, and introduced them to a mentor. We’d like to do something similar for other key areas mentioned above. We can also increase the rate at which people get involved in their local effective altruism community by adding better calls to action to the guide.
3. Improve user tracking. We’d like to get a better understanding of how the highest-value plan changes come about. There are probably opportunities to dramatically decrease the rate at which people disengage. We’d also like to start systematically tracking the most engaged users, so we can match them with specific job opportunities e.g. now we now have a list of people interested in AI risk research, and we could do this for more areas within our CRM (software for tracking users).
4. Make the online guide more engaging and persuasive. There’s a lot we could do to improve the guide. For instance, we could improve the video content, which could make it significantly more engaging and help us gain traffic through YouTube (Ben’s TEDx talk gets over 30,000 views per month, so we’re capable of producing highly shared content). If this works well, we could put the guide on Coursera, or another MOOC platform, which could plausibly double usage, as well as increase the completion rate. We could also improve our decision tool.
5. Scale up outreach with the aim of saturating most of our target universities. We’d start by doing intensive outreach to one university in spring, then apply what we learn to 5-10 universities in our annual September outreach campaign. There’s scope to double or triple the number of students at these universities who use our advice, which could hugely grow the rate of plan changes. If we can fundraise a marketing budget, we could use digital advertising to gain new subscribers from target universities for under £4 (more info).

Roughly, we’d work on these in order – first third of the year is focussed on upgrading (increasing the proportion of high-value plan changes), the second third on improving the introductory content, and the final third on outreach (and Nov-Dec we also do the annual review and fundraising).

With hiring, the roles we’re mostly strongly considering are research and coaching. The researcher would create career reviews and problem profiles. We’re confident these are valuable, and we’ve already had some success delegating them to freelancers. The coach would give workshops and do one-on-one follow up with our most engaged users.

Some projects we considered but decided against

• Writing an advanced career guide. It would cover the same content as the advanced workshop above. We think our audience is still a little small for this to be the highest-impact project. We prefer to keep thinking about what would be covered while giving live workshops, then revisit this project in a year.
• Scale up workshops. We think there’s more we can gain with the online guide supported by the community, so we’ll come back to the workshops later. In-person time will initially focus on “upgrading” people who are already engaged.
• Scale up one-on-one advice. As we argue above, this is dominated by workshops, with the possible exception of really high-potential people who have already read the guide, who we intend to speak to. We also think it’s more effective to set up mentor networks than hire lots of coaches ourselves.
• Launch a MOOC. We’ll start by creating more video content, then turn it into a MOOC if that’s successful.

Where this is going

Our aim is to become the default source of career advice for talented graduates in their 20s who want to do good. This could enable an entire generation of leaders in science, business and politics to work together to have a far greater social impact.Link to paper</a>; <a href=https://80000hours.org/2016/12/annual-review-dec-2016/"http://www.businessinsider.com/how-many-rich-powerful-people-went-to-elite-colleges-2014-6?IR=T%22>link to summary in Business Insider</a>.</p> " rel="footnote" class="footnote-link no-visited-styling" aria-label="Footnote">⁵ If we did this, we could make a major contribution to reducing global poverty, ending factory farming, mitigating existential risks, and tackling new global challenges.

This is achievable in the next couple of years. Reaching the 30% of students who are most interested in us at the world’s top 20 English-speaking universities with paid marketing would only cost about £60,000 per year. With a team of 10, we could run most of them through workshops. This would cost about £800,000 per year.⁶ We may be able to achieve all this much more cheaply, however, by improving the online guide and having it spread by word of mouth.

From there, we could expand into advice for a wider range of ages. Ultimately, we could provide career advice to everyone, expanding the fraction who consider social impact.

Budget and funding needs

Targets

Update: March 2017: We made our target! See more information here.

We’d like to raise at least £1.7m. Here’s how that breaks down:

• Cover our existing commitments to 6 full-time staff and freelancers over 2017 (£585k).
• Maintain at least 12 months’ reserves to give us financial security (to have 12 months’ reserves when we next fundraise in Dec ’17, we need enough to cover all of our 2018 baseline costs, which are projected to be £670k).
• Increase salaries about 30% to match comparable organisations and attract better new staff (£350k over two years).
• Hire two additional entry-level staff members to work on writing career reviews, giving workshops, or design (or one senior staff member) (£175k over two years)
• Expand our marketing budget (£165k).
• Then we’ve subtracted £250k of cash we already have on hand.

This is four-times what we raised last round, so we need to bring in new donors or we won’t make the target. This means we have substantial room for funding.

Moreover, even if we made this target, it wouldn’t exhaust our room for more funding. If we raised more, we could increase our reserves, which would make it easier to attract staff, or we could pursue our expansion opportunities more aggressively (e.g. hire more, larger marketing budget).

2017 budget (provisional)

The budget is still being confirmed so the exact amounts could change.

Historical budget summary

The 2017 baseline is higher than 2016 because (i) we have 40% more staff, because two people joined during 2016 and one joins early 2017 (ii) we moved to the Bay Area which increased costs about 25% (iii) we didn’t have an office over 2016, but now we do, which increases costs about 10% (iv) standard cost and salary increases of about 10% per year (v) we’ve added 5% contingency.

The expansion budget is higher again because it includes (i) £160,000 for marketing (ii) salary increases (iii) hiring new staff.

See more details on our historical budgets.

Conclusion

2016 was a great year, and we feel like 80,000 Hours is starting to fulfill its potential. We made major progress on our online guide, developed a highly effective workshop, more than doubled our newsletter, and made some great hires. This meant we made our aggressive growth targets, and reached over 1,000 plan changes. These include 2 high net-worth donors, 9 employees effective altruist organisations, 46 potential AI safety researchers, 115 Giving What We Can pledgers, and more. Past plan changers have continued to succeed: the Global Priorities Project was involved in a major policy change, Founder’s Pledge doubled total pledges, and Animal Charity Evaluators tripled its money moved.

This was achieved with only a 15% increase in costs, so we think 80,000 Hours is more cost-effective than ever.

But there’s far more we can do to make the advice more concrete, comprehensive and engaging. For instance, we could add in-depth reviews and mentor networks covering the most high-potential paths, like policy. And there’s much more we could do to reach our entire audience – we’ve spent little effort on outreach in the past and now we have an expert on the team. We’re really excited to take 80,000 Hours to the next level of scale.

If you’d like to enable us to grow in 2017, find out how to donate.

To receive monthly updates on progress, join our Google Group.

Other documents in this review:

• Plan change metrics update.
• Has 80,000 Hours justified its costs.
• Why donate to 80,000 Hours
• Financial reports

See all our previous evaluations.