This repo originates with a wish to promote Global Equality for people who want to do a PhD in NLP, following the idea that mentorship programs are an effective way to fight against segregation, according to The Human Networks (Jackson, 2019). Specifically, we wish people from all over the world and with all types of backgrounds can share the same source of information, so that success will be a reward to those who are determined and hardworking, regardless of external contrainsts.
One non-negligible reason for success is access to information, such as (1) knowing what a PhD in NLP is like, (2) knowing what top grad schools look for when reviewing PhD applications, (3) broadening your horizon of what is good work, (4) knowing how careers in NLP in both academia and industry are, and many others.
Contributor: Zhijing Jin (PhD student in NLP at Max Planck Institute & ETH, co-organizer of the ACL Year-Round Mentorship Program). You are welcome to be a collaborator, -- you can make an issue/pull request, and I can add you :).
Twitter: If you prefer to get more updates on Twitter, feel free to follow @ZhijingJin and @aclmentorship.
Contributor: Zhijing Jin (PhD student in NLP at Max Planck Institute & ETH, co-organizer of the ACL Year-Round Mentorship Program).
Endorsers of this repo: Prof Rada Mihalcea (University of Michigan). Please add your name here (by a pull request) if you endorse this repo :).
- Top Resources
- Stage 1. (Non-PhD -> PhD) How to Apply for a PhD?
- Stage 2. (Doing PhD) How to Succeed in PhD?
- Overall Guides
- Timely Topic: Surviving NLP in the Era of LLMs
- What Are Weekly Meetings with Mentors/Advisors like?
- How to Read Papers
- How to Express Our Ideas: Writing Papers, Visualization, etc
- Reviewing, Publishing, Attending Conferences, Networking
- Memoir-Like Narratives
- Excel Your Research
- Coming Up with Good Research Ideas
- Internship
- Grad School Fellowships
- Books
- Stage 3. (After PhD -> Industry) How is life as an industry researcher?
- Stage 4. (Being a Prof) How to get an academic position? And how to be a good prof?
- Stage 5. (Whole Career Path) How to live out a life career as an NLP researcher?
- Further Readings: Technical Materials to Improve Your NLP Research Skills
- Others
- Contributions
- How to Cite This Repo
- Online ACL Year-Round Mentorship Program: https://acl-mentorship.github.io (You can apply as a mentee, as a mentor, or as a volunteer. For mentees, you will be able to attend monthly zoom Q&A sessions hosted senior researchers in NLP. You will also join a global slack channel, where you can constantly post your questions, and we will collect answers from senior NLP researchers.)
- (Organized by PhD students in NLP across the world) NLP with Friends Online Seminar Series (recordings available). [Seminar] (A great way to learn about what others are doing in NLP)
- (John Hewitt, PhD@Stanford) Undergrad to PhD, or not - advice for undergrads interested in research (2018). [Suggestions]
- (Prof Jason Eisner@JHU) Advice for Research Students (last updated: 2021). [List of suggestions]
- (Nelson Liu, PhD@Stanfard) Student Perspectives on Applying to NLP PhD Programs (2019). [Suggestions Based on Surveys]
- (Prof Dragomir Radev@Yale) Advice for PhD Applications, Faculty Applications, etc (2023). [List of Suggestions]
- [(Roma Patel PhD@Brown, Prof Nathan Schneider@Georgetown University) PhD Application Series of the NLP Highlights Podcast) (2021). [Podcast] (A new series they launched that addresses all aspects of PhD application. Besides, it is just a great podcast in general that talks about recent NLP advances)
- (Albert Webson et al., PhDs@Brown University) Resources for Underrepresented Groups, including Brown's Own Applicant Mentorship Program (2020, but we will keep updating it throughout the 2021 application season.) [List of Resources]
- A Princeton CS Major's Guide to Applying to Graduate School. [List of suggestions]
- (Tim Dettmers, PhD@UW) Machine Learning PhD Applications — Everything You Need to Know (2018). [Guide]
- (Kalpesh Krishna, PhD@UMass Amherst) Grad School Resources (2018). [Article] (This list lots of useful pointers!)
- (Prof Mor Harchol-Balter@CMU) Applying to Ph.D. Programs in Computer Science (2014). [Guide]
- (CS Rankings) Advice on Applying to Grad School in Computer Science. [Pointers]
- (Prof Scott E. Fahlman@CMU) Quora answers on the LTI program at CMU (2017). [Article]
- (Prof Philip Guo@UCSD) Finding CS Ph.D. programs to apply to. [Video]
- [(Tim Dettmers, PhD@UW) How to Pick Your Grad School (2020). [Guide]
- (Zhijing Jin, PhD@MPI & ETH) Tips on PhD Applications with Max Planck Institute and/or ETH in AI (2021). [Suggestions]
- (Prof Nathan Schneider@Georgetown University) Inside Ph.D. admissions: What readers look for in a Statement of Purpose. [Article]
- (Nelson Liu, PhD@Stanfard) PhD Statement of Purpose. [Article]
- (Suchin Gururangan, PhD@University of Washington) Personal Statement Advice. [Article]
- (Prof Shomir Wilson@Penn State University) Reference Letter Procedure. [Suggestions]
- (Andrew Kuznetsov, PhD@CMU) CS/HCI PhD Opportunity Tracker from Twitter (Developed in 2021). http://www.andrewkuz.net/hci-opportunities-2022.html
- (Eugene Vinitsky, PhD@UC Berkeley) A Guide to Cold Emailing (2020). [Article]
- (Prof Shomir Wilson@Penn State University) Guide for Interacting With Faculty (2018). [Suggestions]
- (Prof Shomir Wilson@Penn State University) Reference Letter Procedure. [Suggestions]
Existing Summer Research Opportunities:
- Summer research opportunities for Undergrads (2021). [Twitter Thread]
- (ETH) ETH Summer Research Fellowship (every summer). [Apply]
- (MPI) Summer Research Internship with MPI (CaCTüS) (every summer). [Apply]
- Many people use Colab (and its Pro version costs 9.99$ per month)
- For more computationally intensive projects, you can apply to AWS with credits for research.
- For general computational needs, you can use a Digitalocean droplet (5-10$ per month), and run your ubuntu machine with any service for some time.
- (MIT 2020) The Missing Semester of Your CS Education (e.g., master the command-line, ssh into remote machines, use fancy features of version control systems).
Check all the specific suggestions under "Stage 2. (Doing PhD) How to Succeed in PhD?", such as the sections "What Is Weekly Meeting with Advisors like?", "How to Read Papers", and many others that you might need :).
- (Steve Yegge@Google) Get that job at Google (2008). [Article]
- (Carlos Bueno) Get that job at Facebook (2021). [Article]
- Coding practice from CareerCup. [Link]
- (Prof Isabelle Augenstein@UCopenhagen) Increasing Well-Being in Academia (2020). [Suggestions]
- (Sebastian Ruder@DeepMind) 10 Tips for Research and a PhD (2020) . [Suggestions]
- (Maxwell Forbes, PhD@UW) Every PhD Is Different. [Suggestions]
- (Prof Mark Dredze@JHU, Prof Hanna M. Wallach@UMass Amherst) How to be a successful PhD student (in computer science (in NLP/ML)). [Suggestions]
- (Andrej Karpathy) A Survival Guide to a PhD (2016). [Suggestions]
- (Prof Kevin Gimpel@TTIC) Kevin Gimpel's Advice to PhD Students. [Suggestions]
- (Prof Marie desJardins@Simmons University) How to Succeed in Graduate School: A Guide for Students and Advisors (1994). [Article] [Part II]
- (Prof Eric Gilbert@UMich) Syllabus for Eric’s PhD students (incl. Prof's expectation for PhD students). [syllabus]
- (Marek Rei, Lecturer@Imperial College London) Advice for students doing research projects in ML/NLP (2022). [Suggestions]
- (Prof H.T. Kung@Harvard) Useful Thoughts about Research (1987). [Suggestions]
- (Prof Phil Agre@UCLA) Networking on the Network: A Guide to Professional Skills for PhD Students (last updated: 2015). [Suggestions]
- (Prof Stephen C. Stearns@Yale) Some Modest Advice for Graduate Students. [Article]
- (Prof Tao Xie@UIUC) Graduate Student Survival/Success Guide. [Slides]
- (Mu Li@Amazon) 博士这五年 (A Chinese article about five years in PhD at CMU). [Article]
- (Karl Stratos) A Note to a Prospective Student. [Suggestions]
- (UMich; led by Prof Rada Mihalcea) A PhD Student's Perspective on Research in NLP in the Era of Very Large Language Models (2023). [Paper]
- (Prof Julian Togelius@NYU, Prof Georgios Yannakakis@UMalta) Choose Your Weapon: Survival Strategies for Depressed AI Academics Julian Togelius, Georgios N. Yannakakis (2023). [Tweet] [Paper]
- (Prof Jason Eisner@JHU) What do PhD students talk about in their once-a-week meetings with their advisers during their first year? (2015). [Article]
- (Brown University) Guide to Meetings with Your Advisor. [Suggestions]
- (Prof Jia-Bin Huang@UMaryland) How to have effective regular meetings with your advisors/mentors? (2023). [Suggestions]
Literature review tools:
- Connected Papers (which shows a graph of related papers) [link]
- (Built by Papers with Code) Glactica (using language models to generate literature review) [link]
Paper reading suggestions:
- (Prof Srinivasan Keshav@Cambridge) How to Read a Paper (2007). [Suggestions]
- (Prof Jason Eisner@JHU) How to Read a Technical Paper (2009). [Suggestions]
- (Prof Emily M. Bender@UW) Critical Reading (2003). [Suggestions]
- (Prof Jason Eisner@JHU) How to write a paper? (2010). [Suggestions]
- (Simon Peyton Jones@Microsoft) How to write a great research paper: Seven simple suggestions (2014). [Slides] [Talk]
- (Prof Jennifer Widom@Stanford) Tips for Writing Technical Papers (2006). [Suggestions]
- (Prof Shomir Wilson@Penn State University) Guide for Scholarly Writing. [Suggestions]
- (Prof Jia-Bin Huang@U Maryland) How to write the introduction (and also the What-Why-How figures). [Tweet]
- (Prof Jia-Bin Huang@U Maryland) How to write a rebuttal for a conference? [Tweet]
- (Prof Michael Black@Max Planck Institute) Twitter Thread about "Writing is laying out your logical thoughts". [Tweet]
- (Prof Shomir Wilson@Penn State University) Guide for Citations and References [Suggestions]
- (Carmine Gallo, a bestselling author) The Storytellers Secret (2016). [Video]
Takeaways: Writing the Introduction section and giving talks can also be like telling a Hollywood story: the setting (what problem we are solving; how important it is), the villian (how difficult this problem is; how previous work cannot solve it well), and the superhero (what we propose). For giving talks, starting with personal stories (e.g., a story of grandma telling the kid not to drink and persist the right thing leading to the person's life pursuit on social justice) is very helpful to get the audience involved. - (Maxwell Forbes@UW) Figure Creation Tutorial: Making a Figure 1 (2021). [Suggestions]
- UI design as a medium of thought: see Michael Nielsen's explanation of why UI is important for science, Bret Victor's work, Miegakure that visualizes a 4D environment.
- (Prof Jia-Bin Huang@U Maryland) How to write math in a paper? (2023). [Tweet]
-
(Prof Yang Liu, Trevor Cohn, Bonnie Webber, and Yulan He) Advice on Reviewing for EMNLP (2020). [Suggestions]
-
(Dr. Anna Rogers, Prof. Isabelle Augenstein@University of Copenhagen) What Can We Do to Improve Peer Review in NLP? (2020). [Paper]
-
(Prof Shomir Wilson@Penn State University) Guide for Publishing in Conferences and Attending Them [Suggestions]
-
(Prof Emily M. Bender@UW) On Using Twitter (2020). [Suggestions]
-
(Timothy Ferriss, author of 4-Hour Workweek) 5 Tips for E-mailing Busy People (2008). [Suggestions]
-
(Prof Philip Guo@UCSD) The Ph.D. Grind: A Ph.D. Student Memoir (last updated: 2015). [Video] (For the book, you have to dig deeply, and then you will find the book.)
-
(Maithra Raghu, PhD from Cornell, Senior Research Scientist@Google Brain) Reflections on my (Machine Learning) PhD Journey (2020). [Article]
-
(Prof Tianqi Chen@CMU) 陈天奇:机器学习科研的十年 (2019) (A Chinese article about ten years of research in ML). [Article]
-
(Jean Yang) What My PhD Was Like. [Article]
-
The most important step: (Prof Jason Eisner@JHU) How to Find Research Problems (1997). [Suggestions]
-
(Christopher Olah, OpenAI) Research Taste Exercises (2021). [Article]
-
(Prof Richard Hamming, Turing award winner) You and Your Research (How a research journey is like & how do end up with great research) (1995). [Talk] [Transcript] [Transcript2]
Interesting snippets: "Knowledge and productivity are like compound interest. Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity.", "One of the chief tricks is to live a long time!", "I made the resolution that I would never again solve an isolated problem except as characteristic of a class.". -
(Prof Stuart Card@Stanford) The PhD Thesis Deconstructed (2016). [Article]
Interesting snippets: "People basically read your paper to write theirs. Your ideas are more likely to spread if you help out.", "We once had a Nobel Laureate come talk to us about his ideas for how to do research. His surprising number 1 recommendation: Don’t work in an area that doesn’t have good funding.", "Table 1. Seismic scale of impact" -
(Sam Altman, CEO of OpenAI) How To Be Successful (2019). [Article]
Interesting snippets: "I think the biggest competitive advantage in business—either for a company or for an individual’s career—is long-term thinking with a broad view of how different systems in the world are going to come together... In a world where almost no one takes a truly long-term view, the market richly rewards those who do.", "One of the most powerful lessons to learn is that you can figure out what to do in situations that seem to have no solution.", "you also have to be able to convince other people of what you believe.", "My other big sales tip is to show up in person whenever it’s important. When I was first starting out, I was always willing to get on a plane.", "An effective way to build a network is to help people as much as you can.", "One of the best ways to build a network is to develop a reputation for really taking care of the people who work with you.", "The best way to make things that increase rapidly in value is by making things people want at scale." -
(Tim Dettmers, PhD@UW) On Creativity in Academia (2019). [Article]
-
(Prof Jia-Bin Huang@UMaryland) How to come up with research ideas? (2021). [Suggestions]
-
(John Schulman, co-founder of OpenAI) An Opinionated Guide to ML Research (e.g., horning your taste) (2020). [Suggestions]
Interesting snippets: "Goal-driven. Develop a vision of some new AI capabilities you’d like to achieve, and solve problems that bring you closer to that goal.", "If you are working on incremental ideas, be aware that their usefulness depends on their complexity.", "Consider how the biggests bursts of impactful work tend to be tightly clustered in a small number of research groups and institutions. That’s not because these people are dramatically smarter than everyone else, it’s because they have a higher density of expertise and perspective, which puts them a little ahead of the rest of the community, and thus they dominate in generating new results.", "Early on in your career, I recommend splitting your time about evenly between textbooks and papers. You should choose a small set of relevant textbooks and theses to gradually work through, and you should also reimplement the models and algorithms from your favorite papers." -
(Prof Fei-Fei Li@Stanford) De-Mystifying Good Research and Good Papers (2014). [Suggestions]
Interesting snippets: "This means publishing papers is NOT about “this has not been published or written before, let me do it”, nor is it about “let me find an arcane little problem that can get me an easy poster”. It’s about “if I do this, I could offer a better solution to this important problem,” or “if I do this, I could add a genuinely new and important piece of knowledge to the field.” You should always conduct research with the goal that it could be directly used by many people (or industry). In other words, your research topic should have many ‘customers’, and your solution would be the one they want to use. A good research project is not about the past (i.e. obtaining a higher performance than the previous N papers). It’s about the future (i.e. inspiring N future papers to follow and cite you, N->\inf)."
- (Elman Mansimov, Research Scientist@Amazon) Tips on summer industry research internships in ML (2021). [Suggestions]
List of fellowship opportunities to track:
-
(Compiled by CMU) Graduate Fellowship Opportunities [link]
-
(Robbie Allen) CS/HCI PhD Fellowships [link]
-
List of scholarships for different disciplines. [link]
-
CYD Fellowship for Grad Students in Switzerland [link]
How to Write a Good Fellowship Application:
-
(Daricia Wilkinson, PhD@Clemson) Towards Securing the Bag: Tips for Successful PhD Fellowship Applications (2019). [Article]
-
(Meta) Five tips for a successful Meta Research PhD Fellowship application from the people who review them (2020). [link]
-
(Meta) The six most common Fellowship questions, answered by Facebook Fellow Moses Namara (2020). [link]
-
(Meta) Fellowship 101: Facebook Fellow Daricia Wilkinson outlines the basics for PhDs (2020). [link]
-
(Meta) Applying twice: How Facebook Fellow David Pujol adjusted his application for success (2020). [link]
-
(Meta) Fellow spotlights, career advice, and more (up to date). [link]
-
The craft of Research by Wayne Booth, Greg Colomb and Joseph Williams.
-
How to write a better thesis by Paul Gruba and David Evans
-
Helping Doctoral Students to write by Barbara Kamler and Pat Thomson
-
The unwritten rules of PhD research by Marian Petre and Gordon Rugg
- (Wes Weimer@UMich, Claire Le Goues@CMU, Zak Fry@GrammaTech, Kevin Leach@Vanderbilt U, Yu Huang@Vanderbilt U, and Kevin Angstadt@St. Lawrence University) Finding Jobs (2021). [Guide]
- (Mu Li@Amazon) 工作五年反思 (A Chinese article about reflections on the five years working in industry) (2021). [Article]
- Academic Jobs Online https://academicjobsonline.org/ajo
- (Prof Devi Parikh and Prof Dhruv Batra@GaTech) Humans of AI: Stories, Not Stats [YouTube]
- (Prof Manuel Rigger@NUS) Getting Academic Positions (GAP) Interviewing Series (2021). [YouTube Interview Videos]
- (Prof Philip J. Guo@UCSD) Philip’s notes on the tenure-track assistant professor job search (2013). [Suggestions]
- (Prof Dragomir Radev@Yale) Advice for PhD Applications, Faculty Applications, etc (2023). [List of Suggestions]
- (Wes Weimer@UMich, Claire Le Goues@CMU, Zak Fry@GrammaTech, Kevin Leach@Vanderbilt U, Yu Huang@Vanderbilt U, and Kevin Angstadt@St. Lawrence University) CS Grad Job and Interview Guide (2021). [Guide]
Academic job search advice
-
(Prof Nicolas Papernot@University of Toronto, Prof Elissa M. Redmiles@Max Planck Institute) The academic job search for computer scientists in 10 questions. [Suggestions]
-
(Prof Caroline Trippel@Stanford) The Academic Job Search: A Memoir (2020). [Suggestions]
-
(Prof Shomir Wilson@Penn State University) Guide for the Tenure-Track Job Market in Computer/Information Sciences (2018). [Suggestions]
-
(Prof Michael Ernst@University of Washington) Getting an academic job (2000). [Suggestions]
-
(Prof Matt Might@University of Alabama at Birmingham) Academic job search advice. [Suggestions]
-
(Prof Manuel Rigger@NUS) How Did Professors Find Their Jobs? Part 1: Diversity in Experiences (2021). [Suggestions]
-
(Prof Radhika Nagpal@Harvard) The Awesomest 7-Year Postdoc or: How I Learned to Stop Worrying and Love the Tenure-Track Faculty Life (2013). [Article]
Sample Writing
- (Prof Anjalie Field@JHU) Research Statements (2022). [List]
- Different types of schools: https://carnegieclassifications.iu.edu/classification_descriptions/basic.php
- cra.org has a good listing for academic jobs related to CS
- The Chronicle of Higher Education has a listing that tends to be for more teaching-oriented positions
- CS Rankings: http://csrankings.org/
- An academic job portal for professors, lecturers, researchers, post docs, PhDs and academic managers in Europe, Americas, Oceania, Asia and the Middle East: FacultyVacancies.com
-
(Prof Jason Eisner@JHU) How to write an academic research statement (when applying for a faculty job) (2017). [Article]
-
(Prof Jason Eisner@JHU) How to Give a Talk (2015). [Suggestions]
-
(Prof Manuel Rigger@NUS) Interview Questions for Computer Science Faculty Jobs (2021). [Question List]
-
(Prof Austin Z. Henley@University of Tennessee) Faculty interview questions I asked and got asked (2018). [Question List]
-
(Prof Jason Eisner@JHU) Teaching Philosophy. [Article]
-
(Prof Isabelle Augenstein, Emily M. Bender, Dan Jurafsky, and Yoav Goldberg) Panel Discussions on How to Teach NLP Courses (2021). [Video]
-
(Prof Emily M. Bender@UW) Balancing Teaching and Research (2015). [Suggestions]
-
(Prof Shomir Wilson@Penn State University, motivation and usage of writing guides down) Guide to the Advice [Article]
-
(Prof Randy Pausch@CMU) Time management (2007). [Video] [Slides]
-
(Prof Charles Ling@Western University, Prof Qiang Yang@HKUST)Crafting Your Research Future: A Guide to Successful Master's and Ph.D. Degrees in Science & Engineering. [Book]
-
(Prof Michael Nielsen, now an individual researcher) Principles of Effective Research (2004). [Article]
Interesting snippets: "“We are what we repeatedly do. Excellence, then, is not an act but a habit.” Underlying all our habits are models (often unconscious) of how the world works. I’m writing this essay to develop an improved personal model of how to be an effective researcher, a model that can be used as the basis for concrete actions leading to the development of new habits.", "Make sure you’re fit. Look after your health. Spend high quality time with your family. Have fun. These things require a lot of thought and effort to get right.", "Develop a high-quality research environment", "Developing a taste for what’s important: What do you think are the characteristics of important science? What makes one area thrive, while another dies away? What sorts of unifying ideas are the most useful? What have been the most important developments in your field? Why are they important? What were the apparently promising ideas that didn’t pan out? Why didn’t they pan out? You need to be thinking constantly about these issues, both in concrete terms, and also in the abstract, developing both a general feeling for what is important (and what is not), and also some specific beliefs about what is important and what is not in your fields of interest.", "occasionally set time aside to survey the landscape of a field, looking not just for problems, but trying to identify larger patterns. What types of questions do people in the field tend to ask? Can we abstract away patterns in those questions? What other fields might there be links to? What are the few most important problems in the field?"
- (Prof Timothy Gowers and Michael Nielsen) Massively collaborative mathematics (2009). [Article]
- (Prof Jason Eisner@JHU) Technical Tutorials, Notes, and Suggested Reading (last updated: 2018) [Reading list]
- (All kinds of career advice for Cryptography researchers) Mentoring Workshop and Videos (2021) [Videos]
All types of contributions to this resource list is welcome. Feel free to open a Pull Request.
Contact: Zhijing Jin, PhD in NLP at Max Planck Institute for Intelligent Systems, working on NLP & Causality.
@misc{resources2021jin,
author = {Zhijing Jin},
title = {Resources to Help Global Equality for PhDs in NLP},
year = {2021},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/zhijing-jin/nlp-phd-global-equality}}
}