Methodology

Approach

Employment information was sought for all UBC PhD alumni who obtained their degree between 2005 and 2013 from UBC Vancouver. The study was designed to capture a snapshot of current employment information, and anticipated to be repeated at regular intervals (e.g., every 5 years) for the equivalent cohorts of graduates, i.e., 3 to 11 years after graduation.

Graduates were asked to complete an online survey, and internet searches were completed for all those who were not reached or did not respond to the survey. Data were collected in the spring of 2016, with the exception of those obtained through a pilot study in 2015 for the Interdisciplinary Studies Graduate Program (ISGP). Data for the philosophy and English programs were obtained through the national TRaCE project (with additional information sought for non-overlapping cohorts of the two studies) and were subject to the same data cleaning procedures outlined below.

The primary information sought was employment status, employer, job title, academic job category (if appropriate), and location. The survey allowed for both primary and secondary employers, and included the question: "Is your job a useful step along a desired career pathway?" with potential responses of yes, no, and unsure (leading to an answer field allowing free text). A final question asked for any additional comments, and allowed free text responses.

Survey population and response rates

Between 2005 and 2013, 3805 PhD degrees were granted by UBC. As data for the philosophy and English programs (72 graduates) were collected through the TRaCE project, the survey was designed to reach 3733 alumni. Of the 3733 graduates in these cohorts, there were 139 for whom email addresses were unavailable, and 444 whose email addresses were reported by the survey tool as invalid. Of the unavailable emails, 58 were successfully obtained, and of the invalid emails, 145 addresses were obtained through other means.

Thus, 3,358 survey invitations were delivered to recipients. Response rates varied by program and invitation letter (see below), with an overall rate of 55.6% (1866 responses).

Survey design and implementation

In an attempt to maximize response rates, the survey was designed to minimize the time it would take to complete. The questions were purposely limited, the entire survey was branched on the same page to avoid page reloading delays, and the survey was optimized for mobile devices.

Invitations included a survey URL with an anonymized token that allowed data to be linked in the back end. Thus, we did not request information already in the UBC Student Information System. It should be noted that the Student Information System only allows students to identify as male or female.

The data linking was reviewed by the Privacy Office at UBC, and was determined to be appropriate for this purpose, given that the publicly released dataset would be anonymized. The UBC Behavioural Research Ethics Board concurred that the survey qualified as a quality assurance/improvement and program evaluation study and hence did not require review.

The survey questions can be viewed at outcomes.grad.ubc.ca/docs/2016-UBCPhDCareerSurvey.pdf.

The survey for both the ISGP pilot and the main study were the same, and were conducted similarly, although the survey branching on the main study was slightly different, based on feedback we received.

Data collection

The survey was distributed through the UBC enterprise edition of FluidSurveys. Email addresses for all graduates were retrieved from the UBC Student Information System and the alumni database, with preference given to those in the alumni system. Graduates were grouped by program, with smaller programs within a faculty grouped together. Units with more than 35 graduates in the study time frame were invited to write a customized invitation to the survey (both text and sender name).

There were 52 separate invitations: 39 were customized by programs/faculties, and a generic group invitation was sent for the faculties of arts and medicine for very small programs. Otherwise, invitations came from the dean of Graduate and Postdoctoral Studies. The invitations were sent in phases over several months. The first survey (other than ISGP which opened on May 4, 2015) opened on February 16, 2016 and the last, on April 11, 2016. Depending on the wishes of departments, recipients were given deadlines of one to two weeks to complete the survey. However, deadlines were not strictly enforced, and surveys remained open even after the advertised deadline. Two to four days before each deadline, a reminder was sent to all those who had not yet completed the survey. Data collection concluded on April 20, 2016, and all surveys were taken offline at that time.

Internet research

For the second phase of the project (April 28 to June 27, 2016), two temporary staff conducted internet searches to retrieve employment position information for all graduates in the cohorts who had not completed the survey (1900 records). Instructions were provided to the staff, and the results they obtained from one program each were reviewed by G+PS staff for accuracy and adherence to the instructions. A training session was held to address any questions and ensure consistency between their approaches before work resumed.

The researchers were asked to ensure as much as possible that the individual identified online was also identified as having a UBC PhD degree, and preferably also identified with the correct program. They were also instructed to seek an institutional source for information (e.g., university websites). When institutional sources were unavailable, other sources, such as LinkedIn, personal websites, or publications, were used. Confirmation that the identified individuals had been granted a UBC PhD was not always possible; in those cases other information was considered, such as field of study, linkage with UBC individuals, etc. The results were categorized by reliability: high (e.g., data from institutional websites, confirmation of UBC PhD degree), medium (e.g., no primary source confirmation, but multiple sources confirming the same details), low (e.g., LinkedIn or social media account only), and unsuccessful (no results or an unsatisfactory level of reliability). 1127 results were categorized as meeting a high level of reliability, 130 as medium, 221 as low, and 343 as unsuccessful.

Many graduates were employed in multiple institutions and activities. In these cases, a judgment was made to identify an appointment or activity as primary, generally using defined criteria as noted below.

Criteria for determining the primary employment

A consistent approach was sought for the determination of primary employment titles and employers for all graduates with multiple occupations, both for those surveyed and those identified through internet searches. Sessional faculty appointments were generally considered to be primary employment positions, with the exception of those for which there was evidence of substantial engagement in a different occupation (e.g., the primary employment of an individual employed as a nurse who also teaches one course as a sessional would be "nurse"). A clinical faculty stream appointment was generally not considered to be the primary employment, as many or most such positions are non-salaried, and most clinical faculty are employed full-time as health professionals. Likewise, adjunct positions were not generally considered to be primary employment. If no significant alternate employment for clinical, adjunct, or other non-tenured faculty was identified, their primary appointment was considered a term faculty appointment.

Tenure-track and other apparently permanent academic appointments were always considered to be primary employment positions. For those with multiple occupations other than academic appointments, a judgment was made as to which was most likely the predominant one in terms of time and commitment. (For example, the primary employment of a graduate who worked both as a research manager and yoga instructor would be considered to be primarily employed as a research manager.)

The primary employment identified by survey respondents was normally left as such. Clarification was sometimes needed, and in those cases, internet searches were performed. A small number of changes were made for consistency both within the survey and between the survey and internet search approaches. This was relevant for both primary job titles and categorization (sector, academic level), as described further below. For individuals who identified more than one primary job title, the above criteria were used to determine a single title.

TRaCE

Graduates from the philosophy and English programs were tracked through the national TRaCE project, which used internet searches only (with subsequent interviews of selected individuals, not included in this report). An overview of the project may be found at iplaitrace.com/about-trace/, and methodological details are included at iplaitrace.com/wp-content/uploads/2017/01/Stage1_SUMMARY_final.pdf.

Statistical analyses

Statistical analyses were performed only where indicated, and used the Chi Square test, with a significance level of p < 0.05.

Data cleaning

All data, whether obtained through the survey or through internet searches, were reviewed by at least two people who had not participated in the retrieval of public data, and all were subject to the same data cleaning procedures. In addition to the identification of graduates' primary employment as described above, the main alterations were as follows:

General: Clarity was sought on all job titles and institutions, and attempts to obtain information missing from either the survey response or internet search were made. As an example of the former, the single word "professor" was often used as a job title, and follow up searches on these were made to clarify the precise title and role.

Sector: All employment data was categorized into one of four sectors as defined below. Some survey respondents were judged to have misclassified or inconsistently classified their sector (e.g., both "public sector" and "not-for-profit" sectors were selected among graduates working for local hospitals), and some sectors in the manually searched results were judged to have been misclassified. Also, the responses for sectors identified as "other" or "self-employed" (which shouldn't, in retrospect, have been offered as options) were categorized into one of the other sectors according to the criteria below and the comments provided by the respondents. Institutional attributes (e.g., whether publicly or privately owned) were identified through internet searches as necessary.

Employer names: These were standardized as much as possible, using identical names for the same employer (e.g., Google Inc. standardized as Google), and decoding most acronyms. If submissions included specific units or affiliated entities to a parent employer, this information was rolled up into the parent employer and the original submission was kept as a secondary classification (e.g., Department of Physics at University X would have become University X with secondary entry as Department of Physics).

Academic roles: There was substantial inconsistency in the assignment of academic roles to job titles, in data obtained both by survey and by internet search (including the TRaCE derived data). In retrospect, this was due, in part, to insufficiently clear definitions of roles provided to the respondents and researchers, and to the vast array of academic positions found world-wide. All academic roles were ultimately classified as consistently as possible after identifying further information on employer institutions as described in the Definitions section on page 6, and following the criteria and definitions listed there.

Unusual or unexpected data (internet search only): The occasional highly unusual or unexpected result was identified (e.g., a graduate with no medical education identified as being a physician), and these were followed up and corrected as necessary through internet searches.

Typographical errors: These were corrected.

Definitions

Associate researcher: Permanent positions with a primary focus on research carried out under the oversight of faculty members (e.g., research associate, research assistant).

Disciplinary groupings: Programs were grouped according to the Statistics Canada Classification of Instructional Programs 2000, the categorization system used for sharing U15 university data.

Higher education: All post-secondary institutions. Note that these may fall under different economic sectors (public, private or non-for-profit), but all were categorized only as higher education.

Not-for-profit sector: All entities of the economy owned by the private (i.e., non-government) sector, but that are not geared to making profits.

Other, Higher Education: All other occupations within higher education. These include staff administrative and research management positions, service positions, and postgraduate medical training positions.

Postdoctoral fellow (PDF): All positions identified as such or as variants thereof (postdoctoral researcher, postdoctoral associate, research fellow, etc.). These are expected to be temporary research training positions for individuals who obtained their degree within the previous few years.

Private sector: All entities of the economy that are owned by the private (i.e., non-government) sector, and geared to making profits. In general, self-employment (e.g., as a consultant or entrepreneur) was classified in this category, with exceptions including publicly funded health care providers in private practice.

Public sector: All units of the government sector and all publicly controlled non-financial and financial government business enterprises. This includes publicly funded hospitals and research institutes, public schools, Crown corporations, etc.

Research-intensive (RI) faculty: All positions titled assistant professor, associate professor, or professor, or the equivalent terms in non-Canadian countries (e.g., lecturer and its variants in many countries), employed by a PhD-granting institution. The latter stipulation is quite conservative, and was chosen mainly because it was too difficult to differentiate colleges from some universities, and the associated differences in faculty work and responsibilities. The expectation is that these individuals are independent researchers with teaching responsibilities, and generally have the opportunity to gain tenure. Not all appointments in this category, however, were tenure-track, and because the distinction could not usually be made, all positions with the relevant title were included. RI faculty who also held academic administrative positions, such as dean or associate dean, were included in this category.

Teaching-intensive (TI) faculty: Permanent faculty positions at higher education institutions that do not grant PhD degrees, and permanent faculty in PhD-granting institutions with responsibilities primarily for teaching (e.g., instructor stream).

Term faculty: All faculty positions in term appointments, such as sessional lecturers, as well as those in clinical or adjunct appointments for whom no significant other occupation was identified.

Programs by Disciplinary Group

UBC programs were grouped into the following disciplines for the purposes of this study.

Business

Business Administration

Education

Centre for Cross-Faculty Inquiry in Education; Educational and Counselling Psychology and Special Education; Curriculum and Pedagogy; Educational Studies; Language and Literacy Education

Engineering

Biomedical Engineering; Chemical and Biological Engineering; Civil Engineering; Electrical and Computer Engineering; Mechanical Engineering; Mining Engineering; Materials Engineering

Health Sciences

Anatomy and Cell Biology; Audiology and Speech Sciences; Biochemistry and Molecular Biology; Bioinformatics; Dentistry; Genetics; Kinesiology; MD/PhD Program; Medical Genetics; Experimental Medicine; Neuroscience; Nursing; Reproductive and Developmental Sciences; Interdisciplinary Oncology; Pharmacology and Therapeutics; Pathology and Laboratory Medicine; Pharmaceutical Sciences; Physiology; Rehabilitation Sciences; Population and Public Health

Humanities

Art History, Visual Art and Theory; Asian Studies; Classical, Near Eastern and Religious Studies; Central, Eastern and North European Studies; Comparative Literature; English; French, Hispanic and Italian Studies; History; Linguistics; Music; Philosophy; Theatre and Film

Sciences

Agricultural Sciences; Animal Science; Botany; Cell and Developmental Biology; Chemistry; Computer Science; Earth, Ocean and Atmospheric Sciences; Food Science; Forestry; Human Nutrition; Mathematics; Microbiology and Immunology; Physics and Astronomy; Plant Science; Resources, Environment and Sustainability; Soil Science; Statistics; Zoology

Social Sciences

Anthropology; Economics; Geography; Interdisciplinary Studies Graduate Program; Political Science; Psychology; Sociology; Gender, Race, Sexuality and Social Justice

Other Professional

Law; Library, Archival and Information Studies; Community and Regional Planning; Social Work