Four Norwegian universities offer programs in professional studies in psychology. Three of these institutes of higher learning exercise direct admissions through the Norwegian Universities and College Admission Services based on students’ grades from upper secondary school (Danielsen, Pedersen, & Straumsheim, 2011). In this article, direct admission refers to admission with grades from upper secondary school but should not be confused with admission without a pause in time after upper secondary school. The students range from coming directly from upper secondary school to having spent several years after graduation on other pursuits such as retaking exams, doing military service, or working in between. At the University of Tromsø – the Arctic University of Norway, students are admitted based on grades from a one-year introductory program in psychology. This latter form of admission used to be practiced by all Norwegian universities.
The change from indirect admission by introductory courses to direct admission based on grades from upper secondary school most recently took place at the University of Bergen (UiB) in 2016. Half of the students admitted in 2016 were admitted directly and half indirectly. After 2017, all students at the professional psychologist study program at UiB were admitted directly. The change in admission practices was debated over a long period of time. Some argued that students admitted directly would be more homogenous and have less insight into a possible career as a psychologist (Steine, 2010). Concerns also arose about the proportions of male students and socioeconomic status (Strand, 2010). Negative predictions regarding an increased dropout rate were also made, because students would be less satisfied with their chosen future career and study program (Johansen, 2014). Lastly, the earlier selection was predicted to admit students with less life experience, reflecting lower age at admission and fewer major life events outside of school. On the other hand, proponents of the switch argued that the potential changes would be small and pointed out that other institutions had experienced mostly positive results from such an admissions policy (Strand, 2010). These positive changes include fewer students retaking exams to achieve higher grades and students having an integrated six-year study program.
Admitted students are trained to become clinical psychologists and therapists. Their background, personality traits, and personal qualities influence patient outcomes (Von der Lippe, 2014). Studies suggest that between 6% and 8% of therapy outcomes might be explained by therapist effects (Kim, Wampold, & Bolt, 2006; Saxon & Barkham, 2012). The proportion of variance attributable to therapist training and the selection process is not known, but both may contribute to the overall effect. The therapist’s unique contribution to patient outcome makes the selection and admission of students to clinical studies in psychology a topic of interest to the universities supplying education, to the governmental institutions supplying certifications, and to the patients receiving treatment.
SES background and personality
In general, parental socioeconomic status (SES) may affect the student by supplying social capital and by making resources available in their homes (Leppel, Williams, & Waldauer, 2001). Children of highly educated parents experience more encouragement and pressure to attain higher education, and the parental educational level is a significant predictor of children’s future educational and occupational success (Dubow, 2009). Although the Norwegian educational system is, by and large, more egalitarian, the child-parent link in educational attainment is seen here as well (Undheim & Nordvik, 2006). Because of this tendency, it is likely that students admitted to high-status academic subjects, such as the professional psychologist program, come from high SES households. The degree of parental SES may, however, be partially contingent on when the students are admitted, given that the relationship between SES and academic achievement is more pronounced at earlier ages (White, 1982).
It is generally accepted that the degree of educational system stratification is associated with an inequality of educational opportunities (Horn, 2009). The earlier children are selected into educational tracks, the stronger the effect of their socioeconomic background on their educational attainment (Brunello & Checchi, 2007). Although Norway generally has low levels of educational stratification, it is plausible that admittance based on a broad and inclusive first-year introductory program may level out some of the determined socioeconomic differences in grades attained in upper secondary school. It is also possible that it will increase the admittance of students who academically mature later. In this view, late selection into high-status university education may function as an equalizer of early differences (Sirin, 2005). In addition to socioeconomic background, the change in admittance practices may also affect the personality disposition of the students, a sentiment that was discussed before such changes were implemented (Sandsether, 2010; Strand, 2010). Personality morphs from adolescence to early adulthood in girls and boys, and young women tend to be more mature than young men in early adulthood (Roberts, Caspi, & Moffitt, 2001). The change in time of selection, coupled with the possible change in gender ratio, may mean that the change in admittance practice will be associated with a change in the students’ dispositions. All of these potential changes are of interest because personality is linked to student behavior (Poropat, 2009). Also, any change in a student’s personality engenders differences in the graduating psychologist. Research shows that the personality of the therapist affects the patient–therapist bond (Taber, 2011). To the best of our knowledge, there is no evidence that different forms of admission are related to personality.
Objectives
The main aim of the current study was to examine and compare the socioeconomic background and study satisfaction among students admitted directly and indirectly to programs preparing them to become psychologists. Specifically, we wanted to examine four of the main public predictions that were made before the change in admission type was carried out:
- Students admitted directly come from families with higher SES than students admitted indirectly.
- Students admitted directly are less satisfied with the study program and their chosen career.
- Students admitted directly have lower rates of male students in their program of study.
- Students admitted directly have a higher likelihood of dropping out their first year.
Lastly, we wanted to explore possible differences in personality traits between students admitted directly and indirectly. The change in admission process may have led to differences in many traits, but we are specifically interested in possible differences in agreeableness, neuroticism, and conscientiousness due to these characteristics’ link with student behavior and possible therapeutic effects when the students complete their studies and become clinical psychologists.
Method
Participants and procedure
The present study was conducted as an online survey distributed to all students attending programs of professional studies in psychology at the University of Bergen (UiB) (N = 689). Data was collected from February 7, 2018 to May 1, 2018. A total of 202 participants completed the survey, making the participation rate of 29.3%. Of these, 29.8% of all students admitted directly and 28.8% of all students admitted indirectly participated. The sample consisted of 41 males and 160 females. Eighty-seven of the students were admitted under the new admittance practice based on upper secondary school grades, whereas 117 were admitted based on grades from a one-year study in psychology. The mean age was 24.14 (SD = 4.37). Partially completed responses were included in the analysis. Two participants were manually excluded from the sample because of evident mistaken answers given on the question of age.
Ethical considerations
Participants were informed about the voluntary nature of the study and their right to withdraw at any time. They were also assured of anonymity. The study was approved by the Data Protection Official for Research, NSD – Norwegian Centre for Research Data.
Measures
A survey consisting of 60 items was constructed with the intention of measuring several aspects of personality traits, study satisfaction, career choice satisfaction, age, socioeconomic status, and admission type. Additionally, we obtained records of student gender and dropout rates from NSD (www.dbh@nsd.no) and from the University of Bergen to investigate changes in gender and dropout rate before and after the change in admission practices. The findings are presented in Table 1. First-year dropout was calculated by adding together the cumulative drop in percentage from the first and second semesters. The dropout rate is a simple measure of the number of active students in a class, uncorrected for students dropping down or moving up a class. The dropout rate and gender distribution of the students starting in 2016 could not be divided into students admitted directly and indirectly.
Instruments
Personality – We measured personality traits with the questionnaire “Mini-IPIP,” a 20-item scale consisting of five sub-scales measured by four questions each that correspond to personality traits in the five-factor model of personality (FFM) (Donnellan, Oswald, Baird, & Lucas, 2006). Items were presented as statements. For each statement, participants indicated the degree of agreement or disagreement on a five-point Likert-scale (1 = Strongly disagree, 5 = Strongly agree). Higher levels of agreement on items measuring a trait result in an increase in the overall score of the trait. Cronbach’s alpha for Extraversion was 0.837, 0.762 for Conscientiousness, 0.688 for Agreeableness, 0.743 for Neuroticism, and 0.808 for Openness. “Mini-IPIP” has been found to be a valid measure of personality traits (Donnellan et al., 2006).
Satisfaction – We measured study satisfaction with four items based on an online public satisfaction survey for university administrations. The survey included items such as «I would recommend this study program to others.» Cronbach’s alpha for this measure was 0.828. Career choice satisfaction was measured by seven items based on the “Strength of Motivation for Medical School Questionnaire» (Nieuwhof et al., 2004). An example of an item on this questionnaire is «I can’t imagine becoming anything other than a psychologist.» Cronbach’s alpha for the measure was 0.782. All items mentioned above were accompanied by a five-point Likert-scale (1 = Completely disagree, 5 = Completely agree). Also, all items were translated from English to Norwegian. We used the translate-back translation method wherein items were first translated from English to Norwegian and then back to English from Norwegian by a different person. We compared the original and translated questions and found them to have identical or equivalent meanings (Paegelow, 2008).
Sociodemographic measures – The students were asked to rate their parents’ highest completed educational level with the response options: primary school, upper secondary school, bachelor’s degree, or master’s degree. The response options “primary school” and “high school” were collapsed into “basic education” due to few respondents specifying primary school as their parents’ highest level of education. Perceived family finances were measured by the following item: «describe your family’s financial situation.» Responses included: 1 = Very poor; 3 = Average; and 5 = Very good.
Participants were also asked to specify their age and gender. The type of admission was measured by one item asking if the student was admitted based on grades from a one-year introduction program in psychology or from an upper secondary school diploma.
Strategy of analysis
We used an exploratory analysis strategy to investigate the relationship among personality, background, and type of admission. R version 3.5.1 for Mac was used for all analyses (R Core Team, 2018). Chi-square tests (for categorical variables) and Welch two-sample t-test (for continuous variables) were used to examine possible differences between students admitted directly and indirectly on all outcome measures. To estimate effect sizes, the standardized mean difference was calculated for both continuous and categorical variables following the recommendations of Yang et al. (2012).
We simulated a dataset of the total number of students admitted to the professional psychologist program since 2012 (778 students) and generated categorical variables for admission, gender, and dropout. Because we could not separate students admitted directly and indirectly in the year 2016, we used the overall percentage to input male/female ratio and dropout rates for this year. We used two Chi-square tests to examine if being admitted directly was associated with a higher probability of dropping out and being a male student. To correct for multiple comparisons in this exploratory study, we set the Alfa level at 0.01 for all tests of differences between students admitted directly and indirectly. Missing data were handled with listwise deletion.
Results
The number of admitted students, the gender ratio, and dropout rates for the years 2012 to 2016 are displayed in Table 1. The percentage of male students vary from a low of 23% to a high of 29% from 2012 to 2015, when the students were admitted indirectly. The rate of male students dropped to 18% after the change to all students being admitted directly. The results show that the number of male psychology students was higher when students were admitted indirectly (χ2 (1) = 20.83, p < .001). The first-year dropout rates show small variations and a sharp increase in 2016. Students admitted directly are more likely to drop out during the first year of the study program (χ2 (1) = 42.13, p < .001).
Parental education, perceived family finances, and student satisfaction stratified by admission type are presented in Table 2. Having parents with a master’s degree was somewhat more common among students admitted directly (55.2% of mothers and 46.0% of fathers) than among students admitted indirectly (45.2% and 36.5%). Almost twice as many students who were admitted directly reported their family finances to be “very good” (19.5% vs. 10.4%). The effect sizes suggested a small difference between the groups on both parental education and perceived family finances. The groups were, however, not statistically significantly different from each other. We found no differences in study satisfaction, but students admitted directly were less satisfied with their career choice (t (200) = 3.14, p = 0.420). In our sample, students admitted indirectly were older (t (200) = 2.966, p = 0.003), but adjusting for the one-year introduction program in the indirect groups removed this difference (t (200) = 1.327, p = 0.185).
As shown in Table 3, we found no differences in extroversion, neuroticism, agreeableness, or imagination. We did, however, find a difference in conscientiousness (t (200) = 2.60, p = 0.009) with a small effect size (d = 0.379). Students admitted indirectly were found to be more conscientiousness than students admitted directly.
Discussion
The aim of this study was to examine differences between students admitted directly from upper secondary school and students admitted indirectly from a one-year intro psychology program to the professional psychologist program at UiB. Students admitted directly were predicted to come from families with higher socioeconomic status, to display lower satisfaction with the study program and their chosen career, to consist of fewer male students, to have a higher dropout rate, and to possess different personality traits compared to their counterparts who were admitted via the introductory course in psychology. We found partial support for the hypothesis of higher SES among student admitted directly. More students admitted directly had mothers with a high educational level and better family finances. However, the differences were small and non-significant, possibly due to a lack of statistical power. The results also partly confirm the hypothesis of a reduction in student and career satisfaction. Students admitted directly were less satisfied with their chosen career. The reason for this dissatisfaction is likely twofold, and we offer the following interpretation. Firstly, having a year to study psychology and viewing the professional psychologist program more closely provides information on how such a career path pertains to personal preferences. In contrast, students admitted from upper secondary school have limited knowledge about the program’s contents. They also have limited information about psychology as a career and what being a therapist is all about. Secondly, students admitted directly have a generally high grade-point average from their upper secondary school and as such are able to pick and choose from multiple high-ranking study options. Students admitted from the one-year introductory program have high grades from their first-year studies but not necessarily from their upper secondary school. We suspect that the presence of choice between several high-ranking programs leads to reduced career choice satisfaction.
The results confirm the hypothesis that students admitted directly had fewer male students, which implies that the type of admission is related to the ratio of male to female students. The results also confirm the hypothesis that students admitted directly are more likely to drop out during the first year. The change in dropout rates may be partly due to their lower career satisfaction but not their overall program of study satisfaction. It is likely that students admitted directly apply to the professional psychologist program at multiple institutions. They may be content with their study program but transfer when they are accepted into their institution of choice. Lastly, the personalities of the students admitted directly and indirectly were quite similar. We did, however, find a difference in conscientiousness, where students admitted indirectly were more conscientious. This tendency may be related to the selection of students. A hallmark of the one-year study period, which all students admitted indirectly have achieved success in, is its independence from mandatory student activities. An introductory first-year study period may create a high selection of conscientiousness because only conscientious students maintain superior student effort without external pressures. In contrast, achieving high marks in upper secondary school might be less dependent on conscientiousness because students here have more mandatory study activity and external pressure from both the school and their family.
In total, the results indicate that the student mass has changed slightly in the ways predicted before the change in program admittance, but the small nature of the changes does not support the notion of substantial differences in the graduating future psychologists.
Implications for the future therapists
Although research indicates that therapy outcomes are partly attributable to therapist effects (Saxon & Barkham, 2012), the complicated nature of patient/therapist interaction makes outcome predictions due to changes in gender, SES, and conscientiousness of future therapist difficult. There is no overall main effect of gender on patient outcomes (Okiishi et al., 2006) but the decreased rate of male students might prove to change aspects in future psychologist/patient relationships since men elicit different reactions in some therapeutic situations (Shields & McDaniel, 1992). There is also some evidence to suggest gender match between therapist and patient is associated with better outcomes (Luborsky, 1971). The results are mixed, however, and more recent evidence fails to support this notion (Cottone, 2002). Nonetheless, adolescent boys do rate their alliance with a female therapist lower and are more likely to drop out compared to a male-male patient/therapist dyad (Wintersteen, 2005).
To the best of our knowledge, there is no evidence evaluating the effects of therapist SES on therapy outcomes. However, arguments for a more diverse and lower SES therapist population is based on two premises. One, psychologists are in a unique position of understanding psychological phenomena from individual and societal perspectives. Relative increases in SES might lead to therapists with a lower sensitivity to psychological problems that are rooted in societal issues. Higher SES therapists might not be able to relate to or understand the problems faced by lower SES patients or to experience the same life challenges. Second, lower SES patients might be less likely to seek help from therapists from another stratum of society. Low levels of family income are related to decreased treatment-seeking attitudes for psychological problems (Leaf, 1987). Increasing differences between patient and therapist may exacerbate this problem.
There is no direct evidence indicating that changes in conscientiousness in future psychologists will positively or negatively affect the therapeutic process or outcome. However, the trait of conscientiousness is an essential predictor of study habits. It predisposes students to spending more time on their studies as well as applying more effective time management strategies (Ryan, Delaney, & Harmon, 2010). Moreover, procrastination is less common among conscientious students (Lee, Kelly, Edwards, 2006), and in general, conscientious individuals tend to be more hardworking than their less conscientious peers (Bakker, Demerouti, & Ten Brummelhuis, 2012).
Limitations and strengths
As a basis for understanding the possible differences between students admitted directly and indirectly psychologist programs, this study is unique in that it directly compares students during the same time period at the same institution. In a few years, this direct comparison will be impossible, because the students admitted indirectly will have graduated. The main limitation of the current study was the relatively low participation rate of 29.3% that may limit the generalizability of the results. Unfortunately, attrition from survey studies is on the rise(Morton, Bandara, Robinson, & Carr, 2012), and non-response is more common among respondents with lower socioeconomic status and among males (e.g., Fan & Yan, 2010; Galea & Tracy, 2007). Indeed, compared to the official gender distribution obtained from UiB, the current study had slightly lower rates of male respondents. Further, it is quite possible that to a lesser degree, the current study was able to recruit students from lower SES families or that less motivated or satisfied students were more reluctant to participate in the study. Students admitted via the one-year introductory course in psychology have also spent a longer time as psychology students. This tendency may have resulted in a selection process where the least satisfied students have dropped out of the study program by the time of data collection.
Another limitation is the use of a new measure for career choice satisfaction. Although the measure is based on and translated from a similar measure, we cannot confidently ascertain its construct validity. The inter-item reliability does, however, indicate sufficient reliability. We believe that it is pertinent these limitations are considered when interpreting the results of the current study.
Conclusion
This study aimed to examine the associations between admission practice and psychology students’ socioeconomic background, study satisfaction, and personality. The overall finding was that no substantial differences existed between students admitted via the one-year introductory course in psychology compared to those admitted based on grades from upper secondary school. The change in admission process may, however, have created some diverging trends between the two student groups.
Students admitted directly from upper secondary school may be somewhat less diverse.
References
Bakker, A. B., Demerouti, E., & Lieke, L. (2012). Work engagement, performance, and active learning: The role of conscientiousness. Journal of Vocational Behavior, 80, 555–564. doi:10.1016/j.jvb.2011.08.008
Brunello, G., & Checchi, D. (2007). Does school tracking affect equality of opportunity? New international evidence. Economic Policy, 22, 782–861. doi:10.1111/j.1468-0327.2007.00189.x
Cottone, J. G., Drucker, P., & Javier, R. A. (2002). Gender differences in psychotherapy dyads: Changes in psychological symptoms and responsiveness to treatment during 3 months of therapy. Psychotherapy: Theory, Research, Practice, Training, 39, 297–308. doi:10.1037/0033-3204.39.4.297
Danielsen, E., Pedersen, T, B., & Straumsheim, P, A. (2011, July 25). Psykologutdanningen i Norge [Psychologist study programs in Norway]. Retrieved from: https://tinyurl.com/y8jswx2z
Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E. (2006). The Mini-IPIP scales: Tiny-yet-effective measures of the big five factors of personality. Psychological Assessment, 18, 192–203. doi:10.1037/1040-3590.18.2.192
Dubow, E. F., Boxer, P., & Huesmann, L. R. (2009). Long-term effects of parents’ education on children’s educational and occupational success: Mediation by family interactions, child aggression, and teenage aspirations. Merrill-Palmer Quarterly, 55, 224–249. doi:10.1353/mpq.0.0030
Fan, W., & Yan, Z. (2010). Factors affecting response rates of the web survey: A systematic review. Computers in Human Behavior, 26, 132–139. doi:10.1016/j.chb.2009.10.015
Galea, S., & Tracy, M. (2007). Participation rates in epidemiologic studies. Annals of Epidemiology, 17, 643–653. doi:10.1016/j.annepidem.2007.03.013
Horn, D. (2009) Age of selection counts: A cross-country analysis of educational institutions, Educational Research and Evaluation, 15, 343–366. doi:10.1080/13803610903087011
Johansen, S, T. (2014). Inntakssystemet til profesjonsutdanningen ved UiB endres. Retrieved from: https://tinyurl.com/y8qnpfax
Kim, D. M., Wampold, B. E., & Bolt, D. M. (2006). Therapist effects in psychotherapy: A random-effects modeling of the National Institute of Mental Health treatment of depression collaborative research program data. Psychotherapy Research, 16(2), 161–172. doi:10.1080/10503300500264911
Lee, D. G., Kelly, K. R., & Edwards, J. K. (2006). A closer look at the relationships among trait procrastination, neuroticism, and conscientiousness. Personality and Individual Differences, 40(1), 27–37. doi:10.1016/j.paid.2005.05.010
Leaf, P. J., Bruce, M. L., Tischler, G. L., & Holzer III, C. E. (1987). The relationship between demographic factors and attitudes toward mental health services. Journal of Community Psychology, 15, 275–284. doi:10.1002/1520-6629(198704)15:2
Leppel, K., Williams, M. L., & Waldauer, C. (2001). The impact of parental occupation and socioeconomic status on choice of college major. Journal of Family and Economic Issues, 22, 373–394. doi:10.1023/a:1012716828901
Luborsky, L., Auerbach, A. H., Chandler, M., Cohen, J., & Bachrach, H. M. (1971). Factors influencing the outcome of psychotherapy: A review of quantitative research. Psychological Bulletin, 75, 145–185. doi:10.1037/h0030480
Morton, S. M., Bandara, D. K., Robinson, E. M., & Carr, P. E. A. (2012). In the 21st century, what is an acceptable response rate? Australian and New Zealand Journal of Public Health, 36(2), 106–108. doi:10.1111/j.1753-6405.2012.00854.x
Nieuwhof, M. G., ThJ, O. T. C, Oosterveld, P., & Soethout, M. B. (2004). Measuring strength of motivation for medical school, Medical Education Online, 9, 1, doi:10.3402/meo.v9i.4355
Okiishi, J. C., Lambert, M. J., Eggett, D., Nielsen, L., Dayton, D. D., & Vermeersch, D. A. (2006). An analysis of therapist treatment effects: Toward providing feedback to individual therapists on their clients’ psychotherapy outcome. Journal of Clinical Psychology, 62, 1157–1172. doi:10.1002/jclp.20272
Paegelow, R. S. (2008). Back translation revisited: Differences that matter (and those that do not). The ATA Chronicle, 1, 22–25.
Pedersen, T. B., Straumsheim, P. A., & Danielsen, E. (2015, Nov 20). Psykologutdanningen i utlandet [Psychologist study programs abroad]. Retrieved from: https://tinyurl.com/yaqrrf7r
Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135, 322–338. doi:10.1037/a0014996
Roberts, B. W., Caspi, A., & Moffitt, T. E. (2001). The kids are alright: Growth and stability in personality development from adolescence to adulthood. Journal of Personality and Social Psychology, 81, 670–683. doi:10.1037/0022-3514.81.4.670
Ryan, Martin, Delaney, Liam, and Harmon, Colm P., Micro-level determinants of lecture attendance and additional study-hours. IZA Discussion Paper No. 5144. Available at SSRN: https://ssrn.com/abstract=1667768
Sandsether, Ø. (2010). Perspektivløst om opptaksordning [A lack of perspective on admittance practices]. Tidsskrift for Norsk Psykologforening, 47, 962–963.
Saxon, D., & Barkham, M. (2012). Patterns of therapist variability: Therapist effects and the contribution of patient severity and risk. Journal of Consulting and Clinical Psychology, 80, 535–546. doi:10.1037/a0028898
Shields, C. G., & McDaniel, S. H. (1992). Process differences between male and female therapists in a first family interview. Journal of Marital and Family Therapy, 18(2), 143–151. doi:10.1111/j.1752-0606.1992.tb00925.x
Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75, 417–453. doi:10.3102/00346543075003417
Steine, I. (2010). Grunner til å beholde dagens ordning [Reasons to keep the current practice]. Tidsskrift for Norsk Psykologforening, 47, 545–546.
Strand, N. (2010). Bergen beholder opptak fra årsstudiet [Bergen retains admittance from the one-year introduction program]. Tidsskrift for Norsk Psykologforening, 47, 873–874.
Taber, B. J., Leibert, T. W., & Agaskar, V. R. (2011). Relationships among client–therapist personality congruence, working alliance, and therapeutic outcome. Psychotherapy, 48, 376–380. doi:10.1037/a0022066
Undheim, J. O., & Nordvik, H. (1992) Socio‐economic factors and sex differences in an egalitarian educational system: Academic achievement in 16‐year‐old Norwegian students. Scandinavian Journal of Educational Research, 36, 87–98. doi:10.1080/0031383920360201
Von Der Lippe, A., Nissen-Lie, H. A., & Oddli, H. W. (2014). Psykoterapeuten: En Antologi om Terapeutens Rolle i Psykoterapi [The Psychotherapist: An Anthology on the Therapist’s Role in Psychotherapy]. Oslo: Gyldendal Akademisk.
White, K. R. (1982). The relation between socioeconomic status and academic achievement. Psychological Bulletin, 91, 461–481. doi:10.1037//0033-2909.91.3.461
Wintersteen, M. B., Mensinger, J. L., & Diamond, G. S. (2005). Do gender and racial differences between patient and therapist affect therapeutic alliance and treatment retention in adolescents? Professional Psychology: Research and Practice, 36, 400–408. doi:10.1037/0735-7028.36.4.400
Yang, D., & Dalton, J. E. (2012). A unified approach to measuring the effect size between two groups using SAS®. SAS Global Forum, 335, 1–6.