The systematic use of client feedback procedures during therapy is a means of assessing clients’ progress that can be used to supplement clinical intuition (e.g., Lambert, 2010; Lambert & Shimokawa, 2011; Lutz, Bøhnke, & Køck, 2011; Sexton, Patterson, & Datchi, 2012). When therapists acquire information about clients’ lack of progress, they tend to respond with significant adjustments to improve the outcomes for these clients (Anker, Duncan, & Sparks, 2009; Bickman, Kelly, Breda, de Andrade, & Riemer, 2011; Lambert, Whipple, Vermeersch, Smart, Hawkins, Nielsen, & Goates, 2002; Miller, Duncan, Brown, Sorrell, & Chalk, 2006). However, little research has been directed toward therapist-client communication in sessions subsequent to client feedback (Hatfield & Ogles, 2007; Tracey, Wampold, Lichtenberg, & Goodyear, 2014). In this paper, we review the empirical literature on therapists’ own experiences of using feedback procedures. Because feedback systems proliferate, it is important to understand more clearly how therapists are using them.

Feedback can be defined as the process of ‘bringing into the recipient’s awareness the discrepancy between what is thought and what is reality, thereby prompting corrective action’ (Lambert, 2010, p. 110). Several feedback procedures and instruments have been developed over the last two decades (for overviews, see, for example, Lambert, 2010; Overington & Ionita, 2012). Feedback procedures in therapy typically combine a self-report form from the client (in this paper termed ‘structured feedback’) and a less-structured conversation between client(s) and therapist about the obtained feedback information (in this paper termed ‘feedback conversation’).

There are many unexplored facets of feedback procedures, and the focus of the present literature review is the therapists’ experiences when they choose to respond, through a conversation with their client(s), about the information on the self-report form.

The structured feedback may capture data on progress, change, and process, all terms whose interpretation depends to some extent on one’s theoretical frame of reference. However, the use of feedback procedures is considered relevant by many in almost every theory-driven therapeutic approach (Duncan, Miller, & Sparks, 2004; Pinsof & Chambers, 2009), and it is often regarded as pan-theoretical (Overington & Ionita, 2012). The contextual feedback intervention theory (Riemer, Rosof-Williams, & Bickman, 2005) suggests that therapists’ goal for themselves or their client (e.g., a good alliance, a positive outcome, or successful therapy) will influence how they interpret the feedback information (Sapyta, Rieemer, & Bickman, 2005). The hypothesis is that a discrepancy between the goal and the feedback information (e.g., a poor alliance, negative client feedback, or unsuccessful therapy) will motivate the therapist to make changes. Such goals, however, may be strongly influenced by the therapist’s theoretical orientation. Because the development of feedback procedures occurs within theoretical frameworks or schools of thought, the utilization of such instruments do as well. With the inclusion of a feedback conversation about the structured feedback information, the therapists’ and clients’ understanding of the feedback information’s meaning may be merged and integrated into the therapist’s theoretical preferences. Moreover, such understanding may influence the therapists’ clinical impression of the client. This tendency adds a relational aspect of feedback procedures to the individual emphasis in the contextual feedback intervention theory (Sapyta et al., 2005). More specifically, the feedback conversation as an integrated part of the feedback procedure is given less attention.

Feedback procedures add to a mere outcome measure by helping the client and therapist establish and repeatedly adjust a shared understanding of the problems presented. They also aid in the development of strategies for solving those problems together (Pinsof, Breunlin, Russell, & Lebow, 2011). Feedback and the conversation about that feedback are recognized as possible powerful interventions (Pinsof, Goldsmith, & Latta, 2012; Sundet, 2012). In this review paper, these processes are explored within the context of the feedback conversation with a focus on reports of therapists’ first-hand experience with such conversations, regardless of the feedback procedure. The aim of the present paper was to review and discuss the empirical literature under the guidance of the following question: What do therapists say about their experiences with feedback procedures in their therapeutic practices?

Method

Search method
We searched relevant databases for papers that explicitly described, or cited therapists’ experience with, the clinical use of a feedback instrument that included a conversation between therapist and client about the feedback information. The following databases were searched: psychINFO, Ovid MEDLINE (R), Embase, ProQuest, and Web of Science. The search terms were (feedback OR feed-back) AND (assess* OR monitor* OR measure* or instrument) AND (therapist* OR psychotherapist* OR counsellor* OR counselor*) adj3 (experience* OR feeling* OR reaction* OR response* OR cop* OR handl*). In the ProQuest and Web of Science searches, the search function adj3 was substituted with NEAR/2. The search included all publications until December 2014. In all databases, the advanced search option was chosen.

After duplicates were omitted, the search yielded a total of 192 matches, including two book chapters, two reports, one newspaper article, 51 dissertations and theses, and 135 peer-reviewed articles. All abstracts were then read, and the papers were sorted according to our inclusion criterion: that they present empirical data on therapists’ own and first-hand experiences of sharing and discussing structured feedback information with their clients. Thus, we excluded studies on the effect of feedback in therapy (25 papers), studies on feedback provided in clinical supervision (71 papers), studies on computer-generated feedback (15 papers), studies on feedback given to researchers (11 papers), effect studies on therapy or aspects of therapy (12 papers), studies on measures other than feedback measures (eight papers), reviews and descriptions of measures and methods involving therapists giving feedback to clients on, for example, sexual risk behaviour or smoking hazards (25 papers), studies of experience with methods other than feedback (18 papers), and studies on factors affecting feedback (one paper). A total of six papers in the database search met our inclusion criteria.

We also searched the following six websites: www.psychotherapynetworker.org, www.heartandsoulofchange.com, www.psychotherapy.net, www.pluralistictherapy.com, www.family-institute.org, and www.r-bup.no. Through this website search we identified two additional Norwegian reports (in the Norwegian language) that have not been published in a peer-reviewed journal. These two reports were included in the review.

Included papers
A total of eight papers were included in the review. (See Table 1 for an overview of the key features of the included papers.) The total number of therapist participants was 227. (One study did not report participant numbers.) The participants included social workers, psychologists, family therapists and student family therapists, psychiatrists, general practitioners, and occupational therapists. Not all studies reported the ages of the participants; however, labels such as ‘student therapist,’ ‘experienced therapist,’ and ‘senior therapist’ were sometimes used. In the following analysis, all participants in the included papers are referred to as «therapists.» The therapists worked in psychotherapy (six papers), occupational therapy (one paper), and general medical practice (one paper). Five feedback procedures were described across the eight studies: the PCOMS (Miller & Duncan, 2004), the Therapy Personalisation Form/Therapy Personalisation Form – Assessment (TPF/TPF-A; Bowens, Johnstone, & Cooper, 2011; Cooper & McLeod, 2011), the Patient Perspective Survey (PPS; Laerum, Steine, Finckenhagen, & Finseth, 2002), the Psychological Outcome Profiles (PSYCHLOPS; Ashworth, Robinson, Godfrey, Parmentier, Shepherd, Christey, J., . . . Matthews, 2005), and the Children’s Occupational Self-Assessment (COSA; Keller, Kafkes, Basu, Federico, & Kielhofner, 2005).

For further details on the feedback instruments, see Table 2. Although they are different, these feedback procedures all share the purpose of letting the opinions of the client(s) influence the collaborative therapeutic work. In the following analysis, all treatment and collaborative work referred to in the included papers is termed «therapy.»

Analysis
An initial data analysis was conducted by the first author. The identified eight papers were read several times. Initial thoughts, ideas, and emerging themes were recorded. Descriptions and quotations of therapists’ first-hand experience with the utilization of a feedback procedure in therapy were identified. Thus, published descriptions of therapists’ first-hand experiences with feedback procedures constituted our empirical data.

Based on procedures for thematic analysis (e.g., Braun & Clarke, 2006), we analyzed the empirical material by following three steps: (1) The text material was coded through several detailed readings, guided by the research question; (2) The codes were condensed and organized into tentative categories; and (3) The categories were sorted into possible overarching themes. These themes were refined so that the data within each theme cohered meaningfully and the themes sufficiently differed from each other. The analysis was carried out by the first author, with continuous discussions with the third and fourth author throughout the entire process for enhanced internal validity. We adopted a similar approach to that used by Ness, Borg, and Davidson (2014) across seven papers in their review of first-person perspectives on facilitators and barriers in dual recovery. In their literature review, the authors utilized thematic analysis inspired by Braun and Clarke (2006) to examine first-person accounts from a number of different contexts.

Results

We identified four themes related to therapists’ experiences with receiving feedback information and discussing that feedback with their clients.

Theme 1: Adoption of a meta-perspective on therapeutic process and practice
This emerged as a theme based on the subcategories permission to address therapy at a meta-level and opportunity to discuss misunderstandings. The therapists reported that the structured feedback gave both themselves and their clients some sort of permission to address the therapeutic process at a meta-level through the feedback conversation. This tendency is described by Sundet (2012): «…the SRS [part of PCOMS] provided an opening for the service users to voice their frustrations about the therapeutic work and legitimized talk about frustrations» (p. 126).

Permission or the opportunity to discuss the therapeutic process was particularly appreciated in cases where the therapist thought the therapy was not going the way the client(s) or the therapist hoped or wanted. Under these circumstances, the use of structured feedback generated discussion within the frame of feedback conversations that helped to move the therapy forward. Topics of discussion included a more sensitive conceptualization of problem areas, the potential oversight of important aspects of therapy, and developments in the work being done.

The process of listening to clients’ explanations of the structured feedback data gave both the therapist and the client(s) an awareness of misunderstandings or misinterpretations that arose during the sessions. The feedback conversation provided the opportunity to achieve greater agreement about how to proceed with the therapy.

Theme 2: Emergence of incentives to move beyond one’s personal and professional comfort zone
This theme is constituted by the two subcategories enhancement of the therapist’s confidence and feedback stimulated the therapists to reflect on their practice. Based on the feedback conversation, the therapists were given incentive to acquire new skills, such as responding to the client’s wishes, being more confrontational, and giving homework. In this context, therapists experienced both the structured feedback and the feedback conversation as encouraging and reassuring.

The feeling of confidence that feedback conversation engendered was especially important when the therapist was uncertain about what to do and how to proceed. One informant in the study by Sparks, Kisler, Adam, and Bloom (2011) said, «It definitely makes it easier to start…; it just allows that opening to know where to go right from the beginning of a session and not to miss certain areas» (p. 463). However, some therapists reported that use of a feedback procedure increased their confidence and feeling of security only if the client also found the procedure helpful. The therapists further reported that conversations about the structured feedback generated new knowledge for both them and their clients. Direct, immediate feedback from the structured as well as the conversational part of the procedure seemed to have the greatest potential to change the way therapists understood and conducted therapy with a particular client or clients. One of the participants in the study by Bowens and Cooper (2012) reported that she «(…) learned from the TPF that he [the client] was very uncomfortable with silence» (p. 55). This individual also revealed that such feedback directly contradicted her intuitive clinical impression based on previous sessions.

Finally, sometimes the feedback, regardless of its form, presented an emotional challenge for the therapist and led to self-criticism. Such a result also occurred if the feedback was overly positive. Overall, feedback seemed to stimulate therapists to reflect on their own practice. For instance, a feedback conversation with one client could inspire a therapist to reflect on how to develop his or her practice more generally.

Theme 3: Greater awareness of children’s and adolescents’ perspectives during therapy
This theme is based on the subcategories insight into children’s and adolescents’ life and world, structured feedback as a way of showing respect for children’s and adolescents’ life and world, and feedback procedures are not always suitable for children. Two studies used feedback procedures that were specifically designed to assess children’s perspectives (COSA and the child version of PCOMS; see Table 1). Children sometimes gave information about challenges or problems about which both their parents and their therapists were unaware. This information helped the therapists to understand more about the children’s situation in general and the ongoing therapy in particular. Considering this information thus represented a gateway for the therapists to show respect for children’s and adolescents’ perspectives in their therapeutic practice.

Sometimes, the use of feedback procedures created challenging situations. Examples were when the child did not want to give structured feedback or provided ambiguous feedback that he or she found difficult to elaborate on when requested to do so in the feedback conversation. The therapists also reported that feedback often reflected the child’s immediate situation to a greater extent than his or her general situation. In such cases, the use of a feedback procedure engendered questions about how to interpret the child’s feedback, as described in the study by Sundet (2012): «High scores on the SRS elicited questions about whether there was an agenda to please the therapists and whether it was likely for the children to give high scores until they felt safe» (p. 126). These findings illustrates that the feedback conversation was not always helpful in understanding more of children’s and adolescents’ perspectives during therapy.

When information provided by children through a feedback procedure was in accord with the treatment goal formulated by adults, it was considered valuable and was acted upon quickly. However, the feedback procedure sometimes revealed discrepancies between the child’s beliefs and those of his or her parents or therapist. For example, a child might consider a situation normal whereas a therapist might not, as may be the case in a potentially abusive situation. In these situations, as well as in other situations, professional knowledge was usually given precedence over the child’s or parent’s perspective in the context of planning and intervention. However, negotiation, demonstration of capacity, and professional reflection were commonly used in the feedback conversation to reach a shared understanding between the involved parties.

Theme 4: Influence of structured feedback on collaboration with clients
This theme is based on the subcategories feedback stimulated and facilitated collaboration, negative influence on building of alliance, and need for cultural understanding. The feedback procedure was considered helpful in stimulating and facilitating collaborative work with clients in all processes or steps of the therapy, including alliance work. This helpfulness was reported across all studies. Even when the procedure was considered difficult for the client, such as when the client was required to state and identify in his or her own words problem areas to prioritize, the therapists considered the procedure meaningful for both the client and themselves as professionals. The feedback conversation helped to add meaning to the structured and less individualized part of the feedback procedure.

However, in families with a high level of intense emotions (such as in situations of acute crisis or domestic violence) or mistrust, shared feedback between family members was considered to potentially place some family members at risk. To display and share structured feedback as well as to arrange feedback conversations in such situations was viewed as potentially negatively affecting the establishment of a therapeutic relationship. The feedback information obtained in such difficult circumstances was often not trusted.

One participant in the study by Tuseth, Sverdrup, Hjort, and Friestad (2006) said, «One has to be open to the fact that sometimes it may be too threatening to give feedback» (p. 59). This statement reflects the therapist’s openness to feedback as an optional part of therapy, not a condition. It also reflects that not wanting to give structured feedback may be a topic for feedback conversation.

The purpose of the feedback procedure was occasionally not clear to clients. One example was therapies involving newly arrived immigrants. One participant in the study by Tuseth et al. (2006) reported, «If the clients are asylum seekers or in other ways are uncertain about which formal or civil rights they have, forms or an interpreter may create the feeling of insecurity» (p. 59). The therapists offered several explanations for clients’ unwillingness to partake in a feedback procedure, such as the lack of a mutual understanding of important cultural knowledge. Other reasons mentioned in the reviewed papers included the client’s desire to please the therapist, the client’s tendency to place the therapist in the role of the expert, and the client’s hesitation to record his or her personal information in writing. Several therapists reported that the use of an interpreter often exacerbated these problems and complicated the therapeutic situation. However, these situations did not occur that often, and on a daily basis the feedback conversation proved to be of significance to the overall collaborative aspects of the therapy. This significance stemmed from the process of establishing a shared and mutual understanding of the structured feedback information given by the client.

Discussion

This review of empirical reports on what therapists say about their experiences with feedback procedures indicates that integrating the structured feedback into therapy through a feedback conversation may be beneficial for a variety of reasons. The analysis points to the usefulness of discussing feedback information with the client in terms of a shared purpose and meta-perspective of the therapy as well as in terms of collaboration. The therapists found it productive to elaborate on the clients’ structured feedback responses, and conversations about the feedback information may have facilitated the flow of unexpected and precise information from the client to the therapist. Feedback procedures that emphasize both the structured and the not-so-structured parts may help therapists to adjust, correct, and broaden their empathic focus. They may also reinforce the client’s agenda by validating it and promoting its adoption. This may be helpful in the effort to create a collaborative environment. This relational aspect of feedback procedures may be as important as the information obtained from them, as other authors have suggested (Breunlin, Pinsof, Russell, & Lebow, 2011; Sundet, 2012). One main relational aspect is that the use of feedback procedures communicates to the client that the therapist is interested in client feedback, which is an important message. The use of such a feedback procedure may help both the therapist and the client remain focused on collaboration.

Miller, Hubble, and Duncan (2007) suggest that therapist factors exist that are associated with a successful, collaborative approach to the use of structured feedback: an open-minded attitude toward what constitutes good therapy in a particular context; flexibility in the use of interventions; and a willingness to change (Miller et al., 2007). Our review findings are in line with these suggestions, but the limited available evidence does not permit one to draw conclusions. However, one might speculate whether the overall positive experience with the use of feedback may be connected to the therapists sharing a certain willingness to collaborate or a certain open-mindedness.

All the papers we reviewed included circumstances under which the collaborative aspect of the feedback collapsed. Such circumstances arose when the client did not find a feedback procedure relevant to the therapy, when the purpose of the feedback procedure was not clear to the client, when the client did not feel secure enough to give honest feedback in the therapeutic setting (e.g., family therapy, therapy with immigrants), or when the adults involved in the therapy (therapist or parents) overruled a child’s feedback. Although inconclusive, these findings indicate the complexity of contexts in which a feedback procedure may be introduced. Feedback procedures may not be appropriate for every therapy situation. In a wider perspective, one may ask whether a feedback procedure is relevant to all schools of therapy or useful in all phases of a particular therapy. Not every school of therapy would regard a structured feedback procedure as the most appropriate way of allowing clients to express their concerns about their therapist’s therapeutic approach. For example, problems may arise if structured feedback is used alongside psychodynamic approaches. Clients may develop transference (i.e., come to perceive the therapist as a parental or other authority figure from their early life). In many cases, these former authorities would not have welcomed critical feedback, and transference may induce the client to feel as though challenging the therapist’s knowledge could be dangerous, impolite, or ungrateful. Giving structured feedback may not be helpful for working through processes about which the client is currently unaware, and it may even be considered counterproductive. The contextual meaning of the feedback information needs to be explored in collaboration with the client. In situations where the client may shy away from giving honest feedback, the therapist should be cautious about how he or she uses and understands the feedback information.

Another example of a context where problems with utilizing structured feedback may arise is therapy in which one questions the idea of the therapist as the expert, for example, social constructionist and relational approaches (e.g., Anderson, 1997, 2012; Cooper & McLeod, 2011; Seikkula & Arnkil, 2013). These theoretical frameworks emphasize that the client, not the therapist, is the expert on his or her life or experience in terms of facing problems. Using a feedback procedure may directly and indirectly define for the client what the therapist emphasizes in therapy. As a result, it may reinforce the therapist’s role as the expert. However, the feedback conversation may give the therapist the opportunity to tone down the expert role by emphasising how the client has understood the questions and explored the thought process behind the answers (Oanes, Karlsson, & Borg, in press).

Comprehensive structured feedback procedures also systematically compare one client to a pool of similar clients. By contrast, social constructionist and relational approaches typically do not compare clients in terms of pre-defined measures; the process is considered dependent on the specific context and system. How such theoretical underpinnings of therapy may be combined with feedback procedures should be further explored.

Our review indicates that the interactional aspect of feedback procedure usage seemed to go beyond feedback as mere repeated assessment. The potential for collaboration and growth, for both the therapist and the client, that may be embedded in feedback procedures, and perhaps especially in the feedback conversation, should be further explored.

Limitations
The findings should be interpreted with caution. First, the body of relevant papers to include in the review was limited. For example, we were not able to identify studies on how OQ-45 and STIC are used and perceived by therapists. Additionally, most of the studies included a small number of participants, and a variety of therapeutic approaches and professions were represented in the papers. Because of the small sample size, we could not explore possible important distinctions, for instance, between therapy approaches, between professions, and between trainees and very experienced therapists.

Second, we want to note certain limitations of the analysis. The reviewed studies used quantitative, qualitative, and mixed methods, which presents a challenge owing to the accumulation of biases nested within the various methods. The majority of the studies were qualitative, which limits the generalizability of the results. However, this is not the main focus of this paper nor of qualitative data per se. Moreover, with a larger number of identified papers, we would have been able to follow defined procedures for literature reviews more strictly (e.g., Boote & Beile, 2005; Grant & Booth, 2009). Our construction of themes reflects our frames of reference, and other researchers might have interpreted the same data differently. Some nuances and themes in the selected papers were not reflected in our four themes owing to space limitations. Finally, the search and selection process may have limited the scope of the review and may have thus skewed the thematic findings.

Need for further research
There are many unexplored areas in this therapeutic field, and we want to point to five of them.

First, the utilization of feedback procedures designed to provide a meta-perspective on ongoing therapy through a discussion of the feedback information with the client requires a certain degree of mental and social maturity. Many children, and some adults, are not able to take a meta-perspective. More evidence is needed on how children and young adults perceive and use feedback procedures in psychotherapy settings.

Second, there is a growing evidence base on the utility of feedback procedures in therapy with couples and families (Anker et al., 2009; Pinsof, Goldsmith, & Latta. 2012). This research shows that sharing feedback information in family therapy is regarded as a powerful intervention (Oanes, Borg, & Karlsson, 2015; Pinsof et al., 2012). Experiences of giving and receiving feedback alongside fellow clients, however, require more in-depth exploration.

Third, there is a need for research on the micro-processes involved in the use of feedback procedures. Any therapy involves constant informal feedback. Collecting and discussing structured feedback with the use of a dedicated instrument does not eliminate the micro-feedback processes that operate in every therapeutic relationship. Such micro-feedback may be delivered through body postures, tone of voice, and so forth as well as through explicit questions intended to elicit an evaluation of the ongoing process. More research on how the use of structured feedback and the content of conversations on such feedback are related to the constant micro-feedback processes involved in every therapeutic relationship is needed.

Fourth, user involvement is defined as an aspect of evidence-based practice (Presidential Task Force on Evidence-Based Practice, 2006), and the use of structured feedback procedures is line with this approach. The inclusion of a feedback procedure is considered to enhance user involvement (Duncan et al., 2004; Pinsof et al., 2012). The routine use of a feedback procedure might engender the belief that the client is automatically involved in his or her therapy. However, from a user-involvement perspective, there is a need to explore the implications of the fact that the therapist (or institution) inevitably controls which feedback procedure is used and how it is used. We also believe that therapists and researchers must consider the perspective of former clients to improve both the design and utilization of feedback procedures. Engagement with user organizations would be an effective way to accomplish this aim (Davidson, Tondora, O’Connell, Lawless, & Rowe, 2009; Gehart, 2012a, 2012b; Marshall, Oades, & Crowe, 2010; Slade, 2009).

Fifth, the overall finding in our review was that the therapists reported positive experiences with the inclusion of the various feedback procedures. This tendency supports our reading of the literature on feedback procedures – one assumes that therapists want honest feedback and that they will communicate this desire to the client. It is, however, necessary to be aware that therapists differ. For example, although the use of a feedback procedure signals the desire for honest feedback, such use may be motivated by the therapist’s need for positive feedback – something the client may or may not observe. We believe that the field would benefit from a more thorough discussion of therapist factors, such as feelings therapists may develop during the course of delivering therapy (e.g., anxiety, rivalry, envy, competition, flow, or confidence), and the interaction of such factors with the use of feedback procedures. Such feelings are highly relevant both to the feedback conversation and to the therapist’s decision to include or incorporate a feedback procedure into his or her practice.

Conclusion

Despite the widespread use of feedback procedures, surprisingly little research has been published on therapists’ experience with such procedures. Our review indicates that feedback procedures influence both the therapeutic process and the persons involved. Structured feedback is introduced into a complex therapeutic and social context. This review reveals a strong need to examine – both theoretically and empirically – the interactional consequences of introducing structured feedback procedures into therapy. Important elements in such interactions are not only the client, the therapist, and the context but also the type of instrument.

References

Anderson, H. (1997). Conversations, language, and possibilities. A postmodern approach to therapy. New York, NY: Basic Books.

Anderson, H. (2012). Collaborative relationships and dialogic conversations: Ideas for a relationally responsive practice. Family Process, 51, 8–24. doi: 10.1111/j.1545-5300.2012.01385.x.

Anker, M., Duncan, B. L., & Sparks, J. A. (2009). Using client feedback to improve couple therapy outcomes: a randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77, 693–704. doi: 10.1037/a0016062.

*Ashworth, M., Robinson, S. I., Godfrey, E., Parmentier, H., Shepherd, M., Christey, J., . . . Matthews, V. (2005). The experiences of therapists using a new client-centred psychometric instrument, PSYCHLOPS (Psychological Outcome Profiles). Counselling and Psychotherapy Research, 5, 37–42. doi: 10.1080/14733140512331343886.

Bickman, L., Kelly, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62, 1423–1329. doi: 10.1176/appi.ps.002052011.

Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34, 3–15. doi: 10.3102/0013189X034006003.

*Bowens, M., & Cooper, M. (2012). Development of a client feedback tool: A qualitative study of therapists’ experiences of using the Therapy Personalisation Forms. European Journal of Psychotherapy and Counselling, 14, 47–62.

Bowens, M., Johnstone, D., & Cooper, M. (2011). The Therapy Personalisation Form (Appendix 2). In M. Cooper & J. McLeod (Eds.), Pluralistic counselling and psychotherapy (pp. 47–62). London, England: Sage.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101. doi: 10.1191/1478088706qp063oa.

Breunlin, D. C., Pinsof, W., Russell, W. P., & Lebow, J. (2011). Integrative problem-centered metaframeworks therapy I: Core concepts and hypothesizing. Family Process, 50, 293–313. doi: 10.1111/j.1545-5300.2011.01362.x.

Cooper, M., & McLeod, J. (2011). Person-centered therapy: A pluralistic perspective. Person-Centered & Experiential Psychotherapies, 10, 210–223. doi: 10.1080/14779757.2011.599517.

Davidson, L., Tondora, J., O’Connell, M. J., Lawless, M. S., & Rowe, M. (2009). A practical guide to recovery-oriented practice: Tools for transforming mental health care. New York, NY: Oxford University Press.

Duncan, B., Miller, S. D., & Sparks J. (2004). The heroic client. A revolutionary way to improve effectiveness through client-directed, outcome-informed therapy. San Francisco, CA: Jossey-Bass Publishers.

Gehart, D. R. (2012a). The mental health recovery movement and family therapy, part I: Consumer-led reform of services to persons diagnosed with severe mental illness. Journal of Marital and Family Therapy, 38, 429–442. doi: 10.1111/j.1752-0606.2011.00230.x.

Gehart, D. R. (2012b). The mental health recovery movement and family therapy, part II: A collaborative, appreciative approach for supporting mental health recovery. Journal of Marital and Family Therapy, 38, 443–457. doi: 10.1111/j.1752-0606.2011.00229.x.

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26, 91–108. doi: 10.1111/j.1471-1842.2009.00848.x.

Hatfield, R. D., & Ogles, B. M. (2007). Why some clinicians use outcome measures and others do not. Administration and Policy in Mental Health and Mental Health Services Research, 34, 283–291. doi: 10.1007/s10488-006-0110-y.

*Johansen, A., & Sundet, R. (2012). Bruken av KOR-skjemaer i «Familiehjelpa I Stange». En forskningsbasert evaluering [The use of PCOMS at «The Family Help» in the Stange community in Norway. A research-based evaluation (Report 3)]. Drammen, Norway: BUC, Institute of Research within Mental Health and Substance Abuse.

Keller, J., Kafkes, A., Basu, S., Frederico, J., & Kielhofner, G. (2005). The Child Occupational Self-Assessment (COSA) version 2.1. Chicago, IL: MOHO Clearinghouse, Department of Occupational Therapy, College of Applied Health Sciences, University of Illinois at Chicago.

Keller, J., & Kielhofner, G. (2005a). Psychometric characteristics of the child occupational self-assessment (COSA), part one: An initial examination of psychometric properties. Scandinavian Journal of Occupational Therapy, 12, 118–127. doi: 10.1080/11038120510031752.

Keller, J., & Kielhofner, G. (2005b). Psychometric characteristics of the Child Occupational Self-Assessment (COSA), part two: Refining the psychometric properties. Scandinavian Journal of Occupational Therapy, 12, 147–158. doi: 10.1080/11038120510031761.

*Kramer, J., Walker, R., Chon, E. S., Mermelstein, M., Olsen, S., O’Brien, J., & Bowyer, P. (2012). Striving for shared understandings: Therapists’ perspectives of the benefits and dilemmas of using a child self-assessment. Occupational Therapy Journal of Research, 32, 48–58.

*Laerum, E., Steine, S. D., Finckenhagen, M., & Finseth, A. (2002). The final version of the patient perspective survey (PPS): A new tool to improve consultation outcome and patient participation in general practice patients with complex health problems. Family Practice, 19, 264–271.

Lambert, M. (2010). Prevention of treatment failure. The use of measuring, monitoring, and feedback in clinical practice. Washington, DC: American Psychological Association.

Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E. J. (2005). Providing feedback to psychotherapists on their patients’ progress: Clinical results and practice suggestions. Journal of Clinical Psychology, 61, 165–174.

Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48, 72–79. doi: 10.1037/a0022238.

Lambert, M. J., Whipple, J. J., Vermeersch, D. A., Smart, D. W., Hawkins, E. J., Nielsen, S. L., & Goates, M. (2002). Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clinical Psychology and Psychotherapy, 9, 91–103. doi: 10.1002/cpp.324.

Lutz, W., Bøhnke, J. R., & Køck, K. (2011). Lending an ear to feedback systems: Evaluation of recovery and non-response in psychotherapy in a German outpatient setting. Journal of Community Mental Health, 47, 311–317. doi: 10.1007/s10597-010-9307-3.

Marshall, S. L., Oades, L. G., & Crowe, T. P. (2010). Australian mental health consumers’ contributions to the evaluation and improvement of recovery-oriented service provision. Israel Journal of Psychiatry and Related Sciences, 47, 198–205.

Miller, S. D., & Duncan, B. L. (2004). The outcome and session rating scales: Administration and scoring manual. Chicago, IL: Institute for the Study of Therapeutic Change.

Miller, S. D., Duncan, B. L., Brown, J., Sorrell, R., & Chalk, M. B. (2006). Using formal client feedback to improve retention and outcome: Making ongoing, real-time assessment feasible. Journal of Brief Therapy, 5, 5–22.

Miller, S., Hubble, M., & Duncan, B. (2007). Supershrinks: Why do some therapists clearly stand out above the rest, consistently getting far better results than most of their colleagues?. Psychotherapy Networker, 31(6), 26.

Ness, O., Borg, M., & Davidson, L. (2014). Facilitators and barriers in dual recovery: A literature review of first-person perspectives. Advances in Dual Diagnosis, 7, 107–117. doi: 10.1108/ADD-02-2014-0007.

Oanes. C. J., Karlsson, B., & Borg, M (in press). User involvement in therapy: Couple and family therapists’ experiences with the inclusion of a feedback procedure in clinical practice. Journal of Family Therapy.

Oanes, C. J., Borg, M. & Karlsson, B. (2015). Significant conversations or reduced relational capacity? Exploring couple and family therapists’ expectations for including a client feedback procedure. Australian and New Zealand Journal of Family Therapy, 36(3), 342–355 doi: 10.1002/anzf.1110.

Overington, L., & Ionita, G. (2012). Progress monitoring: A brief guide. Canadian Psychology, 53, 82–92.

Pinsof, W., Breunlin, D. C., Russel, W. P., & Lebow, J. (2011). Integrative problem-centered metaframeworks therapy II: Planning, conversing, and reading feedback. Family Process, 50, 314–336. doi: 10.1111/j.1545-5300.2011.01361.x.

Pinsof, W. M., & Chambers, A. (2009). Empirically informed systemic psychotherapy. Tracking patient change and therapist behaviour during therapy. In J. Bray & M. Stanton (Eds.), The Wiley-Blackwell handbook of family psychology (pp. 431–446). New York, NY: Wiley-Blackwell.

Pinsof, W. M., Goldsmith, J. Z., & Latta, A. T. (2012). Information technology and feedback research can bridge the scientist-practitioner gap: A couple therapy example. Couple and Family Psychology: Research and Practice, 1, 253–273. doi: 10.1037/a0031023.

Presidential Task Force on Evidence-Based Practice (2006). Evidence-based practice in psychology. American Psychologist, 61, 271–285.

Riemer, M., Rosof-Williams, J., & Bickman, L. (2005). Theories related to changing clinician practice. Child and Adolescent Psychiatric Clinics of North America, 14, 241–254. doi: 10.1016/j.chc.2004.05.002.

Sapyta, J., Riemer, M., & Bickman, L. (2005) Feedback to clinicians: Theory, research, and practice. Journal of Clinical Psychology, 62, 145–153. doi: 10.1002/jclp.20107.

Seikkula, J., & Arnkil, T. E. (2013). Åpen dialog i relasjonell praksis. Respekt for annerledeshet i øyeblikket. [Open dialogues in relational practices. Respect for the otherness in the present moment]. Oslo, Norway: Gyldendal.

Sexton, T. L., Patterson, T., & Datchi, C. C. (2012). Technological innovations of systematic measurement and clinical feedback: A virtual leap into the future of couple and family psychology. Couple and Family Psychology: Research and Practice, 1, 285–293. doi: 10.1037/cfp0000001.

Slade, M. (2009). Personal recovery and mental illness. A guide for mental health professionals. Cambridge, England: Cambridge University Press.

*Sparks, J. A., Kisler, T. S., Adams, J. F., & Blumen, G. D. (2011). Teaching accountability: Using client feedback to train effective family therapists. Journal of Marital and Family Therapy, 37, 452–467. doi: 10.1111/j.1752-0606.2011.00224.x.

*Sundet, R. (2012). Therapist perspectives on the use of feedback on process and outcome: Patient-focused research and practice. Canadian Psychology, 53, 122–130. doi: 10.1037/a0027776.

Tracey, T. J. G., Wampold, B. E., Lichtenberg, J. W., & Goodyear, R. K. (2014). Expertise in psychotherapy. An elusive goal? American Psychologist, 69, 218-229. doi: 10.1037/a0035099.

*Tuseth, A. G., Sverdrup, S., Hjort, H., & Friestad, C. (2006). Å spørre den det gjelder. Erfaringer med bruk av klient- og resultatstyrt terapi i psykisk helsearbeid [Asking the individual it concerns. Experiences with the use of client and result-dependent therapy in mental health work]. Oslo, Norway: Regionsenter for barn og unges psykiske helse.

References marked with an asterisk (*) indicate studies included in the review.