Appendix C. Existing Learning Collaboratives and Networks

Name of Collaborative	History	Mission and/or Goal(s)	Tools and Mechanisms	Lessons Learned and Best Practice Reflections	Measures of Success
National Association of State Medicaid Directors (NASMD) Multi-State Collaborative1	Created in 2007 together with States that received Medicaid Transformation Grants to support development, implementation, and operation of electronic health records (EHR) and health information exchanges (HIE). Encourage all States to participate.	Share best practices and lessons learned and partner with other States to leverage Medicaid program and financing capabilities in implementing Medicaid EHR and HIE efforts. Will address a variety of topics over time, including maximizing American Recovery and Reinvestment Act of 2009 (ARRA) funding, implementing meaningful use requirements in Medicaid, and preparation for incentive payments for health information technology (health IT). The Collaboration will provide to its members: Information collection and dissemination. National meetings. Liaison between the Collaboration, Federal agencies, Federal policymakers, and relevant partners.2	Educational forums. State-to-State mentoring. Continuous learning workgroups. Workgroup discussions (calls and Web meetings). Opportunities to comment on draft guidance from CMS and other Federal agencies. Technical assistance opportunities. Web site for sharing references, CMS guidance, environmental scan, health IT State plans, HITECH planning APDs by State, consent materials, workgroup archives, and other relevant resources.3 Conducted survey with participants to assess: most useful technical assistance for HIE, EHR, and e-prescribing; most useful leadership and training opportunities; and level of staff participation in collaborative workgroups.4	Not identified.	Not identified.
Medicaid Medical Directors Learning Network	Since 2006, Medicaid medical directors have been meeting to share ideas and best practices related to issues of access, quality, and costs in Medicaid. Sponsored by the Agency for Healthcare Research and Quality (AHRQ) and supported by NASMD. As of 2009, 42 States had joined the Medicaid Medical Directors Learning Network.5	Improve the quality and safety of health care delivered to the Medicaid population.6 Focused on specific target problems such as reducing C-section rates, improving preventive screening and vaccinations. First project: benchmark antipsychotic and mental health drug use in Medicaid children. Data from 16 States were collected to provide best practices to work with providers of mental health to improve prescribing practices.5 This project resulted in a report Antipsychotic medication use in Medicaid children and adolescents.7 Similar approach used to reduce readmissions.5	Offer clinical programs during NASMD meetings in summer and fall. Collaborate on targeted research and quality improvement projects.	Not identified.	Not identified.
AHRQ Primary Care Practice-Based Research Networks (PBRNs)	Primary care practice-based research networks (PBRNs) emerged between the 1970s and 1990s, all around the United States, primarily led by family physicians. Through partnerships with private foundations, professional societies, academic institutions, and State and Federal agencies, these PBRN have evolved into much more robust national networks.8 Since the 1990s, AHRQ has supported capacity-building among the PBRNs and currently funds the PBRN Resource Center.8 As of 2009, there were 113 primary care PBRNs operating throughout the United States.9	PBRNs are defined as "a group of ambulatory practices devoted principally to the primary care of patients. Typically, PBRNs draw on the experience and insight of practicing clinicians to identify and frame research questions whose answers can improve the practice of primary care. By linking these questions with rigorous research methods, the PBRN can produce research findings that are immediately relevant to the clinician and, in theory, more easily assimilated into everyday practice."10 PBRNs are autonomous organizations, often affiliated with an academic medical center. Some focus primarily on research, while others incorporate member learning activities as well. (MP)	Elements of the Network include: PBRN Resource Center—provides technical support, research methodology, education in best practices of primary care research, and data collection. Peer Learning Groups—Conference calls and presentations on relevant topics such as comparative effectiveness research, quality improvement research, PBRN operations, health IT, and research methods. Annual PBRN Conference—three-day meeting sharing information related to primary care and PBRN research. Opportunity to interact and learn from primary care colleagues. PBRN Literature—virtual library of research conducted through PBRNs. PBRN Secure Portal—Secure Web site for registered PBRNs to share documents, communicate, collaborate, and develop resources with others involved in the network.10	Membership buy-in is critical. There needs to be value-added to maintain members' interest. The PBRN Resource Center helps networks conduct membership surveys to assess member interest, experience, and needs. (CL) Even when strong interest exists, finding time is often an issue. (CL) Many networks have established practice facilitators or practice coaches as part of the infrastructure. (CL) Coordination costs money, and PBRNs may have different cost structures.11 Less experienced networks gain a lot by collaborating with more experienced networks.11 Multisite research increases the profile of the networks.11 A project was conducted  to identify best practices in PBRNs. This project was to be completed by August 2011. (CL)	Annual assessments are completed during the AHRQ PBRN registration process. These self-assessed areas include network governance, network membership, network operations, research issues, and information technology. (CL) PBRNs measure their own successes, using metrics such as level of involvement, number of studies, finances, and number of publications. (CL)
Knowledge Translation (KT) Canada		The vision of the network is "To collectively lead knowledge translation theory and research nationally and internationally by creating a sustainable organizational structure that supports national and international collaboration advancing education, theory and research, and health care delivery service. We see KT Canada as the development of an intellectual commons to spark innovation, debate, theory building, and testing of KT research innovations across boundaries: disciplinary, geographical, institutional, and others."12 The mission of the network is, "To form a national Canadian research network to identify and study solutions to ensure that key stakeholders in the Canadian health care system have the opportunities, tools, and skills necessary to achieve KT in order to improve health for Canadians."12 The networks goals  are to: Improve the communication of research results. Develop a consensus terminology in knowledge translations and approaches to measuring success. Evaluate different approaches to knowledge translation. Engage health professionals, members of the community, and other decisionmaking entities to ensure that the impact of knowledge translation initiatives is sustained.13	KT Canada has four research programs aimed at the loop between knowledge and action (knowledge distillation; determinants of knowledge use; selecting, tailoring, and evaluating effectiveness and efficiency of KT interventions; and sustaining KT). These research projects are targeted at three key stakeholder groups (consumers, health care professionals and managers, and policymakers). Each research program and stakeholder group has a study group that publishes the methodological insights drawn from the corresponding research program. Several training programs are offered by KT Canada for graduate students, postdoctoral fellows, physicians, and the general public.14	Not identified.	Not identified.
Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network	The Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network was created in 2002, when investigators in pediatric clinical care who were participating in four multi-site research projects in the United States and Canada decided to join together and establish one research network.15 Participation in the network is voluntary. Participating sites must secure funding for their research projects from government, foundation, and industry sources.15	PALISI is a collaboration of clinical researchers from 79 pediatric intensive care units throughout North America. Together, the researchers aim to "identify optimal supportive, preventive, and therapeutic strategies for acute lung injury, sepsis, multi-organ failure, and other acute, life-threatening pulmonary or systemic inflammatory syndromes that affect infants and children."16 The goal of the PALISI Network is "to perform multi-center research studies to better describe disease processes and outcomes in pediatric patients and to evaluate interventions in this population."16	Hold 3-day conferences, twice per year. Opportunity to share research and get feedback from fellow investigators. (SV) Also have subgroups that focus on specific topic areas and have smaller group meetings during the biannual conferences. (SV) Currently developing a Web site to be able to share research and accommodate the needs of the growing network. (SV)	Not identified.	Not identified.
Robert Wood Johnson Foundation Clinical Scholars	The Robert Wood Johnson Foundation (RWJF) Clinical Scholars program supports physician leaders in their efforts to conduct innovative research and work with communities, organizations, practitioners, and policymakers.17	"The goal of the program is to integrate Scholars' clinical expertise with training in program development and research methods to help them find solutions for the challenges posed by the U.S. health care system, community health, and health services research."17 Through the typical 2-year training program, scholars receive a master's degree, with significant time allotted for research.17 Up to 29 scholars are selected each year to participate at one of four universities across the country. A critical part of the program is the mentorship.	The RWJF Clinical Scholars program maintains a strong network of current and past scholars. Several mechanisms are used to maintain the scholar network. Clinical Scholars Online Directory.18 RWJF Alumni Network creates connections between alumni of the RWJF's scholars, fellows, and leaders programs to support their work improving health and health care in the United States.18 Interactive Alumni Scholars Map.18 Alumni Ambassadors program to support new scholar recruitment.18 Social networking resources include Facebook, LinkedIn, and a private Wordpress blog, where scholars and alumni can post information and interact.19 Web site offers several resources, including job opportunities, funding resources, and a call for abstracts.	Not identified.	Not identified.
Community Health Care Association of New York State (CHCANYS) and Primary Care Development Corporation (PCDC) Learning Collaborative for Meaningful Use (health IT) and the Patient-Centered Medical Home (The PCMH/MU Collaborative)20	Initiated in July 2010. Combines the expertise of CHCANYS, which has the buy-in of community health centers (CHCs) across New York State with PCDC, which is a trusted entity within the primary care community (fits well with CHCs) and has a demonstrated record of success in building collaboratives.	Provides CHCs with training and tools to: Attain a level of PCMH recognition/designation that reflects capacities of CHC. Enable collection of data that will demonstrate CHCs have satisfied Stage I Meaningful Use Standards and Measures. Identify targets for improvement for CHCs to progress in PCMH and MU recognition.	Features an "action-learning" approach that "meets providers where they are" while focusing on teambuilding and identifying specific, measurable changes (i.e., process) that can be directly linked to corresponding outcomes. Sustainability is "built into" this effort.20 Specific action elements of the collaborative (occurring over 6 months): Four learning sessions (4 days) Three activity periods (8 weeks each) Coaching (TBD by situation) Phone conferences, Webinars, Webinettes, Sharepoint Tools (password-protected, shared e-workspace).	Not identified.	Feedback: "Does this meet your needs?" Detailed PCMH-MU assessments (Note these assessments are related to PCMH and MU achievements for individual CHCs within the collaborative, not the work of the collaborative, per se. However, the success of CHCs may be a measure of success of the collaborative)21
Novel Methods Leading to New Medications in Depression and Schizophrenia (NewMeds).22 Collaboration between pharmaceutical industry and academic institutions.	"Research academic-industry collaboration" initiated in 2009 to explore "new methods for the development of drugs for schizophrenia and depression."22	The collaboration seeks to address bottlenecks in the drug discovery, development, and clinical trials processes to bring more medications for schizophrenia and depression to market more rapidly.	10 "Workpackages" introduced at various points during the drug discovery and development phases (e.g., pre-clinical, Phase I-III)23 Each workpackage has a deputy (administrative) and academic (scientific) "lead," both of whom are responsible for the deliverables of the individual workpackage.23 Notifies members of upcoming conferences relevant to their research interests. Publications (potentially also a measure of success).	"The one and only supporting action for every workpackage is communication... Nothing has more power than meeting someone face-to-face." (KS)	Not identified.
Centre of Excellence for External Drug Discovery (CEEDD)24	Idea stage: 2000-2005. In 2005, initiated first "autonomous scientific investment team."24	As a "pioneer in the quest to find new medicines," CEEDD brings together external collaborators to form alliances for drug discovery, development, and clinical trials at any point in the process.18 Through this process, CEEDD hopes to "bring more medicines of value to patients."24	Provides formal guidance to potential collaborators to "form multi-program, risk-reward sharing alliances" that maintain member companies' "independence and creativity."24,25 Seeks out collaborations at any point in the drug discovery, development, and clinical trials processes. CEEDD carefully vets potential partners for an alliance and only selects those that strongly align with its mission.26 CEEDD approach:24-27 Semi-autonomous, flexible team (of 25) Two centers (U.S. and U.K.) Work across multiple therapy areas Form alliances at any point in clinical development "Full strength of GlaxoSmithKline resources" "CEEDD of Innovation" and the "Media Center" are the online voices and platforms through which news about new alliances and progress in existing alliances is communicated.28,29	Not identified.	Not identified.
MindTree, Ltd	Located in India and the United States (New Jersey), MindTree was founded in 1999 as an information technology and business process outsourcing (ITBPO) company. Within 6 years, they earned $100 million in revenues. The 4,000 employees were called MindTree Minds. Building a culture of transparency and knowledge management, the basic organizational unit became a community of practice.30	"95% of the people should have 95% of the information, 95% of the time."30 The company was founded on principles of creating emotional security, ensuring global communication of information, and developing capacity to absorb information by making it interesting. To continually share learning with the world at large. The purpose of knowledge management is helping people to do their jobs better and to develop as professionals.	To ensure knowledge is created and disseminated, the organization supported the notion of voluntary communities of practice.31 The community evolves over time and is motivated by two questions: 1. What do we want to learn? 2. What do you have to share? When individuals share an interest and enjoy talking online, they become a "community of interest." This is just sharing not learning. The next level of maturity is "competency-building." Here people meet face-to-face and learn from each other. The next level (called capability-building) was building relationships between other communities, and directly impacting processes. Finally, the last level called 'capacity- building' shifts attention from the internal to the external world.	Communities of practice are informal. If they can be found on an organizational chart, "they cease to be a community." All communities have multiple champions who are self-selected and lead the group. Communities of practice should have some face-to-face interactions for brainstorming, telling stories, solving problems, sharing best practices and service visions, and offering new work tools. Communities of practice allow for virtual interactions when people are geographically dispersed. Shared goals and passionate people create successful communities of practice. The key to innovation is collaboration; before you can become collaborative, you need a social network process.	At the beginning of every project, they create a Knowledge Map (K-Map) to identify the knowledge requirements and the source of knowledge and an action plan. Teams that adopted Kmaps outperformed projects that did not use them, in terms of on-time, resource utilization, and, higher gross margins and fewer overruns. No organizational result is expected, so there are no metrics. 3% of the workforce were leading a community of practice, and 75% belonged to at least one community of practice.
Intermountain Health Care	In 1975, a group of 15 nonprofit hospitals came together to form a system called Intermountain Health Care. Today, they have 150 facilities, 22 hospitals, 25 health centers, and more than 70 clinics. In 1991, they had a strong consensus that quality improvement and clinical care management were central to their mission, and they began to roll out clinical process management throughout their system. In order to do so, they developed a sophisticated learning collaborative. As a learning collaborative, they strived to "make it easy, to do it right."32	To actively manage clinical care delivery and to get physicians to learn how to use standardized, evidence-based clinical practices. To become a national leader in health quality and quality improvement.	They identified the 60 most common diagnoses and procedures and the core clinical care programs. In these areas, they established learning collaboratives that conducted literature reviews, defined protocols, and established key decision points to define current best practices. They brought groups of clinicians together to review the protocols and relied on medical directors and nurses to oversee implementation. They developed tools to manage patients according to protocols, and monitor performance. To integrate clinical services, they designed an organization with a governance system, a toolbox with electronic protocols, decision support, and patient health records. To manage the learning collaborative, they allowed physicians to override protocols by managing uncommon cases and documenting what needs to be improved. Collaborative projects were based on outcome variance and then RCTs to improve protocols.	Knowledge exists not only in the literature, but it is embedded in the experiences of practitioners. To make sure that knowledge is not only disseminated but also used, a process is needed to support the creation of new knowledge. New knowledge emerges if outliers are observed when one benchmarks performance data. Knowledge that exists in a community of practice should be codified. To create a learning collaborative, learning should come from "real" experience and scientific testing.	Intermountain dominates Utah's health system. They achieved strong commitment to adopt evidence-based practice, and much higher than average clinical outcomes. After developing and implementing protocols, the learning collaborative, achieved significant reductions in mortality and readmissions and patients discharged from cardiovascular units achieved 90% compliance with appropriate medications. They achieved huge savings in fast-track extubation, diabetes management, adverse drug event prevention, and ventilator management.
Wisconsin Collaborative for Health Care Quality	In October 2002, nine physician-led health care organizations and their employer-partners founded a collaboration of major hospitals and physician groups. WCHC is a "voluntary consortium of organizations learning and working together to improve the quality and cost-effectiveness of health care for the people of Wisconsin by developing and publicly reporting measures of health care performance." Steady growth has been reported since the collaborative was launched.33	The primary goal is to improve the quality of health care in Wisconsin by publicly reporting performance measures. By making public outcomes data, increasing transparency of outcomes data, and working together, that will drive internal managerial and clinical process improvements.	Defined a set of performance measures that incorporates safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity.34 The group publishes the data at their Web site. They also convene improvement meetings of providers and payers and promote frequent communication and informal interactions to share ideas. To ensure the quality of the data, they hired an external auditor to check on data reliability and validity.	As a learning collaborative, the willingness to be open and reveal their weaknesses creates value. Adopting a set of common performance measures with common definitions meant providers could not 'pick and choose' the outcomes to show. Learning how to communicate and share ideas when physical distances do not allow face-to-face interactions requires relying on telephones and computers. A learning collaborative should ultimately benefit the customer.	Baseline performance data and tracking performance over time have accelerated change and led to a greater sense of organizational responsibility. Comparative benchmarking has had a "positive influence" on health care delivery. Over time, publicly reporting data created a sense of urgency to improve.

Personal communications are indicated in the table by initials. See details below.

CL—Carol Lange, Email communication, May 3, 2011
KS—Kathrin Stoller, Email communication, May 9, 2011
MP—Michael Parchman, Email communication, April 21, 2011
SV—Stacey Valentine, Email communication, April 19, 2011

References

1. American Public Human Services Association (APHSA). APHSA health services division: Multi-State Collaborative; 2011. Available at http://hsd.aphsa.org/Home/Multi-State.asp . Accessed March 21, 2011.

2. National Association of State Medicaid Directors. Multi-State collaboration for the planning and development of State Medicaid electronic health record and health information exchange initiatives. Washington, DC: National Association of State Medicaid Directors; 2007.

3. American Public Human Services Association (APHSA). NASMD Multi-State Collaborative; 2011. Available at http://hsd.aphsa.org/issues/medicaid_transformation.asp . Accessed March 21, 2011.

4. National Association of State Medicaid Directors. NASMD multi-state collaboration: Medicaid transformation survey results. Washington, DC: National Association of State Medicaid Directors. Powerpoint presentation available at http://hsd.aphsa.org/issues/docs/NASMD-Transformation-Survey.ppt [Plugin Software Help]. Accessed March 13, 2012.

5. Medicaid Medical Directors Learning Network. Policy & Practice 2009; 67(3):31.

6. Agency for Healthcare Research and Quality. Medicaid Medical Directors Learning Network. 2011; http://www.ahrq.gov/news/kt/ktnetworks.htm#mmd. Accessed March 28, 2011.

7. Medicaid Medical Directors Learning Network and Rutgers Center for Education and Research on Mental Health Therapeutics. Antipsychotic medication use in Medicaid children and adolescents: Report and resource guide from a 16-state study. New Brunswick, NJ: MMDLN/Rutgers CERTs; 2010.

8. Agency for Healthcare Research and Quality. AHRQ support for primary care practice-based research networks (PBRNs). 2011; http://www.ahrq.gov/research/pbrn/pbrnfact.htm. Accessed March 28, 2011.

9. Agency for Healthcare Research and Quality. AHRQ practice based research networks (PBRNs); 2011. Available at http://pbrn.ahrq.gov/portal/server.pt/community/practice_based_research_networks_%28pbrn%29__about/852. Accessed March 28, 2011.

10. Agency for Healthcare Research and Quality. AHRQ practice based research networks (PBRNs); 2011. http://pbrn.ahrq.gov/portal/server.pt/community/practice_based_research_networks_%28pbrn%29__about/852. Accessed March 28, 2011.

11. Oppenheimer C. Lessons learned from the PBRN NCS pilot study. Paper presented at: AHRQ 2005 PBRN Research Conference; 2005.

12. Knowledge Translation Canada. Governance and organizational chart; 2011. Available at http://ktclearinghouse.ca/ktcanada/about/governance . Accessed April 20, 2011.

13. Knowledge Translation Canada. About KT Canada; 2011. Available at http://ktclearinghouse.ca/ktcanada/about . Accessed April 20, 2011.

14. Knowledge Translation Canada. Education; 2011. Available at http://ktclearinghouse.ca/ktcanada/education . Accessed April 20, 2011.

15. Children's Hospital Boston. Critical care medicine: PALISI. Available at http://www.childrenshospital.org/clinicalservices/Site530/mainpageS530P12.html . Accessed April 22, 2011.

16. Pediatric Critical Care Medicine. Pediatric Acute Lung Injury and Sepsis Investigators (PALISI); 2010. Available at http://www.pedsccm.org/PALISI_network.php . Accessed April 21, 2011.

17. Robert Wood Johnson Foundation. About us: Who we are. Available at http://rwjcsp.unc.edu/about/index.html . Accessed May 12, 2011.

18. Robert Wood Johnson Foundation. Alumni. Available at http://rwjcsp.unc.edu/alumni/index.html . Accessed May 12, 2011.

19. Robert Wood Johnson Foundation. Resources: Social networking. Available at http://rwjcsp.unc.edu/resources/networking/index.html . Accessed May 12, 2011.

20. Community Health Care Association of New York State. The PCMH/MU Collaborative: What you need to know; 2010. Available at http://www.chcanys.org/clientuploads/2010_pdfs/1-Collab%20Overview%20Slides_PC.pdf [Plugin Software Help]. Accessed April 15, 2011.

21. Community Health Care Association of New York State. Conducting Your Detailed PCMH & MU Assessments; 2010. Available at http://www.chcanys.org/clientuploads/2010_pdfs/7-conducting-detailed-assessments_CR-PC-LP.pdf [Plugin Software Help]. Accessed April 15, 2011.

22. Novel Methods Leading to New Medications in Depression and Schizophrenia. About NewMeds; 2009. Available at http://www.newmeds-europe.com/en/news.php . Accessed April 15, 2011.

23. Novel Methods Leading to New Medications in Depression and Schizophrenia. Workpackages and their Impact; 2009. Available at http://www.newmeds-europe.com/en/projectstructure.php . Accessed April 15, 2011.

24. Centre of Excellence for External Drug Discovery. Who We Are; 2009. Available at http://www.ceedd.com/about_us/index.aspx . Accessed April 15, 2011.

25. Centre of Excellence for External Drug Discovery. What We Do; 2009. Available at http://www.ceedd.com/about_us/what_we_do.aspx . Accessed April 15, 2011.

26. Centre of Excellence for External Drug Discovery. What We Look For; 2009. Available at http://www.ceedd.com/working_with_us/how_we_do.aspx . Accessed April 15, 2011.

27. Centre of Excellence for External Drug Discovery. Our Team; 2009. Available at http://www.ceedd.com/about_us/our_team.aspx . Accessed April 15, 2011.

28. Centre of Excellence for External Drug Discovery. CEEDD of Innovation; 2009. Available at http://www.ceedd.com/ceeddofinnovation/index.aspx . Accessed April 15, 2011.

29. Centre of Excellence for External Drug Discovery. News Bank; 2009. Available at http://www.ceedd.com/media_center/index.aspx?alliance=all . Accessed April 15, 2011.

30. Mindtree. Making of Mindtree; 2011. Available at http://www.mindtree.com/about-us/making-mindtree/making-mindtree . Accessed May 13, 2011.

31. Wenger EC, Snyder WM. Communities of practice: The organizational frontier. Harvard Bus Rev 2000 Jan-Feb; 139:145.

32. Bohmer RJ, Edmondson A. Intermountain Health case. Cambridge: Harvard Business School Press; 2006.

33. Wisconsin Collaborative for Healthcare Quality. About WCHQ; 2011. Available at http://www.wchq.org/about/ . Accessed May 13, 2011.

34. Hatahet MA, Bowhan J, Clough EA. Wisconsin Collaborative for Healthcare Quality (WCHQ): Lessons Learned. Wisconsin Medical Journal 2004; 103(3):45-48.

Return to Contents
Proceed to Next Section