About the Medical Group Ratings
Meeting National Standards of Care
It is important to know if patients are getting the right medical care. To find out, we studied information from records that medical groups and HMOs keep for patient services. The information is for patients who were members of the medical groups in 2006.
Meeting National Standards of Care means, for example, that people who have a heart problem or diabetes have their cholesterol tested. It means that children get immunizations, or shots, to prevent illnesses and that women get Pap smears to test for cervical cancer. It also means that people with asthma get medicine to avoid asthma attacks.
Each medical group gets a single summary rating for Meeting National Standards of Care. The summary rating is based on the medical group’s average score across thirteen measures:
The score for each of these topics shows the percent of the patients in a medical group who should receive a certain kind of care actually got it. For example, if there were 100 children who should have gotten immunizations, or shots, and 85 got the shots, then the medical group got a score of 85%. If only 50 of the 100 children got shots, then the medical group got a score of 50%.
Back to top
Scoring and Rating Methods: Meeting National Standards of Care
The Meeting National Standards of Care summary rating is calculated by combining the medical group’s scores for the thirteen measures listed above. First, scores are calculated for each of the thirteen measures – the score is the percent of patients who got the recommended service. Then, the scores for all of the measures are combined, giving equal weight to each measure, by calculating the average percent across the measures. If the medical group has too few patients to calculate a score for a particular measure a formula is used to estimate a score for the missing information. The formula assumes that the medical group result for the missing measure would be similar to the difference between that medical group’s available scores and the average scores for all medical groups. A summary rating is not produced for medical groups with fewer than seven reportable measures.
The summary rating is given one of four performance grades that are indicated in the report card with stars. The possible grades are:
- Excellent: This means that about 8 of every 10 or more medical group members got the right care.
- Good: This means that about 3 of every 4 medical group members got the right care.
- Fair: This means that about 2 of every 3 medical group members got the right care.
- Poor: This means that fewer than two-thirds of medical group members got the right care.
Using a “buffer zone” adjustment, we account for the error that occurs in measurement and scoring when the formula is based on samples of members rather than all the members in a medical group. This “buffer zone” gives the benefit of the doubt to the medical group—if a score falls below a performance threshold, (e.g., a score of 84 marks an excellent grade) but within one-half point of that threshold, the medical group is assigned that next highest grade.
For additional information on OPA's scoring methodology for IHA effectiveness of care measures, refer to this pdf document.
Back to top
Patients Rate Medical Group
It is important to know what patients think about the care and service they get from their medical groups. To find out, surveys were done with more than 70,000 California patients between the ages of 18-64. All were patients who had a visit with a doctor in a medical group during 2006.
Patients answered many questions about their experiences with their doctors and medical groups. Then, we organized the answers into four topics:
The medical groups were rated on these topics based on patients’ survey answers. Typically, 35% or more of patients who are contacted answered the survey. Four of the five topics were formed by grouping sets of related questions and combining the scores for these questions into a topic score. The Patients Rate Medical Group summary rating was formed by combining the results of the other four topics into a single summary score.
Back to top
Scoring and Rating Methods: Patients Rate Medical Group
Medical group scores were based on patient responses to the Patient Assessment Survey (PAS). Adult patients from more than 180 California medical groups and Independent Practice Associations (IPAs) responded to the survey. Patients are asked to rate the care and service provided by their doctors and other staff in the medical group during the past year (2006). The mailed questionnaire was available in English, Spanish, Chinese, Korean and Vietnamese. About 15% of the patients who responded completed the survey by phone and the remaining 85% answered and returned a mailed survey.
Scores for the four topics are based on the mean score of the patient’s responses to the survey questions. For most questions, responses are scored using one of six possible answers ranging from the patient “always” had a positive experience with a particular need like getting an appointment to the patient “never” had a positive experience. The responses are scored on a scale that assigns a 100 to “always”, 80 to “almost always”, 60 to “usually”, 40 to “sometimes”, 20 to “almost never”, and 0 to “never”. Each patient’s responses for a set of related questions—like “coordinating patient care”—are combined to create a per-patient topic mean score; then the average of all of a medical group’s patients’ scores is calculated to create a medical group score for that topic. The scores represent the average or typical experience that that medical group’s patients reported.
The Patients Rate Medical Group score is calculated by combining the scores for the four topics. The four topic scores are combined by giving them equal weight and calculating an average score. The score for Patients Rate Medical Group is given one of four performance grades that are indicated in the report card with stars. The possible grades are:
- Excellent: This means that more than 8 of every 10 medical group members reported a positive experience.
- Good: This means that about 8 of every 10 medical group members reported a positive experience.
- Fair: This means that about 3 of every 4 medical group members reported a positive experience.
- Poor: This means that somewhat higher than 2 of every 3 medical group members reported a positive experience.
Using a “buffer zone” adjustment, we account for the error that occurs in measurement and scoring when the formula is based on samples of members rather than all the members in a medical group. This “buffer zone” gives the benefit of the doubt to the medical group—if a score falls below a performance threshold, (e.g., a score of 85 marks the excellent grade) but within a half-point of that threshold, the medical group is assigned that next highest grade.
The scores are adjusted for a set of patient characteristics—age, gender, mental health status, education, overall health status, race/ethnicity, language spoken and number of chronic conditions—that have been shown to influence patients’ ratings of their care experience. Scores also are adjusted for the specialty of the physician seen by the patient and by the type of survey—answered in print, online or by phone. These adjustments allow us to make apples-to-apples comparisons across groups whose patients may differ. Through the adjustments, which result in very small changes in a medical group’s scores, we can represent the groups’ results as if they all had a similar mix of patients.
For additional information on OPA's scoring methodology for PAS, refer to this pdf document.
Back to top
“Too Few Patients,” “Not Willing to Report” and “No Report Due to Incomplete Data”
On the medical group rating charts, most columns have a score from one to four stars, with four stars being the highest rating. Sometimes you will see one of the following:
Too Few Patients: This means that the medical group did not have enough members who had the experience to be scored.
- For the Meeting National Standards of Care measures a medical group must have at least 30 patients who needed a certain kind of care to be scored.
- The Patients Rate Medical Group measures are not available for all medical groups for several reasons. In some instances too few patients answered particular questions to report that result. This typically occurs with some smaller medical groups that have fewer patients; their results may not be accurate because of the low number of completed surveys. In other situations, the question is about an experience that is relevant to fewer patients in that group and we are not confident that the results represent typical patient experiences with that medical group.
Not Willing to Report: This means that the medical group would not report its results. This usually means that the medical group did not do well and does not want to share the information.
No Report Due to Incomplete Data: This means that the medical group results were not reported because of uncertainty about the completeness of the available data for that group.
Back to top
How Reliable Are the Scores?
Enough people replied to the survey and we looked at enough records from the medical groups to give a clear picture of how well each medical group is doing. However, anyone can have different experiences with their medical group. Your experiences may be different from the ratings shown here.
The Meeting National Standards of Care rating tells an important story about how well the medical group and its doctors do in meeting national standards for good care. But, because the rating is limited to particular areas of care there are many aspects of medical care that are not part of this rating.
The Patient Rates Medical Group rating is based on patient surveying that is done in a way to show the typical experience of patients in each medical group. The patients who were surveyed were randomly drawn from the medical group’s full list of commercial HMO members. Patients who had a medical visit in 2006 were included on the survey list. Results were from patients with visits to their primary care doctor and other patients who had visits with specialist doctors. Nonetheless, your experience with a particular doctor or medical group staff may differ from the experiences reported in the survey.
Back to top
Comparing Medical Group and HMO Ratings
An HMO is a type of health insurance. A medical group is a group of doctors who work with an HMO to give the HMO’s members their medical care. You should not compare the ratings for HMOs and medical groups because:
- HMOs and medical groups keep different kinds of records. This means that the information we get from them is different.
- We did not always study the same members when we looked at medical groups and HMOs, even when we were rating the same health topic. For example, the person whose diabetes care was reviewed for her HMO may not be the same person whose diabetes care was reviewed for her medical group.
- We used medical charts to get information about HMO performance. We did not use medical charts to get information about medical group performance.
- The quality of the care that HMOs and medical groups give is measured in different ways. The way HMOs and medical groups are rated also differs.
Back to top
California Cooperative Healthcare Reporting Initiative (CCHRI)
The California Cooperative Healthcare Reporting Initiative (CCHRI) is a group of employers, health plans, and health care providers across the State. More than 90 percent of HMO members in California belong to plans that are part of CCHRI. CCHRI makes sure that people get information they can trust on how well health plans and medical groups provide care and service.
CCHRI also directed the Patient Assessment Survey (PAS) to measure and report patients’ care experiences. The results of that survey are shown in the Patients Rate Medical Groups part of this report card. Along with the participating medical groups, the following HMOs financially supported this survey:
Back to top
Integrated Healthcare Association (IHA)
The Integrated Healthcare Association is a not-for-profit statewide collaborative leadership group of California health plans, physician groups, and healthcare systems, plus academic, consumer, purchaser, pharmaceutical and technology representatives. IHA promotes quality improvement, accountability, and affordability for the benefit of all California consumers through special projects, policy innovation and education. The IHA-sponsored Pay for Performance (P4P) program generates the measures used in Meeting National Standards of Care.
The Pay for Performance program is the nation’s largest involving over 40,000 physicians in 220 physician organizations that care for more than 12 million individuals enrolled in eight major health plans (Aetna, Blue Cross, Blue Shield, CIGNA, Health Net, Kaiser Permanente, PacifiCare and Western Health Advantage). An organizing principle behind P4P is the uniform evaluation of physician groups’ performance across multiple health plans with a common set of quality measures. The clinical quality measures are adapted from NCQA’s Healthcare Effectiveness Data and Information Set (HEDIS®), the most widely used set of performance measures in health care. The measures include breast cancer screening, child immunizations and the treatment of chronic conditions such as diabetes and asthma. Patient experience is evaluated by asking patients who saw their doctors during the year for their views on factors such as communication with their doctor and access to specialists. Finally, physician groups were rated on their investment and adoption of information technology (IT) to support patient care. This includes building patient registries for those with chronic illnesses and using physician or patient reminder systems at the point of care.
Back to top