About the HMO Ratings
Meeting National Standards of Care
Each year, a number of sample groups of HMO members are selected and their records are reviewed to determine if members received care that meets nationally recognized standards for good care. The member's care is documented using the HMO’s records and/or the person’s medical chart. Often, HMO records include information from the patient billings by doctors and others for medical and prescription drug services.
Information from the HMOs’ records and the members’ medical charts are collected and scored based on standards established by the HEDIS® (Healthcare Effectiveness Data and Information Set) performance measurement system. HEDIS® is described in greater detail below.
More than thirty HEDIS® measures are combined into a set of familiar topics, like Heart Care and Maternity Care, to score HMOs on providing the right care across a range of important health conditions. Then, these topic scores are combined to calculate a single summary rating for the HMO: Meeting National Standards of Care.
Back to top
Members Rate HMO
Each year, a sample of HMO members is contacted by mail or phone to complete a survey called CAHPS® which is described in greater detail below. Typically, about one-third of the members who are contacted answer the survey. The Members Rate HMO rating is based on a single CAHPS® survey question that asks members to rate all of their experience with the health plan. In addition to this summary rating, HMOs are scored on various aspects of patients' experiences of care and service using the patient survey answers. Though an HMO’s summary rating and the specific topic ratings are calculated separately, the results are very consistent. That is, the Members Rate HMO result is a good summary of members’ reported experiences across areas ranging from customer service to finding a doctor.
The Helping Smokers Quit measures also are CAHPS® survey questions. These measures are not available for some HMOs because too few members smoked or had smoked recently to answer the survey questions about smoking.
Back to top
Scoring and Rating Methods
HMO quality scores were constructed using the HEDIS® and CAHPS® quality performance systems. The quality measures are based on the services, care, and experiences of samples of commercial HMO members who were enrolled in the HMO throughout 2006. Medical chart and service records were collected and HMO members were surveyed in a standardized way through the coordination of the California Cooperative Healthcare Reporting Initiative (CCHRI).
The Meeting National Standards of Care scores represent the percent of members who got the right care. Scoring a topic requires several steps. First, scores are calculated for a number of important measures of good medical care – like are patients with harmful, high blood pressure seeing good results in lowering their blood pressure. Next, a number of these measures, that concern related aspects of a health condition, are combined into a topic score. The measures are combined by giving them equal weight and calculating an average score. Last, the topic scores are combined into a single summary rating using the same “equal weight and average score formula.” The score is given one of four performance grades that are indicated in the report card with stars. The possible grades for the single summary rating Meeting National Standards of Care are:
- Excellent: This means that about 8 of every 10 or more HMO members got the right care.
- Good: This means that about 3 of every 4 HMO members got the right care.
- Fair: This means that about 2 of every 3 HMO members got the right care.
- Poor: This means that fewer than 3 of every 5 HMO members got the right care.
For additional information on OPA's scoring methodology for HEDIS, refer to this pdf document.
The Members Rate HMO rating is based on a single CAHPS® survey question that asks members to rate all of their experience with the health plan. Members are asked to rate their overall experience on a 0-10 scale. The score is the number of members who rated the plan 8, 9 or 10 as a proportion of all members who answered the question.
Scores for the various CAHPS® topics are based on the proportion of the members’ who gave a positive response ("always" or "usually" for most questions) to the survey question. For most questions, responses are scored using one of four possible answers ranging from the member “always” had a positive experience with a particular need like getting an appointment to the member “never” had a positive experience. Each member’s responses for a set of related questions like “paying claims” are combined to create a per-member topic score; then the average of all of the members’ scores is calculated to create an HMO score for that topic. The scores represent the average or typical experience that that HMO’s members reported. The score for the single summary rating Members Rate HMO is given one of four performance grades that are indicated in the report card with stars. The possible grades are:
- Excellent: This means that about 3 of every 4 or more HMO members rated the health plan highly.
- Good: This means that about 2 of every 3 HMO members rated the health plan highly.
- Fair: This means that about 3 of every 5 HMO members rated the health plan highly.
- Poor: This means that only about half of HMO members rated the health plan highly.
A “buffer zone” adjustment is used for the summary measures Meeting National Standards of Care and Members Rate HMO. This adjustment accounts for the error that occurs in measurement and scoring when the formula is based on samples of members rather than all the members in an HMO. This “buffer zone” gives the benefit of the doubt to the HMO — if a score falls below a performance threshold, but within a half-point of that threshold, the HMO is assigned that next highest grade.
For additional information on OPA's scoring methodology for CAHPS, refer to this pdf document.
Back to top
“Too Few Patients” and “Did Not Report”
On the HMO rating charts, most columns have a score from one to four stars, with four stars being the highest rating. Sometimes you will see one of the following:
- Too Few Patients: This means that the HMO did not have enough members who had the experience to be scored.
- Did Not Report: This means that the HMO would not report its results. This usually means that the HMO did not do well and does not want to share the information.
Back to top
How Reliable Are the Scores?
Enough people had their medical charts reviewed or replied to surveys to give a very clear picture of how well each HMO provides care and service to its members. However, your experiences may be different. These ratings concern members who typically get insurance through their job. The ratings do not include members of Medi-Cal, Healthy Families, or Medicare HMOs.
The Meeting National Standards of Care rating tells an important story about how well the HMO and its doctors do in meeting national standards for good care. But, because the rating is limited to a particular set of health conditions there are many aspects of medical care that are not part of this rating.
The Members Rate HMO rating is based on member surveying that is done in a way to show the typical experience of HMO members. The people who were surveyed were randomly drawn from the HMO's full list of commercial members. Adults who were members of the HMO throughout 2006 were included on the survey list. Nonetheless, your experience with doctors or other providers who belong to the HMO or with the health plan’s staff may differ from the experiences reported in the survey.
Back to top
Information Inquiries and Complaint Rates by Health Plan 2004-2006
The following methodology was used to calculate the rate of HMO enrollee information inquiries and complaints with the DMHC Help Center. More information on what was measured, why it is important, and what was the source can be found on the Information Inquiries and Complaint Rates by Health Plan 2004-2006 page.
Number of information inquiries and complaints
The information inquiries and complaints in this report come from all DMHC contacts logged into the DMHC Help Center information system that are not excluded for one of the reasons listed below. An information inquiry or complaint could be received by telephone, mail, e-mail or fax. If there were multiple information inquiries or complaints from a single enrollee, each one was counted when determining the number of information inquiries and complaints per health plan.
Information inquiries and complaints that were not counted:
- Information inquiries and complaints from individuals whose plan does not fall within DMHC jurisdiction, such as Medicare, plans regulated by the Department of Insurance, and Health Net PPO enrollees
- Contacts from individuals requesting information about COBRA or general information about DMHC, OPA, HIPAA, or health care in general
- Contacts to inform DMHC about a case (“cc DMHC”)
- Message/call back requests for DMHC staff
- Wrong numbers
- Contacts from enrollees requesting copies of their complaint file
Contacts classified as “status calls” on pending DMHC complaints
- Contacts from providers for the provider complaint unit
- Hang ups
- Contacts classified as “Unknown Health Plan”
Number of enrollees
To determine the rate enrollees contact DMHC about their HMO with information inquiries and complaints, we must know how many enrollees each HMO has. The number of enrollees in this report is based on the number of enrollees under DMHC jurisdiction in each health plan. The number of enrollees is based on enrollment figures as of December 31 as reported to DMHC by each health plan. This enrollment figure is the same as the number of enrollees under DMHC jurisdiction, and is the same as the health plan enrollment reported in the DMHC Annual Report. To eliminate duplication, only full scope health plans are considered, while dental and behavioral health plans are excluded. This analysis includes enrollees in the following product lines:
- Group ( commercial)
- Medi-Cal Risk
- Individual
- Point of Service
- Small Group
- Healthy Families
- AIM
- PPO (Blue Cross of California and Blue Shield of California only
The following categories were not used to determine the total number of HMO enrollees in this report:
- Medicare enrollees (DMHC does not have jurisdiction for contacts from Medicare enrollees).
- Other categories of enrollees that are not under DMHC jurisdiction. These include “administrative services only” (ASO) and “plan to plan” enrollees. Health plans utilize the determination ASO as an identifier for “self-funded” health plans organized and operated by employers, trade unions, and other entities. ASO enrollees were excluded from the OPA Report Card analysis because under an ASO arrangement, the employer or other entity typically bears the risk for payment of claims associated with the delivery of health care, and the health plan simply serves an administrative, or ASO, function. “Plan to plan” enrollees were excluded, however there were no enrollees in this category.
Back to top
Rate of Service Denials Overturned by DMHC IMR Process 2004-2006
This rate was determined by dividing the plan’s total number of overturned denials by the total number of HMO members within DMHC’s jurisdiction over a three year period (2004-2006). For every 100,000 members a plan has, there are a certain number of denials overturned by the DMHC Independent Medical Review Process. For example, a rate of 5 means the HMO averaged 5 decisions overturned for every 100,000 members over a three year period. More information on what was measured, why it is important, and the source of information is found on the Rate of Service Denials Overturned by DMHC IMR Process 2004-2006 page.
Number of IMRs that overturned an HMO’s decision
IMRs provide HMO members the opportunity to appeal experimental or investigational services for life-threatening or seriously debilitating medical conditions or medically necessary services that were denied by their plan. An IMR is a review of your case by doctors who are not part of your health plan. DMHC contracts with an independent external review organization to conduct the IMR. If the independent review organization overturns the plan’s denial, the plan must give you the service or treatment you requested. Generally the service must be authorized within 5 days of the decision.
The DMHC annual reports provide the number of each HMO’s medically necessary and experimental/investigational denials overturned by IMR.
Number of HMO members
To determine the rate of HMO denial overturns, we must know how many members are in each HMO. Each health plan is required to report enrollment figures in their quarterly financial reports submitted to DMHC. A health plan’s number of members is based on the average of its enrollment figures reported over four quarters in a given year. This enrollment figure is the same as the number of members under DMHC jurisdiction, and is the same as the health plan enrollment reported in the DMHC Annual Report. To eliminate duplication, only enrollment in full service health plans are considered, while dental and behavioral health plans are excluded. This analysis includes enrollees in the following product lines:
- Group (commercial)
- Medi-Cal Risk
- Individual, including MRMIP
- Point of Service
- Small Group
- Healthy Families
- AIM
- PPO (Blue Cross of California, and Blue Shield of California only)
The following categories were not used to determine the total number of HMO members in this report:
- Medicare members (DMHC does not have jurisdiction over the Medicare appeals process).
- Other categories of members that are not under DMHC jurisdiction. These include “administrative services only” (ASO) members. Health plans utilize the determination ASO as an identifier for “self-funded” health plans organized and operated by employers, trade unions, and other entities. ASO member enrollment figures were excluded from the OPA Report Card analysis because under an ASO arrangement, the employer or other entity typically bears the risk for payment of claims associated with the delivery of health care, and the health plan serves an administrative, or ASO, function.
Back to top
Comparing Medical Group and HMO Ratings
An HMO is a type of health insurance. A medical group is a group of doctors who work with an HMO to give the HMO’s members their medical care. You should not compare the ratings for HMOs and medical groups because:
- HMOs and medical groups keep different kinds of records. This means that the information we get from them is different.
- We did not always study the same members when we looked at medical groups and HMOs, even when we were rating the same health topic. For example, the person whose diabetes care was reviewed for her HMO may not be the same person whose diabetes care was reviewed for her medical group.
- We used medical charts to get information about HMO performance. We did not use medical charts to get information about medical group performance.
- The quality of the care that HMOs and medical groups give is measured in different ways. The way HMOs and medical groups are rated also differs.
Back to top
California Cooperative Healthcare Reporting Initiative
The California Cooperative Healthcare Reporting Initiative (CCHRI) is a group of employers, health plans, and health care providers across the State. More than 90 percent of HMO members in California belong to plans that are part of CCHRI. CCHRI makes sure that people get information they can trust on how well health plans and medical groups provide care and service.
The CCHRI commitment to precise standardization supports “apples to apples” comparison of HMO and medical group performance so consumers can more easily make informed choices about their health care. CCHRI uses a common approach to measuring quality that is based on both the services members receive and on members’ experiences. As described above, HEDIS® is used to measure the care members receive, while CAHPS® assesses members’ experiences of their care and service. The collection of this information also is audited to be sure that the same approach is used for all HMOs.
HMOs that are a part of CCHRI voluntarily provide the information that is used for this report card. These HMOs allow outside experts to score their care and service and make that information public. We are grateful to CCHRI for providing the data that our experts used to score quality results. We thank the CCHRI HMOs for their commitment to quality measurement and public reporting.
Back to top
HEDIS®, CAHPS®, and the National Committee for Quality Assurance
HEDIS® and CAHPS® measures are important components of a national system of accreditation of HMOs and some physician organizations that is administered by the National Committee for Quality Assurance (NCQA). NCQA is “a private, not-for-profit organization dedicated to improving health care quality everywhere.” The NCQA-sponsored accreditation process is voluntary but many health plans participate.
The HEDIS® measures are based on randomly selected lists of members who are eligible to be included in an evaluation of quality for a particular condition or need, such as members who have had a heart attack or members who are children. The HMO supplies the information on whether or not the member received a particular service or the results of a test for that member. HMOs gather this information from the member’s medical chart or an administrative record or both. The accuracy of this information is independently checked. The score typically is the proportion of members whose records indicate that they obtained a particular service or test result.
Most HEDIS® measures are collected once a year based on the health plan members’ experiences in the prior year(s). However, HMOs are allowed to report on some HEDIS® measures every other year because the results do not change greatly over the span of just one year and collecting the HEDIS® data is expensive. A HEDIS® measure that can be reported every other year is known as a “rotated” measure. Some HMOs chose to collect and report the rotated measures information this year while others plans did not and instead reported results for the past year. This report card uses the results from either year because generally results do not vary much from year to year.
To get information about members’ experiences with their HMO, randomly selected members of the HMO are asked to complete the CAHPS® survey. These members were mailed a copy of the CAHPS survey and asked to report about their experiences with the HMO and its doctors. Follow-up phone calls also were used to interview some members who do not respond by mail. A research firm collected the survey responses and independent researchers scored the answers. The CAHPS® score typically is the proportion of members who answered the survey reporting a particular experience-like not having a problem getting needed care. For more information about HEDIS®, CAHPS®, or NCQA visit www.ncqa.org
Back to top