A Reference Method Laboratory Network for Cholesterol: A Model for Standardization and Improvement of Clinical Laboratory Measurements

Gary L Myers, Mary M Kimberly, Parvin P Waymack, S Jay Smith, Gerald R Cooper, Eric J Sampson, A Reference Method Laboratory Network for Cholesterol: A Model for Standardization and Improvement of Clinical Laboratory Measurements, Clinical Chemistry, Volume 46, Issue 11, 1 November 2000, Pages 1762–1772, https://doi.org/10.1093/clinchem/46.11.1762

Abstract

Background: Accurate and precise measurement of blood cholesterol plays a central role in the National Cholesterol Education Program’s strategy to reduce the morbidity and mortality attributable to coronary heart disease. Matrix effects hamper the ability of manufacturers to adequately calibrate and validate traceability to the National Reference System for Cholesterol (NRS/CHOL). CDC created the Cholesterol Reference Method Laboratory Network (CRMLN) to improve cholesterol measurement by assisting manufacturers of in vitro diagnostic products with validation of the traceability of their assays to the NRS/CHOL.

Methods: CRMLN laboratories established the CDC cholesterol reference method (modification of the Abell-Levy-Brodie-Kendall chemical method) and are standardized using CDC frozen serum reference materials. CRMLN laboratories use common quality-control materials and participate in monthly external performance evaluations conducted by CDC. The CRMLN performance criteria require member laboratories to agree with CDC within ± 1.0% and maintain a CV ≤2.0%.

Results: From 1995 to 2000, the CRMLN laboratories met the accuracy criterion 97% of the time and the precision criterion 99% of the time. During this time period, the CRMLN maintained an average bias to CDC of 0.01% and an average collective CV of 0.33%.

Conclusions: CDC established the CRMLN as the first international reference method laboratory network. The CRMLN assists manufacturers in the validation of the calibration of their diagnostic products so that clinical laboratories can measure blood cholesterol more reliably. The CRMLN can serve as a model for other clinical analytes where traceability to a hierarchy of methods is needed and matrix effects of the field methods with processed calibrators or reference materials are present.

Coronary heart disease (CHD) 1 accounts for an estimated 500 000 deaths annually in the US; it is the leading cause of death among men and women in all states for all racial and ethnic groups. Symptomatic CHD is present in more than 6 million Americans ( 1), and most US adults have some degree of atherosclerotic narrowing of their coronary arteries ( 2). CHD profoundly affects not only the US healthcare system, but also its economy; the current economic burden is more than $50 billion per year for illness care and lost earnings and productivity ( 3). The three most important modifiable risk factors that provide opportunities for individuals and populations to reduce their likelihood of developing CHD are cigarette smoking, high blood pressure, and high blood cholesterol.

In 1985, the National Heart, Lung, and Blood Institute launched the National Cholesterol Education Program (NCEP) to reduce the prevalence of increased blood cholesterol, thereby contributing to the reduction of CHD morbidity and mortality ( 4). In 1988, the NCEP’s Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults [i.e., the Adult Treatment Panel (ATP)] issued a report describing a national strategy to identify persons at risk for CHD ( 5). This national strategy is based on conclusive scientific evidence that lowering high blood cholesterol reduces the risk for CHD ( 6)( 7). The ATP report classified cholesterol concentrations into three regions:

The association between high blood cholesterol and the risk for CHD has become one of the most widely known medical relationships among healthcare professionals and the general public ( 3)( 8)( 9). All of this attention to blood cholesterol concentrations increased interest in cholesterol testing and focused unprecedented national attention on clinical laboratories and the need to provide reliable measurement of blood cholesterol ( 10). Given the central role of cholesterol measurement in the diagnosis and management of CHD, the reliable laboratory measurement of blood cholesterol became a national public health priority.

While the ATP was deliberating over its approach to reduce morbidity and mortality from CHD, the NCEP also convened the Laboratory Standardization Panel (LSP) to assess the state of reliability of blood cholesterol measurements in the US. The LSP subsequently issued two reports describing the status of blood cholesterol measurements and the need for adequate precision and accuracy in cholesterol testing. The first report concluded that the quality of cholesterol testing was inadequate and recommended analytical performance specifications for precision and accuracy ( 11). In a second more extensive report, the LSP made further recommendations for improving cholesterol measurements ( 12). The LSP recommended that, as a national goal, clinical laboratories should ultimately achieve an overall precision consistent with a CV ≤3% and that bias from the true value should not exceed ± 3% ( 11). The LSP also recommended that all clinical laboratories adopt the uniform cholesterol decision points issued by the ATP and that all cholesterol measurements be standardized and traceable to the National Reference System for Cholesterol (NRS/CHOL) ( 13)( 14). The introduction of these recommendations was very important, particularly the performance goals, because for the first time clinical laboratories had specific performance criteria by which to judge the reliability of cholesterol results. These performance criteria, along with other recommendations from the LSP, provided the first major steps toward ensuring accurate and precise laboratory measurements of cholesterol.

traceability to the nrs/chol

The accuracy of a field method depends on the relationship between the patient result and an accepted standard through a hierarchy of methods and materials. This relationship establishes the traceability of the field methods as defined by a transfer protocol where values are assigned with sufficient accuracy and precision along the entire chain of materials, from primary standards to reference materials to manufacturers’ calibrators and controls to patient samples.

The hierarchy of methods and materials for cholesterol is known as the NRS/CHOL ( 13)( 14). The NRS/CHOL was established by the NCCLS as part of the National Reference System for the Clinical Laboratory and has become the accuracy base for cholesterol measurements in the US. The NRS/CHOL consists of the NIST isotope dilution-mass spectrometry (IDMS) definitive method ( 15), NIST-certified pure cholesterol standard (SRM 911b), and the CDC reference method ( 16)( 17), a modification of the Abell-Levy-Brodie-Kendall method ( 18). Although the NIST IDMS method sits atop the cholesterol reference system hierarchy, the CDC cholesterol reference method has served as the standard for cholesterol testing for >20 years. Indeed, the medical decision points established by the NCEP are based on data obtained from clinical studies performed in research laboratories that were standardized and traceable to the reference method at the CDC.

The NIST IDMS definitive method serves as the accuracy base for the cholesterol reference method and as such plays an important role in the cholesterol reference system. Accuracy of the reference method is ensured through calibration using NIST SRM 911b, a pure cholesterol material, and by periodic comparisons with the NIST IDMS definitive method. Comparisons of these two methods have documented a small but consistent +1.5% bias for the reference method relative to the definitive method ( 19). Further investigative studies of the reference method have found that small contributions from cholesterol precursor sterols and phytosterols, which are measured by the reference method but not the definitive method, may account for more than one-half of the difference in cholesterol values determined by the two methods ( 20). Because the observed bias is small and reproducible, the reference method provides a practical basis for standardizing cholesterol measurement field assays.

The traditional approach for establishing traceability to the NRS/CHOL has been through the use of secondary reference materials, based largely on the assumption that these reference materials and calibrators are “commutable”, that is, they exhibit interassay properties comparable to real patient samples ( 21). In other words, if the relationships between the values obtained by the reference method and the field method on the calibrators or reference materials and on the patient samples are not significantly different, commutability has been demonstrated. Assuming commutability, then calibrators and reference materials value assigned by the definitive and/or reference methods can be used to establish analytical system set points and assess system performance, respectively.

In a study evaluating the performance of several commercially available cholesterol testing systems, acceptable performance by NCEP criteria was found when patient samples were used to compare enzymatic cholesterol results with results from the cholesterol reference method ( 22). However, significant discrepancies were found in results from control products, calibrators, and reference materials when the same routine methods were compared with the reference method ( 22). These discrepancies between the processed materials and the patient samples are matrix-induced biases or matrix effects resulting from differences between an analyte’s reactive properties in processed materials (usually lyophilized) and in patient specimens ( 23). As a result, when used for calibrating set points, the processed materials will generate inaccurate patient results, and when used for assessing accuracy, they will give erroneous conclusions regarding system performance. The presence of matrix effects compromises the commutability of calibrators or reference materials for use in establishing traceability and/or accuracy assessment. Not all methods are affected by matrix effects, but when matrix effects are present, an alternative approach for establishing the chain of traceability from the reference method to a field method is required. Because the ultimate goal in standardizing cholesterol results is to achieve accurate results on patient specimens, the use of processed materials to accomplish this purpose is no longer valid. Until commutable materials that are free of matrix effects are available, an interim approach to establish traceability to the NRS/CHOL is through a direct comparison of the routine method with the reference method by the analysis of fresh patient specimens.

In the US, an estimated 157 000 sites perform clinical laboratory testing, including cholesterol ( 24). Standardizing cholesterol testing in this many laboratories through direct comparisons using fresh patient specimens was impractical and beyond the analytical capacity of the CDC Lipid Reference Laboratory. A more practical approach was needed that would allow both comparison with the reference method and validation of the performance of instrument systems used in the clinical laboratories. Although thousands of testing laboratories exist, a limited number of manufacturers provide the instruments and reagents used in these laboratories. Developing a program that focuses on assisting the manufacturers of cholesterol diagnostic products to properly calibrate their systems would potentially have a greater impact on improving cholesterol testing than interacting with laboratories directly.

To provide the reference method analytical capacity needed to conduct such a program, CDC took a unique approach and, in 1989, established a network of cholesterol reference method laboratories. The goal was to “clone” the CDC reference method and provide analytical performance exactly as at CDC. Through this approach, manufacturers would have a mechanism to directly evaluate the calibration bias of their analytical methods relative to the NRS/CHOL, and by using these calibrated systems, clinical laboratories would be traceable to the NRS/CHOL as recommended by the NCEP ( 11).

Materials and Methods

method

The cholesterol reference method is a modification of the Abell-Levy-Brodie-Kendall chemical method ( 18). The cholesterol reference method uses chemical hydrolysis with alcoholic KOH, hexane extraction, and color development with Liebermann-Burchard reagent (acetic anhydride-acetic acid-sulfuric acid).

equipment

All instruments and equipment used in Cholesterol Reference Method Laboratory Network (CRMLN) laboratories must meet accuracy and precision specifications described in the procedure. When alternative procedures and equipment are used, the performance must be comparable to the corresponding procedures or equipment used by CDC.

cdc reference materials

The serum pools used for standardization and monthly monitoring are CDC-prepared serum pools. Blood is collected in glass containers from human donors with no anticoagulant, water, or other chemical added. The serum yields from bottles of individually collected blood are combined according to the quantity and concentration of lipids desired in the final pool. All pools are pressure-filtered through a 0.22 μm filter to assure sterility. The serum is dispensed into glass vials that are sealed with a stopper and an aluminum seal. CDC pools are prepared to last for ∼2 years and are stored in freezers at −60 °C or below to maintain stability.

fresh-frozen serum materials

These materials were used to monitor external performance. The fresh-frozen serum materials were prepared from units of blood collected from fasting, healthy donors. Collected blood was allowed to clot, and the serum was then separated by centrifugation. All samples used in the surveys were visually free of hemolysis and turbidity. Storage was maintained in freezers at −60 °C or below for no more than 3–4 months.

crmln

At its inception, the CRMLN consisted of nine US laboratories. These laboratories were selected from a core group of laboratories that all had previous experience with the reference method: they either participated in the interlaboratory validation transferability study of the reference method or participated in the Lipid Research Clinics Program. Since initiation of the CRMLN, five laboratories have dropped and one laboratory has been added. The current five active US laboratories are listed in Table 1 . Shortly after the formation of the US-based network, interest grew to expand the CRMLN membership internationally; six international laboratories are CRMLN members ( Table 1 ). To be selected as an international member, a laboratory must have documented support from either its country’s government or a professional society from within its country indicating that the laboratory will be recognized and supported as an international lipid reference laboratory.

Performance requirements for CRMLN laboratories.

The premise behind CDC’s approach to providing traceability to the NRS/CHOL through the CRMLN is that all of the member laboratories must perform the cholesterol reference method the same as it is performed at CDC. As part of the effort to accomplish this, CDC conducted numerous meetings, conference calls, method audits, and on-site training to assist in transferring the method to the CRMLN laboratories.

The performance requirements for CRMLN laboratories have evolved since the CRMLN was formed. In the beginning, performance specifications were arbitrary and established on the basis of previous experience with the reference method. The first set of performance criteria was based on the performance of the 14 laboratories that participated in the interlaboratory transferability study of the reference method. Twelve of the 14 laboratories that participated in the study obtained a bias compared with CDC of

These criteria for CRMLN laboratory performance were also consistent with performance requirements needed to validate traceability of clinical laboratory measurements to the NRS/CHOL. Validating traceability to the NRS/CHOL through comparisons between the CRMLN and manufacturers requires that all CRMLN laboratories maintain a low bias to CDC. Bennett et al. ( 25) showed that reference laboratory bias is the most important parameter in programs of this type in which traceability to a reference method is established through a secondary reference laboratory. In March 1993, the bias requirement for CRMLN laboratories was additionally tightened to its current value of ≤1.0%. This change in criteria was based on actual performance of the CRMLN laboratories and the need to minimize the overall bias of the CRMLN to CDC as described by Bennett et al. ( 25).

Performance of CRMLN laboratories.

To establish the required performance, each member CRMLN laboratory must successfully pass a series of performance tests. First, each laboratory must successfully complete parts I and II of the CDC-National Heart, Lung, and Blood Institute Lipid Standardization Program ( 26). Once standardized and traceable to CDC, CRMLN laboratories must achieve and maintain the strict bias and precision values needed for this program. Acceptable performance of CRMLN laboratories is maintained through a combined program of external blind quality assessment (monitoring) conducted by CDC and uniform internal quality-control (QC) requirements followed by all CRMLN laboratories.

External performance monitoring of CRMLN laboratories.

The most critical performance parameter in this type of traceability validation scheme (where accuracy is not based on group consensus but rather on a direct line of traceability from a field method through a single CRMLN laboratory to CDC) is the bias of the CRMLN laboratories to the reference method at CDC. To evaluate bias compared with CDC, the performance of CRMLN laboratories is monitored monthly through a blind external quality assessment program.

From 1991 through 1998, the external monitoring scheme consisted of a combination of CDC-prepared frozen serum reference materials and a survey of fresh-frozen serum samples (prepared on a rotating basis by different CRMLN laboratories). In each quarter, CDC provided materials for external monitoring for 2 months, and CRMLN provided fresh-frozen materials for the third month. For the CDC frozen materials, reference values were assigned on the basis of quadruplicate measurements in 12 in-control runs (n = 48) by the CDC Lipid Reference Laboratory. Reference values for the fresh-frozen materials were assigned by CDC based on quadruplicate measurements in four in-control runs (n = 16). Both CDC materials and the fresh-frozen samples were provided in three concentrations and were analyzed by each CRMLN laboratory in duplicate over four analytical runs, preferably with no more than two runs in any week. In 1995, the survey scheme was changed to three concentrations in duplicate in each of two analytical runs, and in 1998 the fresh-frozen survey pools were discontinued in favor of only CDC-supplied materials. A low total-cholesterol sample

Beginning in 1999, all materials for external monitoring of the CRMLN laboratories were prepared under contract by Solomon Park Research Laboratories, Kirkland, WA, according to NCCLS Guideline C37-A ( 27). Reference target values for pools prepared by Solomon Park Research Laboratories were established in the Lipid Reference Laboratory at CDC, following the same value assignment protocol (quadruplicate analyses in 12 in-control runs; n = 48) used for CDC prepared pools.

Results

Monitoring data for the CRMLN laboratories for December 1995 through February 2000 are presented in Fig. 1 . As mentioned previously, in 1995 the monitoring scheme was changed from four analytical runs to two. Over a 4-year period, the CRMLN has maintained an average bias to CDC of 0.01% and an average collective CV of 0.33%. During this time period, the CRMLN laboratories met the accuracy criterion 97% of the time and the precision criterion 99% of the time. Fig. 2 shows the long-term stability (average percentage bias vs CDC) for the CRMLN laboratories from December 1995 through February 2000.

Univariate analysis showed the distribution of the biases obtained by the CRMLN laboratories during the monthly surveys. Results of univariate analysis of the survey data collected from December 1995 through February 2000 showed a gaussian distribution. The total number of events (one survey material, analyzed by one laboratory in 1 month) during this time period was 1530. The mean bias was 0.01%, with a SD of the bias of 0.45%. The biases, by percentile, were −0.7% for the 5th, −0.2% for the 25th, 0.0% for the 50th (median), 0.3% for the 75th, and 0.7% for the 95th.

The typical long-term performance of the individual CRMLN laboratories for this same period is shown in Fig. 3 . For clarity and readability, we divided the CRMLN laboratories into two groups. The data shown are for the high-concentration pools used in the monthly surveys. Percentage bias is plotted for each laboratory. The concentrations in the high-concentration pools used during this period were 5.27–8.61 mmol/L (204–333 mg/dL), with a mean concentration of 6.77 mmol/L (262 mg/dL) and a median concentration of 6.59 mmol/L (255 mg/dL).

remedial actions for unacceptable performance

A caution alert is issued to a CRMLN laboratory when results for one pool in any monthly survey challenge are outside acceptable criteria. A CRMLN laboratory’s performance is deemed unacceptable when results from two pools on any single monthly survey challenge are outside acceptable performance criteria or when at least one of the pools fails to meet performance criteria on any two of three consecutive monthly challenges. When performance is deemed unacceptable, the CRMLN laboratory is required to immediately discontinue providing reference method analytical services until corrective measures are taken. Troubleshooting includes a QC evaluation, performance of a special fresh-frozen sample comparison with CDC, and an intensive method audit to evaluate possible sources of bias or imprecision. CDC provides a set of six frozen reference materials as check samples with CDC targets to the deficient laboratory for analysis in duplicate in four runs (n = 48). At the CVs that are typical for the CRMLN laboratories, this gives sufficient power to determine a 1% difference from the CDC reference value. If the laboratory does not meet the criteria of ± 1% with the check samples, CDC will continue to provide samples in sets of six until the laboratory improves its performance with the reference method. If poor performance lasts over 3 months, the CRMLN laboratory is advised to go on inactive status until it can demonstrate acceptable performance over a period of 3 months.

qc in crmln laboratories

With proper care and attention to detail, the cholesterol reference method can be exceptionally accurate and precise. In addition to external performance monitoring, a rigorous internal QC system, which is followed by each laboratory, is key to this performance. The statistical QC procedure used in the CRMLN laboratories is described in the laboratory manual used in the National Health and Nutrition Examination Survey central laboratory at CDC ( 28). Before 1992, CRMLN laboratories used their own in-house QC materials and systems for monitoring routine performance. The experiences learned during the 10-year Lipid Research Clinics Program demonstrated that to obtain the needed inter- and intralaboratory precision, all CRMLN laboratories should use the same QC materials. As a result, in 1992 CDC began providing two QC materials with different cholesterol concentrations to all CRMLN laboratories for analytical-run evaluation: a low-concentration material with a cholesterol concentration of 4.01–4.65 mmol/L (155–180 mg/dL) and a high-concentration material with a concentration of 6.72–7.24 mmol/L (260–280 mg/dL). The QC materials prepared by CDC are frozen pooled human serum with no additives. Using the National Health and Nutrition Examination Survey guidelines, each laboratory sets its own control limits. These limits are based on the calculation of upper control limits (UCL) and lower control limits (LCL) for the mean and range limits at the 95% and 99% confidence levels. Maximum network limits are set for how high and how low the mean UCL and LCL and the range limits can be within the CRMLN. No analytical run may be accepted as “in control” if it exceeds the 99% UCL or LCL, or range limits for the CRMLN. These limits are based on three times the “standard error of the mean” (SE) of the UCL, LCL, and range limits for the entire network.

Representative QC results for pool Q23, with a concentration of 4.1 mmol/L (157 mg/dL), are summarized in Table 2 . Data for Q23 were collected from May 1998 through November 1999. Fifty analytical runs from each laboratory were used to establish the maximum network limits. Listed are the results of the analysis of variance for the QC data for each laboratory. In addition to the 95% and 99% LCLs and UCLs and the range limits, among-run standard deviation (S_ar), within-run standard deviation (S_wr), total standard deviation, and the CVs are listed. The last column in Table 2 lists the maximum network limits. The S_ar and S_wr can be helpful in diagnosing precision problems. In general, laboratories that have wider range limits tend to have higher S_wrs_. In addition, laboratories that have wider mean limits tend to have higher S_ars than do all laboratories.

When QC pools are replaced, the CRMLN laboratories are provided with the new materials in time to perform overlap runs with the appropriate old material. New limits are established after most of the laboratories have compiled a minimum of 25 analytical runs. In our experience, laboratories whose 95% and 99% LCLs or UCLs or range limits exceed the network’s maximum allowable limits are more likely to fall outside the criteria on the monthly performance criteria.

validation of traceability

The main focus of the CRMLN is to provide a mechanism by which manufacturers can evaluate instrument system calibration bias and validate traceability to the NRS/CHOL. By focusing on the manufacturers of in vitro diagnostic instrument systems, CDC can directly impact the quality of cholesterol tests in the nation’s clinical laboratories. However, although the main focus and emphasis are on certifying manufacturers, the CRMLN also provides access for individual clinical laboratories that may desire or need to establish a more direct link to the NRS/CHOL. Both programs are based on performing split-sample comparisons with the reference method.

Manufacturer certification program.

Manufacturers of in vitro diagnostic instruments and reagents play a pivotal role in standardizing cholesterol measurements. CDC believes that working with these manufacturers is the most efficient approach, providing the greatest impact, to achieve the needed accuracy in cholesterol testing. Manufacturers can validate traceability, minimize calibration bias, and establish method-specific set point values for calibrators or reference materials that provide accurate analysis of patient specimens.

The NCCLS has developed a guideline to give conceptual help when designing experiments to evaluate the bias between two methods or devices that measure the same analyte, NCCLS EP9-A ( 29). The CRMLN adopted this guideline as the basis for validating traceability to the NRS/CHOL. The certification protocol requires analysis of 40–50 fresh patient specimens by both the manufacturer’s assay and the reference method at a CRMLN laboratory. The patient specimens are selected to cover a clinically meaningful range of cholesterol concentrations, 3.1–10.3 mmol/L (120–400 mg/dL). The cholesterol concentrations of these specimens should be distributed over a clinically meaningful range, using the following distribution: 20% of samples from 3.10 to 4.67 mmol/L (120 to 180 mg/dL), 30% of samples from 4.68 to 5.71 mmol/L (181 to 220 mg/dL), 30% of samples from 5.72 to 6.74 mmol/L (221 to 260 mg/dL), and 20% of samples from 6.75 to 10.34 mmol/L (261 to 400 mg/dL).

Certification of Traceability is defined here as the ability of the instrument system to measure cholesterol within the performance criteria established by the NCEP’s LSP (bias compared with the reference method must be ≤3.0%; CV must be ≤3.0%). One within-method outlier is acceptable; between-method outliers are not acceptable. The within-method outlier test compares both the absolute and relative differences between duplicate measurements. If an individual sample difference is greater than four times the average difference, the sample is flagged as an outlier. A sample must be flagged as an outlier for both the absolute and relative tests before it fails the within-method outlier test. Similarly, for the between-method outlier test, both the absolute and relative differences between the test method and the reference method mean are compared. If an individual sample difference is greater than four times the average difference, the sample is flagged as an outlier. A sample must be flagged as an outlier for both the absolute and relative tests before it fails the between-method outlier test. Because the primary goal of the CRMLN is to evaluate accuracy, between-method outliers are important to properly assess this parameter.

When the results demonstrate that the NCEP criteria are met, the CRMLN provides documentation in the form of a “Certificate of Traceability”. The certificate means that the system (a specific combination of instrument, reagents, and calibrators) has a documented traceability to the NRS/CHOL and can provide results consistent with the NCEP guidelines for cholesterol measurement. The comparison performed against the reference method represents only one point in time; therefore, manufacturers are expected to maintain traceability for subsequent lots of calibrators and reagents, using their own internal good manufacturing practices. However, manufacturers should recertify whenever instrument modifications are made and are encouraged to recertify new lots of reagents and calibrators whenever possible. The Certificate of Traceability issued by the CRMLN is valid for 2 years.

As mentioned above, the bias of the CRMLN laboratory is the most important factor that needs to be controlled to make appropriate decisions about the performance of manufacturers. Although the CRMLN has achieved a very low bias compared with CDC, some bias still exists. Using univariate analyses, we have determined that the CRMLN can allow manufacturers an additional 0.3% bias above the NCEP accuracy limit. This decision was based on the fact that 50% of the CRMLN laboratory bias lies between ± 0.3%.

As a service to the laboratory community, the CRMLN publishes annually a list of manufacturers that have successfully completed the split-sample comparison and received a Certificate of Traceability. The report of CRMLN certified instruments and reagents is accessible on the Internet at http://www.aacc.org/standards/cholesterolinfo.html. This list includes all systems that have current Certificates of Traceability.

Clinical laboratory certification program.

Clinical laboratories use either homogeneous or heterogeneous component systems. Those that use a homogeneous system should request information from the manufacturer of the system’s performance and documented traceability to the NRS/CHOL. However, a manufacturer’s demonstration of a product’s traceability does not in itself guarantee the accuracy of that product in the hands of every user. Therefore, these clinical laboratories may also want to validate traceability to the NRS/CHOL and participate with a CRMLN laboratory in a split-sample comparison to standardize their cholesterol measurements. If a heterogeneous system is used, components may come from a variety of manufacturers; thus, the laboratory must take primary responsibility for validating traceability of the system to the NRS/CHOL. Laboratories with heterogeneous systems are strongly encouraged to compare the performance of their system with that of a CRMLN laboratory.

For practical and economic reasons, the clinical laboratory protocol is not as extensive as the protocol used by the manufacturers. Clinical laboratories are required to analyze six samples in duplicate to qualify for clinical laboratory certification. A Certificate of Traceability, valid for 6 months, is issued to any clinical laboratory that meets the NCEP guidelines for accuracy and precision. In this program, a clinical laboratory has met the NCEP guidelines when the following conditions are satisfied for the comparison: (a) the bias compared with the reference method is ≤3.0%; (b) the CV is ≤3.0%; (c) there are no within- or between-method outliers; (d) the biases at the medical decision points of 5.17 mmol/L (200 mg/dL) and 6.21 mmol/L (240 mg/dL), calculated from linear regression, are ≤3.0%; and (e) the correlation coefficient for the regression is ≥0.975.

Discussion

Increased blood cholesterol is a significant risk factor for developing CHD. The NCEP has developed a national strategy to reduce morbidity and mortality resulting from CHD. Reliable cholesterol measurements are necessary for successful implementation of a public health program to identify, classify, and treat an individual’s risk for CHD based on his or her cholesterol concentration. The CDC reference method for total cholesterol is considered the “gold standard” for cholesterol measurement. It served as the accuracy base for all of the epidemiologic studies and clinical trials on which the relation of increased blood cholesterol to CHD is based and the medical decision points were derived. Therefore, obtaining cholesterol results that are linked to this epidemiologic database requires standardization of cholesterol measurement systems and validation of traceability to the reference method for cholesterol at CDC. Validating traceability of a field method back to a reference method requires an unbroken chain through which the accuracy of results on patient samples is guaranteed. Matrix effects observed in lyophilized and other processed samples presented new challenges for validating traceability of field methods to a reference method. The establishment by CDC of the CRMLN is the first such network of laboratories designed to provide a direct link between an established reference measurement system and manufacturers of in vitro diagnostic instruments and reagents, using patient samples. It is also the first time an international group of laboratories has cooperated with a goal of establishing a common reference method that in turn would provide an accuracy base to make possible the global standardization of a clinical laboratory test. Creating and sustaining a network for this purpose has required substantial commitment from each of the laboratories’ institutions.

The CRMLN has had a major impact on improving the reliability of cholesterol testing in US laboratories. The test systems used by nearly 95% of the participants in the proficiency testing surveys of the College of American Pathologists (CAP) and the American Association of Bioanalysts have validated traceability and been certified for total-cholesterol performance through the CRMLN. In 1994, the CAP conducted a study to determine calibration biases, matrix effect biases and survey biases in results from the Comprehensive Chemistry Survey ( 30). A fresh-frozen serum sample was tested in conjunction with the CAP Comprehensive Chemistry Survey in a 10% subset population of the ∼7000 survey participants. Results from the participants on the fresh-frozen sample were compared with a reference value established by the cholesterol reference method at CDC. The average calibration bias for the cholesterol methods surveyed was −0.05%, indicating excellent agreement of the cholesterol field methods with the cholesterol reference method.

In 1996, Thienpont et al. ( 31) evaluated the performance of four frequently used cholesterol test systems, using patient split-sample comparisons with isotope-dilution gas chromatography–mass spectrometry as the comparison method. Results demonstrated excellent performance for some of the most widely used cholesterol measurement systems. The authors of the study credit the CRMLN, in part, for the excellent performance of the four systems evaluated and state that the CRMLN can serve as a model for other clinical analytes where traceability to a hierarchy of methods is needed and matrix effects of the field methods with processed calibrators or reference materials are present ( 31). In fact, a laboratory network patterned after the CRMLN was formed in 1996 to standardize the measurement of glycohemoglobin ( 32).

Total cholesterol is only part of the NCEP’s strategy to lower the morbidity and mortality associated with CHD. Also included in this strategy is the measurement of HDL-cholesterol, triglycerides, and LDL-cholesterol. Because of the difficulty in transferring the CDC reference method for HDL-cholesterol, the CRMLN developed a designated comparison method for HDL-cholesterol to make comparison services for HDL-cholesterol more accessible to manufacturers ( 33). Three of the CRMLN laboratories have established the reference method for LDL-cholesterol and been standardized by CDC. As a result, the CRMLN now provides analytical services for validating traceability for both HDL- and LDL-cholesterol. Standardization of HDL and LDL is especially relevant because of the new homogeneous assays for HDL- and LDL-cholesterol that have become available in the clinical laboratory testing market. These assays also present challenges for validating traceability because they use different principles for quantifying HDL- and LDL-cholesterol from the traditional lipoprotein precipitation methods.

The CDC reference method for triglycerides is being modified for ease of operation and transfer. Reference analytical services are expected to be available in early 2001 to assist manufacturers in the validation of traceability of their triglyceride assays.

CRMLN participating laboratories.

CRMLN participating laboratories.

Performance plot of the CRMLN for total cholesterol measurement: results from December 1995 through February 2000.

Criteria for acceptable performance: bias ≤1% vs CDC; imprecision (as CV) ≤2%. n = 1509 (each point represents the bias and CV of one CRMLN laboratory on one material in one monthly survey). Data from 10 laboratories are included. Each laboratory contributed 146–153 points, with an average of 151. Average bias = 0.01% compared with CDC; average CV = 0.33%.

Bias plot of total cholesterol for all-laboratory average from December 1995 through February 2000.

Format of date: yyyymm. ▾, low-concentration pools; +, medium-concentration pools; ▴, high-concentration pools.

Bias plots of total cholesterol for individual laboratories for the high-concentration pool December 1995 through February 2000.

For readability, the laboratories have been divided arbitrarily into two groups (A, laboratories A–E; B, laboratories F–J). Format of date: yyymm. Concentration range of pools used in these surveys was 5.27–8.61 mmol/L (204–333 mg/dL); mean concentration of pools was 6.77 mmol/L (262 mg/dL); median concentration of pools was 6.59 mmol/L (255 mg/dL). Laboratories: ▪, A; ▴, B; +, C; ○, D; ▾, E; □, F; ▴, G; ×, H; •, I; ▿, J.

Summary of CRMLN performance on 50 analytical runs of a low concentration QC material.

Calculated from 3 SE of the laboratory values.

S_t, total standard deviation.

Summary of CRMLN performance on 50 analytical runs of a low concentration QC material.

Calculated from 3 SE of the laboratory values.

S_t, total standard deviation.

We thank Charlene Griffin (CDC), who for more than 20 years has provided outstanding service as the analyst responsible for the cholesterol reference method. We also thank Stephen Ethridge, Robert Cheek, and Wenxiang Chen (all from CDC) for able technical assistance, and Merle Holstun (CDC) for valuable assistance in data analysis and daily activities supporting the CRMLN. In addition to the current members of the CRMLN listed in Table 1 , we express our appreciation to all past CRMLN members.

Nonstandard abbreviations: CHD, coronary heart disease; NCEP, National Cholesterol Education Program; ATP, Adult Treatment Panel; LSP, Laboratory Standardization Panel; NRS/CHOL, National Reference System for Cholesterol; IDMS, isotope-dilution mass spectrometry; CRMLN, Cholesterol Reference Method Laboratory Network; QC, quality control; UCL, upper control limit; LCL, lower control limit; S_ar, among-run standard deviation; S_wr, within-run standard deviation; and CAP, College of American Pathologists.