Methodology behind CNN's exclusive story, 'The more opioids doctors prescribe, the more money they make'

(CNN)To examine the relationship between which doctors were writing prescriptions for opioids and which were receiving money and gifts from the opioid industry, CNN collaborated with a pair of top academic medical researchers: Dr. Michael Lawrence Barnett of the Harvard T.H. Chan School of Public Health and Dr. Anupam B. Jena of Harvard Medical School.

The source data

The CNN and Harvard team analyzed a series of government records to investigate the issue, primarily using two databases maintained by the US Centers for Medicare and Medicaid Services.
One captures all prescription claims nationwide written under Medicare's Part D program (Medicare's prescription drug insurance program). The database includes tens of millions of records of the number of prescriptions written by any medical provider nationally, broken down by the specific medication prescribed. For this story, we used 2014 and 2015 data, the most recent available at the time.
    The other database contains millions of records of payments (e.g. speaking fees, research funds) and gifts (e.g. travel, meals, etc.) made by pharmaceutical companies to doctors in the United States. Under the Affordable Care Act, the federal Open Payments program requires all drug and medical device companies to report such payments annually to the government, which then makes them available to the public online on a searchable website. CMS also makes the entire database available. For this story, we used 2014 and 2015 data, to match the Part D prescription data timeframe.
    Additionally, several government databases were also used to help further identify and classify the drugs, manufacturers and doctors examined in the analysis. The US Food and Drug Administration's Orange Book includes all drug variations considered opioids by the FDA, as well as the companies that manufacture them. The CMS-maintained National Plan and Provider Enumeration System's National Provider Identifier database includes unique identifiers, known as NPI numbers, for all US physicians, as well as information such as location and practice specialties.

    Building bridges

    To perform this analysis, we first had to get the two main databases -- the prescription records and the payment records -- to talk to each other. Ideally, such databases would use a commonly shared unique identifier, such as the national provider identifier (NPI) number of each physician, to allow analysts to match up doctors in both sets of records. However, in this instance, that was not the case. Though the Part D prescription records used the NPI to identify a doctor, the Open Payments database did not (instead, it had its own separate set of ID numbers for its doctors).
    To bridge this gap, we needed to add an NPI number to the Open Payments records. We conducted a series of programmatic queries designed to match street address and other location information between those listed for a doctor in the payments database and those listed in the National Provider Identifier database. Since the payment records dated to 2014, when a physician may have practiced at a different location or even in a different region, we also ran secondary matches against archived version of the NPI database as it existed in 2014 and 2015, in order to capture doctors whose locations may have changed and thus would not match the current NPI listing for them.
    We decided to err on the conservative side when it came to any doctors who showed up as duplicates, with more than one NPI number at the same address: They were excluded from the analysis. When the matching analysis was complete, this method was able to match NPI records for about 96% of the total amount paid to doctors in the Open Payments database for 2014-15.
    Using the combined dataset, we examined doctors prescribing opioid medications and which doctors were receiving money or gifts from opioid companies. To determine which drugs should be considered opioids for the purpose of the analysis, we used the FDA's Orange Book to pull in all the variations on drug names that the agency classifies as opioids. Additional FDA records helped show which companies manufactured each of the drugs.

    Determining the providers

    We limited our analysis to physicians whose specialties required their payments to be reported to the Open Payments program. This allowed for a more accurate and fair "apples to apples" comparison with the Part D prescription records, which included many non-physician medical providers.
    For the Open Payments program, only doctors (M.D.s and D.O.s), dentists, eye/vision providers and podiatric medicine providers are included.
    Additionally, there were some instances in which a provider's NPI taxonomy code -- his or her specialty -- had a code that did not exist in the associated code sheets. Because many of these providers were indeed determined to be physicians on manual review, we decided to include providers with such non-conforming specialty codes in the analysis, in order not to inadvertently exclude relevant physicians. As a result, a small number of non-physicians may also have been included in the analysis.

    Determining the manufacturers

    A challenge was presented by the Open Payments data in how they account for which drugs are supposedly tied to the payments. It is well-recognized that many drug manufacturers do not list the name of the drug the payment is associated with in these data, despite the federal program's direction that they do so. Indeed, the majority of total payments recorded in the 2014-15 database contained no associated drug name.
    Because of this large amount of missing data on the drug associated with individual payments, CNN and the Harvard researchers decided to focus on opioid manufacturers themselves rather than individual drugs. We flagged all payments by manufacturers who produced opioids and discarded the unreliable drug "name" field in the database. As a result, some of the records considered opioid-related include records where there was no drug listed or where other drugs may have been listed.
    We performed additional checks to make sure that the relationships we observed were not explained by non-opioid payments from opioid manufacturers who make other drugs. For example, in an analysis of pain medicine specialists, who largely focus on pain management and therefore may naturally prescribe more opioids to their patients, we still observed the same trends among the manufacturers making payments as we did for the overall physician population. Relationships between opioid prescribing and payments were also very similar.