Get in touch
555-555-5555
mymail@mailservice.com
Systems Approach to Solution Development & Implementation
Contact Us
T: +1-(631)-791-8129
Curious look into the quality of evidence produced in clinical trials
Veronika Valdova • December 14, 2019
Interventional studies: quality of evidence
As of today, there are 256,092 interventional studies
in the ClinicalTrials.gov registry. This NIH-operated database is not the only one but the most relevant and complete source of information on ongoing and completed clinical trials. The available information includes basic study descriptors such as phase, type, interventions, funding, and sponsors and co-sponsors; number of enrolled subjects and their demographic characteristics such as gender and age, and more detailed descriptors of study design such as masking, intervention model and allocation; as well as primary purpose and endpoint classification. This article is the first piece from a series of analyses of interventional trials on this database.
In interventional trials, the participants are assigned to receive one or more interventions or none at all, as assigned in the protocol. The objective is to gather and evaluate data from comparable groups of participants who only differ in the intervention they receive so that researchers can assess the effects of these interventions on biomedical or health-related outcomes. In general, interventions can be diagnostic, therapeutic, or behavioral.
Level of Evidence
For a biomedical researcher, the level of evidence produced in clinical trials is essential to determine the overall impact of the study on benefit-risk assessments of the studied interventions, treatment guidelines, recommendations for reimbursement, and other assessments. Most organizations have a structured and comprehensive way of assessing scientific evidence that is coming in. However, there are no universal standards or generally accepted guidelines for doing so, and these evaluations differ from one organization or journal to another.
Level I evidence includes:
- High-quality randomized trial or prospective study;
- Testing of previously developed diagnostic criteria on consecutive patients;
- Sensible costs and alternatives;
- Values obtained from many studies with multiway sensitivity analyses;
- A systematic review of Level I RCTs and Level I studies.
Some past scandals illustrate the extent of the problem very clearly: scientific opinions on the value, power, and level of evidence produced by the RECORD study meant the difference between acceptance and rejection of GSK’s drug Avandia. Assessment of quality of evidence produced in a study has real-life consequences, both medically and financially.
Unfortunately, ClinicalTrials.gov does not provide an assessment of evidence produced by studies included in the registry. The NIH leaves it solely to the assessor to decide how valuable each trial is in the context of all other available information.
Data
- Processing the dataset before analysis included deletion of extra columns, standardization of null values, and removal of trials scheduled to start after 2016. The material, therefore, does not serve as a prediction but solely as a review of studies that already started.
- Enrollment figures are skewed by a small number of studies with a very high number of participants (millions of subjects). Three studies listed the total number of participants at 99,999,999. These outlier figures skew the trends. Moreover, the number of planned and enrolled subjects can differ significantly, so the reliability of this data depends on the stage of the trial (plan, estimate, in progress, or completed and updated).
Type of Interventions
About half of all clinical trials studied drugs as the intervention in question. Biologics, devices, procedures, and other types of interventions account for most of the remaining 50%. A relatively small number of interventions concerned radiation therapy and dietary supplements. Breakdown by phase shows similar distribution by each phase: the majority of interventions for trials in phase I to IV are drugs, followed by biologics, devices and procedures. A relatively high number of trials does not have any phase stated; for these, the interventions fall almost evenly fall into categories “other”, “behavioral”, “device”, “drug” and “procedure”.
Phases
As mentioned previously, about one-third of clinical trials have no phase stated. Early-stage trials (phase I and II) create one third, and the remaining third goes to phase III and IV studies. One would expect that the highest number of enrolled subjects would be in late-stage and post-marketing studies, but that’s not the case: the few previously mentioned studies with a very high declared number of subjects are Phase I and undeclared.
Primary purpose
Nearly half of the studies are conducted for treatment purposes. About 40% of all studies have no primary purpose stated.
Funding
The highest share of studies, about a half, is funded by entities categorized as “other”, even more, if we include “other” in cooperation with other entities. The category "other" is a very broad category that includes academic institutions, individuals, and community-based organizations, including hospitals. About one-third of all trials are funded by industry.
Endpoint classification
The majority of all studies are conducted to satisfy safety and efficacy-related questions. More than a third of studies have no endpoint stated.
Quality attributes
The main quality attributes readily available for scrutiny are characteristics of study design: masking, allocation, and intervention model.
Allocation
is a clinical trial design strategy used to assign participants to an arm of a study. In a Randomized Controlled Trial, the participants are assigned to intervention groups by chance. If the trial is non-randomized, the participants are expressly assigned to intervention groups through a non-random method, such as physician choice. Allocation is not relevant for single-arm studies where all participants receive the same treatment. The gold standard scientific evidence is produced through randomized controlled trials.
Masking
is a design strategy in which one or more parties involved in the trial, usually the investigator or participants, do not know which participants have been assigned which interventions. The primary purpose of masking is to prevent placebo effect, post-randomization confounding bias, selection bias, or group differences n loss to follow-up and information bias - that is bias due to differences in reporting of symptoms.
- "Open-label" study describes a clinical trial in which masking is not used, and all parties involved in the trial know which participants have been assigned which interventions.
- In single-blinded studies, one party involved in the clinical trial, typically the investigator or participants, does not know which participants have been assigned which interventions.
- In double-blinded studies, two or more parties do not know which participants have been assigned which interventions. Typically, the parties include the investigator and participants, but sometimes also the assessor or caregiver.
Finally, the intervention model describes the design of the strategy for assigning interventions to participants in a clinical study.
- Parallel Study Design means that two or more groups of participants receive different interventions "in parallel“.
- In a Cross-Over Study design, groups of participants receive two or more interventions in a particular order, so the participants "cross over" from one drug to the other.
- Single Group Study Design means that all participants receive the same intervention.
- In Factorial Study Design, groups of participants receive one of several combinations of interventions so, during the trial, all possible combinations of the two drugs are given to different groups of participants.
Randomized Cotrolled Trials, the gold standard
In Randomized Controlled Trials, direct comparison is made between two or more treatments groups. Randomization eliminates baseline differences in risk between treatment and control groups. Randomization should make all groups similar in terms of the distribution of risk factors whether these risks are known or unknown. The larger the groups, the greater the probability of equal baseline risks. However, participants in randomized controlled trials are often not representative of the target population, which introduces selection bias and generalizability.
Data available in the ClinicalTrial.gov registry cannot satisfactorily answer the poignant question of quality of evidence studies generate. It would make life of health professionals and researchers much easier, if study results and level of evidence became standard part of all trial reports.
Nařízení (EU) 2017/745 o zdravotnických prostředcích vstoupilo v platnost 26. května 2021. Platnost certifikátů vydaných podle stávajících směrnic vyprší nejpozději 24. května. Toto je třeba provést ve lhůtě, která odpovídá době nezbytné pro přezkoumání dokumentace a před vypršením platnosti současných certifikací. Česká republika stále nemá svůj oznámený subjekt. Jakmile dojde k akreditaci, bude ITC zavalen ohromným počtem nevyřízených podání v českém jazyce. Zajisdtěte si pomoc s podáním od týmu Arete-Zoe, který má s přípravou klinické dokumentace dle MDR bohaté zkušenosti.
History of EU device legislation, definitions, obligations of economic subjects
Introduction & Orientation
More than 500,000 medical technologies are available on the European market, from hospitals to community care settings and people's homes. The products range from syringes, pregnancy tests, and wheelchairs to X-Ray machines and body scanners, pacemakers, hip implants, and pharmacogenomic tests. The medical technology industry is the source of a constant flow of innovations. The sector spends about 8% of its sales on R&D. Typical product lifecycle is about 18 to 24 months when a new, improved version becomes available. In 2020, the European Patent Office (EPO) accepted nearly 14,200 patent applications in the medical technology sector, trumping pharmaceutical patents (8,500 applications) and biotechnology (7,200). European and U.S. entities filed almost 80% of the applications (38% EU and EEA, 39% U.S.) [ 1 ]. The European medical technology sector employs more than 760,000 people, mainly in Germany (210,000), the United Kingdom (102,800), Italy (94,000), France (89,000), Switzerland (63,000), and Ireland (40,000), accounting for 0.3% of total employment. In comparison, the European pharmaceutical industry employs around 795,000 people. These jobs reach a value-added of about €184,000 per employee. More than 33,000 medical technology exist in Europe, of which 95% qualify as small, medium, and micro-sized companies (SMEs). The majority of these enterprises employ less than 50 people [ 1 ]. In 2020, Europe had a positive trade balance in the medical technology sector of € 8.7 billion. Compared to 2019, the European trade balance dropped by 27.5% (€ 12 billion in 2019). The most important trading partners for Europe are the United States, China, Japan, and Mexico. Germany, Ireland, the Netherlands, Belgium, and Switzerland have the highest trade share, both within and outside the EU [ 2 ]. Until May 2021, the medical device sector was regulated by Medical Device and In Vitro Diagnostic Device Directives 93/42/EC and 90/385/EEC (MDD and IVDD), when the new Regulations replaced these: Medical Device Regulation (EU) 2017/745 (MDR) and In Vitro Diagnostic Device Regulation (IVDR) 2017/746 [ 3 ],[ 4 ]. The new regulations introduced numerous changes, including the reclassification of some devices, requiring additional obligations for manufacturers to comply with. About 85% of in-vitro diagnostic devices will now require Notified Body involvement, compared to ~20% under the IVDD. Existing MDD/IVDD certificates remain valid until May 2024. After this date, all devices on the EU market must comply with the new MDR/IVDR regulations [ 5 ]. For some manufacturers, the costs associated with keeping some of their devices on the market under MDR/IVDR may no longer justify the expense considering their profitability. Others may not be able to obtain new CE Mark certification in time due to decreased capacity of notified bodies. These factors combined are already reducing the number of devices on the EU market and limiting the certification of innovative products. The number of notified bodies available to review new certifications and recertifications dropped significantly under the MDR/IVDR. Of 51 notified bodies designated to MDD [ 6 ] and ten to AIMDD [ 7 ], only 29 obtained designation for MDR, of which seven operate in Germany, seven in Italy, and three in the Netherlands [ 8 ].
The Medical Device industry produces a vast number of products, ranging from bandages and surgical instruments to life function monitors to imaging technology. The technology currently in use varies from devices that have been in use for decades to highly innovative products. Innovations in the medical device field are frequent and typically incremental in response to feedback from physicians. Mordor Intelligence estimated the global Medical Devices market at USD 532.62 billion in 2021, growing at a CAGR of around 5.5%, to reach USD 734.39 billion in 2027 [1]. Fortune Business Insights projects the global medical devices market growth from $455.34 billion in 2021 to $657.98 billion in 2028 at a CAGR of 5.4%. This development follows a decline from 2020, when CAGR dropped to 3.7% due to the pandemic [2]. The key drivers of market growth include the rising prevalence of chronic diseases, increased disability throughout the population, technological advancements in medical devices, and population aging. The U.S. medical device market was valued at USD 186.5 billion in 2021 and is anticipated to exhibit a compound annual growth rate (CAGR) of 5.0% over the forecast period to reach USD 262.4 billion in 2028 [4]. The Medical Device sector is even larger than Biopharmaceuticals, that by comparison, employed over 224,000 people, earning $21.2 billion [3]. The rising prevalence of chronic diseases and the increasing geriatric population in the country are the key market drivers. The Medical Device industry has a significant impact on the U.S. economy and supports hundreds of thousands of jobs. More than 80 percent of U.S. medical device companies have fewer than 100 employees [5]. In 2020, the U.S. Medical Device industry supported over 329,000 jobs, with an annual payroll of $25.8 billion [3]. Employment and payroll for Medical Device Subsectors, 2020 ( SelectUSA ):
U.S. Pharmaceutical Dependency on Foreign Sole-Source Production of Essential Materials Imposes Vulnerable Exposure to Interruption by Both Natural and Man-Made Threats
In the high consequence environment of pharmaceutical development, any assumption made earlier in the process can prove extremely costly if uncorrected once more information becomes available. From a business perspective, it is essential to create a safe avenue for communication of concerns regarding the drug candidate’s efficacy, safety, toxicity, or pharmacological function immediately as the researchers become aware of them.
Innovation always involves the risk of failure. It is an art to see what the data show, and what they don't, and which projections are the result of our wishful thinking or unsubstantiated assumptions. It may be just my impression that 14th-century logician William of Ockham whispers in my ear that entities shall not be multiplied unnecessarily.
Companies are growing in size due to acquisitions and mergers. Operations routinely span across geographical, jurisdictional, and cultural boundaries. The trend of industry consolidation continues in 2015 and 2016: the total number of deals flattened and remained even at around 600 mergers a year. Geographically, mergers, and acquisitions have been shifting from the U.S. to Western Europe. This shift is the result of transactions driven by the need to add complementary products to the core business areas and tax inversions.
Yes indeed. The reason is that critically important products such as active pharmaceutical ingredients and medical devices are manufactured in regions that are being shut down. Just-in-time supply chain and very few alternative suppliers for many off-patent generics make significant, long-term disruption much more likely.