May 2026

PS6: The Quest for Quicker Queries

As interest in real-world data grows, so do the number and complexity of questions that networks such as PaTH are asked to answer. These questions, or queries, can range from simple population identification, like “How many people in the system have a diagnosis of diabetes?”, to more complex queries, like “How many people in the system have a diagnosis of diabetes and comorbid kidney disease, have taken a GLP-1 medication in the past twelve months, and have discontinued that GLP-1 within the past 6 months”. Some queries can be run in minutes or hours, while others can take days to return data.

Here at PaTH, we are always looking for ways to improve the services that we provide to investigators, so as part of PCORnet 2.0, we proposed to undertake an initiative to improve the efficiency of query execution. This proposal, also known as PS6, used two approaches. First, we upgraded our computer hardware across all PaTH sites. Second, we investigated non-hardware approaches to improve our query speeds.

Drawing from our informatics experience across PaTH and PCORnet, we profiled the hardware needs across PaTH sites and successfully deployed upgrades, which resulted in improvements in query speeds over and above our initial targets. While investigating and building our own non-hardware solutions,  Eugene Sadhu, MD, Associate Chief Research Informatics Officer and Assistant Professor in the Department of Biomedical Informatics at the University of Pittsburgh School of Medicine, noticed that the SAS software that is used to program many data queries includes something called the Scalable Performance Data Engine (SPDE), which can leverage a variety of technical approaches (native multi-CPU scaling with automatic partitioning, parallel reads and execution, and hybrid storage architecture) to help queries run faster. Furthermore, importantly, SPDE allows us to run more multiple queries at once.

The PaTH teams tested the effect of the hardware and software improvements by looking at how long it took to run standard data quality (curation) queries from the PCORnet Coordinating Center. They found an average reduction of query time of 13.5 hours across PaTH sites! We were happy to share what we found with other PCORnet sites, so that PCORnet can continue to support research that answers questions that impact people’s lives.

«—- Back To News