HHippocratic Club

EHR Build Archaeology: Same Vendor, Wildly Different Experience

Software explains only 19.8% of clinician EHR satisfaction variance. Two hospitals on identical Epic builds score wildly differently because the analyst who made the local configuration choices left, and nobody documented why.

15 minutes read 3,133 words
EHR Build Archaeology: Same Vendor, Wildly Different Experience

A newly hired clinical informaticist is staring at a sepsis order set she did not build, at a hospital she has worked at for six weeks, trying to figure out why it requires four extra clicks that the identical order set at her previous hospital, running the same version of the same vendor's software, does not require.

There is no comment field explaining the choice. There is no ticket linking to a rationale. There is a name attached to the last modification, from a change made three years ago, and that name belongs to an analyst who left the health system eighteen months ago for a job at a different hospital in a different state.

She could call him. Nobody has built a way to find out that she should, or how, or whether he would even remember. So instead she does what every informaticist in her position does: she spends the next two weeks reverse-engineering a decision someone else already made, for reasons that were probably good, that are now completely unrecoverable, before she is allowed to touch it.

Her hospital and her previous hospital are running, by any formal definition, the same product. The clinicians at one are measurably more satisfied than the clinicians at the other, and the gap has almost nothing to do with which vendor either hospital chose. It has to do with a decision made years ago by a person who is gone, whose reasoning left with him, because nobody ever built a way for that reasoning to survive a job change.

The vendor is not the variable

The single most important fact in this problem, and the one that should reframe how every hospital thinks about its EHR budget, is this: the software itself is a minor contributor to how clinicians experience it.

KLAS Research's Arch Collaborative, drawing on more than 300 organizations and over 700,000 clinician surveys, finds the average physician Net EHR Experience Score (NEES) sits at 23.4, well short of the 60-point threshold required for a Pinnacle Award. The average nurse score is 47.3, against a 75-point award threshold. These are organizations mostly running the same handful of certified platforms.

A peer-reviewed analysis of that same dataset, published by Longhurst and colleagues in Applied Clinical Informatics in 2019, decomposed exactly where satisfaction variance comes from. The EHR software itself explained only 19.8 percent of the variance in clinician satisfaction. Individual factors explained 50.6 percent. Organizational and local-configuration factors explained 15.1 percent. Specialty explained 14.4 percent.

Read that carefully. The choice of Epic versus Oracle Health versus any other certified vendor explains roughly one-fifth of why a clinician is satisfied or miserable using it. The rest is the clinician's own characteristics and, critically, local, organization-specific choices about how the software was configured and how people were trained to use it.

What actually moves the number, and by how much

The same study identifies exactly which local choices matter, and the magnitude is not subtle.

Template-personalization adopters scored 29.3 points higher than non-adopters. Organizations providing 10 to 16 hours of local training scored 27.5, versus 6 for organizations providing under 4 hours. These are not marginal differences on a 100-point scale. A 29-point swing from template personalization, on a scale where the award threshold sits at 60, is close to the entire gap between an average hospital and an award-winning one.

The consequences of that gap are not confined to satisfaction surveys. Eschenroeder and colleagues, in JAMIA in 2021, found that physicians whose organization "excelled at implementing, training on, and supporting the EHR" were roughly twice as likely to report lower burnout (adjusted odds ratio 2.14, 95 percent CI 2.01 to 2.28), against a burnout rate ranging 22 to 34 percent by specialty in the same study.

And it is not just a wellbeing metric. Classen, Longhurst and colleagues, in JAMA Network Open in 2023, examined 112 hospitals and 5,689 physician surveys and found Leapfrog EHR/CPOE safety scores correlated with Arch Collaborative user-experience scores (adjusted beta 0.011, 95 percent CI 0.006 to 0.016, P<.001). The local build quality that determines whether a physician likes using the system is statistically linked to a measure of how safely the system actually functions.

Local configuration, training investment, and template personalization, none of it vendor choice, is the swing factor across satisfaction, burnout, and a validated safety measure.

Why the knowledge disappears instead of accumulating

Here is where the story turns from "local configuration matters" into an actual missing-infrastructure problem, because none of this would be a crisis if the knowledge behind good local configuration simply stayed inside the institution that built it.

It does not. The standard lifecycle looks like this: an analyst or a consulting team makes a set of build decisions at go-live, under time pressure, weighing tradeoffs that made sense given the clinical workflows, the specialty mix, and the politics of that specific hospital at that specific moment. Those decisions get documented, if they are documented at all, as a closed ticket in a change-management system: what was changed, rarely why, and almost never what alternative was rejected and why it lost. Then the analyst leaves, for a promotion, a relocation, or simply because informatics analysts and CMIOs move between health systems at a normal professional rate. The next person to touch that build inherits an artifact with no attached reasoning.

Vendor community resources do not solve this, because they were never built to. Epic's UserWeb and Community Library let customers share build artifacts, an order set here, a flowsheet there, but they share the object, not the provenance. They tell you what another hospital built. They do not tell you who built it, what they tried first and abandoned, or what happened clinically after it shipped. And they are platform-exclusive: an Oracle Health shop gets nothing from Epic's library, and vice versa, regardless of how similar the underlying clinical problem is.

KLAS's Arch Collaborative measures the outcome at scale and does not connect it to the person who could fix it. It can tell a hospital, with real statistical confidence, that its physician NEES score is below where it should be. It has no mechanism for connecting that low-scoring hospital to the specific analyst at a high-scoring peer institution who solved the equivalent local build problem years earlier and would, if asked, explain exactly how.

The result is a field where the same expensive mistake, and the same expensive fix, gets made and remade at different hospitals running identical software, because the only channel that could carry the fix across institutions has never been built.

The structural failure: everyone measures a different piece

This is a coordination failure with an unusually clean anatomy, because each of the plausible builders has a clear, specific reason not to solve it.

Epic and Oracle Health monetize the platform and their certified consulting partners. Cross-institution analyst reachability is not a product either vendor has a reason to build; if anything, an easier path to another hospital's build knowledge slightly undercuts the value of paying the vendor's own consulting arm to do the same reverse-engineering work.

KLAS measures outcomes at the institution level and is funded by the vendor ecosystem it studies. Its Arch Collaborative dataset is the closest thing that exists to a map of where the problem is worst. It was not designed, and its funding model is not structured, to add individual-level, cross-institution reachability on top of that map.

Optimization consultancies are paid specifically to perform the reverse-engineering work this article describes, hospital by hospital, engagement by engagement. A consultancy that solved the underlying provenance problem, so hospitals never needed to pay for archaeology again, would be solving itself out of a recurring revenue line. There is no commercial incentive anywhere in that business model to fix the root cause.

Individual analysts and CMIOs hold the actual knowledge and have no professional infrastructure, and often no professional norm, for staying reachable to the person who inherits their old build after they leave. Their expertise simply exits with them, unless a personal relationship happens to survive the job change.

Every party closest to the problem is positioned to profit from, or is structurally indifferent to, the fact that the knowledge does not travel.

Why this is getting worse, not better

Two forces are compounding the cost of this gap right now.

The generation of analysts who did the original Epic and Cerner go-lives during the meaningful-use wave of the early 2010s is now turning over. Builds that are a decade old, already thinly documented at the point of creation, are losing the last people who remember the original reasoning at exactly the moment those builds are being asked to absorb new complexity.

Ambient AI and in-basket automation tools are being layered onto those same decade-old builds, by teams who frequently do not fully understand the configuration choices underneath the layer they are adding. A new tool built on top of an undocumented foundation inherits every unexplained quirk in that foundation, and adds a new one nobody will be able to explain in five years either.

The optimization budgets available to fix any of this are also under new pressure, which means the reverse-engineering time this article describes, expensive, billed at consulting rates, repeated at every hospital independently, is happening at precisely the moment hospitals can least afford to keep paying for it.

What would actually work

A build-provenance record attached to the build element itself, not buried in a closed ticket. Every order set, flowsheet, or workflow should carry a persistent, findable record of who built it, what alternative was considered and rejected, and what happened after it shipped, not just a timestamp and a name in a system nobody searches for context.

A callable author, not just a credited one. The value of provenance collapses if the named author cannot actually be reached. A "who built this and how do I ask them" primitive, with the author opted in and reachable regardless of which institution currently employs them, is the actual product this gap requires.

Cross-referencing outcome data to authorship, the thing KLAS's model cannot do on its own. The Arch Collaborative already identifies which institutions score well and badly. Pairing that outcome data with documented, reachable authorship at the high-scoring institutions is the missing link between "we know where the problem is" and "we know who already solved it."

Rationale shared, not licensed build content. Any credible product here has to respect that literal build exports are frequently vendor-licensed intellectual property under contract restrictions. What is shareable, and valuable, is the reasoning: what was tried, what was rejected, and why, not the underlying configuration file itself.

Independence from vendor and from KLAS's funding structure. The credibility of any cross-institution provenance resource depends on it not being funded by the same vendors whose products it is implicitly evaluating, which is exactly the constraint the current KLAS model, however valuable, operates under.

Coverage organized by module and specialty, not by hospital count alone. Build knowledge is EHR-version and module specific. A resource that reaches a meaningful share of active analysts and CMIOs across the common Epic and Oracle Health module and specialty combinations is far more useful than one spread thin across every possible configuration.

A norm, not just a tool, that documenting rejected alternatives is part of the job. Even the best product cannot manufacture rationale that was never written down in the first place. Institutions that start requiring analysts to record what they rejected, and why, at the moment of the decision, are building the raw material this entire fix depends on.

What you can do now

If you are a clinical informaticist or EHR analyst

Before you change a build you inherited, try to find the person who made it. A name attached to a change-management ticket is a real lead. AMDIS and CHIME informal networks, LinkedIn, and simple professional outreach solve this more often than most people expect, and a five-minute conversation can save two weeks of reverse-engineering.

Document what you rejected, not just what you shipped. The single highest-value thing you can leave behind for whoever inherits your build is a record of the alternative you considered and why it lost, because that is exactly the information every community library and ticketing system currently fails to capture.

Stay reachable after you leave. The analyst who built the sepsis order set at your old hospital is the single most efficient source of truth for whoever inherits it. A norm of staying findable, even loosely, for former colleagues is a low-cost professional courtesy with an outsized payoff.

If you lead a CMIO office or IT department

Pull your own Arch Collaborative NEES score and ask specifically how much of the gap is training versus configuration. The Longhurst decomposition (19.8 percent software, the rest individual and local) means most of the fixable gap is inside your own control, not the vendor's.

Invest in local training hours before you invest in another optimization consulting engagement. The 27.5-versus-6 point gap between 10-16 hours of local training and under 4 hours is one of the largest, cheapest levers identified in the entire Arch Collaborative dataset.

Require rationale documentation as part of your build change process, starting now. You cannot recover the reasoning behind a decade-old order set. You can guarantee that the decisions your team makes this year do not become next year's archaeology project.

If you are a hospital or system executive

Treat EHR dissatisfaction as a local-configuration and training problem before a vendor problem. Fifteen years of switching or threatening to switch vendors has not resolved a variance that the peer-reviewed evidence attributes mostly to what happens after the vendor is chosen.

Ask how much your organization has spent on optimization consulting to reverse-engineer builds nobody currently understands, and treat that number as a symptom, not a fixed cost. It is direct evidence of exactly the knowledge-loss problem this article describes.

Frequently asked questions

Why do hospitals with the same EHR have such different clinician satisfaction? Because the software itself explains only about 19.8 percent of satisfaction variance; individual clinician factors (50.6 percent), organizational and local-configuration factors (15.1 percent), and specialty (14.4 percent) account for the rest, according to a 2019 analysis of the KLAS Arch Collaborative dataset by Longhurst and colleagues in Applied Clinical Informatics.

What is a KLAS Arch Collaborative score? A Net EHR Experience Score (NEES) derived from surveys of 700,000-plus clinicians across 300-plus organizations. The average physician score is 23.4, well below the 60-point Pinnacle Award threshold; the average nurse score is 47.3, below a 75-point threshold, according to KLAS Research's 2026 published data.

How much does local EHR configuration affect physician burnout? Substantially. A 2021 JAMIA study by Eschenroeder and colleagues found physicians whose organization excelled at implementing, training on, and supporting the EHR were about twice as likely to report lower burnout (adjusted odds ratio 2.14), in a population with burnout rates ranging 22 to 34 percent by specialty.

Who builds hospital EHR order sets, and does it matter who they are? Local clinical informatics analysts and CMIO-office staff typically build and configure order sets and workflows during and after go-live. It matters significantly because template personalization by these analysts was associated with a 29.3-point higher clinician satisfaction score than non-personalized builds, per the Longhurst et al. 2019 analysis.

What happens when an EHR analyst leaves a hospital? The rationale behind the build decisions they made typically leaves with them, since vendor community resources (like Epic's UserWeb and Community Library) share build artifacts but not the reasoning behind them, and internal ticketing systems rarely document why an alternative was rejected. Successor analysts then spend significant time reverse-engineering intent before making further changes, an informally recognized practice sometimes billed by consultancies as build reconstruction work.

Does EHR local configuration affect patient safety, not just satisfaction? There is a measured association. A 2023 JAMA Network Open study by Classen, Longhurst and colleagues, covering 112 hospitals and 5,689 physician surveys, found Leapfrog EHR/CPOE safety scores correlated with Arch Collaborative user-experience scores (adjusted beta 0.011, P<.001), suggesting the same local build and training quality that drives satisfaction is statistically linked to a validated measure of medication-order safety.

The bottom line

The informaticist staring at that sepsis order set is not dealing with a bad vendor choice. Her hospital and her previous one are running the same platform, and the Longhurst decomposition says the software itself explains barely a fifth of why the experience differs. The other four-fifths is a person: an analyst, a training budget, a template someone did or did not bother to personalize, a rationale that either got written down or didn't.

That person is gone. The reasoning is gone with him, because nowhere in the fifteen years American hospitals have spent obsessing over EHR dissatisfaction did anyone build a way for build rationale to survive a job change. Epic's and Oracle Health's community libraries share the artifact and not the provenance. KLAS measures the outcome and cannot reach the person who could fix it. Consultancies are paid, engagement by engagement, to redo the reverse-engineering rather than to solve the problem that makes it necessary.

So she spends two weeks rebuilding a decision someone already made well, at a hospital eighteen months and one state away, where an analyst who has moved on remembers exactly why he built it that way and has never once been asked.


Part of a series on the missing professional infrastructure of healthcare. Previously: The Unmatched Year

Evidence note: sources include KLAS Research's Arch Collaborative published data (2026, 300-plus organizations, 700,000-plus clinician surveys); Longhurst CA et al., "Local Investment in Training Drives Electronic Health Record User Satisfaction," Applied Clinical Informatics 10(2):331-335, 2019; Eschenroeder HC et al., JAMIA, 2021; and Classen DC, Longhurst CA et al., JAMA Network Open, 2023, examining 112 hospitals and 5,689 physician surveys. The variance-decomposition figures (19.8% software, 50.6% individual, 15.1% organizational, 14.4% specialty) come from a single peer-reviewed analysis of the Arch Collaborative dataset and have not been independently replicated by a separate research group in this pass. KLAS Arch Collaborative data is collected and published by a vendor-adjacent research firm whose funding model includes participation fees from the health systems and vendors it studies; this article treats its outcome figures as credible but notes the potential for selection bias among participating organizations. Claims about analyst turnover rates, the specific mechanics of Epic UserWeb and Community Library access scope, and the prevalence of "build archaeology" as a named consulting engagement type are drawn from the underlying research dossier's synthesis and were not independently re-verified against primary vendor or workforce-survey sources in this pass.