Case Study: Experian and AI Team Solve Peak Server Problems With Existing Software

Download As PDF

Experian Health Revenue cycle management is based in Franklin, Tennessee and serves over 13,000 healthcare organizations, including nearly 60% of all US hospitals.

As a subsidiary of Experian plc. a company most known for its credit reporting services, the Experian Health subsidiary provides revenue cycle management, identity management, patient management, and care management solutions designed to help providers make better-informed decisions while limiting risk.

experian

The ChalLENGE

Experian Health processes the prequalification of medical benefits for health providers. For example, when a physician concludes that surgery is needed, the patient’s records are first sent to the insurance company to determine coverage. Experian Health provides a clearing house for those “prequal” transactions.

Experian Health's system was hitting its capacity every month when organizations would run their prequals to plan the coming month. The prequal results were needed in order to schedule medical procedures and associated resources such as doctors, nurses, operating rooms, etc.

Experian Health’s 316-server solution was Enterprise Service Busbased (NServiceBus), but the interface with payment processors was single-threaded — each transaction handled one at a time. The system used RavenDB to house the underlying “saga data” needed to track the state of each transaction as it was processed by the system.

Before AI was called in, Experian Health’s high water mark was six million transactions per day, and usage was increasing rapidly. The underlying payment processors were also taking transactions from services other than Experian’s, and some of them were unable to keep up.

During the monthly peak, the system would slow to a crawl and crash. Though its system was already over capacity, the company was adding clients at a rapid pace and usage was going up. Experian Health’s CTO realized they were at a turning point and brought in The AI Team to help.

Results at a glance

In just four weeks, AI Team….

  • Eliminated service interruptions during monthly peak periods
  • Leveraged existing software and technology
  • Created a smooth transition with minimal training or post-deployment support

 

THE SOLUTION

During their initial assessment, the AI Team looked at the rate that requests were going into each of the payment processors and took measurements from their queuing systems. From this, it was easy to see which processors were working well and which were causing problems. It allowed the team to identify the 25 payment processors that were most likely causing performance issues. Experian’s system was then re-architected so that rather than one queue feeding all payment processors, each of the top 25 processors now has their own queue. This prevents any one backlogged processor from bottling up the entire system.

experian-before[1]
THE SOLUTION

AI also looked at the “saga data” being tracked for each transaction. As it turned out, there was much more data being tracked per transaction than necessary, greatly increasing the burden on the underlying database. AI’s recommended solution was to eliminate unnecessary saga data and upgrade the database used to track it. They switched from RavenDB to Microsoft SQL Server and NHibernate.

The database changes alone got Experian Health over the six million transactions-per-day hump. The queuing changes have the company well-prepared as continuing increases in business demand higher levels of scalability.

SOLUTION DETAILS

  • Decreased the amount of data tracked for each transaction
  • Migrated from RavenDB to Microsoft SQL Server and NHibernate
  • Re-architected NServiceBus configuration for maximum scalability, enabling requests to be fed to multiple payment processors in parallel

RESULTS

Before the project, Experian Health’s prequalification processing system would become bogged down and crash during monthly peak processing, and new clients were being added at a rapid pace.

The AI Team worked together with Experian for four weeks to modify the system, which now easily handles peak monthly workloads. Plus, it is architected to scale well as the business continues to grow.

The new system uses much of the same technology stack as it did previously, including the NServiceBus ESB. Much of the processing software was preserved as well, and massive migration and retraining was necessary. When AI left, Experian was well-equipped to maintain the system with their existing personnel and has not made a "911" call to AI for support since the project’s end.