This guide outlines a structured approach for hospital CFOs and IT decision-makers to select health IT solutions in 2025. It covers a step-by-step procurement process, best practices for RFPs and demos, scoring and risk assessment, and detailed evaluation criteria for major system categories (EHR, RCM, Telehealth, etc.).
The guide emphasizes 2025 trends (cloud, AI/ML, value-based care, cybersecurity) and US standards (HIPAA, HL7/FHIR, HITECH, HITRUST) while noting global applicability.
Guide to Evaluating and Shortlisting Health IT Vendors
Identify Organizational Needs
Form a Cross-Functional Team: Assemble stakeholders from clinical, IT, finance, and administration to guide requirements gathering. This ensures all perspectives (e.g. physicians, nurses, billing staff, CIO) inform the process.
Assess Current State: Document existing systems and workflows. Map data flows, integrations, and cost drivers (software, hardware, support) to establish a baseline total cost of ownership. Understanding “what we have now” clarifies where new solutions can add value.
Define Strategic Objectives: Align technology needs with organizational goals (e.g., improving patient outcomes, supporting value-based care, complying with new regulations). Prioritize needs such as enhancing telehealth, patient engagement, analytics, or cyber-resilience.
Perform Gap Analysis: Identify functionality gaps in current systems. Ask: What workflows are missing? Which data remains siloed? What clinical or operational problems need solutions? This list of “must-have” features will drive the RFP and vendor selection.
Prioritize Requirements: Rank needs by impact (clinical quality, revenue, patient satisfaction) and feasibility. Use a scoring matrix or weighted criteria (see Section 4). AHRQ and HIMSS recommend assigning weights to each criterion based on its importance.
Envision Future State: Consider emerging trends (cloud deployment, AI/ML, remote monitoring, interoperability via FHIR) and how they support long-term strategy. For example, value-based care demands robust analytics and population health tools.
Prepare Effective RFPs
Set Clear Objectives: Before writing the RFP, ensure everyone understands why the new system is needed. Clarity on goals (e.g., “automate coding to reduce denials”, “enable virtual visits across specialties”) keeps proposals focused and comparable.
Involve Stakeholders: Engage all relevant departments in drafting the RFP. Clinical, IT, revenue cycle, and compliance teams should review requirements to capture diverse needs and ensure no critical feature is omitted.
Outline Requirements: Include sections on: background (organization profile, mission); project scope (functional requirements, interfaces, user roles); data standards (e.g., HL7, FHIR, DICOM); technical requirements (cloud or on-premise, mobile access); and regulatory compliance (HIPAA/HITECH, local laws).
Specify Evaluation Criteria: Publish the criteria and weights within the RFP. Common categories include functionality, interoperability, security, support, cost, and vendor track record. For healthcare, explicitly require certification (e.g., ONC-certified EHR, HITRUST, or SOC 2 for security).
Define Submission Requirements: Clarify format and content of responses (e.g. answer-by-section, budget breakdown, implementation plan, reference list). Standardize templates or forms to streamline comparisons.
Allow Adequate Timeline: Set realistic deadlines for proposal submission and evaluation. Vendors need time to tailor responses; too-short windows limit detail and increase the risk of omissions.
Ensure Transparency and Fairness: RFPs should create a level playing field. Provide the same information to all bidders, allow Q&A, and consider using RFP management software to track documents and communications.
Conduct Vendor Demos and Reference Checks
Prepare Realistic Scenarios: For the shortlist of vendors, define concrete “day-in-the-life” use-cases (e.g. admitting a patient, billing a complex claim, a telehealth visit) that reflect your workflows. Ask each vendor to demonstrate those scenarios.
Use Structured Scoring: Develop a scoring sheet for demo evaluations. Score each vendor on key criteria (usability, performance, configurability, etc.) during the demo to allow objective comparison.
Engage End-Users: Include end users (clinicians, billing staff, patients) in demos to gauge usability and fit. Multiple stakeholders help uncover issues that a purely technical review might miss.
Evaluate Solution Fit: During demos, note whether the vendor’s solution meets the previously defined needs. Check how well it addresses identified gaps (e.g., does the patient portal send reminders? Does the RCM module handle specific payer rules?).
Update Total Cost Analysis: As demos proceed, refine cost models (subscription vs. license, cloud fees, implementation services). BDO recommends a 10-year TCO analysis, including staffing costs and potential savings from retiring legacy systems.
Conduct Reference Checks: After demos, contact each vendor’s client references – ideally with similar size, specialty, or workflow. Ask about real-world performance: vendor responsiveness, ease of implementation, post-live support, frequency of updates, and any unexpected issues. This verifies claims made in demos and reveals vendor reputation.
Site Visits (Optional): Where feasible, visit reference sites to observe the system in action and talk to users on-site. These visits can highlight integration nuances and user satisfaction that phone calls may miss.
Score and Rank Vendors
A disciplined scoring matrix ensures transparent, objective decisions. Develop a decision matrix with criteria down the rows and finalist vendors across the columns.
For example, rate each vendor 1–5 on categories like cost, functionality, support, security, and interoperability. Use a relative weight for each category based on strategic importance. Score independently, then reconcile differences in a review meeting. Key tips:
- Assign higher weights to mission-critical factors (e.g. regulatory compliance, data security). The Agency for Healthcare Research and Quality (AHRQ) advises weighting criteria by their importance to your organization.
- Include representatives from IT, clinical, finance, and other groups to capture all viewpoints.
- Multiply each score by its weight and sum to get a total for each vendor. This quantifies the best fit.
- Use the total scores to rank vendors. Consider both the raw score and qualitative insights. The top 2–3 vendors become finalists for contract negotiation.
- Blank out vendor names during initial scoring (“Vendor 1/2/3”) to prevent preconceptions from influencing scores.
Risk Assessment and Compliance Evaluation
Regulatory Compliance: Verify that each solution meets all applicable standards. In the US, ensure HIPAA/HITECH compliance for data privacy/security.
For international organizations, consider GDPR or other local laws. Check whether the vendor has HIPAA training, a privacy policy, and an incident response plan.
Security Certifications: Favor vendors with recognized security certifications (HITRUST CSF, SOC 2 Type II, ISO/IEC 27001) as evidence of robust cybersecurity practices. Confirm their encryption, access controls, and audit logging meet industry best practices.
Vendor Due Diligence: Perform a high-level due diligence on each finalist – review company background, financial stability, and liability insurance. Thoropass recommends a risk-based approach: for high-impact vendors, gather financial statements and check for any litigation or sanctions.
Data Governance: Ensure clear contract terms on data ownership, data migration on exit, uptime guarantees, and service-level agreements (SLAs). Evaluate the vendor’s disaster recovery and business continuity plans.
Third-party Risk: If the solution involves sub-contractors (e.g., cloud hosting, analytics partners), confirm their compliance as well. Ask about their security controls and whether the main vendor monitors them.
Risk Scoring: Incorporate vendor risk factors into your decision matrix or a separate risk log. Consider security posture, regulatory risk (e.g., liability for a data breach), and patient-safety risk (for clinical systems). Prioritize vendors with lower overall risk.
Shortlist Presentation
Organize your findings in a concise shortlist report or slide deck for decision-makers. A sample format might include a comparison table like:
| Vendor | Key Features / Strengths | Pricing Model | Compliance/Standards | Weighted Score | Comments |
|---|---|---|---|---|---|
| Vendor A | Comprehensive EHR/PM suite; strong interoperability (FHIR API) | Subscription per provider, minimal upfront fees | ONC-certified EHR; HITRUST, HIPAA | 82 | Top score; best clinical fit |
| Vendor B | Cloud-native RCM with AI claims scrubbing; multi-language patient portal | Percentage of collections + tiered modules | HIPAA, GDPR for EU clients | 75 | Good ROI; lower adoption risk |
| Vendor C | Telehealth & RPM focus, well-rated patient engagement tools | Per-seat licensing, extra for add-ons | HIPAA; ISO 27001 | 68 | High telehealth capability, but missing some hospital modules |
In the table above, “Weighted Score” is derived from the scoring matrix. Include notes on unique pros/cons (e.g., “Vendor A has the best integration but the highest cost”). This concise format lets executives quickly see how finalists compare on features, cost, and risk.