You may have seen the article in NZ Doctor (Feb 2 issue) headlined “Panel puts Medtech fourth out of four”.

The team at Medtech feels mistreated and maligned by its publication because:

• Medtech did not participate in the evaluation process because the criteria was flawed
• Medtech did not complete a self-assessment and the Review Panel did not have access to technical descriptions of the Medtech32 system, its architecture, its internal workings or its interfaces
• The Review Panel acknowledged the possibility that statements made about Medtech32 are factually incorrect • The report is inaccurate in material respects
• The Review Panel did not follow the Terms of Reference mandated.

The article refers to a report commissioned by the National Health IT Board and produced by Patients First, a joint programme of work between the RNZCGP and General Practice New Zealand. The report referenced by the NZ Doctor article was withdrawn several days prior to NZ Doctor’s publication.

Background

In June of last year, Patients First initiated a process to evaluate the four Practice Management Systems (PMS) in New Zealand (the “PMS Requirements Project”).

The stated outcome of the PMS Requirements Project was to provide “a credible, independent and accurate appraisal of PMS systems for the health sector to help inform purchasing decisions”. The process to provide “a credible, independent and accurate appraisal....” was by means of an evaluation that was to be derived through:

• Vendor self-assessment of their respective PMS product(s) against the criteria; and
• An independent and facilitated process of evaluation by a Review Panel against the criteria”.

The “independent” Review Panel was provided with a Terms of Reference, which among other things, set out the process to be followed for the review of PMS products. This Terms of Reference provided for the Review Panel to review the vendor self-assessments by:

• The vendors providing access to the PMS systems (including technical descriptions, architecture, internal workings and interfaces);
• Inviting vendors to meet with the Panel for interactive discussions around any areas of dissent;
• Enabling vendors to present their case to the Panel, to avoid incorrect conclusions going forward and supporting the view that this is not a process, which is being done to the vendors.

Facts

Medtech did not undertake a self-assessment of its Medtech32 product and nor did Medtech get invited to meet the Review Panel (as provided for in the Review Panel’s terms of reference) to discuss any assumptions the Panel made about Medtech32 in the absence of a self-assessment.

Medtech’s decision not to participate in the evaluation process was carefully considered, and included input from the independent Medtech Advisory Group (MAG). There were a number of reasons for non-participation, which included:

1. Patients First required Medtech to provide them with detailed, commercially sensitive information, much of which was our specific intellectual property. On this basis, we felt that participating was not an option.
2. The evaluation criteria were limited to five predefined areas of focus, mainly around the area of interoperability
3. Given our extensive experience, we did not believe the five evaluation criteria aligned with what our customers were telling us were important to them in choosing a PMS. Our independent Medtech Advisory Group and key Medtech users also validated this.

In short, Medtech had significant concerns about the effectiveness of the proposed process, and accordingly chose not to be involved. The fact that Medtech’s non-involvement has now led to Medtech32 being inaccurately assessed on different criteria to that proposed, and on the basis of incomplete information, is, to say the least, disappointing.

Examples of inaccuracies

Examples of inaccuracies in the assessment of Medtech32 include:

• The report finds that Medtech32 has no banking module. This is just plain wrong. In fact it has a full banking module available within the application.
• It states that searching for Polyp does not return Polyps. Polyp* will return both sets of data (along with other matches such as Polypharmacy and Polyposis). This exemplifies a lack of knowledge by the panel.
• Medtech32 is not able to traverse back-up the Read Code tree. It certainly can.
• Medtech32 could only tabulate, but not graph, screening results. The application can both tabulate and graph screening results. It can even graph results against one another to display trends over time.

Medtech’s position

Given that Medtech did not:

• undertake a self assessment;
• have the same opportunity as other vendors to meet with the Review Panel, and
• provide access to technical descriptions of the Medtech system, its architecture, its internal workings or its interfaces,

the comparison of Medtech’s PMS product with other PMS vendor products, which had the benefit of the mandated evaluation process, is manifestly unfair. Furthermore, as the Review Panel did not have access to critical information and explanations from Medtech, the reports findings, in respect of Medtech32, are substantially inaccurate.

In the circumstances, Medtech believes it is impossible to rank Medtech32 in a table with other PMS systems, which had the benefit of the self-assessment and the opportunity to provide further explanations and information to the Review Panel.

Furthermore, Patients First did not evaluate ManageMyHealth or CAT, which when used in conjunction with Medtech32, provide a compelling set of solutions for today’s market, unmatched by any of our competitors. Nor did Patients First evaluate Medtech Evolution, the soon to be released successor to Medtech32.

Medtech is confident that any fairly conducted assessment of its Medtech32 product would rank the product very positively in relation to its competitors.

In the meantime, we are working with the National Health IT Board to seek to have our concerns acknowledged ahead of the proposed release of the Patients First report on February 10, in the hope that further action is not required.

Yours sincerely

Russell Clarke
Executive Director