Some Observations on the more important points made by

Miss Ellen Goudsmit PhD in her “formal response” of 10th May 2005

to concerns about the ‘London’ criteria

 

Margaret Williams

 

11th May 2005

 

  

Background

 

 

The key matter here at issue remains: why is the Medical Research Council (MRC) intending to use the so-called “London” criteria for secondary analysis in the PACE trials on “CFS/ME” when those criteria have not been defined or published and when even the authors have not been established?  How does this accord with the MRC’s frequent claim of its requirement for (quote) “high scientific standard” as contained in its press release and in its pro-forma replies to the ME community (as, for example, in the letter of 15th April 2005 from Simon Burden of the MRC to Neil Brown)?

 

The selection of participants for the PACE trials is to be via the Oxford 1991 criteria which deliberately include those with psychiatric disorders: however, by definition, those with a neurological disorder are excluded from the Oxford criteria so one would assume that those with myalgic encephalomyelitis (ME) would, by definition, be automatically excluded from entry into the PACE trials.  If there were to be strict adherence to the entry criteria, this would indeed be so.

 

It seems that there is not to be strict adherence to the Oxford criteria:  Miss Goudsmit has stated --- correctly --- that the psychiatrists involved with the PACE trials do not accept that ME is a neurological disorder and on this basis those with ME will be included in the PACE trials (presumably under the umbrella of “chronic fatigue).

 

Miss Goudsmit maintains that because those with ME will be included in the PACE trials, it is necessary for the MRC to use the “London” criteria for secondary analysis to identify those with ME; that the MRC will indeed use the “London” criteria has been publicly confirmed by Chris Clark, CEO of AfME.

 

Is it not the case that if the MRC is serious about using the “London” criteria in the PACE trials, it will first have to issue and publish a pro-forma assessment sheet of the “London” criteria that has been standardised for its researchers to use, and all groups of PACE trial researchers will have to use the same criteria, and the criteria used in the trials will have to be available and referenced?

 

Whilst desirous of an end to what might be deemed unseemly squabbling in public, this is a matter of such grave concern that it seems necessary to address some of the matters provided by Miss Goudsmit in her “formal response”, because (even though she herself has removed it from various internet sites) it remains in the public domain.  For her claims to be valid, they must be seen to stand up to scrutiny. 

 

 

Observations

 

Miss Goudsmit refers to the “London” criteria as “LC” and her “formal response” is in capital letters.

 

“(The LC) HAVE BEEN PUBLISHED ON THE INTERNET….THEY HAVE NOT BEEN PUBLISHED IN A PEER-REVIEWED JOURNAL BUT NEITHER WERE THE OTHERS”: it is the case that by convention, “publication” means publication in a scientific journal.  It is incorrect for Miss Goudsmit to state that “the others” were not published in a peer-reviewed journal: both the Holmes et al 1988 criteria and the Fukuda 1994 criteria were published in the Annals of Internal Medicine, which is a peer-reviewed journal, with the Editors selecting reviewers from a database of about 13,000 reviewers. It was only the Oxford 1991 criteria (formulated mostly by psychiatrists) that were published in a non peer-review journal (the Journal of the Royal Society of Medicine).

 

“GIVEN THE LC WERE FROMULATED (sic) FOR IN-HOUSE USE AS OPPOSED TO GENERAL USE, THERE WAS NO NEED FOR THEM TO BE PUBLISHED IN A JOURNAL AT THAT TIME”: “in-house use” means that the material has not been published and is not available for general use; this being so, other researchers would not use them because  a) they would not know anything about “in-house” criteria  and  b) other researchers would not accept findings produced by “in-house” criteria without a published reference to the specific criteria. If the “London” criteria were intended only for limited “in-house” use by Action for ME and not for international application, how would adding to the existing eight sets of criteria a further set of criteria that were non-accessible to the international research community advance understanding and lessen existing confusion?  For Miss Goudsmit to claim that the “London” criteria were intended only for “in-house” use seems not to accord with what the Report of The National Task Force stated about them in September 1994; it states that: “Action for ME, the ME Association and The International Federation of ME Associations” (IFMEA) proposed the “London” criteria.  (IFMEA was created and run by Miss Goudsmit herself as a means of collaboration between international ME associations). If the  “London” criteria were not intended for “GENERAL USE” as here claimed by Miss Goudsmit, why does the National Task Force include the International Federation of ME Associations” as one of the proposers of the “London” criteria?

 

“THEY COULD NOT HAVE BEEN PUBLISHED EVEN IF WE HAD WANTED TO, GIVEN THE BLANKET BAN ON ANYTHING POSITIVE REGARDING ME IN BRITISH JOURNALS, THE ONLY JOURNALS WHO MIGHT HAVE BEEN INTERESTED”:  If the “London” criteria represented an advancement in defining those with ME, why would they have been of interest only to British journals?

 

“SADLY, THE INFORMATION (in the Westcare National Task Force Report) WAS NOT CHECKED WITH ANYONE WHO KNEW THE FACTS AND WAS WRONG”:  Is Miss Goudmsit implying that, because she thought of the concept of the “London” criteria, only she knew the facts and was able to check them?  If so, given the eminence and experience of the members of the Task Force (which included Dr William Weir, then of The Royal Free Hospital and successor in post to Dr Melvin Ramsay), this is an astonishing presumption by Miss Goudsmit.  Neither Dr Anne Macintrye nor Dr Charles Shepherd was a member of the Task Force but the Report acknowledged their assistance, so it is presumed that they approved the information regarding the “London” criteria contained within the Report.

 

(In her response to the claim that “Merely being mentioned in a Report or published document does not equate with criteria being published and used in research”):  “THIS IS TRUE, BUT THEY WERE USED IN RESEARCH”:  unless the “London” criteria can be identified by virtue of being defined, published and referenced, how can they be “used” as a credible basis for research by anyone? 

 

“IF IN 1994 YOU WANTED TO STUDY ME USING RESEARCH CRITERIA, (the LC) WERE THE ONLY ONES AROUND, COMPILED BY SCIENTISTS WITH A DEGREE OF EXPERTISE IN THE SUBJECT”:  Is Miss Goudsmit referring to herself as a “scientist with a degree of expertise in the subject”?  The reality is that Miss Goudsmit is not now and was not then in formal employment in the field of ME and she has no professional clinical expertise in ME: she is a patient who has degrees in psychology and she appears never to have held a salaried post in the field in which she claims to be an expert.  On her own admission, she states that she has worked in clinics in a voluntary capacity.   Although qualified medical practitioners, neither Dr Macintyre nor Dr Shepherd holds an NHS consultant post: Dr Macintyre worked for a time as an eye surgeon in a third world country before succumbing to ME, since when she has worked intermittently as a GP, and Dr Shepherd sees ME patients privately at his home when not giving talks around the country as Medical Adviser to the ME Association. 

 

“THE CRITERIA WERE AS OPERATIONALISED AS OTHERS AT THE TIME”:   given that Miss Goudsmit herself states that the “London” criteria were designed only for “in-house use”, how can it be true that the “London” criteria were as widely “operationalised” as the Holmes et al (1988) criteria, the Sharpe et al (Oxford 1991) criteria or the CDC (1994) criteria?

 

“IF WILLIAMS BELIEVES (Dr Costa) USED DIFFERENT CRITERIA (from the “London” criteria), CAN SHE PROVIDE A REFERENCE?  A COPY?”:  the reference supplied by Costa et al confirms about the population studied that “All ME/CFS patients were clinically assessed and diagnosed according to standard criteria (Oxford), CDC and ME Action”.  The reference in the paper for the ME Action criteria is number 14.  That reference states:  “Criteria for a diagnosis of ME for use in the ME Action funded research.  Based on the criteria suggested by WRC Weir in Postviral Fatigue Syndrome by Jenkins & Mowbray pp 248-9”.  The Jenkins & Mowbray textbook at pp 248-249 sets out Dr Weir’s own modification of the Holmes et al 1988 criteria and makes no mention of any “London” criteria, nor of the authors of the criteria being Miss Goudsmit, Dr Macintyre and Dr Shepherd.  From this, it is apparent that the “ME Action” criteria used by Costa, Tannock and Brostoff were based on criteria suggested by Dr Weir.  For comparison, in September 2004 Miss Goudsmit stated about the use by Costa et al of the “London” criteria:  “In the Costa paper, the LC were listed as their ref 14.  No full reference because we don’t have major egos and at that time, did not insist people write our names”, whilst in November 2004 Miss Goudsmit stated about the use of the “London” criteria by Costa et al: “Costa did not refer to the criteria as the LC but he did refer to the criteria developed by MEAction and guess what they are?”.  In her “formal response”, however, Miss Goudsmit now states: “IF ONE COMPARES THE LC TO THOSE FROM WEIR, ONE SEES SOME NOTABLE DIFFERENCES. NO-ONE COMPARING THEM WILL BE CONFUSED”.  It must be emphasised that Costa et al maintained that they had used the MEAction criteria, but Costa et al referenced those MEAction criteria to Dr William Weir.

 

“I AM PLEASED THAT MS WILLIAMS HAS NOT REPEATED THE ERROR THAT IT (ie. Miss Goudsmit’s dissertation) RECEIVED THE LOWEST SCORE, AS IT DID NOT”: in the table published in JAMA, 19th September 2001:286:11:1360-1368, Miss Goudmsit’s dissertion was awarded 2 out of a possible 20 points on the validity score; the paper by Perrin et al quoted by Miss Goudsmit as having used the “London” criteria (which does indeed state that participants were selected according to both the CDC and the “London” criteria) scored 0 on the validity score.  In relation to the Perrin et al study, the lead author has confirmed to more than one person that he was unaware that the “London” criteria had never been defined or published; he also confirmed that he had relied entirely upon Drs Macintyre and Shepherd for information about the “London” criteria; that it was Dr Macintyre who selected patients for inclusion in his study from the database of members held by Action for ME, and that it was Dr Macintyre who assured him that the “London” criteria had been published and validated and that the reference for this was Costa et al (QJM 1995) as asserted by Miss Goudsmit.

 

“PROF SCHOLEY WOULD HAVE TOLD ANYONE WHO ENQUIRED THAT HE IS HOPING TO SUBMIT THIS STUDY FOR PUBLICATION.  BUT THE ISSUE I WAS ASKED TO ADDRESS WAS NOT, HAS THAT STUDY BEEN PUBLISHED, BUT DID IT USE THE LC.  IT DID!”:  the question remains: how can anyone credibly use criteria that have not even been defined and, since they remain unpublished, are not available for comparison?  If a researcher wanted to publish the results, s/he would have no valid published reference point on which to base the selection of patients for the study.

 

“ALL PATIENTS FULFILLED THE LC”:  what are the LC?  Where (apart from the internet) can they be found?  Who are the authors?  Where have they been published and made publicly available for researchers’ use?  Where are the comparative results of studies that used them?  What is their validity?

 

“ONE HAS TO FULFIL A NUMBER OF CRITERIA TO BE CITED AS AUTHOR.  YOU HAVE TO MAKE A SUBSTANTIAL CONTRIBUTION.  I DID NOT ACTUALLY WRITE A WORD OF THE LC BUT I WAS SUFFICIENTLY INVOLVED TO MERIT INCLUSION AS AUTHOR. THE MRC WILL KNOW THE RULES.”:  this should be compared with what Miss Goudsmit states as being “ACCURATE” four paragraphs later:  “The London criteria are research criteria.  I know this as I was one of the co-authors”.  Since the “London” criteria have never been published, they have no authors as far as the outer world is concerned.

 

“I DID NOT SEE VERSION 2 (of the “London” criteria) AND DO NOT KNOW WHO WROTE THEM”:  who did write “version 2” of the “London” criteria?   What are the stipulated criteria to which researchers must conform?

 

“PROF. WHITE IS AWARE OF THIS.  AND IF HE USES VERSION 2, I WILL NOT DEFEND HIM”:  on what grounds is it Miss Goudmit’s responsibility to “defend” Professor Peter White (lead researcher in the MRC PACE trial)?  If the “London” criteria are to be used, the MRC and Professor Peter White should now issue a statement clarifying which version of the “London” criteria they intend to use, and why, and should make available the “London” criteria assessment sheet that will become the standard instrument for use by all researchers for the PACE and FINE trials.

 

“IF MS WILLIAMS’ LOGIC IS EXTENDED TO MERGE, ALL THEIR RESEARCH IS EQUALLY LACKING IN LEGITIMACY.  IT IS BECAUSE WE KNOW THEY USE ADDITIONAL CRITERIA, UNPUBLISHED, NOT PEER REVIEWED, THAT WE KNOW THEY HAVE STUDIED ME”:  for the avoidance of doubt, Dr Neil Abbot, Director of Research at MERGE, has provided the following clarification:  “All patients were assessed by a medically qualified researcher to confirm that they fulfilled the CDC 1994 criteria.  Those that did not were excluded. Additional measurements, for example, basic nerve function tests, were also performed for subsequent comparisons but study entry was on the basis of fulfilling the CDC 1994 criteria alone”.

 

Since Miss Goudsmit has provided no evidence that the “London” criteria have been defined or published, and since they are not on PubMed, it remains to be established how the MRC can justifiably use them in the PACE trials.  If, as she claims, the “London” criteria were just for AfME’s in-house use, how can they legitimately be used for official MRC research?

 

As things stand, the MRC trials would seem to be open to potential abuse and the results to misinterpretation, but the two major adult ME charities seem to be disinterested and even oblivious to this possibility.