Methods
The methodology proposed for the achievement of the set objectives is an adaptation of the Delphi qualitative technique (modified Delphi), widely used to assess in a structured and systematic way the opinion on a specific topic. The main purpose of the Delphi technique is to generate a reliable consensus opinion from a group of experts through an iterative questionnaire process with controlled feedback.
For the construction of the consensus process, the workflow is summarized in the following diagram:
Two types of statements will be assessed:
- a) Warning signs that should be included in the algorithm – Likert scale from 1 (total disagreement) to 9 (total agreement).
- b) Specific weight that each WS should have in the algorithm (scale from 10 to 80).
For the warning signs, the statistical analysis will consider: (i) The median of the scores for each item. (ii) The interpretation of the “consensus” levels of the items. Thus, an item is considered consensual when there is “agreement” of opinion in the panel, i.e., when the experts scoring outside the 3-point region ([1-3], [4-6], [7-9]) contained in the median are less than one third of the respondents. In such a case, the value of the median will determine the group consensus reached: majority “disagree” with the item (if the median is ≤ 3) or majority “agree” with the item (if the median is ≥ 7). Cases where the median is in the 4-6 region will be considered “doubtful” items for a representative majority of the group.
The remaining items, where agreement is not reached, will be considered as having an “indeterminate” level of consensus, and are those which will go into the second round. The items in which there is a high dispersion of opinions among respondents are also subject to re-evaluation for the panel, with an interquartile range ≥ 4 points (range of scores contained between the p25 and p75 values of the distribution).
For the weight of the WS, the statistical analysis will consider mean and median.
Questionnaire
Experts included in the DELPHI can answer pediatric or adult or both questionnaires. It is compulsory that the expert respond only to the same questionnaire that you completed in each round, or to both if applicable.
The role of an expert is to assign weights for all the statements to achieve consensus. Please, consider the following:
- For the DELPHI method, a consensus is reached when the majority of votes fall between 7 and 9 on the Likert scale. For signs that you believe should be considered during medical history taking and contribute to diagnostic suspicion but are not specific to PI/IEI, we recommend assigning them a lower weighting in terms of diagnostic importance, while still giving them a high score on the Likert scale as a warning sign.
- The weighting of each warning sign, derived from its corresponding ICD code/codes (present or absent), aims to assign lower impact to less specific codes and higher scores to more specific ones, thereby reflecting their diagnostic relevance. 75 or more gets a direct immunology referral while less point will need the presence of other warning signs to add up to 75.
We want to remind all the participants that this method has some limitations in the current model:
- Only some diagnosis can be excluded but ICD codes are limited, for example HIV or cystic fibrosis were excluded here but not MRSA or others.
- Timeframe is hard to codify well and the number of episode registered in the EHR rather than the timeframe was taken into account.
- The weights are not assigned per episode, but points are assigned to the individual who meets the criteria. For example, for warning sign “3 or more pneumonias”, if the weight is 80 and individual A has had 3 pneumonias they have the same score as individual B with 8 pneumonias: 80 points. In contrast, individual C having 2 pneumonia gets 0 points.
As a reference, we provide the weightings from the Catalan Delphi here:
https://link.springer.com/article/10.1007/s10875-024-01825-3
Each warning signs is matched with ICD-10 codes. The list of ICD-10 is available here:
Please answer the different questions based on your professional or personal experience and, for the participation to be valid, you must answer all of them.
Your participation and thoughtful input are greatly appreciated.