The Animal Health Committee maintains the policy and procedure for the development and evaluation of new laboratory tests for animal health laboratories in Australia.
Procedure to develop and evaluate new veterinary laboratory tests
The Subcommittee on Animal Health Laboratory Standards (SCAHLS) is a subcommittee of the Animal Health Committee (AHC). SCAHLS with administrative support from the Department of Agriculture, Fisheries and Forestry, seeks to sustain and improve the quality of livestock and livestock products and to assure market access through the application of best practice to veterinary laboratory services.
Guidelines have been developed to assist laboratory scientists in the development and approval of new diagnostic tests through to their inclusion in the Australian and New Zealand standard diagnostic procedures (ANZSDPs).
The first step in this process is the development of the test to the point where it is eligible for submission to SCAHLS. If approved, the test is expected to be included in the relevant ANZSDP at its next revision unless relevant information and development in the transition period suggest otherwise.
The purpose of this procedure is to detail the process and responsibilities for managing new test development and evaluation.
This procedure applies to the development, evaluation and approval of new laboratory tests for diseases of production, feral, captive and companion animals.
4.1 New test development records
4.1.1 The Subcommittee on Animal Health Laboratory Standards (SCAHLS) will consult annually for the purpose of updating records. Each member will receive a list of tests under development and will be required to provide an update to SCAHLS.
4.1.2 The SCAHLS, with administrative support from the Department of Agriculture, Fisheries and Forestry, will maintain and update a spreadsheet with lists of the following:
- new tests being developed
- tests in routine use in diagnostic laboratories (often validated in-house, to at least World Organisation for Animal Health (WOAH) Stage II, but not submitted to SCAHLS for inclusion in ANZSDPs).
4.1.3 SCAHLS, with administrative support from the Department of Agriculture, Fisheries and Forestry, will develop and maintain validation templates for submissions of new tests for evaluation.
4.2 New test evaluation
4.2.1 Assessment by SCAHLS
Evaluation of new tests will be undertaken by SCAHLS. SCAHLS consists of members with relevant expertise in test types and/or validation who will have adequate knowledge of the policy and procedures for new test development and evaluation. All assessors are requested to ensure confidentiality and should declare any conflicts of interest to SCAHLS before receiving dossiers and when conflict of interest arises during the evaluation process.
4.2.2 New test approval process
4.2.2.1 Submitters forward dossiers to SCAHLS in electronic (PDF) format.4.2.2.2 SCAHLS reviews the submitted dossier and other relevant information to determine if it falls under the scope of SCAHLS to review and is in the correct format with sufficient validation data. If not, it will be returned to the submitter with feedback on reason(s) for rejection or corrective action(s) required.
4.2.2.3 SCAHLS will assemble a review panel and nominate a coordinator of the review panel. This panel should consist of at least 2 independent technical assessors who are experts in a relevant field and/or assay validation and may be members of SCAHLS. The coordinator is responsible for ensuring the assessors complete the review within the given timeframe and, if required, identify additional assessor(s) or replacement assessor(s).
4.2.2.4 Written reports from the assessors will be submitted to, and considered by, the SCAHLS. Where reviewers have differing opinions the SCAHLS will make the final decision to recommend, or not, the new test for submission to the AHC for endorsement.
4.2.2.5 The coordinator of the review panel will complete the New Test Evaluation Recommendation Template for consideration by laboratory task group members. The document will be forwarded to members out of session with the following time frames for approval: 10 business days to provide comments or approval and, if required, another 10 business days to confirm approval for the updated recommendations (with members’ comments addressed). There will be a period for the review panel and the submitter to address SCAHLS members’ comments between these 2 consultation rounds subject to the nature and level of issues raised by members.
4.2.2.6 The new test will be approved by SCAHLS through consensus. Approvals may be granted with specific conditions.
4.2.2.7 SCAHLS will communicate with the submitter via the SCAHLS Secretariat about the progress and outcome of the assessment, for example if there are extenuating circumstances which may cause a delay in completion of the assessment. This also includes the provision of a decision and specific conditions if appropriate. If not approved, feedback or recommendations for remedial action will be provided. The assessors will remain anonymous to the submitter in all communications unless prior permission is obtained.
4.2.2.8 Submitters may choose to resubmit the test after inclusion of further information. Where this is done within 6 months, the same review panel should be used for consistency.
4.2.2.9 Approved new tests will be submitted to the AHC for endorsement within 10 business days of approval.
4.2.2.10 Should the AHC provide any variations to an approved test and/or decide not to endorse the test (partly or fully), SCAHLS will seek reasons and/or clarification from the AHC. SCAHLS will circulate AHC feedback to all members, and communicate the outcome of the AHC process to the submitter, subject to directives from the AHC and the laboratory task group members.
4.2.2.11 Once a test is approved by SCAHLS and endorsed by the AHC it will be listed on the Department of Agriculture, Fisheries and Forestry website—see Approved tests.
4.2.2.12 The test protocol will be made available on the department’s website for access by the public.
4.2.3 New test evaluation records
SCAHLS, with administrative support from the Department of Agriculture, Fisheries and Forestry, will maintain a summary list of tests under consideration, approved tests and relevant dates, correspondence and decisions. It will present this information as a paper at each SCAHLS face-to-face meeting.
4.2.4 Expected timeframes (under normal circumstances) for the assessment process following submission of dossier:
- Decision by SCAHLS to proceed with review upon receiving a validation dossier - up to 2 weeks
- Formation of an independent review panel - up to 2 weeks
- Completion of the panel’s review process - up to 2 weeks (or more, subject to additional data/information required to be submitted by the test developer)
- Submission of the panel’s full report, including recommendations, to SCAHLS - up to 2 weeks
- Decision by SCAHLS on the panel’s recommendations - up to 2 weeks
- Total timeframe from receipt of a dossier to the decision by SCAHLS on the panel recommendations - approximately 10 weeks
- If relevant (i.e. dossier favourably assessed), submission of SCAHLS’ recommendation to the AHC for final approval - up to 2 weeks.
4.2.5 Costs
Charges are due at the time of submission to SCAHLS and will be invoiced once the submission is sent for independent review by an assessment panel. Payment for submissions will be used to cover costs of the assessors and the administration service provided by Animal Health Australia.
The costs of a submission may be reviewed by SCAHLS, in consultation with the administrative service provider, as needed. Currently, it is $3,450 for a new submission. Extensions of intended purpose for existing SCAHLS tests might be considered where the minor changes would not impact significantly on the fitness for purpose of the test involved. The cost for review of an extension to an existing assay will be $2,300. Tests submitted as extensions which are not accepted by SCAHLS, may be resubmitted as a new test for review with payment of an additional $1,150. SCAHLS’ decision on the evaluation of submitted dossiers will be final.
Submitters may review and resubmit within 6 months and will incur a cost of $500.
For tests not accepted by SCAHLS, their submitters may review the panel feedback and resubmit their dossiers within 6 months, which will incur a cost of $575.
The above costs include a 15% rate for the administrative service.
For additional information or query please contact the SCAHLS Secretariat at Animalhealthlaboratories@aff.gov.au
These documents supporting this procedure can be found on the department’s website:
Policy to develop and evaluate new veterinary laboratory tests
This policy establishes agreed mechanisms and guidelines for the development, evaluation and communication of new tests developed in Australia and New Zealand for infectious animal diseases, for use in veterinary laboratories. It is consistent with international (OIE) guidelines and is intended to ensure a uniform understanding of the way in which new tests are developed, evaluated and communicated.
This policy applies to new tests developed in Australia and New Zealand for use in production animals as well as feral, captive and companion animals.
The Animal Health Committee laboratory task group, with administrative support from the Department of Agriculture and Water Resources, seeks to sustain and improve the quality of livestock and livestock products and to assure market access through the application of best practice to veterinary laboratory services.
The laboratory task group has a mandate to facilitate the development, evaluation and adoption of new diagnostic methods presented for consideration to Animal Health Committee. New test development is facilitated by:
- maintaining a register of new test methods under development to reduce duplication
- promoting the establishment of collaborative links between institutions.
New test evaluation is facilitated by:
- developing and communicating information on validation requirements for new tests that will be included in the Australian and New Zealand standard diagnostic procedures (ANZSDPs), including validation templates and guidelines for authors
- coordinating independent evaluation of new test methodologies and validation data for tests that are to be included in the ANZSDPs
- providing feedback to submitters following the evaluation process.
Adoption of new diagnostic methods is facilitated by:
- monitoring and communicating new test developments
- providing a transparent and auditable process for evaluating new diagnostic tests developed for the livestock industries and assisting their inclusion, as appropriate, into standard test procedures (e.g. ANZSDP)
- promoting the uptake of new tests by Australian and New Zealand laboratories.
This policy is designed to assist laboratory scientists in the development and evaluation of new diagnostic tests through to their inclusion (where appropriate) in ANZSDPs.
5.1. Examination of fitness for purpose (the reasons why a test is performed)
When developing a new test, consideration of the purpose for which the test will be used must be considered. This particularly applies to test evaluation/validation because it is important to ensure that the test is evaluated using samples from the same population as those that the test will be used on.
The 6 levels described by OIE for ‘Fitness for Purpose’ and which are accepted in Australia and New Zealand are to:
- demonstrate population 'freedom' from infection (prevalence apparently zero) in the following categories:
-
- 'free' with or without vaccination
- historical 'freedom'
- re-establishment of 'freedom' following outbreaks
- demonstrate freedom from infection or agent in individual animals or products for trade purposes
- demonstrate efficiency of eradication policies
- confirm diagnosis of clinical cases:
- in groups of animals, or
- in individuals
- estimate prevalence of infection to facilitate risk analysis (surveys, classification of herd health status, implementation of disease control measures)
- to determine immune status in individual animals or populations (post-vaccination).
5.2. Priorities for test approval process
Consistent with best current practice all tests used in Australian/New Zealand animal health laboratories should be adequately validated. Test developers should seek laboratory task group approval for new tests applicable to:
- national disease control/eradication and surveillance programs
- health certification testing (for livestock destined for international export)
- tests developed for exotic disease and required to be transferred to other laboratories. The above tests (when approved) will be included in ANZSDPs (as a matter of course) and will carry the statement ‘Approved by AHC laboratory task group for the stated purpose’—the stated purpose being any one or a combination of 1–6 above.
5.3. Requirements for test approval
It is recognised that there are 5 stages in the validation process as outlined by the OIE. These include:
- feasibility
- development and standardisation
- assay performance characterisation
- monitoring performance (including transferability—tested in another lab)
- maintenance/extended validation criteria.
Submitters are expected to complete the relevant validation template for serological or nucleic acid detection methods, which detail the continuum of the developmental approach in a standard format. Tests may be submitted as extensions to an existing assay where the minor changes would not be expected to impact significantly on the fitness for purpose of the assay involved. (Tests on different animal species could reasonably be expected to have quite different specificity.) In the case of an extension, submitters must complete the validation template for extension of existing assays. Completed templates should be provided to the laboratory task group.
New tests submitted for approval must have reached at least stage 4 before submission to the laboratory task group. Assuming the submission contains all of the elements of the validation template in sufficient detail and quality, the test may be approved for the stated purpose.
The process for test evaluation and the cost of the assessment by the independent reviewers are outlined within the procedure to develop and evaluate new veterinary laboratory tests. The laboratory task group may approve inclusion of a new test in the appropriate ANZSDP.
New tests that are developed for use in individual laboratories that do not fit with the priorities outlined above, do not require laboratory task group approval, but should nevertheless be validated.
Laboratory managers responsible for the introduction of such new tests should develop a system of independent review prior to implementation, as a routine practice.
In cases where individual reagents are changed in a standard test, managers should ensure that the laboratory verifies that any new critical reagent performs adequately against standard reagents before the new reagent is used in routine diagnostic testing.
It is desirable but not essential for a new test to be published in a peer-reviewed scientific journal prior to submission to the laboratory task group for test evaluation.
Neither the laboratory task group, nor its members, are legally liable for any repercussions for conferring ‘Approval’ for any given test or for not granting ‘Approval’. By submitting a dossier for laboratory task group evaluation the submitter agrees to current procedures and policies.
The evaluation process applies to all tests as nominated in section 5.3. Tests already in common use will be assessed for approval on a priority basis.
Although the laboratory task group has no formal authority to force laboratories to use an ‘Approved’ test, it recommends through its members and associated networks that approved tests are used where possible. NATA Supplementary Requirement for Veterinary Testing states that laboratories ‘should’ use ANZSDPs ‘where appropriate’.
Laboratories are encouraged to collect and store samples that may be useful for validation purposes, especially for uncommon diseases, so that when validation is needed, sufficient samples are available, even if it means aggregation of samples from various laboratory sources.
In the context of non-standard methods, in-house methods and standard methods that have been modified or are being used outside their intended application:
Validation is the demonstration of technical soundness, method performance and limitations of the method and an indication of the suitability of the method for its intended use. Validation studies should include evaluations of uncertainty of measurement (where relevant), detection limit (or sensitivity), selectivity of the method (or specificity), linearity, limit of repeatability and/or reproducibility and robustness against external influences.
In the context of ‘standard’ methods (i.e. kit methods validated by the manufacturer, methods in the ANZSDPs):
Verification is the demonstration of the ability to achieve the performance specifications of the manufacturer/standard method under the testing laboratory’s own conditions.