Industry Refund - Final Report

Callida was engaged to:

• Assess the accuracy of the calculations performed to date by the Department to establish an estimate of the incorrectly billed charges for each of the affected entities based on available data; and
• Based on that analysis, determine whether the estimate represents a reasonable payment to each affected Industry participant. Department of Agriculture, Water and Environment Industry Refund 4

The initial scope of the review was to analyse the worksheets and any associated documentation provided and determine their accuracy and completeness as to whether an accurate estimate can be made of any potential overcharging based on the extracted baseline data.

Callida was then requested by the Department to extend the scope to address the identified issues and extended the Department’s calculations to provide dollar estimate payments due to affected Industry participants noting that this analysis would not consider the accuracy and completeness of the source data and based on the data available at the time of the review and work undertaken by the Department to date.

Given the volume of data and the compressed timeframe, Callida has had to make several assumptions in undertaking the review and as such, this has resulted in several scope limitations including the following:

The Department advised that a review of invoicing was undertaken in 2012-13 and any issues that have previously been identified in relation to overtime prior to August 2013, including overcharging Industry participants, have been corrected. Callida was provided with industry notices and relevant industry committee papers relating to this review which Callida accepted as having demonstrated action had been taken to address the overcharging and as such no work was undertaken to provide further assurance over errors prior to August 2013.

Callida has relied on the extracted baseline data being the most accurate data the Department can provide. The scope of the review did not include any testing from the extracted baseline data back to source documents such as overtime sheets completed by FSMAs or Vets, nor did we track the hours in the extracted baseline data through to actual Processor invoices generated by the Department. Without this testing back to source documentation or through to final invoices, Callida has had to rely on the accuracy and completeness of the Department’s data as provided.

The review of relevant documentation such as EAs, Fees Orders and Departmental briefings and notes was limited to the extent that they could provide context for the Industry charging.

Callida was briefed on the methodology and assumptions made in arriving at the data/spreadsheets provided by Departmental Officers. Documents supporting those briefings, where available, were provided and considered in the analysis of this engagement.

Any recalculation undertaken by Callida has been undertaken on the extracted baseline data provided by the Department. No further assumptions or additional data have been provided/created by Callida in arriving at the conclusion in this report.

Noting the scope limitations, Callida achieved the objectives by undertaking the following fieldwork activities:

• Reviewed the numerous and detailed spreadsheets provided by the Department which contained both the original shift loading and overtime extracted from various Departmental systems as well as the calculations performed on this extracted baseline data to determine the extent, in hours, of the potential error in charging.
• Reviewed relevant contextual information including EAs, Fees Orders, Departmental briefings and relevant Industry notifications.
• Compared the contents of each of the original worksheets containing source data to the corresponding departmental worksheets containing the revised/corrected data.
• Traced the formula and additional worksheets to confirm the process that was being undertaken.
• Followed through from the extracted baseline data tables to the results/summary worksheets that were provided to ensure that they continued to represent the calculations performed on the extracted baseline data.
• Met with the Departmental Officers who had undertaken the work to extract and format the extracted baseline data and then undertake the calculation of the estimate of the overcharging.
• Formulated several observations/findings which impacted on the Departments ability to arrive at an estimate of what would be a reasonable payment to affected Industry participants.
• Conducted a status meeting with the engagement sponsor and to provide feedback on preliminary findings and recommendations and confirm next steps.
• Undertook remedial work as requested by the engagement sponsor.
• Used the hours estimates for overcharging for both overtime and shift loading calculated by Departmental staff to provide a dollar estimate of what would be reasonable payment to affected Industry participants. Individual dollar estimates for Industry participants have been provided separately to the Department but are not published here to ensure privacy and confidentiality.
• Prepared a draft report for consideration and feedback from the engagement sponsor.
• Prepared a final report.

We note that, despite the difficulty in arriving at the estimates, the Department has clearly undertaken significant work to provide an estimate of what would be a reasonable payment to affected Industry participants impacted by the overcharging. Our review identified the use of reasonable assumptions, and the application of clear logic and transparent and traceable processes to arrive at an estimate for each affected Industry participant.

The accuracy and completeness of the source data is limited due to the number of years of historical billing and time recording information that needs to be checked, and the lack of a single system with all of this information in it. To assist in mitigating this risk, consideration may be given to seeking assurance over the accuracy of the source data and/or providing Industry participants with the opportunity to submit additional and more robust data to support their calculations.

Our review identified several issues with the data as presented by the Department which were raised and subsequently resolved. With the remediation of the issues identified, the final estimated refund amounts are $7,680,711 for shift loading and $2,153,009 for overtime, a total of $9,833,719. This estimate covers the amount of overcharging up to March 2018, when billing for both shift loading and overtime was corrected to be consistent with the Fees Order. Whilst not within the scope of this review, the Department has advised that no overcharging has occurred since March 2018.

Our engagement with the Department included the remediation of the issues noted above (based on the baseline data provided by the Department) and extended the calculations to provide dollar estimate payments due to affected Industry participants. As noted previously, this analysis has not considered the accuracy and completeness of the source data, however, review of the calculations found that they support the estimated refund amounts above.

When presented with the remediated spreadsheets, a residual issue was identified by the Department. Correction of the date span for both overtime and shift allowance based on advice from the Department resulted in a reduction to the estimate of $461,629. Callida verified the dates and refund reduction which is reflected in the estimated refund amounts above.

A summary of the issues/observations identified, their corresponding rating and some recommended mitigation strategies that have subsequently been implemented are included in the table below:

Summary of Issue/Observation	Summary of Recommendation	Risk Rating	Current Status
An error in the extracted baseline data has carried through to the Departmental data that was used for calculations resulting in overtime charges being incorrectly recorded against shift loadings.	Review the extracted baseline data to recode the identified overtime to the overtime worksheet.	Medium	Resolved
The dollar calculation for overtime has taken into account public holidays whereas the dollar calculations for shift loading do not.	The data contained in the extracted base line spreadsheet be utilised to calculate the appropriate shift loading for public holidays.	Medium	Resolved
Several variations were found between the hours in the Departmental detailed worksheet and the hours in the Cost Summary worksheet that could not be explained at the time of the review, suggesting there were additional steps not provided for review or transposition errors between the two worksheets.	The cost estimate be recalculated to ensure a consistent and accurate transition of hours to dollars.	Medium	Resolved
The worksheets use a number of inefficient and redundant formulas which do not add value for the outcome being sought and add unnecessary complexity.	Further work be undertaken to remove redundant data and where possible, simplify and streamline formulas across the worksheets.	Low	Resolved

Data errors impacting final calculations

Risk Rating: (Medium)

Observation

There is an issue which originates in the extracted baseline data and has subsequently been carried through to the Departmental data that was used for calculations of both hours and dollars. In March 2016 (1,309 entries) and March 2017 (2 entries) there are codes for overtime being included in the shift calculations. This issue had previously been identified by departmental staff, but the data as presented had not been altered to correct the issue at the time of undertaking fieldwork presumably to maintain data integrity through this stage of the process.

Callida had excluded these calculations from the overtime estimate calculation on the basis that they were shift codes. The Department undertook further work in relation to shift codes incorrectly included in overtime calculations. The Department advised that whilst they were the incorrect codes, they did reflect actual overtime billed. The Department then adjusted the spreadsheets to reflect the inclusion of those amounts. Callida is not in a position to determine the accuracy of that rectification as we have not had access to or reviewed the additional data through which the errors were clarified. The rectification has added an additional $94,604 to the overtime refund identified by Callida.

Implication

Data integrity issues with the extracted baseline data being carried through to the revised calculations will impact on the reliability that can be placed on estimated payments to affected entities and may result in incorrect payments if not addressed.

Recommendation

Review the baseline extracted data to identify incorrectly coded overtime that has carried through to the Departmental detailed worksheets and correct extracted baseline data accordingly.

Status

Resolved.

Inconsistent treatment of public holidays

Risk Rating: (Medium)

Observation

When the Departmental detailed worksheets have been used to calculate the financial impact of the adjustment in hours charged, there is a difference in accounting for public holidays. In calculating the correct dollar charge for overtime, public holidays have been included for overtime, whereas the same calculation for shift loading does not.

Both overtime and shift loadings are impacted by when the hours were worked. Normal overtime, weekends and public holidays attract different multipliers of the normal rate. Not taking into account public holidays presents an anomaly in the way the two allowances have been calculated and will result in the dollar estimate for shift loading to be understated.

Implication

Any incorrect application of multipliers for public holidays will result in inaccurate calculations for shift loading. If not rectified any further work based on current data to calculate refunds or to determine next steps will be flawed through data inaccuracies.

Recommendation

The data contained in the original spreadsheet should be reviewed and revised formulas be utilised to calculate the appropriate shift loading for public holidays and included in the final calculation of overcharging estimates.

Status

Resolved.

Variation between hours calculation and hours used in dollar calculation

Risk Rating: (Medium)

Observation

A separate worksheet which was provided subsequent to the testing of the extracted baseline data Departmental and detailed worksheets was developed to provide a dollar value to the overtime and shift loading hours calculations. A review of the cost summary worksheets identified slight differences between the Departmental detailed worksheets, and the hours used in the cost summary worksheet. These errors have been corrected in the final calculations

Implication

Incorrect transposition of hours or between the respective worksheets will create errors in determining the final refund due. Similarly, if there is an explanation for the variance, a failure to document that reasoning may call into question the integrity of the data being used to support the refund estimate.

Recommendation

As part of the additional work Callida has been asked to undertake, the cost estimate be recalculated to ensure a consistent and accurate transition of hours to dollars.

Status

Resolved.

Inefficient and complex use of formulas and the use of redundant data

Risk Rating: (Low)

Observation

It is clear from reviewing the work undertaken to date and discussions with Departmental staff that a significant amount of work has been undertaken to ensure a thorough review of available data and the subsequent use of that data to calculate an estimate of potential refunds.

Whilst the work undertaken is detailed, it has resulted in a large amount of complexity in relation to the formulas used and the manipulation of the source data. In addition, and in an effort to ensure data integrity, there is an amount of redundant data that has been carried forward that adds little value to final calculations. The worksheets use several inefficient formulas including in some instances different formulas being used to calculate the same outcome. Despite the complexity, Callida did not detect any errors resulting from the way formulas were applied to the source data.

Implication

As well as creating unnecessary complexity when trying to verify the accuracy of calculations, the redundant data and inefficient formulas make the worksheets difficult to navigate and, in the future, properly document.

Recommendation

Further work be undertaken to remove redundant data and where possible, simplify and streamline formulas across the worksheets.

Status

Resolved.

Each recommendation has been rated using the following risk matrix.

The table below provides information on what each rating means in relation to the recommendations made.