2000 Statistical Clearing House Agency Report
January 2000 to December 2000
Contents
Background
Introduction
General Observations
General SCH Report
For Further Information
Appendix A. More information on the creation and purpose of the SCH
Appendix B
PDF File
Attachments
Tables
- Table 1: Reviews: Review Status, Respondent Load by Agency, 2000 (a)
- Table 2: Review Status, by Type, 2000
- Table 3: Completed Reviews (a) - Approval Status, 2000
- Table 4: Feedback received, 2000
- Table 5: Completed Reviews (a), Changes to Methodology, 2000
- Table 6: Average Elapsed Time for Review, Average Elapsed Time from Receipt of Complete Information for reviews completed, 2000(a)
- Table 7: Collections Investigated, Found In Scope but Not to be Reviewed, by Reason, 2000
1 The SCH is dedicated to the reduction of respondent burden on business, especially small business, from the statistical activities of Government. All surveys that involve 50 or more businesses are required to be reviewed by the SCH.
The SCH was created in 1997 by a cabinet decision and was given explicit backing by the Prime Minister. A joint letter with the Australian Statistician noting the objectives of the SCH was sent to Government Ministers and department heads informing them of the decision on 24 October 1997. More information on the creation and purpose of the SCH can be found in Appendix A.
Introduction
2 This Report summarises the surveys reviewed by the SCH during the period 1 January to 31 December 2000. This period incorporates those surveys that provided their first information template during this period and all reviews carried over from the previous year.
3 Since the last Agency Report, SCH staff have made several observations concerning the review process and have identified common problems amongst surveys that have been reviewed.
Timeliness In many cases, there was little time to conduct the review or to identify changes required for the survey before the questionnaire was printed or, worse still, was due in the field. Survey timetables often allow inadequate time for planning and development, to the detriment of likely survey outcomes.
Sample size SCH staff have found that sample sizes are rarely well justified. Sample sizes are often larger than what is likely to be needed to meet the objectives of the survey.
Response rates Particularly in the case of smaller, one off surveys, there is a lack of understanding about the implications of low response rates and the effect of non-response bias.
Questionnaire testing SCH staff stress that the questionnaire should always be tested at least once prior to the main survey, even if only on a limited number of respondents, to ensure the questions are understood, and the required data is actually available.
Duplication The SCH are ensuring that the duplication between surveys is minimised. More time should be invested by agencies to identify other organisations that are examining similar populations and use this knowledge to coordinate strategies. This would lead to better use of the information being collected, reduced sample sizes and efficiency through reduced project costs.
Application for review The small number of surveys notified by some agencies is a cause of concern.
Use of Consultants SCH staff have found that when consultants are engaged to conduct a survey they are not always aware of the requirement to have the survey approved by the SCH (and it has not always been built into their contract to conduct the survey).
Post Survey Information The SCH has only received responses to 35% of the requests for post survey information in 2000. SCH reviews are undertaken prior to survey implementation, whereas the post survey information is an important measure of how well the survey was actually undertaken.
Confidentiality SCH staff have found that some confidentiality restrictions that survey areas are placing on themselves are impractical for editing and follow up procedures.
General SCH Report
4 Table 1: Reviews: Review Status, Respondent Load by Agency, 2000 (a)
| Agency (b) | Completed Reviews (c) | In Progress | In Scope - Not to be reviewed(d) | Not in Scope (e) | Total | Estimated Load (hrs) (f) |
| ABARE | 11 | 0 | 1 | 1 | 13 | 4883 |
| ABS | 38 | 2 | 9 | 4 | 53 | 109017 |
| ACA | 2 | 0 | 0 | 0 | 2 | 350 |
| AFC | 0 | 0 | 1 | 0 | 1 | 0 |
| AFFA | 9 | 1 | 2 | 0 | 12 | 7156 |
| AG | 1 | 0 | 1 | 1 | 3 | 538 |
| AGO | 0 | 0 | 0 | 1 | 1 | 0 |
| ANTA | 6 | 1 | 3 | 1 | 11 | 1351 |
| ATO | 5 | 0 | 1 | 5 | 11 | 1453 |
| ATSIC | 0 | 0 | 0 | 1 | 1 | 0 |
| AusAID | 0 | 0 | 0 | 1 | 1 | 0 |
| Austrade | 2 | 0 | 2 | 0 | 4 | 1058 |
| Defence | 2 | 1 | 1 | 0 | 4 | 132 |
| DEH | 2 | 0 | 0 | 1 | 3 | 102 |
| DETYA | 4 | 1 | 1 | 6 | 12 | 853 |
| DEWRSB | 2 | 1 | 10 | 3 | 16 | 74 |
| DFAT | 2 | 0 | 0 | 1 | 3 | 345 |
| DHAC | 2 | 1 | 0 | 2 | 5 | 552 |
| DISR | 6 | 0 | 7 | 6 | 19 | 619 |
| DOCITA | 2 | 1 | 0 | 5 | 8 | 200 |
| FACS | 2 | 0 | 1 | 2 | 5 | 235 |
| HIC | 4 | 0 | 0 | 0 | 4 | 160 |
| HRDC | 0 | 1 | 2 | 1 | 4 | 0 |
| NOHSC | 2 | 0 | 1 | 0 | 3 | 300 |
| Other(g) | 0 | 0 | 0 | 1 | 1 | 0 |
| PC | 0 | 0 | 0 | 2 | 2 | 0 |
| DOTRS | 1 | 0 | 2 | 0 | 3 | 125 |
| Total | 105 | 10 | 45 | 45 | 205 | 129503 |
(a) This table includes all surveys which the SCH reviewed during the period 1 January to 31 December 2000.
(b) The acronyms for the agencies are explained in Appendix B.
(c) The number of completed reviews includes surveys where review action was only partially completed after the review commenced (for reasons such as: surveys cancelled or changed to out of scope, that were conducted without SCH approval).
(d) Surveys are classified as "In Scope - not to be reviewed" for reasons such as: product evaluation surveys, lack of SCH resources leading to smaller surveys not being reviewed.
(e) Surveys are classified as "Not in Scope" for reasons such as: less than 50 business respondents, not a Commonwealth Government survey, administrative collection.
(f) These figures indicate the estimated respondent load incurred by surveys which began SCH review during the period 1 January 2000 to 31 December 2000 only. The estimated load is based on completed reviews only, and is as estimated by the agency conducting the survey.
(g) Not Commonwealth nor State Government.
Graph 1: Total Surveys Considered by SCH
5 The number of surveys considered by the SCH in the year 2000 increased by 38% from 1999 (graph 1). This was mainly due to a large increase in the number of non-ABS surveys considered over this period. In particular, DISR, DEWRSB and DETYA contributed to the increase in reviewed surveys by at least doubling the number of surveys submitted to the SCH.
Graph 1: Total Surveys Considered by SCH
Table 2: Review Status, by Type, 2000
6 Table 2: Review Status, by Type, 2000
| Type | Completed Reviews (a) | In Progress | In Scope - Not to be reviewed | Not in Scope | Total |
| Large Repeated Public Interest Survey (b) | 13 | 0 | 0 | 0 | 13 |
| Small Repeated Public Interest Survey (c) | 39 | 4 | 10 | 15 | 68 |
| Once Only Public Interest Survey | 22 | 6 | 16 | 18 | 62 |
| Pilot Test or Post Enumeration Survey | 3 | 0 | 5 | 0 | 8 |
| Programme Evaluation | 28 | 0 | 14 | 12 | 54 |
| Total | 105 | 10 | 45 | 45 | 205 |
(a) The number of completed reviews includes surveys where review action was only partially completed after the review commenced (for reasons such as: surveys cancelled or changed to out of scope, that were conducted without SCH approval).
(b) Large surveys are surveys which planned 2,000 or more respondents.
(c) Small surveys are surveys which planned fewer than 2,000 respondents.
The number of surveys that were not applicable for review (i.e. "In Scope - Not to be Reviewed" and "Not in Scope") rose from 19 in 1999 to 90 in 2000. The number of "Completed Reviews" and "In Progress" reviews decreased by 12% (15 surveys in total) from 1999 and 2000. This decrease is due to less Small Repeated Public Interest Surveys (16 fewer surveys in 2000).
Table 3: Completed Reviews (a) - Approval Status, 2000
7 Table 3: Completed Reviews (a) - Approval Status, 2000
 | Approved | Not Approved | Other (b) | Total |
| ABS | 38 | 0 | 0 | 38 |
| Non-ABS | 60 | 6 | 1 | 67 |
| Total | 98 | 6 | 1 | 105 |
(a) The number of completed reviews includes surveys where review action was only partially completed after the review commenced (for reasons such as: surveys were cancelled or changed to out of scope, surveys that were conducted without SCH approval).
(b) One review was completed but was neither approved nor not approved. This survey was cancelled after the review had commenced.
The number of not-approved surveys increased by 3 from the previous year to 6 in 2000 (table 3). Not approved surveys are usually those that entered the field without prior approval from the SCH. Usually this is due to the survey managers lack of knowledge about the SCH, but in some cases, the survey manager has decided that the survey will go into the field regardless.
Table 4: Feedback received, 2000
8 Table 4: Feedback received, 2000
| Received Compliment (a) | Received Complaint (a) | Total Feedback Received (b) | Post Survey Information Received |
| ABS | 17 | 8 | 22 | 9 |
| Non-ABS | 37 | 28 | 54 | 29 |
| Total | 54 | 36 | 76 | 38 |
(a) The compliments and complaints are feedback received in writing. Complaints and compliments can occur on the same feedback form i.e. these two columns should not be aggregated.
(b) Feedback was received from survey managers for 76 of the surveys reviewed.
Agency Report Graph 2
9 The numbers of complaints and compliments are well up on 1999. In total an increase of 36 compliments (graph 2) and 13 complaints (graph 3) regarding the SCH were recorded following a rise in total feedback received (total of 76 or an increase of 17 between 1999 and 2000). The increased feedback is believed to be due to a more proactive follow-up of feedback requests in 2000.
Graph 2: Compliments Received
Agency Report Graph 3
10
Complaints varied from the template taking too long to complete through to such issues as the SCH process being a "necessary evil". Compliments were mainly focused around the level of professionalism of the SCH staff or that it gave the survey areas a chance to clarify their methodology and purpose of the exercise.
Table 5: Completed Reviews (a), Changes to Methodology, 2000
11 Table 5: Completed Reviews (a), Changes to Methodology, 2000
| Respondent Load Reduced (b) | Improvements Suggested (c) | Improvements Made (c) | Total (d) |
ABS | 3 | 8 | 4 | 38 |
Non-ABS | 7 | 32 | 29 | 67 |
Total | 10 | 40 | 33 | 105 |
(a) The number of completed reviews includes surveys where review action was only partially completed after the review commenced (for reasons such as: surveys were cancelled or changed to out of scope, surveys that were conducted without SCH approval).
(b) This is the number of surveys where the sample or the survey content has been explicitly reduced by the clearance process. It does not include reductions in load due to questionnaire testing, leading to improved forms design, or reduction due to not approved or cancelled surveys.
(c) The quality improvements suggested include:
- changes to questionnaires
- changes to sample design methodology
- increasing follow-up procedures to improve response rates, and reducing initial sample sizes
- undertaking questionnaire testing
- addressing duplication with other surveys
- replacing census methodology with a sample survey
(d) Total number of completed reviews in 2000.
Graph 4: Occasions where Respondent Load was Reduced
12 Graph 4: Occasions where Respondent Load was Reduced
Graph 5: Improvements Suggested
13 Graph 5: Improvements Suggested
Graph 6: Improvements Made
14 Graph 6: Improvements Made
Table 6: Average Elapsed Time for Review, Average Elapsed Time from Receipt of Complete Information for reviews completed, 2000(a)
15 Table 6: Average Elapsed Time for Review, Average Elapsed Time from Receipt of Complete Information for reviews completed, 2000(a)
| Average Elapsed Time for Review (Calendar Days) (b) | Average Elapsed Time from Receipt of All Information to Approval Status Assignment (Calendar Days) (c) |
| ABS | 68 | 1 |
| Non-ABS | 31 | 1 |
| Total | 45 | 1 |
(a) The time taken to review a survey officially starts from the date of receipt of all information (ie template fully completed) to when the approval status is granted. However, in practice the SCH commences review action from the date of receipt of the initial information template and works with survey managers to ensure that all the information required is provided and queries are resolved during this time. Hence by the time that the SCH has received all information, there is a relatively short turn around time. Note that "reviews completed" only includes those reviews which result in either an "Approved" or "Not approved" status.
(b) These figures are based on the average elapsed time from the receipt of the initial information template to the notification of approval. These figures include the time taken for survey managers to respond to requests for clarifying information.
(c) These figures are based on the average elapsed time from the receipt of the final and complete information template to the notification of approval.
The majority of reviews are now completed within 0 to 20 working days from receipt of the intial imformation template, decreasing from 6 to 40 days in 1998 and 1999 (see graph 6). The average elapsed time for reviews was 45 days for the year 2000 (table 6) though the midpoint of these elapsed times provides a better indication of the time taken for reviews as a few very large elapsed times unduly affect the mean average elapsed time. The midpoint for reviews to be completed in 2000 was 18 calendar days. This number is a reduction from previous years (a midpoint of 30 calendar days in 1998 and 22 days in 1999). The SCH attempts to complete all reviews within 20 working days of starting the process. The longest review times (those in excess of 500 days) were delayed for such reasons as a survey manager proposed change in methodology, delays in contracting a consultant and delays in clearing the survey through the project board (ie. reasons outside the control of the SCH).
Graph 7: Number of Reviews Completed, Time Taken
16 Graph 7: Number of Reviews Completed, Time Taken
Table 7: Collections Investigated, Found In Scope but Not to be Reviewed, by Reason, 2000
17 Table 7: Collections Investigated, Found In Scope but Not to be Reviewed, by Reason, 2000
| ABS | Non-ABS | Total |
| Small surveys(a) | 7 | 25 | 32 |
| Product Evaluation | 2 | 6 | 8 |
| Other | - | 5 | 5 |
| Total | 9 | 36 | 45 |
(a) Due to constraints on SCH resources, some smaller surveys are not reviewed even though they involve 50 or more businesses.
Graph 8: Number of Reviews Started and Completed(a), Progressively, 2000
18 Graph 8 shows the differences between completed reviews and those that are either started throughout the year or continued from 1999. The progressive totals show that the highest increase of survey reviews started are in the months before the financial year begins (reflecting surveys covering the financial year). There is also a sharp rise in the number of reviews started at the end of the year, reflecting surveys covering the calendar year.
Graph 8: Number of Reviews Started and Completed(a), Progressively, 2000
(a) The number of completed reviews includes surveys where review action was only partially completed after the review commenced (for reasons such as: surveys were cancelled or changed to out of scope, surveys that were conducted without SCH approval).
For Further Information
19 The Statistical Clearing House can be contacted by:
E-mail: statistical.clearing.house@abs.gov.au
Phone: (02) 6252 6812
Fax: (02) 6252 8008
Website
Appendix A. More information on the creation and purpose of the SCH
20 In 1996, the Small Business Deregulation Task Force recommended that 'statistical collections affecting 50 or more businesses and run by, or on behalf of, all Commonwealth Government departments and agencies be subject to a central clearance process', that non-approved surveys not be permitted, and that approved surveys receive a registration number to be displayed on collection material. The report noted that clearance should not apply to statistics produced as by-products of administrative processes, and that it should be administered by the Australian Bureau of Statistics (ABS), in view of its coordination role.
This recommendation was endorsed by the Prime Minister to all Ministers in a letter dated 11 October 1997 informing them about the SCH and asking them to inform their departments and agencies within their portfolios to comply with the clearance process. This was followed up by a letter and accompanying annex sent by the Australian Statistician to departmental heads on October 24 that provided details of SCH procedures. Clearance of surveys began in November 1997. The SCH formally endorsed its objectives on 16 July 1997.
The primary objective of the SCH is:
through the development and operation of a clearance process for Commonwealth Government statistical collections with 50 or more business respondents, to minimise provider load and maximise the utility of the data collected, by reducing duplication of data requests, improving the quality of collections and ensuring the results are visible and accessible.
Appendix B
21
| ABARE | Australian Bureau of Agricultural and Resource Economics |
| ABS | Australian Bureau of Statistics |
| ACA | Australian Communications Authority |
| AFC | Australian Film Commission |
| AFFA | Agriculture, Fisheries and Forestry (Department of) |
| AG | Attorney-General's Department |
| AGO | Australian Greenhouse Office |
| ANTA | Australian National Training Authority |
| ATO | Australian Taxation Office |
| ATSIC | Aboriginal and Torres Strait Islander Commission |
| AusAID | Australian Agency for International Development |
| Austrade | Australian Trade Commission |
| Defence | Defence (Department of) |
| DEH | Environment and Heritage (Department of) |
| DETYA | Education, Training and Youth Affairs (Department of) |
| DEWRSB | Employment, Workplace Relations and Small Business (Department of) |
| DFAT | Foreign Affairs and Trade (Department of) |
| DHAC | Health and Aged Care (Department of) |
| DISR | Industry, Science and Resources (Department of) |
| DOCITA | Communications, Information Technology and the Arts (Department of) |
| FACS | Family and Community Services (Department of) |
| HIC | Health Insurance Commission |
| HRDC | Horticulture Research and Development Corporation |
| NOHSC | National Occupational Health and Safety Commission |
| PC | Productivity Commission |
| DOTRS | Transport and Regional Services (Department of) |
PDF File
approx. 157kb
The complete document in Portable Document Format to view online or download.
