ACA Telecommunications Consumer Awareness and Information Needs Survey, Financial Year 1999/2000
Contents
- Overview *** You are here ***
- Quality Framework
- A. General Information
- B. User Requirements, Survey Objectives and Budget
- C. Alternative Sources of Data
- D. Data Outputs
- E. Questionnaire Design and Data Collection
- F. Respondent Load
- G. Population / Frame
- H. Sample Design
- I. Estimation and Analysis
Overview
 | Portfolio: | Communications, Information Technology and the Arts | ||
| Agency: | ACA - Australian Communications Authority | ||
| Contact Person... | Name: | Judy Eastman | |
| Email: | judy.eastman@aca.gov.au | ||
| Phone: | 03 9963 6954 (International +61) | | |
Summary
|
Planned Outcome: Outputs: It is envisaged that this research will be quantitative only, as results from the previous research survey will be used to ascertain which areas are in need of information. Further qualitative research may be undertaken in subsequent surveys. From the research we intend to determine areas where consumer information strategies, such as public education campaigns, would be beneficial to ensure the community is well informed about telecommunications matters and how previous consumer information strategies have contributed to increased consumer awareness of those issues. The information is for ACA purposes only. The second phase of research (quantitative survey) will enable the ACA to compare the outcomes with benchmark results obtained in the first phase. It will provide the ACA with a better understanding of the relevant information needs of the public, will be used to determine whether the objectives, target audiences and consumer information strategies currently in place, are still appropriate. The 2000 research will also assist the ACA to assess whether there has been any changes in consumers’ awareness of aspects of the telecommunications environment, particularly in relation to: Overview of topics the survey will cover: |
Return to top
Reference Period & Frequency
| Reference Period: | Financial Year 1999/2000 |
| Frequency: | Annual |
| First run: | 1998/1999 |
Return to top
Response Rates
| | | As planned | Actual |
| Number of businesses... | | ||
| | | | |
|
| | 2100 | 2813 |
|
| | 1302 | 1097 |
|
| | 300 | 327 |
| | | | |
| Average time to complete questionnaire: | | 20 mins | 19 mins |
| Statistics... | | | |
| Response rate: | | 23% | 30% |
Return to top
Clearance Information
| Survey reviewed by | |
| Stat. Clearing House: | Financial Year 1999/2000 |
| Status: | |
| For additional clearance information, please contact the Statistical Clearing House by phone, fax, post or email. | |
Return to top
|
ACA Telecommunications Consumer Awareness and Information Needs Survey, |
 |
 |
|||
 |
 |
|
|||
 |
 |
 |
 |
|
|
Quality Framework
This data quality framework has been published internationally (Brackstone G., Managing Data Quality in a Statistical Agency, (1999) Survey Methodology, Vol. 25, no. 2, Statistics Canada) and has been recommended by the ANAO as 'better practice' in specifying performance measures (ATO Performance Reporting under the Outcomes and Outputs Framework, Australian Taxation Office, Audit Report No.46 2000-01, pp63-64.) on advice from the ABS Statistical Consultancy Unit.
Return to Survey Contents
Relevance
The relevance of statistical information reflects the degree to which it meets the needs of the clients. It is usually described in terms of key user needs, key concepts and classifications used, the scope of the collection and the reference period. |
Why do you need to conduct a survey? (B1)
- The principle aim of the research is to assess the ACA’s Corporate Plan 1999/2002
- an informed community with consumer safeguards and Corporate Plan
- consumer information and consumer safeguards.
Planned Outcome:
Outputs:
It is envisaged that this research will be quantitative only, as results from the previous research survey will be used to ascertain which areas are in need of information. Further qualitative research may be undertaken in subsequent surveys.
From the research we intend to determine areas where consumer information strategies, such as public education campaigns, would be beneficial to ensure the community is well informed about telecommunications matters and how previous consumer information strategies have contributed to increased consumer awareness of those issues.
The information is for ACA purposes only.
How will the survey outputs meet the needs specified in B1? (B2)
- The second phase of research (quantitative survey) will enable the ACA to compare the outcomes with benchmark results obtained in the first phase. It will provide the ACA with a better understanding of the relevant information needs of the public, will be used to determine whether the objectives, target audiences and consumer information strategies currently in place, are still appropriate.
- choice in products and services now available; and
- array of consumer safeguards in place.
- consumer safeguards (Telecommunications Industry Ombudsman, universal service obligation, Customer Service Guarantee);
- emergency service issues;
- mobile telephone services;
- numbering and number portability;
- cabling;
- awareness of the ACA; and
- consumer information requirements.
The 2000 research will also assist the ACA to assess whether there has been any changes in consumers’ awareness of aspects of the telecommunications environment, particularly in relation to:
Overview of topics the survey will cover:
- The main data items to be produced by the survey include:
- local calls (number made, provider, reasons used)
- long distance calls (number made, providers, reasons used)
- mobile calls (number made, providers, reasons used)
- internet connection (providers)
- views and awareness on various communications issues.
- by geographic area (metro/non-metro, state, etc)
- by sex and position/job function of respondent
- by number of telephone lines to business.
Cross tabulations :
Attitudinal segmentation:
Final written interpretive report with comparisons between sub-groups in the Small Business sample; and Residential consumer market.
- Refer responses to previous questions.
- Attitudinal segmentation to be conducted utilising cluster and factor analysis.
- Significance testing between proportions at 95% confidence level has been applied to the study.
- ABS definition of Small Business:
Manufacturing: | 100 employees or less |
Other industries: | 20 employees or less |
- The target population is consumers of telecommunication services in Australia, who are small businesses (small business is defined as any businesses with 20 or less employees except for manufacturing businesses, which will include businesses with 100 or less employees).
- Electronic Yellow Pages. Random selection of sample items utilising a software programme (OZ on Disk and internally developed software).
Call patterns i.e. return calls to outs/engaged/answering machines etc. along with booked appointments, programmed to yield highest possible response rates from random numbers selected.
Once randomly selected, sample is split into State locations allowing proportional sampling throughout the study.
- The type of unit is 'business phone number'.
- manufacturers with less than 100 employees
- non-manufacturers with less than 20 employees.
Two screening questions then allow us to identify small businesses using these criteria:
Businesses who do not meet the above criteria are out-of-scope and screened out of the survey.
- Random telephone interviewing (from electronic Yellow Pages) with relevant screening for Small Business qualification (refer questionnaire).
Most cost efficient and timely methodology to obtain representative sample of Small Business population.
- 2000
- It is intended that the survey be repeated every year, to compare results over a
financial year period.
Return to Quality Framework
Accessibility
The accessibility of statistical information refers to the ease with which it can be referenced by users. It includes the ease with which the existence of information can be ascertained, as well as the suitability of the form or medium through which the information can be accessed. |
How will the results of the survey be explained / presented to the major users? (I8)
- Various forms as appropriate:
- Cross tabulations and derived tables.
- Written interpretative report published by the ACA and distributed to a broad audience, including through associations representing small business.
- Graphs, pie charts, etc.
- Australian Communications Authority,
Consumer Affairs Group,
Consumer Awareness Team (Melbourne Central Office)
- Judy Eastman,
Consumer Awareness Team,
Tel: 03 9963 6954,
Email: judy.eastman@aca.gov.au
- Telephone numbers and respondent names are not linked to the final data Output. Interviewers and all staff are trained and abide by the code of ethics preventing disclosure of respondent or client information.
Return to Quality Framework
Timeliness
The timeliness of statistical information refers to the delay between the reference point to which it pertains and the date on which the information becomes available. |
To what reference peiod(s) does the survey refer? (A6)
- 2000
Date questionnaire finalised | 17/7/00 |
Date data collection commenced | 19/7/00 |
Date data collection finished | 26/7/00 |
When results available | 7/8/00 |
Return to Quality Framework
Accuracy
The accuracy of statistical information is the degree to which the information correctly describes the phenomena it was designed to measure. It includes measures of both sampling and non-sampling error. |
What is the expected total sample size to be approached and how has it been set? (H4)
- 6-8 times the final base of n=300 will be approached in order to yield the final sample of n=300 ie around 2100 calls for 300 responses.
Budget and ACA brief requirement.
Same size as in Wave 1.
Same ratio to Residential as in Wave 1.
- Refer previous questions.
Weighting not possible due to lack of population data.
- N=300 - refer H4/5.
- No weighting applied to Small Business study, due to lack of population data. Refer to H7.
- 13% expected to no longer be in business, or changed telephone number.
65% expected to meet the scope of the project but will not be included in the
sample frame, due to time constraints, i.e. booked appointment does not
eventuate during the life of the project, or attempts to the sample item constantly
yield voice mail, answering machine or rings out.
Duplicate numbers and mobile numbers were removed from the sample; If decision process was controlled by a head office, only the head office was interviewed regardless of State; The questionnaire screened out those that did not meet the "small business" criteria this equated to 25% of those contacted. Those with multiple numbers listed for the one business, if contacted, did not complete the interview as second time.
- Logic checks have been applied to appropriate questions within the CATI computer programme. Code Frames developed in conjunction with the fieldwork Manager, Consultant and Coder. All code frames are reviewed and approved by the consultant prior to coding all open questions. Fieldwork Supervisor edits, and checks open responses for logic with previous answers to closed questions before passing same onto the Coder.
- Pilots of both Residential and Small Business questionnaires were conducted utilising CATI (computer assisted telephone interviewing) prior to commencement of main fieldwork. Minor refinements made to wording of a couple of questions.
Questionnaire was again tested at the briefing stage for clarity and ease of administration by interviewers. Briefing duration 1.2 hours.
- None
- Electronic Yellow Pages business population nationally = 1,337,795 . This is inclusive of duplicate listings, and mobile numbers which were removed on extraction of sample.
What actions have been taken to overcome known frame deficiencies? (G7)
- Sampling is random with issue of numbers controlled by computer program. Each sample item is attempted up to 10 times during the life of the study. Therefore allowing all sample items equal chance of being included in the final sample and thus yielding a representative final sample.
- 23% of sample items approached are expected to meet scope and conduct the interview. Statistics on number of calls are kept and reviewed throughout the project.
What evidence is there that the expected respondents will be representative of non-respondents? (H7)
- The sampling protocol ensures each sample item is randomly chosen for approach and the call pattern is programmed to be applied evenly to each sample item. Therefore each sample item has the same chance of being included in the final sample, refusal rates being the only determining factor. No one particular group of respondents would present with a higher Refusal, out, or engaged/voice mail rate than any other group of respondents. Ensuring through the computer programme, that contact rates are evenly distributed, ensures that the final sample is as representative as can be logically expected.
Return to Quality Framework
Interpretability
The interpretability of statistical information reflects the availability of the supplementary information and metadata necessary to interpret and utilise it appropriately. It includes appropriate presentation of data such that it aids correction interpretation. |
How will the results of the survey be explained / presented to the major users? (I8)
- Various forms as appropriate:
- Cross tabulations and derived tables.
- Written interpretative report published by the ACA and distributed to a broad audience, including through associations representing small business.
- Graphs, pie charts, etc.
- Simple random sampling applied. With a quota cell applied to each State, proportional to population. 'Population' determined by listings in the Electronic Yellow Pages.
- Stratified to State only. A purely random national sample frame would not allow, given the required base of n=300, for proportional representation within each State, thus State was controlled by applying a quota.
- No weighting applied to Small Business study, due to lack of population data. Refer to H7.
- Each question analysed and interpreted independently, therefore:
- Depends on whether responses conform to ‘other’ categories established for individual questions.
- The proportion of ‘unusual’ and/or ‘suspicious’ responses to each question.
- Cross tabulations will be produced from the survey data.
- Statistical significance testing will be applied to identify any differences (not attributable to sampling error) between sub-groups/segments.
- No population data is available for weighting to total population.
- Refer responses to other questions relating to analysis, sample definition, population, etc.
- Need further clarification to expand on response to this item.
- Refer previous questions.
Weighting not possible due to lack of population data.
- Refer responses to previous questions.
- Attitudinal segmentation to be conducted utilising cluster and factor analysis.
- Significance testing between proportions at 95% confidence level has been applied to the study.
Return to Quality Framework
Coherence
The coherence of statistical information reflects the degree to which it can be successfully brought together with other statistical information within a broad analytical framework and over time. The use of concepts, classifications and target populations promotes coherence, as does the use of common methodology across surveys.
- Repeating survey, first conducted 1998/1999
- It is intended that the survey be repeated every year, to compare results over a
financial year period.
- Intention would be to retain the same definition and criteria for consistency.
- ABS definition of Small Business used in initial wave of ‘Telecommunications Awareness and Information Needs’ survey. However, SOHO excluded in Wave 1 (1999) but not in Wave 2 (2000), therefore not directly comparable.
The consumer awareness/information needs survey is to be conducted annually. While it is intended that the ACA be able to track certain issues over time, the survey is also a dynamic document that will probe awareness and understanding of different issues each year, partly because issues will lose relevance over time. For example, we did not probe awareness of competition in the telecommunications industry this year as the 1999 research and other public information campaign research conducted by AUSTEL/ACA consistently revealed almost universal awareness of competition. To this end, it will not be directly comparable in its entirety from year to year. Considerable effort was taken to ensure that the purpose of the questions in the 2000 survey would produce quality responses and in that respect, it is not directly comparable to the 1999 survey.
Those aspects that may be directly comparable include the services used by households/small business; the number of lines per household/small business; awareness of other service providers; and attitudes to new technology and innovations in telecommunications products and services.
- The survey is undertaken annually to meet corporate objectives and legislative functions. The survey targets specific issues relevant to the ACA’s functions and powers as well as aiming to probe awareness of general competition issues and consumer attitudes/motivations.
This research may be impacted upon by other ACA research conducted for specific campaigns. Where other ACA research data exists and is relevant to consumer awareness/information needs, it is utilised.
The ACA also utilises other external research/survey work where it is publicly available (eg. relevant ABS surveys) but is not aware of research/survey work that has been undertaken by communications carriers and service providers. Other data sources are utilised/examined where possible but are not the primary data source for determining the focus of the ACA’s information campaigns and strategies. Other data sources of relevance to the ACA include research undertaken by the Office of Telecommunications (OFTEL) in the UK. OFTEL has recently undertaken a number of surveys on residential and small business use of telecommunications, which are compared against the ACA’s own data.
The ACA is also required by legislation to report annually on the performance of the industry. Some of the data obtained from this consumer awareness survey and further survey work by the ACA on consumer satisfaction is utilised for this report. This report also sources a large range of data from industry - it is used by the ACA to the extent that the performance report data is relevant/useful to meeting corporate objectives (eg. providing information to consumers) in addition to its legislative reporting obligation, for which it is primarily sought.
- As indicated above, the ACA has specific legislative functions and its own corporate objectives, which include providing information to the public about telecommunications and radiocommunications issues and ensuring that the Australian community is well-informed about telecommunications matters. These issues and the methods for providing information are partly determined by the ACA’s consumer awareness research. Industry research (ie. conducted by telecommunications carriers and service providers) is not widely available and in any case would arguably be ill-suited to the ACA’s needs where conducted for those companies’ own marketing purposes. OFTEL research (and other overseas research) is useful for comparative purposes but cannot be utilised as a primary data source because it is specific to the UK (or other country), which operates under its own distinct telecommunications regulatory environment.
- ABS definition of Small Business:
Manufacturing: | 100 employees or less |
Other industries: | 20 employees or less |
- Other research data is utilised where possible but as outlined above, is not relied upon as a primary data source due to the fact that it may not be related to the ACA’s functions and legislative responsibilities. Where possible, the ACA utilises its contacts within the telecommunications industry to obtain data that may be collected by other organisations and liaises with the industry about our role and data needs. In particular, the ACA has a close relationship with the Department of Communications, Information Technology and the Arts, which is kept informed of the ACA’s data needs and proposals for meeting those needs.
Return to Quality Framework
