NSW Population Health Survey: Description of methods 2024

​​​​​​​​​​​​​​​​​​​​​ ​​​​​On this page​

1. Introduction
2. Survey instrument​
3. Survey sample and sampling frame
4. Weighting strategy
5. Data collection
6. Data analysis
7. Respondent distribution
8. Response rates
9. Biases and uncertainty
10. ​ References​​​

​Introduction

The New South Wales Population Health Survey (the Survey) is a continuous survey of the health of people of NSW, conducted annually since 2002.

The main aims of the Survey are to provide detailed information on the health status of adults and children in NSW and to support the planning, implementation and evaluation of statewide health services and programs in NSW.

In 2024, the survey used a mixture of computer assisted telephone interview (CATI) and, for the first time, a trial of a Push-to-Web (PtW) postal survey in the final quarter of the calendar year. The trial of the PtW methodology was initiated due to declining response rate of the CATI methodology. The trial aimed to understand whether PtW methodology could be a suitable complementary or alternative collection methodology for the Survey in the future.

This page describes the Survey's collection methodology for 2024.

Survey instrument

The survey instrument consists of question modules covering health behaviours, health status, health service utilisation, and attitudes toward health behaviours and service use. The questionnaire is reviewed annually within the Centre for Epidemiology and Evidence and by expert policy groups. Questionnaires for various years can be accessed from the NSW Population Health Survey website. The instrument was translated into five languages for the telephone survey: Arabic, Simplified Chinese, Greek, Italian and Vietnamese. Due to the limited timeframe for conducting the PtW version of the survey, the online questionnaire was only made available to respondents in English.

The online survey replicated the existing CATI survey as closely as possible, with minor adjustments made where necessary for clarity or operational efficiency, given the absence of an interviewer.

Survey sample and sampling frame

The target population for the Survey included all residents of NSW. For the CATI survey, the sampling targets were established at both the local health district level and for children and adults at a statewide level. For the PtW survey, collection targets were established at the local health district level, as well as for children (0–15 years), and young people (16–24 years).

For the CATI survey, respondents were sourced from a mobile phone sampling frame, using a sampling frame consisting of list-assisted mobile phone numbers and random digit dial numbers purchased from a phone number provider. List-assisted mobile phone numbers include information relating to the likely postcode of the mobile phone owner and are used to stratify that part of the sampling frame by local health district. Random digit dial phone numbers do not include any geographical information. No further information was available on the sampling frame to support screening of respondents.

For the PtW sample, a raw address file (called the Postal Address File) was purchased from an Australia Post authorised third-party data provider.¹ The Postal Address File was filtered to addresses flagged as residential, then geocoded and boundary tagged to stratify the sample by local health district. The sampling frame was supplemented with information from the 2021 Census General Community Profiles to identify small areas (Statistical Area 1) that had a higher proportion of residents that were either children or young people.² Areas that were flagged as having a higher proportion of children or young people (more than 20% of the total population in an area) were specifically flagged to support testing of recruitment strategies specific to those respondent groups.​

Data collection

For the CATI survey, interviews were completed throughout the year, from January through to the week before Christmas 2024. Interviews were conducted every day of the year, except Sundays and public holidays, with calls made between 10am and 8pm. All CATI respondents received a pre-contact SMS, giving individuals who lived outside of NSW the opportunity to opt out of further contact.³

In addition to directly interviewing individuals aged 16 years and over, the survey also captures information relating to children aged 0–15 years in two ways:

• as a follow-up interview to one completed by an adult where they consented to further contact (described as a ‘child callback’)
• as a separate survey specifically targeting households that include at least one child under 16 years of age (described as a ‘child booster’).​
  

For interviews relating to respondents aged 2–16 years, if the parent or carer was unable to identify the height and weight of their child or gave a biologically implausible height, the interviewer offered the parent or carer the opportunity to provide that information via a follow-up survey sent via SMS.

For the PtW survey, respondents were sent a primary invitation letter and one reminder letter. As the address file does not include any information on the occupants of the household, all letters were addressed to the household.

The invitation letter included instructions to select a respondent within the household (and for children and young people, a respondent within that specific age range), using an alternating next or most recent birthday methodology.⁴

Weighting strategy​

For the CATI survey, the weighting strategy was the same as the previous year. It is described in detail in the Description of methods 2012–2023.⁵ 

For the PtW survey, separate selection weights were estimated for the three recruitment streams: the general population, the ‘child booster’ and the ‘young people booster’. The overlap between the three frames that arose from this design was accounted for by determining a respondent’s frame memberships and then averaging the selection weights across the frames they were a part of.^6,7 Selection weights were post-stratified to match population projections by local health district, age group and sex. Population projections were based on Australian Bureau of Statistics estimates and population projections based on data from the NSW Department of Planning, Housing and Infrastructure and were used to calibrate weights to the population within each local health district.

A final set of analyses weights were derived by combining the two surveys in proportion to the number of respondents achieved by each collection method.

Data analysis

Respond​ent distribution

Selected characteristics of respondents to the 2024 NSW Population Health Survey

Overall​

CATI Survey

• Number: 12,684

• Per cent: 100.0%

PtW Survey

• Number: 965

• Per cent: 100.0%

Combined Survey

• Number: 13,649

• Per cent: 100.0%

By age group

CATI Survey

PtW Survey

Combined Survey

​​​[a]​ Excludes not stated and another term.

​​​[b] The question used to define Aboriginal status was: Are you/Is your child Aboriginal or Torres Strait Islander? With response categories: Aboriginal but not Torres Strait Islander, Torres Strait Islander but not Aboriginal origin, Aboriginal and Torres S​​trait Islander, Not Aboriginal or Torres Strait Islander, Don’t Know and Refused/Prefer not to say. For the purposes of this web page, people who answered “Aboriginal but not Torres Strait Islander”, “Torres Strait Islander but not Aboriginal origin”, or “​Aboriginal and Torres Strait Islander” were reported as Aboriginal, with people that responded “Not Aboriginal or Torres Strait Islander” reported as non-Aboriginal. Excludes Don’t Know and Refused/Prefer not to say.

​[c] Remoteness area is based on the Australian Bureau of Statistics Australian Statistical Geography Standard (ASGS) Edition 3. The category ‘Outer regional an​d remote’ includes people living in ‘very remote’ areas.

​Response rates​​

Response rates measure the proportion of individuals who completed a survey divided by the total number of eligible people who were approached from the sample. The lower the response rate, the higher the likelihood of non-response bias. Response rates for the Survey were calculated using the American Association for Public Opinion Research (AAPOR) Response Rate 3 which estimates the proportion of cases of unknown eligibility that are eligible.⁹

For the CATI survey, response rates have been declining since 2017, however response rates for 2024 were similar to those achieved in 2023. For the PtW survey, response rates were estimated using the same methodology that was used for the telephone survey. Response rates for the PtW survey were consistently higher than those achieved in the CATI survey.

​Table 2. Response rate (AAPOR RR3*) for NSW Population Health Survey 2017–2024 (CATI)

Adult (16+) Survey

Child Booster Survey

* American Association for Public Opinion Research, Response Rate 3.

​​​ ​Biases and uncertainty

Data for the Survey is collected from a random selection of NSW residents. The probabilistic nature of the selection process reduces the risk of selection bias that would have otherwise resulted from the use of non-probabilistic selection processes, such as those that make use of convenience panels. Sampling introduces some uncertainty to estimates derived from the data, which can be quantified using confidence intervals (CIs), unlike data arising from a convenience panel. A wider CI indicates greater uncertainty in the estimate, while a narrow CI suggests a more precise estimate.

The 2024 survey represented the first time that the Survey was conducted using CATI and PtW methods. A mode effect refers to a systematic difference in estimates that is attributable to the differences in the mode of data collection. Prevalence rates obtained from health surveys can vary according to the mode of data collection, primarily between interviewer and self-administered modes.¹⁰ Factors contributing to mode effects may include differences in the demographic characteristics of respondents for each mode, the involvement or not of an interviewer and the perceived anonymity of responses, especially for sensitive topics such as drug use.¹¹

The Survey is also subject to non-response bias. Non-response can occur when a phone number is selected, but the person who owns the phone does not pick up or decides not to respond to the survey. While a low response rate, such as that achieved in the Survey, can be indicative of the potential for non-response bias, the degree to which non-response impacts on the findings of a survey depends on whether there are meaningful differences between responders and non-responders, in a way that is related to the measure of interest.¹² Non-response is somewhat mitigated by using weighting procedures to ensure that the sample represents the NSW population on basic demographics.

In addition to non-response bias, the Survey is subject to the risk of inaccurate responses. Inaccurate responses may arise when a respondent fails to understand a question, fails to recall an event accurately, or feels uncomfortable responding truthfully to a question. Inaccurate responses may arise for questions where the question relates to a behaviour or health issue that is stigmatised or is potentially embarrassing.¹¹ In the Survey, all answers remain completely confidential, and respondents may choose to refuse to answer any question put to them, which encourages respondents to answer truthfully.

These potential biases described above and the potential for other forms of error in surveys should be considered when drawing conclusions and making decisions based on the Survey results available via the NSW Population Health website​ or from HealthStats NSW​​​.

​[back to top]

​References

1. Australia Post. Address data (accessed 12/05/2025).
2. Australian Bureau of Statistics. Community Profiles: Census data tools and products​​ (accessed 12/05/2025).
3. Dal Grande E, Chittleborough CR, Campostrini S, Dollard M and Taylor AW. Pre-Survey Text Messages (SMS) Improve Participation Rate in an Australian Mobile Telephone Survey: An Experimental Study. PLoS One; 2016.
4. Gaziano C. Comparative Analysis of Within-Household Respondent Selection Techniques. Public Opinion Quarterly 2005; 69(1): 124–57.
5. NSW Ministry of Health. NSW Population Health Survey: Description of methods 2012–​2023​ (accessed 12/05/2025).
6. Lohr SL. Alternative survey sample designs: Sampling with multiple overlapping frames. Survey Methodology 2011; 37(2): 197-213.
7. Lohr SL. Sampling: Design and Analysis (3rd ed.). Chapman and Hall/CRC; 2021.
   
8. Wolter KM. Introduction to variance estimation. Vol 53: Springer; 2007.
9. The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys, 10th edition. AAPOR; 2023.
10. Hoebel J, von der Lippe E, Lange C and Ziese T. Mode differences in a mixed-mode health interview survey among adults. Archives of Public Health 2014; 72: 46.
11. Tourangeau R, Yan T. Sensitive questions in surveys. Psychological Bulletin 2007; 133(5): 85.
12. Bianchi A, Shlomo N, Schouten B, Da Silva DN, Skinner C. Estimation of response propensities and indicators of representative response using population-level information. Survey Methodology 2019; 45: 231–64. ​
    ​[back to top]​​​