2023 Household Disability Survey Data Collection

Introduction The New Zealand Household Disability Survey (HDS) is currently the most comprehensive source of information on disabled people in New Zealand and allows for comparisons between disabled and non-disabled people on several key social and economic outcomes. The survey provides estimates of the prevalence of functional difficulty for the total population and for sub-populations such as age groups, gender and ethnic groups. It also provides information about the outcomes and experiences of disabled people compared with non-disabled people across various aspects of life including employment and education, housing and transport. Further information is available on: • the support disabled people use and need in their daily life • the experiences of disabled people regarding their housing and transport use • participation in education and paid work • levels of education and income • experiences of leisure and wellbeing • unmet need for support and care • difficulty in more than one functional domain • the prevalence of HDHD and Autism Estimates from the 2023 HDS are not comparable with those from previous household disability surveys. Changes in the way in which disabled people are identified in the survey mean that Stats NZ does not view the surveys as providing a valid time series. Survey content Survey responses were collected either by phone or face-to-face interviews by trained interviewers using an electronic questionnaire. NZSL trained interviewers were available, as well as other methods by which hearing impaired people could participate in the survey. By default, all sample members were initially to be interviewed by telephone unless: • there was no phone number on their 2023 Census form, in which case an interviewer visited them at home • they indicated on their Census form that they had a hearing impairment; in which case they were contacted by a member of the NZ Sign Language interviewer team. Respondents could switch from one mode of interviewing to the other, by request. In total, 26 percent of respondents completed their survey in person, and 74 percent by phone. Wherever possible, adults selected for the survey (aged 15 and over) answered the interview themselves, although if they were unable to be interviewed a proxy respondent such as a family member or caregiver completed the interview on their behalf. In the case of children selected for the survey, all interviews were conducted with an adult parent, guardian, or caregiver on behalf of the child. Primary content The HDS questionnaire has 11 topics or modules: • Screening • Assistive equipment and technology • Personal and Household Support. • Health Services • Employment • Education • Transport • Housing • Leisure • Wellbeing • Child’s Main carer Changes to the survey content Previously, the Disability Survey was an amalgamation of two surveys: the Household Disability Survey (HDS), which surveyed adults and children living in private dwellings or group homes; and the Disability Survey of Residential Facilities (DSRF), surveying adults living in residential care facilities. The DSRF surveyed a much smaller number of people, who were given a much shorter version of the questionnaire. The DSRF was not run in 2023. Only the HDS was run in 2023, and the content described in this report all relates to the HDS. Stats NZ recognises that the lived experiences of people in residential facilities are important and that a comprehensive survey providing insights about those experiences would be of value. However, the DSRF is not such a survey and redeveloping it to meet this need is not within the scope or objectives of the NZDS. It would require a much more targeted and purpose-designed survey, which is outside the current scope of Stats NZ’s work. The DSRF has aways been a reduced version of the HDS, providing only a limited amount of data about a small section of the disabled population. The DSRF only had two full modules: screening and assistive equipment and technology, plus a handful of questions from the personal and household support module. It is important to note that when we say ‘residential facilities’ we are not talking about community-based housing for disabled people. For the purpose of the 2023 survey, community housing (also known as supported living) is treated the same as any other private, residential home. That is, people living in community housing are in scope to be selected for the HDS coverage and not the DSRF. Given that disabled people increasingly live in the community rather than in residential facilities or institutions, and the acknowledged limitations of the survey, we decided not to run the DSRF in 2023. Data Collection: The data collection took place over a period of 12 and a half weeks, from 23 August 2023 to 19 November 2023. Survey responses were collected either by phone or face-to-face interviews by trained interviewers using an electronic questionnaire. NZSL trained interviewers were available, as well as other methods by which hearing impaired people could participate in the survey. By default, all sample members were initially to be interviewed by telephone unless: • there was no phone number on their 2023 Census form, in which case an interviewer visited them at home • they indicated on their Census form that they had a hearing impairment; in which case they were contacted by a member of the NZ Sign Language interviewer team. Respondents could switch from one mode of interviewing to the other, by request. In total, 26 percent of respondents completed their survey in person, and 74 percent by phone. Children selected for the HDS had their interview completed by an adult proxy. These were typically the child’s parent or another primary caregiver. Wherever possible, adults completed the survey for themselves, including being supported to do so by someone they trusted. However, if an adult was unable to participate due to a health condition or disability, a proxy was sought to complete the survey on their behalf. That proxy would typically be their carer. The HDS electronic questionnaire was designed in flowcharts. It was then built using Blaise® (a computer-assisted interviewing system and survey processing tool developed by Statistics Netherlands). Salesforce was used to manage sample members while the survey was in the field. The 2023 Census collected phone numbers so that people could be contacted if selected for the HDS. Contact details were cleaned in preparation for fieldwork. Contact details collected in the Census were reviewed in preparation for fieldwork and any obvious scanning errors or typos were corrected. Five people were removed from the sample during this review process because they had not provided enough of their name on the Census form for them to be identified by interviewers. Before being contacted by Stats NZ, a letter was sent to all sample members at the address they gave on their Census form. The letter alerted them that they’d been selected for the HDS and encouraged them to call Stats NZ to complete the survey or otherwise wait for Stats NZ to contact them. Sample design information The HDS is referred to as a post-censal survey. A post-censal survey uses a census to support the survey sample design. This enables people in a particular subpopulation (like disabled people) to have a higher chance of being included in the sample than would occur with a random selection. It is a cost-effective way to survey smaller population groups. The 2023 Census of Population and Dwellings (Census) included questions about activity limitations and limiting health conditions. During HDS sample selection, those questions were used to indicate those people who were more likely than others to meet the threshold for being disabled in the survey. Selecting more generously from this group compared with others increases the likelihood of having enough disabled people in the sample. Without this approach a larger and more expensive survey would be required to achieve estimates of the same quality. In addition, information collected in Census could be used in the analysis of the survey which reduces the burden on respondents and data collection costs. The sample selection for the HDS was conducted in two stages: Selecting primary sampling units (PSUs). PSUs are small geographic areas designed to contain a standard number of dwellings. Selecting people from within PSUs based on information from their Census forms. In step one, a sample of PSUs were selected from a list or ‘sampling frame’. This frame of PSUs is used to select samples and manage overlap for all Stats NZ household surveys. The frame lists PSUs with attributes determined by data collected in the Census. PSUs were divided into groups or ‘strata’ based on which region they belonged to and whether they were classified as urban or rural. A proportionate number of PSUs were selected from each region; this is done with the aim of making the sample representative of New Zealand’s regional population distribution. For this survey, 1,805 PSUs were selected. During PSU selection, efforts were made to improve the quality of survey estimates for Pacific people. This was done by deliberately oversampling PSUs that were known to have a relatively high number of Pacific people living in them. The oversampling of “Pacific-dense” PSUs was expected to increase the number of Pacific people in the HDS and, therefore, create better quality estimates for Pacific people. More direct methods were taken to help ensure that there were sufficient children, Māori, and disabled people in the sample, see 7.5.2 Selecting from strata for more information. In step two, the Census records were retrieved for dwellings and people within the selected PSUs to create a sampling frame of people who could be selected for the HDS. Sample population A ‘target population’ is the population about which information is being sought, and a ‘survey population’ is all the members of the target population who have a chance of being selected for a survey. So, the survey population is the population from which information about the target population can be obtained (via the survey). The survey population needs to very closely resemble the target population because it will be used to produce information that represents the entire target population. The HDS survey population was the same as its target population (the usually resident population of New Zealand, living in permanent private dwellings or residential and community care facilities on the main islands of New Zealand during the 2023 Census) but with some exclusions made, typically for cost and practicality reasons. These exclusions were a small percentage of the target population, so any bias introduced is minimal. The exclusions were: People living in non-permanent private dwellings (such as tents or caravans) People living on islands other than the North, South, and Waiheke islands (such as Great Barrier, Kawau, Chatham, and Stewart islands) People who did not complete the 2023 Census. After these exclusions, the HDS survey population was: The usually resident population of New Zealand who lived in permanent private dwellings or residential and community care facilities on the North, South or Waiheke Islands when the Census was taken and who completed the Census. A ‘sampling frame’ is simply the list of units within the survey population (‘units’ may be individuals, dwellings, business, etc.). By being on this list, every unit in the sampling frame has a chance of being selected. The HDS sampling frame was created by taking Census responses for individuals living in the selected 1,805 PSUs and using the HDS eligibility criteria to remove anyone who was not eligible to be the survey because they were not members of the survey population (for example, people who were not usual residents, people who did not live in either a private dwelling or a residential an community care facility, people living in non-permanent private dwellings, people living on islands other than the North, South and Waiheke Islands). It was from the sampling frame that the HDS sample was selected. Selecting from strata With the sampling frame created, all individuals included in the frame were then placed into strata. Strata are groups that a population is divided into, based on certain shared characteristics. Everyone in the HDS sampling frame was placed into exactly one stratum based on their age, ethnicity, and whether they were more likely than others to be disabled (as determined by their Census responses). A set number of individuals were then selected from within each stratum. When a survey selects individuals from within strata, it is called ‘stratified sampling’. Stratified sampling is used for several reasons including: • increasing the efficiency of a sample design with respect to lowered costs and improved estimator precision • ensuring that all relevant subgroups within a population are adequately represented in the sample • targeting subgroups by disproportionate sampling (or over-sampling) certain strata. To ensure adequate representation of children, Māori, and disabled people in the HDS, the subgroups used for stratification were age group, Māori, and likelihood of disability (from Census). The strata were : Non-Māori aged 0–14 years who were more likely to be disabled Non-Māori aged 15–44 years who were more likely to be disabled Non-Māori aged 45–64 years who were more likely to be disabled Non-Māori aged 65+ years who were more likely to be disabled Māori aged 0–14 years who were more likely to be disabled Māori aged 15–44 years who were more likely to be disabled Māori aged 45+ years who were more likely to be disabled Non-Māori aged 0–14 years who were not more likely to be disabled Non-Māori aged 15–44 years who were not more likely to be disabled Non-Māori aged 45–64 years who were not more likely to be disabled Non-Māori aged 65+ years who were not more likely to be disabled Māori aged 0–14 years who were not more likely to be disabled Māori aged 15–44 years who were not more likely to be disabled Māori aged 45+ years who were not more likely to be disabled People were selected from each strata using systematic random sampling. The number of people selected from each of the 14 strata was determined by the need to oversample some people. These were children, Māori, and people more likely to be disabled. In total, 25,000 people were selected for the HDS. Five had to be immediately discarded because insufficient name information was provided by them in the Census. Because people were selected to be in the survey using age strata, people who had not given their age on their Census form and had not had their age found through admin data at the time of the sample selection, were unable to be selected because they were excluded from all strata (and the sampling frame). People missing either ethnic information and/or disability information from Census were not excluded but were assumed to be non-Māori and/or less likely to be disabled for the purposes of placing them into strata. There was no limit set on how many people from a particular dwelling could be selected for the HDS. Weights Of the 25,000 people selected for the 2023 HDS, 21,636 successfully completed the survey. Making 21,636 the number of valid respondents. The HDS aimed to represent the target population and not just the group who responded. To do this each valid respondent is assigned a ‘weight’ to denote how many people in the target population a single respondent represents. A person with a weight of 200 is representing 200 people in the target population. The weights for each respondent sum to the total target population. Determining the size of the weight for each respondent was done in three distinct steps: Assign an initial selection weight to account for some respondents having had a higher probability of having been selected. Carry out a non-response adjustment to reduce bias by compensating for the 3,364 people who were selected to be in the survey but did not answer it. Calibrate with the target population to make the sum of weights equivalent to the target population within specified subgroups. For the first step, each person selected into the sample was given a ‘selection weight’. Selection weights were based on both the PSU and stratum that a person belonged to because people in different PSUs and strata had different probabilities of being selected. Selection weights are calculated as the inverse probability a person had of being selected so people who came from strata that were oversampled (children, Māori, or likely to be disabled) or who came from PSUs that had a higher probability of being selected (Pacific-dense or non-rural) had lower selection weights. The second step in working out the weight for each respondent is non-response adjustment. A total of 3,364 people were in the sample but did not provide a response. Non-response in surveys can happen for several reasons including the person being overseas for the entire collection period; the person refusing to take part; the phone number and address held by Stats NZ being incorrect or out-of-date (leaving Stats NZ with no way to contact the person); or the person having serious, long-lasting health problems that meant they were too unwell to answer the survey for the entire collection period and they did not have anyone who could do a proxy interview. People who were selected for the survey but did not respond need to be represented in the survey estimates by people who did respond. To work out how best to do this, a statistical model was used. The model produced an estimate of how likely each selected person was to respond to the survey, based on their personal characteristics. This is known as their ‘response propensity’ and it was expressed as a percentage. The estimated response propensity was used to adjust the weights of people who did respond so that they represent the people who did not. The adjustment involves multiplying each individual’s selection weight by the multiplicative inverse of their estimated response propensity. Once the weights have been adjusted for non-response, the third and final step of weighting involves calibration to target population benchmarks. Benchmarks are independent counts of people in the target population. The benchmarks used in the HDS were estimates of the March 2023 population based on 2018 Census counts. This means that benchmarks used were for sex rather than gender as gender was not available from the 2018 Census. After being calibrated, the weights will sum to each set of benchmarks at the same time. Calibration was done so that survey weight totals matched the benchmark data for: Five-year age groups by sex Five-year age groups by sex for Māori ethnic group 12 regions. Calibration adjusts for coverage errors in the sampling frame and the under- or over-representation of groups in the sample. Presentation of estimates Like all sample surveys the HDS estimates are affected by both sampling error and non-sampling error. The effects of non-sampling error are not quantifiable. It arises due to respondents or interviewers misinterpreting questions, respondents giving incorrect answers, or mistakes in the recording, collecting, coding or processing of data. The impact of non-sampling errors is minimised by applying good survey techniques. Sampling error can be estimated. The estimate is a measure of the variability that occurs by chance because a sample, rather than an entire population, has been surveyed. The absolute sampling error (ASE) that is reported by Stats NZ is the margin of error in percentage points at 95 percent confidence. For example, if the estimated total proportion of people with some characteristic is 27 percent and the estimate is subject to an ASE of 2 percent, that means the 95 percent confidence interval on the estimate of 27% is plus or minus 2 percentage points (i.e. the confidence interval at 95% is 25 to 29 percent). The relative sampling error (RSE) is a percentage showing how big the ASE is compared to the estimate itself. RSE is calculated as ASE/estimate x100. In the Excel tables, estimates with an RSE of 30 percent or higher are marked with an asterisk and estimates with an RSE of 50 percent or higher are marked with two asterisks. It is recommended that estimates marked with an asterisk be used with caution. The following rules apply to all HDS estimates: • Count estimates and their attached ASEs are rounded to the nearest 1,000 when published. • Percentage estimates are rounded to 0 decimal points. The ASE for rates or percentages are rounded to 1 decimal point. This is done to reduce the amount of rounding to zero in the tables. • Percentages are always calculated using unrounded counts and exclude residual responses (i.e. ‘don’t know’ or ‘refused’). • Estimates with very few contributors are deemed both a risk to respondent confidentiality and unreliable for national statistics. Count estimates are supressed if the weighted count is less than 1,000 or if less than 6 unweighted respondents contributed to the estimate. The same rules apply for percentage estimates: percentages are suppressed if the numerator or denominator is less than 1,000 weighted or less than 6 unweighted. Suppressed estimates are marked with an ‘S’ in tables. en-NZ