New York City Housing and Vacancy Survey (NYCHVS)

analyze the new york city housing and vacancy survey (nychvs) with r for those interested in the real estate and rental markets of the big apple, the census bureau's nyc housing and vacancy survey might be your key to the city. if you care about how many new york residents live more than one person per room (a lot), how many structures are dilapidated (a few, phew), or what rent prices run these days (cha-ching), start here. way back in 1965, new york law began requiring the enumeration of the city's heavily-regulated rental market, establishing this complex sample survey of about twenty thousand households, both occupied and vacant. nowadays it's triennial, it's publicly-downloadable, and it's free n easy to analyze with the r language. although the census bureau employs the survey administrators and produces the main how-to documents (both faq and overviews), city government actually pays the bill and gets the glory: the preliminary 2011 report with all the fun facts and the older but more complete 2008 report. the microdata include four exciti ng files: a person-level file for occupied units, a household-level file for occupied units, a household-level file for vacant units, and a household-level file for units that didn't yield an interview (solely for adjusting the vacant-unit statistics). most urban planning and policy wonks line up the occupied and vacant household-level files to calculate a vacancy rate, but depending on your mission, you might need some person-level action as well. by the way, the nyc.gov report is six months older than the latest 2011 microdata, so don't panic if your stats are off by a whisker. this new github repository contains three scripts: 2002 - 2011 - download all microdata.R download, import, save each of the four data files into a single year-specific .rda file back to 2002 bumper sticker idea for nychvs data users: if you can read this, thank a furman center for the sas import scripts. 2011 analysis examples.R load all available tables for a single year of data construct the complex sample survey object, but it's fake - see note below. run example analyses that calculate perfect means, medians, quantiles, totals replicate contract items 2008.R load all available tables for a single year of data construct the complex sample survey object, but it's fake - see note below. thoroughly explain a back-of-the-envelope cal culation for standard errors, confidence intervals, variances print statistics that match exactly - and confidence intervals more conservative than - the target replication table click here to view these three scripts for more detail about the new york city housing and vacancy survey, visit: the census bureau's nychvs homepage the new york city government's housing policy analysis homepage notes: hint for statistical illiterates: if the data point you're looking for isn't in the nyc.gov grand report, check the census bureau's copious online tables too. as described in detail in the comments of the replication script, it's impossible to exactly match the census-published confidence intervals. here's one snippet of a longer conversation about how users cannot automate the computation of standard errors (discussed at footnote five ) with the nychvs. the `segment` variable (mentioned in the e-mail) does not get released due to confidentiality concerns. either calculate them by hand with the infuriating generalized variance formula recommended in each year's source and accuracy statement (2008, 2011) or use the back-of-the-envelope method i invented that approxi mates census-published confidence intervals conservatively. when i learned that users couldn't automate the matching of census-published numbers, i tried to be a bootstrapping young lad and come up with some fancy standard error computation methodology. but it turns out that multiplying the un-adjusted errors by two gets as close to the right answer as anything else. if you're writing the final draft of a research product destined to get heavy exposure, you might have to calculate confidence intervals by hand or pay the census bureau for a custom run. but for those of us who can live with an occasional false negative in our lives, try it my way. confidential to sas, spss, stata, and sudaan users: i look at you the way new yorkers look at jersey. time to transition to r. :D对于关注大苹果城（即纽约市）房地产与租赁市场的研究者而言，借助R语言开展纽约市住房与空置调查（New York City Housing and Vacancy Survey, NYCHVS）分析时，美国人口普查局发布的该调查数据集将是您获取这座城市住房相关数据的核心抓手。若您关心每间居室居住超过1人的纽约居民人数（规模可观）、老旧破损建筑的数量（为数不多，着实令人松一口气），或是当前的租金水平（关乎收益），不妨从该调查入手。早在1965年，纽约州通过立法要求对该市监管严格的租赁市场开展摸底统计，由此建立了这项涵盖约2万户已入住与空置住房单元的复杂抽样调查。如今该调查每三年开展一次，数据可公开下载，完全免费，且便于使用R语言进行分析。尽管调查的执行由美国人口普查局负责，且其编制了主要的操作指南文档（包括常见问题解答与概况介绍），但实际经费由纽约市政府承担，相关报告也由市政府发布：其中包含各类趣味数据的2011年初步报告，以及内容更为全面的2008年历史报告。该微观数据集包含四类极具价值的文件：已入住住房单元的个人层面数据文件、已入住住房单元的家庭层面数据文件、空置住房单元的家庭层面数据文件，以及未完成访谈的住房单元家庭层面数据文件（仅用于调整空置住房单元的统计数据）。多数城市规划与政策研究人员会整合已入住与空置住房的家庭层面数据文件，以计算住房空置率；但根据您的研究目标，或许也需要使用个人层面的数据进行分析。顺带一提，纽约市政府发布的报告比最新的2011年微观数据集滞后六个月，因此若您的统计结果存在细微偏差，无需惊慌。本GitHub仓库包含三个R脚本：1. `2002-2011 - download all microdata.R`：下载、导入并将四类数据文件保存为针对单个年份的.rda格式文件，数据可追溯至2002年。 为纽约市住房与空置调查数据使用者设计的趣味车贴标语：若您能读懂此段文字，请感谢弗曼中心（Furman Center）提供的SAS导入脚本。2. `2011 analysis examples.R`：加载单年份可用数据表，构建复杂抽样调查对象（注：该对象为模拟数据，详见下文注释），运行示例分析以计算精准的均值、中位数、分位数与总量。3. `replicate contract items 2008.R`：加载单年份可用数据表，构建复杂抽样调查对象（注：该对象为模拟数据，详见下文注释），详细解释标准误、置信区间与方差的简易估算方法，输出与目标复现表格完全匹配的统计量，且置信区间比目标表格更为保守。若需查看这三个脚本的更多细节，或了解纽约市住房与空置调查的更多信息，请访问：- 美国人口普查局纽约市住房与空置调查官方主页- 纽约市政府住房政策分析官方主页注释：针对统计基础薄弱的使用者提示：若您所需的数据点未在纽约市政府的完整报告中出现，不妨查阅美国人口普查局丰富的在线数据表。如复现脚本的注释中详细说明的那样，无法完全匹配人口普查局发布的置信区间。以下是一段对话片段的节选，内容围绕使用者无法借助纽约市住房与空置调查自动计算标准误展开（详见脚注5）。由于保密考量，`segment`变量（邮件中提及的变量）并未对外公布。您可以选择两种方式：一是根据各年度的数据源与准确性声明（2008年、2011年）中提及的令人头疼的广义方差公式手动计算；二是使用我自行研发的简易估算方法，该方法可保守地近似人口普查局发布的置信区间。当得知使用者无法自动匹配人口普查局发布的统计数据后，我曾像初出茅庐的研究者那样，尝试提出一套精致的标准误计算方法。但最终发现，将未调整的标准误乘以2，就能得到最接近准确值的结果。若您正在撰写即将广泛传播的研究成果终稿，或许需要手动计算置信区间，或是付费向美国人口普查局定制数据服务。但对于我们这些能接受偶尔出现假阴性结果的人来说，不妨试试我的方法。致SAS、SPSS、Stata与SUDAAN用户：我看你们的眼神，就像纽约人看新泽西一样。是时候转向R语言了。:D