Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability

In July 2023, New York City became the first jurisdiction globally to mandate bias audits for commercial algorithmic systems, specifically for automated employment decisions systems (AEDTs) used in hiring and promotion. Local Law 144 (LL 144) requires AEDTs to be independently audited annually for race and gender bias, and the audit report must be publicly posted. Additionally, employers are obligated to post a transparency and opt-out notice with the job listing.

In this study, 155 student investigators recorded 391 employers’ compliance with LL 144 and the user experience for prospective job applicants. Among these employers, 18 posted audit reports and 13 posted transparency notices. These rates could potentially be explained by a significant limitation in the accountability mechanisms enacted by LL 144.

Since the law grants employers substantial discretion over whether their system is in scope of the law, a null result cannot be said to indicate non-compliance, a condition we call "null compliance." Employer discretion may also explain our finding that nearly all audits reported an impact factor over 0.8, a rule of thumb often used in employment discrimination cases. We also find that the benefit of LL 144 to ordinary job seekers is limited due to shortcomings in accessibility and usability. Our findings offer important lessons for policy-makers as they consider regulating algorithmic systems, particularly the degree of discretion to grant to regulated parties and the limitations of relying on transparency and end-user accountability.

The peer-reviewed, final version of this research can be read in open access here: https://dl.acm.org/doi/10.1145/3630106.3658998

于2023年7月，纽约市成为全球首个强制要求对商业算法系统进行偏见审计的司法管辖区，尤其是针对用于招聘和晋升的自动化就业决策系统（AEDT）。地方性法律144号（LL 144）规定，AEDT需每年接受独立审计，以评估种族和性别偏见，且审计报告需对外公开。此外，雇主有义务在职位列表中发布透明度和选择退出通知。在本研究中，155名学生调查员记录了391家雇主对LL 144的遵守情况以及对求职者的用户体验。在这些雇主中，有18家发布了审计报告，13家发布了透明度通知。这些比率可能反映了地方性法律144号（LL 144）实行的问责机制存在重大局限性。该法律赋予雇主对其系统是否属于法律适用范围的较大自主权，因此，无法将无结果视为不遵守，我们称之为“零合规”状态。雇主的自主权也可能解释了为何几乎所有审计报告都指出影响因子超过0.8，这在就业歧视案件中是一个常用的经验法则。我们还发现，由于可访问性和可用性的不足，LL 144对普通求职者的益处有限。我们的研究结果为政策制定者在考虑监管算法系统时提供了重要启示，特别是对受监管方赋予多大自主权的程度以及对依赖透明度和最终用户问责制的局限性的认识。经同行评审的最终研究版本可在以下链接中免费阅读：https://dl.acm.org/doi/10.1145/3630106.3658998