The AHLI Conference on Health, Inference, and Learning (CHIL) solicits work across a variety of disciplines, including machine learning, statistics, epidemiology, health policy, operations, and economics. CHIL 2023 invites submissions focussed on significant problems affecting health. Specifically, authors are invited to submit 8-10 page papers (with unlimited pages for references) to any of the tracks described below.
To ensure that all submissions to CHIL are reviewed by a knowledgeable and appropriate set of reviewers, the conference is divided into tracks and areas of interest. Authors will select exactly one primary track and area of interest when they register their submissions, in addition to one or more sub-disciplines.
Track Chairs will oversee the reviewing process. In case you are not sure which track your submission fits under, feel free to contact the Track or Proceedings Chairs for clarification. The Proceedings Chairs reserve the right to move submissions between tracks and/or areas of interest if the Proceedings believes that a submission has been misclassified.
Important Dates
- Submissions due: Feb 3rd, 11:59 EST
- Bidding closes: Feb 7th, 11:59 PM
- Reviews due: March 3rd, 11:59 EST
- Author/Reviewer Discussion period: March 10th - March 24th
- Meta-review period: March 25th - March 31st
- Author notification: April 3rd
- CHIL conference: June 22-24
Submission Tracks
- Track 1: Models and Methods
- Track 2: Applications and Practice
- Track 3: Policy: Impact and Society
Evaluation
Works submitted to CHIL will be reviewed by at least 3 reviewers. Reviewers will be asked to primarily judge the work according to the following criteria:
Relevance: All submissions to CHIL are expected to be relevant to health. Concretely, this means that the problem is well-placed into the relevant themes for the conference. Reviewers will be able to recommend that submissions change tracks or flag submissions that are not suitable for the venue as a whole.
Quality: Is the submission technically sound? Are claims well supported by theoretical analysis or experimental results? Is this a complete piece of work or work in progress? Are the authors careful and honest about evaluating both the strengths and weaknesses of their work?
Originality: Are the tasks or methods new? Is the work a novel combination of well-known techniques? Is it clear how this work differs from previous contributions? Is related work adequately cited?
Clarity: Is the submission clearly written? Is it well organized? (If not, please make constructive suggestions for improving its clarity.) Does it adequately inform the reader? (Note: a superbly written paper provides enough information for an expert reader to reproduce its results.)
Significance: Are the results important? Are others (researchers or practitioners) likely to use the ideas or build on them? Does the submission address a difficult task in a better way than previous work? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions about existing data, or a unique theoretical or experimental approach?
Final decisions will be made by Track and Proceedings Chairs, taking into account reviewer comments, ratings of confidence and expertise, and our own editorial judgment.
Submission Format and Guidelines
Submission Site
Submissions should be made via the online submission system (Coming soon!). At least one author of each accepted paper is required to register for, attend, and present the work at the conference in order for the paper to appear in the conference proceedings.
Length and Formatting
Submitted papers must be 8-10 pages (including all figures and tables), plus unlimited pages for references. Additional supplementary materials (e.g., appendices) can be submitted with their main manuscript. Reviewers will not be required to read the supplementary materials.
Archival Submissions
Submissions to the main conference are considered archival and will appear in the published proceedings of the conference if accepted. Author notification of acceptance will be provided towards April 2023.
Preprint Submission Policy
Submissions to preprint servers (such as ArXiv or MedRxiv) are allowed while the papers are under review. While reviewers will be encouraged not to search for the papers, you accept that uploading the paper may make your identity known.
Peer Review
The review process is mutually anonymous. Please submit completely anonymized drafts. Please do not include any identifying information, and refrain from citing the authors’ own prior work in anything other than third-person. Violations to this policy may result in rejection without review.
Conference organizers and reviewers are required to maintain confidentiality of submitted material. Upon acceptance, the titles, authorship, and abstracts of papers will be released prior to the conference.
You may not submit papers that are identical, or substantially similar to versions that are currently under review at another conference or journal, have been previously published, or have been accepted for publication. Submissions to the main conference are considered archival and will appear in the published proceedings of the conference if accepted.
An exception to this rule is extensions of workshop papers that have previously appeared in non-archival venues, such as workshops, arXiv, or similar without formal proceedings. These works may be submitted as-is or in an extended form. CHIL also welcomes full paper submission that extend previously published short papers or abstracts, so long as the previously published version does not exceed 4 pages in length. Note that the submission should not cite the workshop/report and preserve anonymity in the submitted manuscript.
Open Access
CHIL is committed to open science and ensuring our proceedings are freely available.
Models and Methods:
Algorithms, Inference, and Estimation
Algorithms, Inference, and Estimation
Track Chairs
- Dr. Michael Hughes, Tufts University
- Dr. Yuyin Zhou, Univ. of California - Santa Cruz
- Dr. Jean Feng, Univ. of California - San Francisco
- Dr. Rahul Krishnan, University of Toronto
- Dr. Samantha Kleinberg, Stevens Institute of Technology
Description
Advances in machine learning are critical for a better understanding of health. This track seeks technical contributions in modeling, inference, and estimation in health-focused or health-inspired settings. We welcome submissions that develop novel methods and algorithms, introduce relevant machine learning tasks, identify challenges with prevalent approaches, or learn from multiple sources of data (e.g. non-clinical and clinical data).
Our focus on health is broadly construed, including clinical healthcare, public health, and population health. While submissions should be primarily motivated by problems relevant to health, the contributions themselves are not required to be directly applied to real health data. For example, authors may use synthetic datasets to demonstrate properties of their proposed algorithms.
We welcome submissions from many perspectives, including but not limited to supervised learning, unsupervised learning, reinforcement learning, causal inference, representation learning, survival analysis, domain adaptation or generalization, interpretability, robustness, and algorithmic fairness. All kinds of health-relevant data types are in scope, including tabular health records, time series, text, images, videos, knowledge graphs, and more. We welcome all kinds of methodologies, from deep learning to probabilistic modeling to rigorous theory and beyond.
Upon submission, authors will select one or more relevant sub-discipline(s). Peer reviewers for a paper will be experts in the sub-discipline(s) selected upon its submission, so please select your relevant disciplines judiciously.
Examples
Janizek, Joseph D., et al. "An adversarial approach for the robust classification of pneumonia from chest radiographs." Proceedings of the ACM Conference on Health, Inference, and Learning 2020.
Choi, Edward, et al. "MiME: Multilevel medical embedding of electronic health records for predictive healthcare." Advances in Neural Information Processing Systems. 2018.
McDermott, Matthew BA, et al. "Semi-supervised biomedical translation with cycle Wasserstein regression GANs." Thirty-Second AAAI Conference on Artificial Intelligence. 2018.
Shalit, Uri, Fredrik D. Johansson, and David Sontag. "Estimating individual treatment effect: generalization bounds and algorithms." Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017.
Futoma, Joseph, Sanjay Hariharan, and Katherine Heller. "Learning to detect sepsis with a multitask Gaussian process RNN classifier." Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017.
Neill, D.B., Moore, A.W., Sabhnani, M. and Daniel, K. "Detection of emerging space-time clusters." In Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining (pp. 218-227). ACM
Applications and Practice:
Investigation, Evaluation, Interpretation, and Deployment
Investigation, Evaluation, Interpretation, and Deployment
Track Chairs
- Dr. Lifang He
- Dr. Tom Pollard
- Dr. Carl Yang
- Dr. Yu Zhang
Description
The goal of this track is to highlight works applying robust methods, models, or practices to identify, characterize, audit, evaluate, or benchmark systems. Additionally, unique deployments and datasets used to empirically evaluate these systems are necessary and important to advancing practice. Whereas the goal of Track 1 is to select papers that show significant technical novelty, submit your work here if the contribution is either more focused on solving a carefully motivated problem grounded in applications or on deployments or datasets that enable exploration and evaluation of applications. Introducing a new method is not prohibited by any means for this track, but the focus should be on methods which are designed to work particularly robustly (e.g., fail gracefully in practice), scale particularly well either in terms of computational runtime or data required, work across real-world data modalities and systems, etc. Contributions will be evaluated for technical rigor, robustness, and comprehensivity. We encourage applications and practice in both traditional and emerging clinical areas (e.g., models in electronic health records as well as applications in emerging fields such as remote and telehealth, integration of omics, etc.)
Computer software submissions should describe the intended use for the software, justify the need for the software, and provide executable examples for other researchers. Software submissions should directly support a healthcare application. Examples include code for summarizing the demographics of a study cohort, deriving meaningful clinical concepts from electronic health records, and natural language processing tools specifically designed for clinical text. All computer software submissions must be open source and released under a suitable open source license. Computer software should adhere to best practices in software development where possible, including the use of unit tests, continuous integration, and diligent documentation of component design and purpose.
We welcome submissions from a wide variety of perspectives, including but not limited to: examination of robustness of ML systems to real-world dataset shift or adversarial shift, scalable and safe machine learning/inference in clinical environments, new ML tools or comprehensive benchmarks for healthcare, development of scalable systems for processing data in practice, bridging the deployment gap, remote, wearable, and telehealth, data or software packages.
Upon submission, authors will select one or more relevant sub-discipline(s). Peer reviewers for a paper will be experts in the sub-discipline(s) selected upon its submission.
Examples
Pollard TJ, Johnson AE, Raffa JD, Mark RG. "tableone: An open source Python package for producing summary statistics for research papers." JAMIA Open. 2018 May 23;1(1):26-31.
Johnson AE, Stone DJ, Celi LA, Pollard TJ. "The MIMIC Code Repository: enabling reproducibility in critical care research." Journal of the American Medical Informatics Association. 2017 Sep 27;25(1):32-9.
Chen, Irene Y., Peter Szolovits, and Marzyeh Ghassemi. "Can AI help reduce disparities in general medical and mental health care?." AMA journal of ethics 21.2 (2019): 167-179.
Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Marklund H, Haghgoo B, Ball R, Shpanskaya K, Seekins J. "Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison." arXiv preprint arXiv:1901.07031. 2019 Jan 21.
Wilson G, Aruliah DA, Brown CT, Hong NP, Davis M, Guy RT, Haddock SH, Huff KD, Mitchell IM, Plumbley MD, Waugh B. "Best practices for scientific computing." PLoS biology. 2014 Jan 7;12(1):e1001745.
Wei Q, Wang Z, Hong H, Chi Z, Feng DD, Grunstein R, Gordon C. "A residual based attention model for eeg based sleep staging." IEEE Journal of Biomedical and Health Informatics. 2020 Mar 3.
Nestor B, McDermott MB, Boag W, Berner G, Naumann T, Hughes MC, Goldenberg A, Ghassemi M. "Feature robustness in non-stationary health records: caveats to deployable model performance in common clinical machine learning tasks." In Machine Learning for Healthcare Conference, pp. 381-405. PMLR, 2019.
Finlayson SG, Bowers JD, Ito J, Zittrain JL, Beam AL, Kohane IS. "Adversarial attacks on medical machine learning." Science 363, no. 6433 (2019): 1287-1289.
Boag W, Suresh H, Celi LA, Szolovits P, Ghassemi M. "Racial disparities and mistrust in end-of-life care." In Machine Learning for Healthcare Conference, pp. 587-602. PMLR, 2018.
Henderson J, Ke J, Ho JC, Ghosh J, Wallace BC. "Phenotype instance verification and evaluation tool (PIVET): A scaled phenotype evidence generation framework using web-based medical literature." Journal of medical Internet research 20.5 (2018): e164.
Wang S, McDermott MB, Chauhan G, Ghassemi M, Hughes MC, Naumann T. "Mimic-extract: A data extraction, preprocessing, and representation pipeline for mimic-iii." In Proceedings of the ACM conference on health, inference, and learning, pp. 222-235. 2020.
Xu Y, Biswal S, Deshpande SR, Maher KO, Sun J. "Raim: Recurrent attentive and intensive model of multimodal patient monitoring data." In Proceedings of the 24th ACM SIGKDD international conference on Knowledge Discovery & Data Mining, pp. 2565-2573, 2018.
Tonekaboni S, Joshi S, McCradden MD, Goldenberg A. "What clinicians want: contextualizing explainable machine learning for clinical end use." In Machine Learning for Healthcare Conference, pp. 359-380. PMLR, 2019.
Impact and Society:
Policy, Public Health, and Social Outcomes
Policy, Public Health, and Social Outcomes
Track Chairs
- Dr. Sanja Šćepanović
- Dr. Stephen Pfohl
- Dr. Dimitrios Spathis
Description
Algorithms do not exist in a vacuum: instead, they often explicitly aim for important social outcomes. This track considers issues at the intersection of algorithms and the societies they seek to impact, specifically for health. Submissions could include methodological contributions such as algorithmic development and performance evaluation for policy and public health applications, large-scale or challenging data collection, combining clinical and non-clinical data, as well as detecting and measuring bias. Submissions could also include impact-oriented research such as determining how algorithmic systems for health may introduce, exacerbate, or reduce inequities and inequalities, discrimination, and unjust outcomes, as well as evaluating the economic implications of these systems. In other words, we invite submissions tackling the responsible design of AI applications for healthcare and public health. System design for the implementation of such applications at scale is also welcome, which often requires balancing various tradeoffs in decision-making. Submissions related to understanding barriers to the deployment and adoption of algorithmic systems for societal-level health applications are also of interest. In addressing these problems, insights from social sciences, law, clinical medicine, and the humanities can be crucial.
We welcome papers from but not limited to following areas of interest: responsible AI for health, fairness, equity, ethics and justice, policy, public health, and societal impact of algorithms, interpretability, system design for implementation of ML at scale, regulatory frameworks, tools for the adoption of ML, evaluation of bias in legal and/or health contexts, and human-algorithm interaction.
Upon submission, authors will select one or more relevant sub-discipline(s). Peer reviewers for a paper will be experts in the sub-discipline(s) selected upon its submission.
Examples
Bolukbasi, Tolga, et al. "Quantifying and reducing stereotypes in word embeddings." arXiv preprint arXiv:1606.06121 (2016).
Obermeyer, Ziad, et al. "Dissecting racial bias in an algorithm used to manage the health of populations." Science 366.6464 (2019): 447-453.
Kleinberg, Jon, and Sendhil Mullainathan. "Simplicity creates inequity: implications for fairness, stereotypes, and interpretability." Proceedings of the 2019 ACM Conference on Economics and Computation. 2019.
Zink, Anna, and Sherri Rose. "Fair regression for health care spending." Biometrics 76.3 (2020): 973-982.
Yang, Wanqian, et al. "Incorporating interpretable output constraints in Bayesian neural networks." Advances in Neural Information Processing Systems 33 (2020): 12721-12731.
Bhatt, Umang, et al. "Explainable machine learning in deployment." Proceedings of the 2020 conference on fairness, accountability, and transparency. 2020.
Pierson, Emma, et al. "An algorithmic approach to reducing unexplained pain disparities in underserved populations." Nature Medicine 27.1 (2021): 136-140.
Seyyed-Kalantari, Laleh, et al. "Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations." Nature medicine 27.12 (2021): 2176-2182.
Panch, Trishan, Heather Mattie, and Leo Anthony Celi. "The “inconvenient truth” about AI in healthcare." NPJ digital medicine 2.1 (2019): 1-3.