Paper Introduction: The Recon Approach: A New Direction for Machine Learning in Criminal Law.

Transcript

1. Learning and Educational Technologies Research Unit
   Learning and Educational Technologies Research Unit
   Monday Lunch Meeting:
   Paper Introduction
   Bell, K., Hong, J., McKeown, N., & Voss, C. (2021).
   The Recon Approach: A New Direction for Machine
   Learning in Criminal Law.
   Berkeley Technology Law Journal, 37.
   Ogata Lab. D3 Hiroyuki Kuromiya
   

   View Slide

2. Learning and Educational Technologies Research Unit
   Abstract
   2
   

   View Slide

3. Learning and Educational Technologies Research Unit
   Notice
   3
   This paper is not from the field of education, but criminal law.
   However, the concept itself seems to be applicable in education.
   

   View Slide

4. Learning and Educational Technologies Research Unit
   Contents
   1. What is RECON approach?
   2. Pilot study (example)
   3. Limitations of RECON approach
   4. Applicability in Learning Analytics
   4
   

   View Slide

5. Learning and Educational Technologies Research Unit
   1. What is RECON approach?
   5
   

   View Slide

6. Learning and Educational Technologies Research Unit
   Current machine learning approach
   6
   Current machine learning approach à predictive approach
   This person will commit a crime in X%.
   Feedback to whom decisions are made.
   

   View Slide

7. Learning and Educational Technologies Research Unit
   RECON approach
   7
   RECON approach à flash lighting on the past
   Feedback to decision makers.
   You were likely to judge poor
   people as guilty than rich people.
   

   View Slide

8. Learning and Educational Technologies Research Unit
   Two functions
   8
   Reconnaissance Reconsideration
   Identifying which factors tend to influence
   human decision making.
   Identifying particular cases that appear to
   be inconsistent with most other decisions.
   ≒ Association mining ≒ Anomaly detection
   

   View Slide

9. Learning and Educational Technologies Research Unit
   Why we need RECON approach?
   1. Humans are far from being perfect.
   • A machine learning from human judgement will likewise be imperfect.
   • Not to replace human judgement with machine learning.
   2. Provide data-driven opportunities “to make things as little
   wrong as possible”
   • Develop tools that act like a flashlight on the past
   • Bring to light potential problems in decisions that humans have already
   made.
   9
   

   View Slide

10. Learning and Educational Technologies Research Unit
    2. Pilot study (example)
    10
    

    View Slide

11. Learning and Educational Technologies Research Unit
    Context: parole decisions
    11
    In California, each year, the Board
    holds 6,000 parole hearings and
    decides whether a given individual
    to be eligible for release on parole.
    The questioning focuses on social
    history, the underlying crime, the
    record of conduct in prison, as well
    as plans for re-entry upon release.
    

    View Slide

12. Learning and Educational Technologies Research Unit
    Dataset and analysis method
    • Dataset
    • They acquired 35,105 parole hearing transcripts from 2007-2019 as well
    as demographic data like race/ethnicity.
    • Reconnaissance
    • They developed a tool to show what factors influence parole suitability
    decisions. Stakeholders are able to query the data for factors of their
    interest in response to the changing social and legislative landscape.
    • Reconsideration
    • They developed a tool that can identify the 10% of cases that are
    anomalous in the sense of having this same combination of factors but
    nevertheless resulting in a denial of parole.
    12
    

    View Slide

13. Learning and Educational Technologies Research Unit
    Reconnaissance tool
    13
    

    View Slide

14. Learning and Educational Technologies Research Unit
    Decision modeling
    • Regression analysis is often used to perform this type of task.
    However, it has at least two limitations: 1) it assumes the input
    and output variables are continuous numerical values, and 2)
    limited ability to capture the way of the decision making.
    • Nearest neighbors or decision trees are particularly well-suited
    to modeling decision making in a multi-step manner.
    14
    

    View Slide

15. Learning and Educational Technologies Research Unit
    Reconsideration tool
    15
    

    View Slide

16. Learning and Educational Technologies Research Unit
    3. Limitations of RECON approach
    16
    

    View Slide

17. Learning and Educational Technologies Research Unit
    Information extraction
    • In our developing of the RECON approach, we have focused a
    great deal on building NLP tools to identify and extract
    information from hearing transcripts.
    • To reliability extract information, NLP methods need to be
    developed to be capable of consuming long text all at once.
    17
    

    View Slide

18. Learning and Educational Technologies Research Unit
    Access to data
    • Nearly all data about a decision-making process is held by the
    agency that makes those decisions.
    • The agency has some incentive to resist disclosing data to
    researchers seeking to implement a Recon Approach.
    • However, while the Recon Approach offers a way to
    improve discretionary decision-making in the longrun, it does so
    by exposing problems with the existing way in which decisions
    are made.
    18
    

    View Slide

19. Learning and Educational Technologies Research Unit
    Regulatory capture
    • As explained above, existing members of the agency have
    an interest in minimizing the risk that the Recon Approach will
    uncover problematic issues that could disrupt the regular
    functioning of the existing agency.
    • This interest may express itself in the form of granting access to
    only selective data points. It may also express itself in granting
    access to data only on the condition that any resulting research
    must be reviewed and approved by the agency prior to
    publication.
    19
    

    View Slide