Law as Data pp. 443–452
DOI: 10.37911/9781947864085.17
17: The Law Search Turing Test
Authors: Michael A. Livermore, University of Virginia; and Daniel N. Rockmore, Dartmouth College
Excerpt
One of the defining features of AI has been the field’s desire and ability to articulate and then surmount increasingly sophisticated challenges toward an overarching goal of humanlevel (or more than human-level) general intelligence. The most famous of these challenges is the so-called Turing test, suggested by Alan Turing in one of the foundational papers in artificial intelligence (Turing 1950; A. P. Saygin 2000). Turing suggested that an honest benchmark of machine intelligence would be a machine capable of engaging in natural language conversation with a person in such a way that the person can’t tell the difference between the computer and another human. With recent advances, Google’s new assistant technology appears to have pulled off the feat, at least within the limited domain of setting up appointments over the phone.
Turing’s test was an adaptation of a party pastime of the day called the Imitation Game, in which the challenge was for an interlocutor to carry on a conversation by passing notes with a hidden player with the goal of determining the gender of the challenger. This framing of a test or competition soon gave rise to other kinds of Turing tests. Recently, these challenges have become less about appearing human than about vanquishing the human. Such “Terminator tests” (as one of the authors here has called them) have included mastering board games: first checkers, then chess, then Jeopardy!, and, most recently, Go and poker.
Each of these challenges offers its own obstacles. The game of poker, for example, requires fairly sophisticated models of human psychology and not just mastery of a set of formal rules. When the program DeepStack showed that it could beat professional-level poker players at heads-up no-limit Texas Hold’em, it showed that machine code could represent models of fairly complex human behavior, such as bluffing (Moravčík et al. 2017). When IBM’s Watson beat Jeopardy! champion Ken Jennings, it demonstrated sophisticated natural language processing in addition to encyclopedic knowledge of a variety of fields of human knowledge and culture (pop and otherwise). The complexity of the game Go and its long history of human play offered another set of challenges that were surmounted by DeepMind’s AlphaGo program, which beat Go world champion Ke Jie in 2017. Moving even further, DeepMind developed a new algorithm, referred to as AlphaZero, that learns the game “solely through self-play reinforcement learning, starting from random play, without any supervision or use of human data” (Silver et al. 2017). This new algorithm has exhibited a level of creativity in its play that is so sophisticated that expert players have characterized its gameplay as “games from far in the future,” “Go from an alternate dimension,” “alien,” and “inhuman.”
Bibliography
A. P. Saygin, V. Akman, I. Cicekli. 2000. “Turing Test: 50 Years Later.” Minds & Machines 10 (4): 463–518.
Blei, D. M. 2012. “Probabilistic Topic Models.” Communications of the ACM 55 (4): 77–84.
Epstein, R., and R. E. Robertson. 2015. “The Search Engine Manipulation Effect (SEME) and its Possible Impact on the Outcomes of Elections.” Proceedings of the National Academy of Sciences 112 (33): E4512–E4521.
Grinstead, C. M., and J. L. Snell. 1997. Introduction to Probability. Providence, RI: American Mathematical Society.
Hannak, A., P. Sapiezynski, A. Molavi Kakhki, B. Krishnamurthy, D. Lazer, A. Mislove, and C. Wilson. 2013. “Measuring Personalization of Web Search.” In Proceedings of the 22nd International Conference on World Wide Web, 527–538. New York, NY: ACM.
Leibon, G., M. A. Livermore, R. Harder, A. Riddell, and D. Rockmore. 2018. “Bending the Law: Geometric Tools for Quantifying Influence in the Multinetwork of Legal Opinions.” Artificial Intelligence & Law 26 (2): 145–167.
Livermore, M. A., F. Dadgosari, M. Guim, P. Beling, and D. N. Rockmore. 2018. “Law Search as Prediction.” Virginia Public Law and Legal Theory Research Paper No. 2018-61. https://ssrn.com/abstract=3278398.
Moravcík, M., M. Schmid, N. Burch, V. Lisý, D. Morrill, N. Bard, T. Davis, K. Waugh, M. Johanson, and M. Bowling. 2017. “DeepStack: Expert-Level Artificial Intelligence in Heads-Up No-Limit Poker.” Science 356 (6337): 508–513.
Noble, S. U. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. New York, NY: NYU Press.
Page, L., S. Brin, R. Motwani, and T. Winograd. 1999. “The PageRank Citation Ranking: Bringing Order to the Web.” In Proceedings of the 7th International World Wide Web Conference, 161–172. Palo Alto, CA: Stanford InfoLab.
Schauer, F., and V. J. Wise. 1996. “Legal Positivism as Legal Information.” Cornell Law Review 82 (5): 1080.
Silver, D., J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, et al. 2017. “Mastering the Game of Go without Human Knowledge.” Nature 550 (7676): 354–359.
Turing, A. M. 1950. “Computing Machinery and Intelligence.” Mind 59 (236): 433–460.