Episode 190 – Artificial Intelligence In The Field Of Law | Interview With Attorney Dimitrios Tsarapatsanis

Lecturer in Law at University of Sheffield and Associate at Panayotis Yatagantzidis Law Offices, Dimitrios Tsarapatsanis shares his research about judicial decision-making using artificial intelligence.

Artificially intelligent judge has been developed and it can predict the court verdicts with an amazing 79% accuracy. Listen to the podcast for more detailed take on this. Make sure to subscribe to the podcast and post a review.

 

Podcast: Play in new window | Download

Hey, this is Richard Jacobs with Speakeasy Authority Marketing and I have today Dimitrios Tsarapatsanis. He is an attorney based out of Greece that has worked in the realm of artificial intelligence and the law. He has got quite a bit of experience, various experiences so I’m going to let him describe who he is because he can do a better job of it.

Richard Jacobs: Welcome Dimitrios. How are you doing?

Dimitrios Tsarapatsanis: I’m good. Thank you very much. It’s great to be with you. Just a little bit of background as you said. I have worked as an attorney back in Athens, Greece where with my first law degree I went to Athens and then I pursued 2 post graduate degrees, one in Athens and one in Paris, France where I also got my PhD. So currently I’m working as a lecturer in New York, the University of Sheffield and that is where my research with artificial intelligence and computer sciences began. There is one in the UK and one in the US actually. I’m still handling some cases in Europe. Some of them to do with European Law and the European Court of Human Rights which would of course mean that those cases which we would contest without adequate members maybe. So thank you again very much.

Richard Jacobs: Okay. Tell me a little bit about. So you work with computer people that used an artificial intelligence machine learning algorithm to evaluate cases in a certain area of law. So go ahead, tell us about the background on the experiment and what happened, what the result was.

Dimitrios Tsarapatsanis: Okay. So the idea was this. I teamed up with Sam Bostocks in Computer Science. Two of them were based in UCL, University College of London and one in the University of Pennsylvania but actually I knew all of them because they did their BSc when I was teaching. All I did was to use a typical natural language processing model which is just one way than an algorithm breaks down text and tries to solve it’s patterns between textual features which is for easiness, words and we used judgements of the European Court of Human Rights which is an international and not a regional court in Europe which tries cases where individuals mostly make applications against their own states on the grounds that some violation of human rights contained within the convention has kind of happened. Then our idea was because those judgements are very well practiced so they are not like common law jurisdiction judgements. We have very specific factors so we begin with the facts of the case. The circumstances of course cause it but you know it’s more or less operated by facts. Then they go on with the legal arguments we practice and then we have the reasons provided by the court to hear and justify the argument and at the end the judgement will have the option. So we began by trying to focus on 3 articles of the convention which corresponded to human rights and cut down those judgements into pieces, chunks of text. Then the algorithm with about 250 cases, we used in total about 584 cases and then for the rest of the cases, we tried to use the textual features to make the algorithm depict what the outcome was just on the basis of text. What was surprising about this; the idea was that we could use this as a proxy for the data within an algorithm. To give you an idea, we didn’t have access either to the applications of the individuals to the courts because generally no one with such data is allowed without having specific permission by the court nor to the briefs by the parties but we had access to the judgements because there is a huge online database which contains all those judgements we used for textual patterns and then we tried to see which of those chunks can help predict the outcome. The surprising part was the algorithm after being trained had an average accuracy of 79% in predicting the outcome. So in a nutshell,

Richard Jacobs: Dimitrios. So you showed it the facts of the case without showing it the final judgement. The briefs, the motions etc. and you are saying the computer would figure out and it would produce a judgement that was 79% similar to what the judge actually said in the case.

Dimitrios Tsarapatsanis: Exactly. That’s what I’m saying

Richard Jacobs: That’s amazing.

Dimitrios Tsarapatsanis: Well, it’s less amazing if you think that one of the reasons that we also chose that; and I have to mention this, one of the reasons that we chose the European Court of Human Rights is that the outcome is finite. So the courts either say that if you enter such a violation, such rights have been replaced or not. So there is no possibility for example of predicting or within using it to predict whether a guy is going to get damages or whenever the court has the possibility to choose between more than 2 calamities or damages. In a nutshell, in fact, it’s an algorithm.

Richard Jacobs: So, it’s a start. It’s definitely a very auspicious start. So out of 580 some odd cases you fed the computer half the cases. How did you instruct the algorithm to understand what was going on in the text? Did you give it rules or how did it figure it out?

Dimitrios Tsarapatsanis: That’s the idea. I would never figure out what the text is. I’m not a computer scientist. I’m going to make it simpler. I’m a lawyer and I wrote the qualitative analysis part of the algorithm and the legal analysis. So the algorithm definitely does understand the words. The algorithm has a fixed dictionary and what it does is work with patterns between words, so it breaks down the text into patterns and by identifying the physical patterns between the legal words, it could relate those with outcomes. So that’s how it learns. It doesn’t really understand what the words mean and that’s the interesting part about it because it can spot patterns. For example, you know sometimes judges use without explicitly saying so or even without them thinking so.

Richard Jacobs: I know, for instance, everyone has a verbal tick or a way of speaking that repeats itself in conversation and I’ve also read and seen in relationships where people know each other, the conversational patterns tend to repeat quite often. So I could see how this could come about in court cases.

Dimitrios Tsarapatsanis: Exactly. That’s the idea. The idea was to try to spot patterns in the natural language because the technical details are most complicated but to spot patterns between words and outcomes which this kind of software does pretty well. After having fed the algorithm or after having trained, I should say, the algorithm, then to extract the outcome, then to provide different judgements and to make it predict what the outcome was. So that’s where the accuracy count came in which was 79%.

Richard Jacobs: Do you think the accuracy can go higher with more cases analyzed? Where would you go from here?

Dimitrios Tsarapatsanis: I really don’t know. Our next step and that’s what we want to do. In academia, we are very cautious. Actually we published this paper in a journal called the Big Journal of Computer Science which is available online and is open source to everyone on visit. Our ambition was only to do what some people call the “Proof of Causes”, to really show that this type of analogy was possible. The next step would be to actually try to generalize with many more; with much bigger data. So I was glad really to first come into contact after the publication of this paper with judges and other people of the European Court of Human Rights and try to fit it with real applications and briefs and motions so that we can try to test. A second thing which would be easier would be to try to work with many more cases so that’s what we will try to do.

Richard Jacobs: What are the computer guys saying? What do they believe may happen if you have a thousand cases? At what level do they think that the accuracy will go way up may be even 100%?

Dimitrios Tsarapatsanis: I really don’t have an answer to that. What I really know is that these algorithms are really good at being trained and the more cases you feed them the more they can actually spot these patterns and try to predict what the outcome was. What was interesting from a legal analysis point of view was that different chunks of text have different scores in terms of accuracy. So generally speaking and that’s interesting for people like me because I’m doing a bit of legal theory. Generally speaking, the chunks that only correspond to the facts of the case were a bit more accurate than the chunks corresponding to the legal arguments of the parties and that actually corroborates many empirical works on how judges decide cases but actually the next step would be to try to enlarge the data input and see where we are after that.

Richard Jacobs: What do you think is going to be the implication if this works? Are you going to next try to apply it to cases where it’s not just binary, where there are damages, where there are shades of right and wrong?

Dimitrios Tsarapatsanis: I’m not sure about that right now because I’m working. Actually my academic division works with the European Court of Human Rights so I’m very keen on enlarging the data and that point on but our algorithm and the article are openly accessible to everyone and we trust the people to try and position over that. So actually it was the idea more or less that was original than the techniques, the specific techniques used if I may put it this way. So people can try to use it in any possible way.

Richard Jacobs: I thought about this. For instance, in the United States, let’s say, Drunk Driving. What if you had an algorithm and you fed it 1000 drunk driving cases in one particular state and all the rulings on them, the police reports, the facts of the case and everything, I wonder if an algorithm can become a recommendation engine where it wouldn’t replace the lawyer but it would be a little advisement of if you file this motion, if you take the case down this path, you have a much higher likelihood of succeeding than if you do that, it would be an adjunct. Do you see that this possibly could become something like that in the future?

Dimitrios Tsarapatsanis: Richard what you said was spot on, I think. I really believe that this is what the future is going to look like. I really believe that. So what these things can do, I mean what these algorithms could do is actually spot patterns very easily and the big difference right now is that we had access to the enormous amounts of data between them. So yes, I’m actually seeing things like that evolving in the future and lawyers using specific passwords, lawyers correlating outcomes with specific judges, clients correlating outcome with specific lawyers and so on and so forth. Yes, I think that in about 10 to 20 years’ time, law firms will be using that sort of thing.

Richard Jacobs: I know good attorneys and good lawyers do this already. They know, Okay, judge so and so tends to do this, so I’ll do that, but this is a computer assisted way of doing it.

Dimitrios Tsarapatsanis: Exactly and it’s much more accurate and it can spot things which for various reasons, we humans cannot spot. So that’s one more thing to it. One thing we have highlighted in our analysis was that the Older Increasing Doctrinal, let’s call them that have grains and patterns in the European Court of Human Rights were more or less identified correctly by the algorithm but the algorithm also identified various other patterns that lawyers can use to do exactly that kind of work. So actually it helps them systemize the kind of knowledge we already have.

Richard Jacobs: Do you think if algorithms, if they become more widespread will actually change interpretation of the law, make it more rigid or change how the law is interpreted and unearth the need for changes to exist in the law?

Dimitrios Tsarapatsanis: I really don’t have a view on that right now. I would like to do a lot more research. I’m trying to understand how this works a lot better before answering your question. What I can tell you though is that the preliminary results from the kind of research we have done seems to suggest that at least the kind of cases that we are talking about is parts of facts that are more important than legal argument results which would be let’s say the interpretive part of this. Actually that corroborates one of the major movements in legal house in the US. The so called legal reality because you have all these great lawyers and things. At the beginning of the century, the US, saying that appellate courts, especially appellate courts decide cases on the basis of the facts of the case not on the basis of legal arguments. So yes, our algorithm corroborated that and it can spot those patterns very easily.

Richard Jacobs: Okay. Very Interesting. Before we leave this subject, any other thing that you think will come out of this? You know, intentional or unintentional, anything that it’s made you aware of or shocked you or put a new thought in your head as to what may happen in the future because of this.

Dimitrios Tsarapatsanis: Yeah. The first thing that shocked me really was the accuracy rate. I was expecting it to be much lower than that. What I believe is that this kind of analogies will surely change how. It’s already changing many other domains. The software has been used for various other kinds of projects. I’ll give you an example. The researchers I have collaborated with have actually tried to use it to independently predict people’s incomes from the kind of content that we have. They have actually done that pretty accurately. Or for example, we have tried to predict flu outbreaks from the content available. So applying it to the law is just one idea among many others. I really believe that things are evolving and changing in a way that we cannot really even imagine right now.

Richard Jacobs: That is really fascinating. Anything else in the unique nature of your work and what you do that you think will be of interest to my audience.

Dimitrios Tsarapatsanis: I have said more or less what this goal was about and how it could be used to support some kind of position in the ever going debate between the so called legal realities and legal formalities. The results seem to corroborate what legal reality among other things says. The thing to think about is that this kind of algorithm can be used to identify topics. So if you read the paper you will see that the algorithm provides scores for words and it ranks words depending on how it believes that they correlate with outcomes. So it could be at the very least an assisting tool for lawyers to try to have spots. Which words will win the day against the kind of words that will eventually lose it even though it’s pretty crude for the time being but it is evolving.

Richard Jacobs: It actually seems like it could be a 3^rd or 4^th party in the courtroom. You have the lawyers, you have the judge, sometimes you have the jury but what if you had in the future, a computer that gives you it’s version of what it thinks the outcomes should be based on the facts and the pattern matches. What if it became a 3^rd voice in the courtroom besides the judge or an addition that was relied upon in the determination of the case. Do you see that happening?

Dimitrios Tsarapatsanis: As things stand right now, I don’t see that happening. It could be used behind the scenes by various people prior to exclusively saying that they like it. I’m saying this because our idea of what a courtroom should be is a process which is reasoned, where there are various ideas that are justified, where the outcome is justified and judges give reasons for the way they decided the case and so on and so forth. So this kind of algorithm cannot work like that. It actually is completely incapable of providing reasons for the kinds of outcomes it predicts. The second thing is that the algorithm is in a sense conservative. In which sense? In the sense that it can only extrapolate from patterns that have already been spotted. So it doesn’t have, let’s call it for lack of a better word, the imagination to invent new doctrine so as to provide a new solution. It extrapolates from past patterns. So I wouldn’t say it for the time being. No one knows what the future will look like. I wouldn’t see it being the 4^th party in a courtroom but I would see it being used by both clerks of judges and lawyers just to see what’s going on.

Richard Jacobs: What about if it was the ultimate in legal research? Lawyers who offer to reference other cases, other case law to make a point. If you had a computer that had all the case law in it, tens of thousands of cases, it could be a tremendous resource for both sides to draw on in a case.

Dimitrios Tsarapatsanis: Exactly. Yeah it could be. It seems to me that this is also one way in which this algorithm could be used. It seems to me that there are some people working to get that extra knowledge already. So yes, I definitely see that happening too.

Richard Jacobs: Okay. Any other questions on the subject that I should have asked you but that I left out?

Dimitrios Tsarapatsanis: It seems to me that we can work all that which was gathered for intelligent and ambitious lawyers.

Richard Jacobs: Okay. Yeah. I really appreciate your feedback. For people to access this paper, do you have a URL? Or do you just Google something that we will find it?

Dimitrios Tsarapatsanis: Either they can Google my name, or they can Google the name of the paper which is entitled “Predicting Judicial Decisions Of The European Court of Human Rights.” I’m not sure on language processing perspective, it’s open access. They can alternatively go on pwrj.com so that would be P W R J dot com on the computer science journal because there is also, it seems to me a live biology and health sciences as well and they can easily find it. They can also find the algorithm plus the whole research history. So as our computer sciences say, it’s up to them really to develop the model further.

Richard Jacobs: For someone reading this paper, do they have to be a computer scientist to understand it or if they are an attorney, do you believe they can understand it?

Dimitrios Tsarapatsanis: The good thing about me being a lawyer and a big part of the team of researchers is that there are a lot of things that a lawyer can understand just by reading it. There is a technical section of the paper which describes the methodology and that section is a bit more demanding for computer scientists but more or less I think it’s a bit accessible to lawyers as it is.

Richard Jacobs: Okay. Alright. Very Good. I appreciate your time doing this call. It was very informative.

Dimitrios Tsarapatsanis: I have to say that you have given me the time, Richard. Thanks a lot.

Richard Jacobs: Are you open to anyone contacting you for your thoughts on this or would you rather they go to the paper?

Dimitrios Tsarapatsanis: Sure. I’m open to that. I have to discuss this especially with people growing more skeptical about the idea because many people were skeptical of me. Back in my room, people said especially lawyers that we are not going to let machines dominate us in this way.

Richard Jacobs: How would people get in contact with you if they wanted to have a conversation? What is the best way?

Dimitrios Tsarapatsanis: The easiest way is to email me my email address is
tsarapatsanis@sheffield.ac.uk

Richard Jacobs: Okay thanks a lot for coming. I really appreciate it.

Dimitrios Tsarapatsanis: A big thanks to you for inviting me.

You’ve been listening to the Secrets of Attorney Marketing Podcast from Speakeasy Authority Marketing. Call (888) 225-8594 or email Richard.Jacobs@speakeasymarketinginc.com to enhance your reputation and out-market your competitors. Make sure you subscribe to the podcast and post a review.

 

Podcast: Play in new window | Download