In this episode Shannon Salter breaks down the technology behind British Columbia’s Online Civil Resolution Tribunal including what it is, its successes and failures, and why designing for the public is different than designing for customers.
You can download a document outlining what Online Dispute Resolution is, as well as a document with multiple case studies on the subject here.
Shannon Salter
Shannon Salter is the Chair of the Civil Resolution Tribunal, Canada’s first online tribunal resolving small claims and condominium disputes. She is also an adjunct professor at the UBC Allard School of Law, teaching administrative law and legal ethics and professional regulation.
You can follow Shannon on Twitter and LinkedIn.
Thanks to Ruby Receptionists and Clio for sponsoring this episode!
Listen & Subscribe
To listen to the podcast, just scroll up and hit the play button (or click the link to this post if you are reading this by email).
To make sure you don’t miss an episode of The Lawyerist Podcast, subscribe now in iTunes, Stitcher, or your favorite podcast player. Or find out about new episodes by subscribing to our email newsletter.
Transcript
This transcript was prepared by Rev.com.
Aaron Street: Welcome to the Lawyerist Podcast, with Sam Glover and Aaron Street. Each week, Lawyerist brings you advice and interviews, to help you build a more successful law practice, in today’s challenging and constantly changing legal market. Now, here are Sam and Aaron.
Sam Glover: Hi, I’m Sam Glover.
Aaron Street: I’m Aaron Street. This is episode 157 of the Lawyerist Podcast, part of the Legal Talk Network. Today, we’re talking with Shannon Salter about British Columbia’s Civil Resolution Tribunal and how to build products for the public at large, similarly in the United States.
Sam Glover: Today’s podcast is brought to you by Law Pay, Fresh Books and Ruby Receptionists. We appreciate their support and we’ll tell you more about them later in the show.
Aaron Street: So, we’ve done this podcast now, for over three years and have provided lots of cool guests and content for our listeners. I feel like we probably, in giving all of that, haven’t done enough asking in return. I think we’ve got such cool ways for lawyers to engage with the broader Lawyerist community, and to help us spread the word about the great guests we have on the show, that we wanted to take a minute and make three asks of you, at our TBD law events that we do every year. We’ve developed a system of having people make unreasonable requests, with the idea that every once in a while, if you ask a group of people to do something for you that’s totally unreasonable and you shouldn’t expect them to do it, that every once in a while they will and that you can have some really great results from that. I’m not sure these requests are unreasonable, but we wanted to make them of you. We want you to be both more deeply engaged with our community, and to help us spread the word about the work we’re doing.
Here are the three things we would love for you to do, if you haven’t already. Number one is to review this podcast in iTunes or Google Play, depending on where you listen to it. Those ratings and reviews both help us have a broader reach within those platforms, but also understand what people like about this show so that we can provide more of that. If you go to Lawyerist.com/podcast, there’s a big button there to click to review the show on one of those two platforms. We would really appreciate it.
Sam Glover: Number two, we have been building out several directories, portals, with information about the kinds of technology products that lawyers need and use. We have a portal for law practice management software, virtual receptionists, credit card processors, time keeping and billing software. Here’s the thing, we really need your input on those portals. Please visit those pages and review the software or the services that you have tried and tell us what you like or don’t like about it. Your input is vital to those pages. You can just go to Lawyerist.com, click on the nav menu under topics and you’ll find those pages. Please find your software and leave a review, we’d really appreciate it.
Aaron Street: Really, not just the software you’re currently using, but any product or service you’ve ever used. Even if you don’t use it anymore, because you don’t like it or it doesn’t work for you, those reviews are useful to other lawyers, too.
Our number three ask is if you haven’t already, we want you to join our community, join the Lawyerist tribe by becoming an insider. It’s a free thing to join, but allows us to engage more deeply with you and understand the needs of your practice and your firms, that we can provide better support to you and so you can get to know other lawyers in our community. To join the insider, again, it’s free. You just go to Lawyerist.com/insider and you can check out. You’ll get some access to free downloads, an invitation to a free Facebook group of some of our favorite people and some other stuff. It’s free and we’d really love to have you join the Lawyerist tribe, as an insider.
Sam Glover: So those are our asks and we’d really appreciate it. Doing this podcast is a lot of fun and this is a great way that you can thank us for it. We’re going to hear briefly from Alexis Martin Neely, who is going to make her case for why most small firm lawyers should consider launching estate planning practices and after that sponsored interview, we’ll have my conversation with Shannon Salter, which I think you’ll love.
Alexis Neely: My name is Alexis Neely. I’ve been in private practice as a lawyer since 2000. I went out on my own in 2003, after working at Munger Tolles & Olsen, as an associate attorney for three years. I realized that the traditional law business model is broken. From 2003 to 2006, I created a new law business model, serving families and small business owners in a new way and built a million dollar practice, while I was also raising my little kids and going through divorce and in my fourth year, worked in that practice just two to three days a week, still bringing in a $1 million of revenue for that year, while I wrote a best selling book on legal planning for families and started doing a bunch of TV and really realized that I needed to teach this model to other lawyers, which is what I do today.
Sam Glover: Great. Alexis, you wanted to tell people, that estate planning is the perfect practice area for most lawyers. I’m a little skeptical, because you say, even if lawyers have never considered it before, they should be doing estate planning, so why? Tell us why.
Alexis Neely: Yeah. There’s really two reasons, two sides to this claim. The one is the impact that you can make on your community. I know that there are so many lawyers out there, who are unfulfilled with the practice of law. They went to law school, they wanted to make a difference in their clients’ lives, they wanted to make a great living. They came out of law school and that’s just not what’s happening. They are tied to the billable hour or they can’t even get a job, or they’re working way too much or they have zero control over their schedule. Estate planning … Or, by the way, they’re not feeling as if they’re really making a difference in their clients’ lives.
Estate planning actually solved all of those problems, when done in the right way. Most lawyers are not doing estate planning in the right way, but when done in the right way it actually makes a huge difference in your community. It is really exciting and I totally did not get that in law school. Most of our Wills and Trust teachers made it seem really boring and dry. It’s not like that at all, when you’re doing it right. You can have complete control over your schedule, and you can make a great living while you’re doing it. You can dial it up or dial it down. You can have a small part-time practice, working from home, totally virtually. You can have a office with multiple attorneys in it, like I did. Today, I have a virtual practice, I don’t have the same office anymore. You get to decide. I think it’s the best practice area because to me, it checks all the boxes, other than going out there and being in litigation. I tried out the litigation thing and while I liked it, I didn’t like having to be in set positions until 11 o’clock at night. I didn’t like dealing with adversarial opposing council, and I didn’t like the woman that I had to be, in order to be a great litigator. I really love the woman I get to be, being an estate planning lawyer.
Sam Glover: Okay. I’m guessing the reasons that you gave for why people ought to try estate planning don’t necessarily resemble all the estate planners practices, who might be listening, or people who dabble in Wills and Trusts. What are the mistakes they’re making?
Alexis Neely: The big mistakes that most estate planning lawyers are making is that they are doing estate planning the way that we were taught in law school, which is forms and documents. You’re right, doing it that way, it does pretty much suck. Not only does it suck for you, it sucks for your clients, too. I learned this first hand when I saw a number of sales estate plans throughout my own life, starting with my own father-in-law, who spent $3,000 on an estate plan when I was in law school. He died and we were stuck dealing with the probate court and his ex-wife. I thought for sure his lawyer must have committed malpractice and then I go to work at one of the biggest law firms in the country and I find out no, this is actually common practice. Then, I survey lawyer throughout the country who are doing estate planning. I find out nope, this is common practice.
That’s when I really began to discover that most estate planning lawyers, or most lawyers who are doing Wills and Trusts, and just dabbling in it, are actually failing their clients. By the way, they’re also failing themselves, Sam. It’s not a great way to practice law. You know you’re not really making a difference. You know that what you’re providing is nothing more than somebody can go online today and do themselves on one of the big document providing services, Rocket Lawyer, Legal Zoom, there’s so many of them today. That is not a way to love your law practice, serving families or small business owners, but there is a way that you can do it. It just requires you to shift the way that you look at what you’re doing for your clients and how you’re doing it. Then, you get to serve families and business owners in your community in a way that has them give you gits, refer you to all their friends, family, clients and colleagues, be so happy to be working with you. That’s pretty rare for us, as lawyers. When you do it right, it actually feels really good to you, feels really good to your clients and lets you have control over your schedule and make as much or as little money as you want.
Sam Glover: If you’d like to learn more about how Alexis thinks you ought to be building your estate planning practice, or if you’re curious and you want to learn how, you can visit estateplanningrules.com. That’s estateplanningrules.com, to learn more and download a white paper that will introduce you to it. Thanks Alexis.
Alexis Neely: You’re so welcome.
Shannon Salter: Hello. My name is Shannon Salter. I’m the chair of the Civil Resolution Tribunal, Canada’s first online tribunal. By way of training, I’m a lawyer and have a history of sitting on administrative tribunals as a decision maker, and also previous tot hat, practicing civil litigation at a Vancouver firm.
Sam Glover: Wow, I am so excited to have you here. Quickly, shout out to Spencer Keys, who is a B.C. lawyer, who I believe you know, who listened to our podcast with Professor Ben Barton and said, “Sam, you’ve got to interview Shannon Salter,” and so here we are.
Shannon Salter: Yes, thank you Spencer.
Sam Glover: I’ve lots of questions, but I think we should probably just start with, what is the Civil Dispute Resolution Tribunal? Is it CRT, is that what we can call it?
Shannon Salter: We can call it the CRT, yes. It’s the Civil Resolution Tribunal.
Sam Glover: What is it?
Shannon Salter: The CRT is the first online tribunal in Canada. As far as we knew, the first one in the world as well. What it is, is an administrative tribunal. We’ve got hundreds of them here in Canda. Each of them have responsibility under legislation for particular kinds of disputes or decisions that need to be made. I think in the U.S., you would call these administrative courts. In Canada, they’re not courts, but they are part of the Public Justice System. The result of somebody that has a dispute, that comes to the CRT, at the end of the dispute resolution process, they get a tribunal order that’s enforceable as a court order. The CRT has jurisdiction over two kinds of disputes, right now. The first are, every day neighbor disputes, that happen in condominium buildings and the secondary of disputes are small claims, $5,000 or under. Small claims is, probably you’re familiar with, are your everyday kind of consumer disputes, debt, contract, personal injury issues, that kind of thing.
Sam Glover: The significance of the tribunal part is that this was legislatively created, not court created.
Shannon Salter: That’s right. One of the things that makes the CRT distinct, compared to other online dispute resolution projects around the world, or ODR projects, are that it is the first one that’s publicly integrated into the justice system. You see a lot of ODR projects, like Modria and others that were really the product of for profit, corporations or not for profit organizations, rather. They operated outside the Public Justice System, to provide people with an alternative to that justice system. That’s been really valuable in all kinds of ways, but in my view, really the potential to really transform the justice system, requires that online dispute resolution and modernization, happen within the Public Justice System.
Sam Glover: Small claim courts are things that we’re all familiar with. It sounds very similar, but is it purely optional or is it the kind of thing where if you want to bring a small claim, the court is going to either demand or strongly encourage you to go through the CRT first?
Shannon Salter: It is … The CRT is mandatory for the kinds of disputes within its jurisdiction. If you have a neighbor dispute in a condominium in B.C., or you have small claims dispute, $5,000 or under, you have to come to the CRT, unless the court gives you an exemption, on a case by case basis. That hasn’t really happened so far, yet. We have exclusive jurisdiction over all of those kinds of disputes, in British Columbia. That doesn’t mean that somebody who is not comfortable with technology or doesn’t have access to a computer, is forced to go online. We offer a variety of ways to engage in the dispute resolution process. We have mail and telephone based services, in addition to online services. One thing that’s really surprised us, is the overwhelming demand for online services. In a nutshell, basically nobody uses our paper forms.
Sam Glover: Wow. From the consumer/user perspective, I’m pissed off at my neighbor because he pulled a tree out of our yard or something like that, that was annoying him and so I want to sue him for the cost of the tree or something like that. How do I even find out about the CRT? Am I likely to walk into court or am I going to be googling how do I sue my neighbor? How will I even find out about it?
Shannon Salter: We know from research that where people go first to find out what their options are, is online, for the most part now. Hopefully, when you google condominium or neighbor dispute, or small claims in British Columbia, our website pops up first. We designed our website and really all of our technology to look very friendly and easy to use. We aim for about a grade six reading level, for all the content we create. Somebody who comes to our website with that kind of issue, first before they file a claim, has to go through something called the solution explorer, which is a guided pathway, or expert system, which is just a tech way of saying it’s an integrated, interactive, questionnaire. We spent a little bit of time, first, with people, or at least our computer system does, helping them to better understand their legal problem and to give them plain language, legal information up front, as well as some self-help tools like template letters that they can use to, for example, send their neighbor a letter outlining their position and offering to resolve the dispute somehow.
Sam Glover: Without even going through the CRT or any other court.
Shannon Salter: Exactly. This is important for a few reasons. One, is that we know that for most people, they will not see a lawyer for these kinds of issues. They may feel very much wronged, but in fact, may not have a legal issue. They have a problem, but not a legal issue.
Sam Glover: Even if they do, it’s way easier to just resolve it.
Shannon Salter: Exactly. The goal of the solution explorer, which is basically we ask you what your problem is, and you select from options and then that tells us what kind of information or what kind of question to ask you next. What it does is allows us to get a pretty high degree of granularity for people’s problems. If you ask people questions online, and you take their answers, it lets you give them very targeted bits of legal information, as well as really targeted tools. The template letter, I talked to you about before, that you can send your neighbor, happens at the end of having asked some questions, and received some answers from the user. That information is used to populate the template letter, for example. We want to get people those tools up front, so that ideally, as you say, they never even file a dispute with us. Even if they have to, our hope is that by having that up front legal information, they can take a more rational or reasonable position in terms of what their next step should be. Is it worth filing a claim? If it is worth filing a claim, what should my position be in the mediation phase, which we haven’t talked about yet, but is really the core of the CRT, is collaborative dispute resolution.
Sam Glover: I want to talk about that, but one thing struck me while we were talking about how people would find it, and they arrive at it. I’m wondering if you spent time thinking about how to give the message that this is an actual official thing. When I’m googling stuff, I land on web pages and one of the first thing I’m always doing with that webpage is, is this real? Let’s face it, your website looks a lot more like something I would expect to see from a fairly tech savvy startup, than from court system. When I went to visit it, I was like am I in the right place? This actually looks too slick for a court. I’m wondering if you considered that.
Shannon Salter: I’m going to take that as a compliment.
Sam Glover: I mean it as a compliment, but it’s also a funny, user, design problem that I hadn’t really thought about, which is if you’re trying to convey officialness, sometimes it can be too good looking and too slick.
Shannon Salter: This is actually something we spent a fair bit of time thinking about, especially because when somebody does file a claim with the CRT, once they determine that we are authentic, and legitimate and file a claim, what we send them is by e-mail, a dispute notice that they then have to serve on the other parties. A dispute notice, too, doesn’t look so much like a court form. It’s actually much easier to understand, it has our logo, it’s pretty clear in plain language.
Sam Glover: Which again is like that’s not what you expect from official documents.
Shannon Salter: Right, and especially if you’re receiving this by e-mail, you may well think this is not actually an official document. We spent a lot of time making sure that we get word out about this. Well before we opened, at least a couple years before we opened our doors, we spent a huge amount of time doing town hall meetings, doing media interviews, webinars with community advocates and librarians, going to all the places where people go to seek legal help and advice and making sure that those intermediaries understand the CRT and can direct people to the right place. A lot of it is an education piece. So far, surprisingly, we haven’t had too much confusion from folks about whether or not this is a “real thing,” they have to pay attention to.
Sam Glover: Thank you, I kind of loved the details about design and stuff like that, that people might not be thinking about. It’s neat that you tried to sort that out. Let me go back to where we were going next, which is okay, so somebody does decide to start a dispute, someone responds and then there’s a mediation, negotiation phase? Say more about those pieces of it. Or did I skip something important?
Shannon Salter: No, you’re exactly on the right track. I should just take a step back and say that we get a lot of attention for being an online tribunal, because I think in the legal world, or the court world, if you do something online it’s very exciting, even though it’s not particularly revolutionary in terms of modern society.
Sam Glover: Throw A.I. in there and people’s brains will explode.
Shannon Salter: The solution explorer is a very basic form of A.I., although we are looking at ways of using A.I. in different ways, as well. In my view, though, the whole online piece is secondary to our greater mission. Our mandate is to provide more accessible, inexpensive, less complicated, more timely, dispute resolution for people. We know that one of the ways to do that is to allow them to participate, wherever, however they are. Using online tools is a good way of doing that. Another good way of doing that, though, is by helping to support people to reach an agreement wherever possible. This is important because we know that people tend to be more satisfied with problems that they solve by agreement, rather than having a tribunal member, like me, or a judge even, make a decision. We also know they’re more likely to stick to their side of the bargain, if they reach an agreement, rather than even if they are subject to a court order, which is a bit surprising. We really try and support people along the way, to reach an agreement, using as few of their resources and as few as our resources as possible, in the early stages. The solution explorer that I described, that question an answer thing, is automated. It doesn’t cost the user anything and it doesn’t use any of our staff time.
The next phase, though, is a negotiation phase. That’s where the parties have filed their claim, they’ve responded and then they’re invited to enter the neutral, virtual forum, where they can basically start talking. Surprisingly, that resolves a considerable number of disputes as well, even though there are no staff members involved at this point. Just providing people with a forum where they can discuss the issue, is pretty effective in certain kinds of cases.
Sam Glover: Does that mean they can invite their opponent, as it were, in and they have a private kind of chat room?
Shannon Salter: That’s right, except we’re the ones that do the inviting, or at least our computer system does. Once a dispute is filed, our computer system will send an e-mail to both parties, inviting them to come into this neutral chat room and start to talk. We incentivize that by waiving the fee for creating an order, if they’re able to reach an agreement. Basically, it’s very low intervention on our part. They’re doing this, while they’re waiting for a mediator to be assigned. It doesn’t take any additional time in then process, but it makes the best use of these natural pauses in the dispute resolution process.
Sam Glover: I notice on the website that you have some language about abusive behavior. I imagine that on the one hand, not being face to face might be very helpful in facilitating an agreement, but I can also imagine being a person who has spent a lot of time on the internet, that it could go wrong and people can kind of be assholes to each other. Does that happen a lot?
Shannon Salter: It hasn’t happened very often. I think we all seen the comments sections for various blogs and newspapers and have that anxiety. I think a key difference there is the anonymity that sometimes encourages people to behave badly online. People are not anonymous in the CRT process. They also are aware that CRT staff are occasionally monitoring these chats. We also have, as you pointed out, codes of conduct for our participants and if they breach those, there could be consequences. They can be subject to a non-compliance order from a tribunal member, they’re participation in the process can be limited and there could be other cost consequences, as well. We have tools to make sure that people adhere to the rules of the tribunal, and treat each other respectfully. We have a zero tolerance for harassment and bullying. I’m happy to say that hasn’t happened very often. If that does happen in the negotiation phase, people start getting too heated, there is a button to report abuse and then the negotiation phase is just paused until a mediator can come in and start mediating.
Sam Glover: How long does that typically … Does the self negotiation, or self motivated negotiation go before a mediator is assigned? How many mediators are there? Is this a big pool or are a lot of things getting resolved before that?
Shannon Salter: I’d say a small percentage of cases are being resolved in the negotiation phase, but that’s its early days. We only rolled out that feature a couple of months ago. We’re experimenting, we’re doing a lot of AB testing, and we’re learning a lot about that. There is a waiting period, for a mediator to be assigned. We have about 14 or 15 mediators and we’re hiring more every few months. People do need to wait, typically about four or five weeks, for mediator to be assigned. We call them facilitators in our world. Once the facilitator is assigned, that person will go in and start communicating with the parties. It’s very, very flexible. For example, if two parties are in a dispute and one of them is only really comfortable on the telephone, and the other is sort of happy to be online, the facilitator will likely organize a telephone conference call because that’s what’s most comparable for both of the parties.
Similarly, the facilitator can accommodate all kinds of different schedules. If you have a single parent who has a day job and kids and is only really available to respond or work on a dispute in the evening and they’re in a dispute with a shift worker who has a totally different schedule, that’s fine. That’s one of the ways that using online tools offers people more accessibility and flexibility because people can go in and respond when they need to. The mediation there might happen through our online portal or through e-mail.
Sam Glover: I was going to ask you about the technology that powers this and it sounds like it is mostly a communications platform. Does that sound right?
Shannon Salter: That’s certainly one of the assumptions. It’s also a case management platform. It has this artificial intelligence component. Pulling back the curtains, the CRT software is built on salesforce.com. I’m not sure if you’re familiar with Sales Force.
Sam Glover: yeah, definitely.
Shannon Salter: For your listeners, who may not be, it’s a customer management database software. It’s designed primarily for E-retailers. If you think about it, the kind of information that yo need to collect in that context, it isn’t that much different from what a court needs to collect for a case management system. You’ve got names and addresses and file details and documents that need to be attached. You have tasks that you need to create for staff members. That’s the cloud-based software that powers the CRT. What the Ministry of Justice in British Columbia has procured to build, and this is pretty much entirely built now, is two lightweight pieces, or web applications that integrate with Sales Force. One powers that solution explorer question and answer platform I talked about earlier and the other one, as you say, has a big communications function. It includes our online intakes form, different tools for citizens to take different steps in the life of their claim, includes evidence handling, case management functions, the negotiation platform, that kind of thing.
Sam Glover: Is this something that you’re constantly developing or was this built and now you’re using it and at some point in the future, you’ll do an update?
Shannon Salter: No, we use an agile development process. It’s never going to end, probably. I think that’s a good thing.
Sam Glover: Absolutely.
Shannon Salter: We do a huge amount of user testing. I can describe our process with that, if you’d like. I think it’s different to undertake rigorous user testing, when you’re developing software in a public justice system, versus a private software company, the [inaudible 00: 26: 27] market for example. None the less, user testing is obviously extremely important and we’ve done tons of it. We keep doing tons of it. We’re really committed to continuous improvement and also ruling out software in a thoughtful kind of incremental way. We started with the solution explorer, we rolled that out in beta. We opened our doors for these neighbor disputes, with a pretty limited set of technology. As we keep moving forward and keep learning more, we develop different functionality and keep rolling that out in an agile way.
Sam Glover: So many questions, but we need to take a break, to hear from our sponsors. When we come back, I think rather than just geeking out on your design process, which I would ordinarily do, I think I want to talk about how you measure success and then talk about the implementation process, how you got this done and how someone might replicate that. We’ll be back in just a few minutes.
Aaron Street: Did you know that attorneys that accept online payments get paid 39% faster on average, than those using traditional payment methods? With Law Pay, the only payment solution offered through the ABA Advantage Program, you can accept client payments online, via e-mail or in person. No equipment needed. Visit lawpay.com/lawyerist to sign up and get your first three months free. Trust the only payment solution developed for attorneys, and recommended by 47 stat bars. Law Pay.
Being a self-employed lawyer is hard enough, which is why dealing with your day to day paperwork, on top of it all, shouldn’t have to be. Fresh Books makes ridiculously easy to use cloud-based time and billing software, that will help you work smarter, get paid faster and become more organized. With Fresh Books invoicing, you can create and send polished, professional invoices effortlessly in mere seconds. Fresh Books can set you up to receive payments online, which can seriously improve how quickly you get paid. You can track your time either by using their mobile app or your desktop, meaning you’ll always know what work you did, when you did it and who you did it for. There’s also a super handy deposit feature, so you can invoice for a payment up front, when you’re kicking off a project. To feel the full impact of how Fresh Books can change the way you deal with your paperwork, Fresh Books is offering our listeners, a 30-day free trial. To claim it, just go to freshbooks.com/lawyerist and enter Lawyerist in the how did you hear about us section.
Sam Glover: Ruby Receptionists is a live, remote, receptionist service that is dedicated to helping lawyers win clients and built trust, one happy caller at a time. From their offices in Portland, Oregon, Ruby’s friendly, professional receptionists ensure exceptional client experiences by, answering calls live, in English or Spanish, transferring calls, taking messages, collecting new client intake, addressing common questions, making outbound calls for you and more. Just like an in-house receptionist, at a fraction of the cost. More importantly, they sound like they’re sitting in your office. For a special offer, visit callruby.com/lawyerist2018 or call 844-715-7829, that’s 844-715-RUBY.
Okay, we’re back. Like I said, I’m tempted to ask you all about user testing. I am curious about that. Maybe just a couple of things about what’s different about user testing in a public environment, because you dropped that and to me that felt like a meaty thing that I wanted to dig into.
Shannon Salter: I think there are some key differences. Going back to what I said at the beginning, the private ODR software companies, or even not for profit ODR projects, have some key differences from a publicly integrated one like ours. One is that a private software company, can pick its market segment. It can decide this product is for middle-classed users who have this particular education background and that particular skill set. When you’re designing technology for the public justice system, you don’t have that luxury. It has to work for everybody. You have to account for everybody, including the most heavily barriered, marginalized user, all the way up to somebody who files 50 disputes a day.
That really drives the way that you do user testing, or at least it drives the way that we do user testing. The way we do user testing, in a whole variety of different ways, but one key continuity is that we always start with the most heavily barriered users in society. We always start with community legal advocates, who serve clients who may be low income, may have mental health issues, may have addiction issues, may not speak English as a first language, may have a physical disability. We have a very generous roster of advocates who are the first step, whenever we design anything new. This could be technology, like the negotiation platform, but it also can include things like our forms, our rules, a new policy on fee waivers, for example. We always start with them. They always are always very generous with their feedback. We make a pile of changes based on their feedback.
Then, we move on to mainstream public testing. We recruit members of the public to test all of these things as well and give us feedback and then we make a ton of changes there. We end with the lawyers, that we test with. Other, more sophisticated, actors in the justice system.
Sam Glover: Really, lawyers don’t actually have much of a role to play in this.
Shannon Salter: They don’t, and our assumption is that if it works for the most heavily barriered members of our society, it will absolutely work for lawyers.
Sam Glover: That sounds a lot like Microsoft’s accessible design principals, which is exactly what you just stated. If you solve the hardest problems by designing for the most challenged people, the rest of it will pretty much take care of itself.
Shannon Salter: I think that’s true. That certainly worked for us and it’s worked, not only to design really simple software and simple forms and simple rules, but it also means that we’ve been able to build a lot of trust with people who haven’t always been well-served by public sector technology. Sometimes it felt quite disenfranchised by it, in fact. That’s been really valuable for us as well. We managed to rub off a lot of the rough edges of the technology before we even start coding. One of the key lessons we learned through watching the solution explorer, is that it’s really tempting, even in an agile process to say, well we’re just going to test that in beta. We’ll wait until beta to figure that out.
Beta is pretty far along. It’s much less expensive and you get much closer to the mark, if you start testing in the conceptual design phase, early with the people who are likely to have the most difficulty. That leads to much more sustainable, usable technology later on.
Sam Glover: It sounds like the order of your testing also, reflects reality. I was having a conversation with a friend, an old family friend, who’s a judge recently. He was expressing frustration about constantly having to deal with pro-say parties. We know each other well enough, that I kind of exclaimed at him, I mean, come on. Only 20% of the people you see are represented. That’s not your core demographic. You need to be building your justice system around the 80% who aren’t represented. I’m a lawyer and I’m an advocate for lawyers and I love lawyers, but I’m also interested in access to justice. I try to advocate for that. It sounds like your system takes reality into consideration, that most of the people using this are not going to be lawyers. Lawyers really ought to be a secondary consideration when you’re designing something meant for the public like this.
Shannon Salter: This is my view, and I absolutely share your perspective on that. I find it very frustrating when lawyers talk about self-represented people as though they are an obstacle to the justice system. If we could just get them out of the way, we can get on with the practice of law.
Sam Glover: Give them a lawyer, any lawyer and then we’ll know what to do with them.
Shannon Salter: Right. As you recognize, that’s not the reality. It’s increasingly not the reality. Moreover, the justice system actually belongs to those people and they’re entitled to use it and it’s not their fault that they come into courthouses and don’t understand the rules of the Latin or the order, or even where to go. Those are design problems. It’s not their fault, they’re not there voluntarily most of them. They wouldn’t chose necessarily, to do this. We know it’s extremely stressful. Our view is, isn’t it our responsibility to build the justice system around the people, to whom it belongs, to go to where they are, to understand that people have a bundle of abilities and skills and challenges and limitations and to account for all of that, while bringing the justice system to them. Sometimes of the physical place, we have big rural areas in British Columbia, as you do in many states in the U.S. as well. Physically, it’s hard to get to the same courthouse for some people.
There’s other challenges, as well, as we’ve discussed. That’s exactly our perspective, is that the justice system belongs to the public. It’s our job, as a public justice entity, to build it around their needs and to make it accessible. That’s what we do through human-centered design.
Sam Glover: If it’s not clear to listeners, who are private lawyers and are wondering if I’ve just thrown them under the bus, this is the exact same design process that you need to go through in your own practices. As Shannon said a minute ago, a private firm gets to choose their target demographic, which private lawyers do, too. Then, you have to design for it. You take your demographic as you find it and yo have to build something for them. I’m now at risk of spending the next hour geeking out with you about design and things.
Shannon Salter: I would definitely go with you, down that rabbit hole.
Sam Glover: Let me switch gears and ask you about success. Especially because of the way you’re doing this, I’m confident that you have been setting targets and trying to assess whether you’re getting to them. When you started out, how were you going to measure success and how has it measured up?
Shannon Salter: It’s an interesting question because it’s a bit of a moving target. When you have a mandate, as we do, to provide dispute resolution services that are more affordable, more timely, more accessible, more proportionate, all of those things. You can think of many different ways that you might measure that. Some of the constants have been, what is the time to resolution? How long does it take? How much does it cost for a person to get there? How complicated is it? How satisfied are people with the process? Do people think the process is fair? How often does the court overturn one of our decisions on appeal? We haven’t talked about the adjudicated phase but if mediation fails, then a tribunal member makes a binding determination and that’s appealable to the courts.
All of those things can be measured and we are measuring all of those things, as well as frankly a million other more granular things as well. Our evaluation framework is highly detailed. I think the key indicators … One of the most important key indicators for me, is how do people feel when they’re going through the process? If you’re designing around the public, and the key is that they are confident in the process and that they think that hey have been treated fairly and they understand what they’ve been asked to do, I think we’re making a pretty big leap forward, in terms of access to justice. We collect all kinds of qualitative data and quantitative data. Some of the qualitative data we collect are through user satisfaction surveys. This is not something that’s typically done in courts or tribunals, here. We ask people, who have gone through the process, not whether or not they agree with the decision, that’s a question for the court on appeal. We ask some questions like, “Did you think that you were treated fairly? Were you treated with respect by CRT staff members? Did you understand what you were being asked to do? How did the technology work for you?”
These kinds of questions … And our satisfaction ratings are really high. So far, about 90% of people say that they feel that they’ve been treated fairly. About 80% of people say that they found the process clear and understandable. I would hazard that that can bare as pretty favorably to the way people tend to feel, going through the typical court process.
Sam Glover: Oh come on, it’s not even close.
Shannon Salter: I’m being diplomatic.
Sam Glover: Nobody feels that good about going through the court system.
Shannon Salter: I guess my point is, the technology is great because it allows you to collect all kinds of data and we use that and analyze that. There’s a story to tell there, as well. I think just as important, is the qualitative data. That effects people’s confidence in the administration of justice. Whether they feel that in a nutshell, they got a fair shake or not, even if they disagree with the outcome. That’s been pretty heartening. We are very cautious and we’re careful to be very humble about this, because it’s always a work in progress. The good thing about all this user testing is that it keeps you really humble and our team is pretty much egoless at this point because we’ve gone back so many times to the public. We had so many cases where some small feature that we thought, collectively, as a group of lawyers and IT professionals, was going to be wonderful and everybody would adore it, only to have the public tell you it’s a disaster and it’s horrible. You should kill it right away.
Sam Glover: I’m still working on that. I’m impressed.
Shannon Salter: I think it’s something you constantly have to keep in check because the temptation is to make assumptions about what the public preference will be and also to get too attached to ideas that you think are great in theory, but just don’t work in practice. Keeping really connected to the public need, through user testing, has been really, really helpful. We keep doing that now, as well. We will never stop doing that.
Sam Glover: Are you able to say anything about the potential burden on the justice system that CRT may have taken up?
Shannon Salter: I can tell you in rough strokes, that when we assumed jurisdiction over small claims disputes, $5,000 and under, in B.C. on June 1st of last year, we anticipated through statistics, that that would take out about 40% of the small claims courts work. In other words, small claims are heavily concentrated at the lower end of the monitoring spectrum. This is born out so far, through our analytics. About 40% of those cases are now taken out of the court system and are being handled through the CRT. That’s important because it allows the court system then, to devote those resources to reducing wait times for criminal trials and family trials, frankly cases where the interests at stake, are pretty important.
Sam Glover: If someone were to try to get something, like the CRT implemented in their own, say, state or county, where would you suggest they start? I know this is legislative so maybe you just start knocking on doors of your legislators. I’m curious how you would recommend people start trying to get something like this done?
Shannon Salter: That’s an interesting question. I seen it start to happen in different ways and different jurisdictions around the world. I mentioned, I think we were on the forefront, but it’s exciting to see that there are projects now, in all kinds of jurisdictions and some of them are being driven by the government, as it was, as with the case here in British Columbia. A number of them are being driven by the judiciary itself. You see that in the courts in England and Wales. You see it in the courts in your jurisdiction in Utah, the Utah State courts are hosting an ODR, small claims project. That was really spawned within the court system.
It can happen, I think, in different ways, but I think the key is having a strong champion, who has decision making authority, who is committed to seeing it through, despite various setbacks and road bumps. It’s never a smooth process. I think the other key important thing there is change management. We spent a huge amount of time, on change management. I think that might be easier as ODR becomes more acceptable and known and has more of a track record anyway. For us, we had to spend a lot of time talking to stakeholders, talking to lawyers, talking to the public, helping them to really understand what the vision is and alleviate some of their anxieties about what it might mean for the justice system.
If your listeners are interested, the National Center for State Courts, Joint Technology Initiative, just released two reports on ODR. One is a set of case studies around the United States and other jurisdictions. The other is a set of recommendations for jurisdictions that are looking at instituting an ODR process. I would recommend both of those reports to you. Not just … Full disclosure, I did sit on the committee that helped write them, but I think they are a solid starting point.
Sam Glover: Very cool. We’ll try to include those links in the show notes. Shannon, thank you so much for being with us today. I really enjoyed hearing about the CRT and how it has been developed, the role that you had in designing it and bringing it to the public, so thank you so much.
Shannon Salter: Thank you very much, Sam. I’ve enjoyed it.
Aaron Street: Make sure to catch next week’s episode of the Lawyerist podcast, by subscribing to the show in your favorite podcast app. Please leave a rating to help other people find our show. You can find the notes for today’s episode on lawyerist.com/podcast.
Sam Glover: The views expressed by the participants are their own and are not endorsed by Legal Talk Network. Nothing said in this podcast is legal advice for you.
Podcast #157: Online Dispute Resolution & Public Interest Design, with Shannon Salter was originally published on Lawyerist.com.
source https://lawyerist.com/podcast-157-shannon-salter/
No comments:
Post a Comment