The What/When/How/Why of Measuring Neutrals - ADRhub - Creighton NCR2024-03-28T09:17:05Zhttp://www.adrhub.com/forum/topics/the-what-when-how-why-of-measuring-neutrals?feed=yes&xn_auth=noThank you, Jo, for inviting m…tag:www.adrhub.com,2014-11-10:4905899:Comment:628992014-11-10T07:16:44.152ZKatherine Newcomerhttp://www.adrhub.com/profile/KatherineNewcomer
<p>Thank you, Jo, for inviting me to participate. While I think that it would be helpful to evaluate the Neutral, I am concerned about the issue of bias based on numerical information alone. I find that I agree with Ben that such a percentage would not properly consider the nature of the diverse fact patterns that the Neutral evaluates within the context of requirements of a particular forum. Beyond that concern, I think that Neutrals could derive great benefit from feedback and an…</p>
<p>Thank you, Jo, for inviting me to participate. While I think that it would be helpful to evaluate the Neutral, I am concerned about the issue of bias based on numerical information alone. I find that I agree with Ben that such a percentage would not properly consider the nature of the diverse fact patterns that the Neutral evaluates within the context of requirements of a particular forum. Beyond that concern, I think that Neutrals could derive great benefit from feedback and an evaluative process. It would provide an opportunity for self-reflection and hopefully lead to better decisions. However, this data should be used cautiously. While it should certainly be provided to the Neutrals, the idea of it being generated for investigative reporters is somewhat chilling. Furthermore, this data should not be provided to the parties in the dispute as it would more than likely be used as a pretext for rehashing disagreements. </p> Good afternoon! I'm sorry to…tag:www.adrhub.com,2014-11-09:4905899:Comment:631132014-11-09T17:55:40.059ZDale Ann Iversonhttp://www.adrhub.com/profile/DaleAnnIverson
<p>Good afternoon! I'm sorry to be so late to this great discussion. Many, many good observations and I didn't want to pipe in until I had time to read them all, albeit not as carefully as I would have liked.</p>
<p>Ben's reply begins to get at some of what I've been thinking. In arbitrating on-line, I try to be a Jiminy Cricket of sorts, sitting on my own shoulder and scrutinizing my biases as I move through a stream of cases. I'm watching those numbers, alright, and I have asked myself,…</p>
<p>Good afternoon! I'm sorry to be so late to this great discussion. Many, many good observations and I didn't want to pipe in until I had time to read them all, albeit not as carefully as I would have liked.</p>
<p>Ben's reply begins to get at some of what I've been thinking. In arbitrating on-line, I try to be a Jiminy Cricket of sorts, sitting on my own shoulder and scrutinizing my biases as I move through a stream of cases. I'm watching those numbers, alright, and I have asked myself, "What's going on here?" Like Ben, I think there are perhaps other things going on besides neutral bias when there are a number of decisions coming down on the same side. Besides the standards for the forum that Ben refers to, there may also be factors at play around the parties and how they participate in the forum. For example, where one type of party regularly fails to "tell their story" at all, or fails to tell it in ways that support their position under the standards, decisions can look slanted.</p>
<p>BUT, all that is to say that doing this kind of evaluation of neutrals seems to me a great step toward continuing improvement of the entire program, as long as evaluators are communicating with each other and with program directors and looking for patterns that suggest other ways for program improvement in addition to addressing neutral bias. Input from neutrals is probably also key.</p> Thank you, Jo, for inviting m…tag:www.adrhub.com,2014-11-07:4905899:Comment:629392014-11-07T20:18:40.138ZBen Schusterhttp://www.adrhub.com/profile/BenSchuster
<p>Thank you, Jo, for inviting me to participate. I'll begin by stating that I agree with many of the sentiments posted above. In particular, I will opine that it is difficult to analyze a Neutral's potential bias based solely on the aggregate percentage of that Neutral's decisions in favor of the claimant or respondent. Not only does this percentage fail to take into account the nature of the fact patterns put before a Neutral, but it also fails to account for how a forum's standards are…</p>
<p>Thank you, Jo, for inviting me to participate. I'll begin by stating that I agree with many of the sentiments posted above. In particular, I will opine that it is difficult to analyze a Neutral's potential bias based solely on the aggregate percentage of that Neutral's decisions in favor of the claimant or respondent. Not only does this percentage fail to take into account the nature of the fact patterns put before a Neutral, but it also fails to account for how a forum's standards are "slanted". It cannot be assumed that a forum is designed to produce "50/50" results even if that percentage intuitively appeals to our sense of fairness. The criminal justice system is designed to err on the side of "over-exoneration" of defendants because the cost of mistakenly convicting an innocent is thought to outweigh the cost of exonerating a guilty person. The bias is "baked into" the criminal justice system through the requirement that a guilty verdict be unanimous; we should examine the biases that are "baked into" the standards governing all forums before automatically attributing these biases to the Neutrals applying the standards.</p>
<p>All this is to say, I don't think it does much good to reveal aggregate data to parties to the dispute. Parties are uniquely predisposed to spotting bias where none exists, and as I detailed above, I think the appearance of bias in a forum is a nuanced issue that is ripe for misinterpretation. Regulators should certainly have the aggregate data at their disposal, but they should understand the purpose behind the forums that they're regulating. When done correctly, examining the biases that appear in a forum can be a wonderful method to get at the true purpose of the forum - why it exists, whom it protects, etc. However, beginning that examination with the assumption that any biases are those of the Neutral would be a mistake. </p> I don't believe that any data…tag:www.adrhub.com,2014-11-07:4905899:Comment:629382014-11-07T19:44:41.387ZAnn Garofolinihttp://www.adrhub.com/profile/AnnGarofolini
<p>I don't believe that any data, individual or aggregate, should be shared prior to the decision. What would the reason be for sharing at that point?</p>
<p>I do believe however, that following the decision, aggregate, anonymous data should be available to parties to the dispute, and neutrals. The information shared should be factual, but very vague as to what cases it applies to.IE: Aggregate data about the time length not considered "submitted in a reasonable time frame". The aggregate data…</p>
<p>I don't believe that any data, individual or aggregate, should be shared prior to the decision. What would the reason be for sharing at that point?</p>
<p>I do believe however, that following the decision, aggregate, anonymous data should be available to parties to the dispute, and neutrals. The information shared should be factual, but very vague as to what cases it applies to.IE: Aggregate data about the time length not considered "submitted in a reasonable time frame". The aggregate data could be complied and released on a monthly basis.</p>
<p></p> Great comments, everyone. Tha…tag:www.adrhub.com,2014-11-07:4905899:Comment:627862014-11-07T19:19:30.500ZJo DeMarshttp://www.adrhub.com/profile/JoDeMars
<p>Great comments, everyone. Thanks for participating.</p>
<p></p>
<p>Today I'd like to open the discussion further.</p>
<p>Question: Should the data always be considered confidential? </p>
<p></p>
<p>If not, what information, when should it be available, and to whom?</p>
<p>Parties to the dispute, regulatory agencies, investigative reporters, others?</p>
<p>A. Prior to the decision?</p>
<div style="margin-left: 2em"><ol>
<li>Individual Neutral performance or aggregate data?</li>
<li>Other?…</li>
</ol>
</div>
<p>Great comments, everyone. Thanks for participating.</p>
<p></p>
<p>Today I'd like to open the discussion further.</p>
<p>Question: Should the data always be considered confidential? </p>
<p></p>
<p>If not, what information, when should it be available, and to whom?</p>
<p>Parties to the dispute, regulatory agencies, investigative reporters, others?</p>
<p>A. Prior to the decision?</p>
<div style="margin-left: 2em"><ol>
<li>Individual Neutral performance or aggregate data?</li>
<li>Other? Please provide examples or suggestion</li>
</ol>
</div>
<p></p>
<p>B. Following the decision?</p>
<p> 1. Individual Neutral performance or aggregate data?</p>
<p> 2. Other? Please provide examples or suggestions</p>
<p></p> Jo, thanks for the invite to…tag:www.adrhub.com,2014-11-07:4905899:Comment:627832014-11-07T19:09:25.954ZAnn Garofolinihttp://www.adrhub.com/profile/AnnGarofolini
<p><font face="Calibri"><font size="3">Jo, thanks for the invite to participate. </font> <font size="3">I do agree that neutrals should be evaluated. If they were informed of the criteria , rules, and standards, in writing,</font> <font size="3"> </font><font size="3">prior to accepting the position.</font> <font size="3"> </font><font size="3">I also think that the 5 performance issues you have laid out would be</font> <font size="3"> </font><font size="3">perfect for evaluation.…</font></font></p>
<p><font face="Calibri"><font size="3">Jo, thanks for the invite to participate. </font> <font size="3">I do agree that neutrals should be evaluated. If they were informed of the criteria , rules, and standards, in writing,</font> <font size="3"> </font><font size="3">prior to accepting the position.</font> <font size="3"> </font><font size="3">I also think that the 5 performance issues you have laid out would be</font> <font size="3"> </font><font size="3">perfect for evaluation.</font><font size="3"> </font> <font size="3">I do believe</font><font size="3"> </font> <font size="3">that customer input is important, but remember that there will be a bias depending on who “won” the case</font><font size="3"> </font> <font size="3">In</font> <font size="3"> </font><font size="3">some of the examples I can think of, I’m not sure that the “losing” customer would be able to provide a fair analysis of the neutra’ls performance. Many times they are only seeing their side of the issue.</font> <font size="3"> </font></font></p>
<p><font face="Calibri"><font size="3">I also think that periodic feedback to neutrals is important for job growth and improvement. Maybe the data that is gathered could be shared </font> <font size="3">anonymously with all neutrals, so that everyone gets to see the “big picture” which could help with impartiality as well as logical reasoning.</font></font></p>
<p><font face="Calibri"><font size="3">I’m not sure whether I agree with judging neutrals based on the percentage of cases decided in each parties favor without knowing the facts of the individual cases, and the statements submitted. Since assignment of cases is arbitrary, it is possible that a neutral could get assigned </font> <font size="3">cases that would, on their face, be favorable to one party or the other. IE:</font><font size="3"> </font> <font size="3">cases where feedback is left 70 days after the purchase, or cases where payment was not completed by the buyer.</font></font></p>
<p></p> I believe that a manager shou…tag:www.adrhub.com,2014-11-07:4905899:Comment:629352014-11-07T18:25:41.512ZBruce Chapinhttp://www.adrhub.com/profile/BruceChapin
<p>I believe that a manager should rely upon objective standards. Trying to create or measure a subjective standard is by definition an impossible task...assuming one wants to get it "right". You can share objective performance with the neutrals exactly because it is objective. ( The first 4 categories). You can have participant in put, which is at least know to come from an advocated source, and also shared with the neutral, with or without identification of the participant.( Do not worry the…</p>
<p>I believe that a manager should rely upon objective standards. Trying to create or measure a subjective standard is by definition an impossible task...assuming one wants to get it "right". You can share objective performance with the neutrals exactly because it is objective. ( The first 4 categories). You can have participant in put, which is at least know to come from an advocated source, and also shared with the neutral, with or without identification of the participant.( Do not worry the arbitrator will know who is reporting, for better or worst, since he/she will recall the case, based on the input.) The notion that a good neutral is authenticated by a balance of outcomes is a faulty premise, and highly unlikely when dealing with consumer contracts, that have carefully, and legally, been crafted to greatly advantage the drafter. We could not possible be doing a good job if we were striving to create a " balance" of outcomes. The 5th category cannot be applied to an arbitrator, any more than it could to a judge, who we hardly expect to balance her opinions just to make pretty.</p> Jo: thanks for sharing these…tag:www.adrhub.com,2014-11-07:4905899:Comment:628502014-11-07T14:19:01.032ZPablo Cortéshttp://www.adrhub.com/profile/PabloCortes425
Jo: thanks for sharing these thought-provoking ideas on evaluating neutrals.<br />
I criteria seem very reasonable. Although, I am with Susan in that it would be important to treat neutrals in a fair manner, and so they should be informed about the criteria for renewal, it makes me wonder whether such criteria may affect their impartiality. In Jo’s example, if I were Chris, I’d try to meet the criteria, and that might influence my view, especially for the difficult cases, where I may be inclined to…
Jo: thanks for sharing these thought-provoking ideas on evaluating neutrals.<br />
I criteria seem very reasonable. Although, I am with Susan in that it would be important to treat neutrals in a fair manner, and so they should be informed about the criteria for renewal, it makes me wonder whether such criteria may affect their impartiality. In Jo’s example, if I were Chris, I’d try to meet the criteria, and that might influence my view, especially for the difficult cases, where I may be inclined to favour one of the parties if that might help me to reach the balanced outcomes required for my renewal. Also, many ethical rules state that the fees of neutrals should not depend on their outcomes –I wonder whether this criterion gets close to breaching such rules.<br />
I very much like the idea of the customer surveys. One would have to be careful in how to gather this data though. It might be also interesting to see whether the adversarial principle was respected (i.e. whether the parties had similar opportunities to present their cases) and how fair they consider the final award to be –naturally, one would expect that the loser will be less happy with the outcome than the winner, but I would be interested to know whether all the arbitrators score the same points. Today I'd like to expand the…tag:www.adrhub.com,2014-11-06:4905899:Comment:628332014-11-06T22:15:34.277ZJo DeMarshttp://www.adrhub.com/profile/JoDeMars
<p>Today I'd like to expand the discussion, and look at how dispute resolution program managers might measure neutrals.</p>
<p>Here is one approach for an arbitration process:</p>
<p>Data is collected on performance issues such as:</p>
<ol>
<li>Decision made on time</li>
<li>Decision includes an explanation of the reasoning, includes compelling factual evidence presented</li>
<li>Decision indicates the Neutral conducted a thorough review of the information presented</li>
<li>Neutral’s…</li>
</ol>
<p>Today I'd like to expand the discussion, and look at how dispute resolution program managers might measure neutrals.</p>
<p>Here is one approach for an arbitration process:</p>
<p>Data is collected on performance issues such as:</p>
<ol>
<li>Decision made on time</li>
<li>Decision includes an explanation of the reasoning, includes compelling factual evidence presented</li>
<li>Decision indicates the Neutral conducted a thorough review of the information presented</li>
<li>Neutral’s performance demonstrates compliance with rules and policies</li>
<li>Neutral’s decisions, over time, demonstrate balance; reflecting a fluctuation or swing in the decisions. In other words, in some cases the Claimant prevails, in some cases the Respondent prevails.</li>
</ol>
<p> </p>
<p>Performance is evaluated as either “pass” or “fail”.</p>
<p></p>
<p>Other programs use a customer survey at the end of the process, in which the parties to the dispute rank (on a scale of 1-5) their opinions on:</p>
<p> -the neutral was prepared for the meeting</p>
<p> -the neutral conducted a professional meeting</p>
<p> -the neutral maintained independence.</p>
<p></p>
<p>I look forward to reading your comments.</p>
<p></p> Thanks for your comments, Ann…tag:www.adrhub.com,2014-11-05:4905899:Comment:626412014-11-05T20:24:08.172ZJo DeMarshttp://www.adrhub.com/profile/JoDeMars
<p>Thanks for your comments, Anne and Susan.</p>
<p></p>
<p>I agree that collecting data and using it for management purposes to help ensure fairness is an important aspect of Administrative oversight.</p>
<p></p>
<p>Susan's point about sharing the information with the Neutral, so they can self-assess regarding possible bias, points to another valuable use of the data.</p>
<p></p>
<p>I expect this information might provide additional insights if the individual neutral is provided the data on…</p>
<p>Thanks for your comments, Anne and Susan.</p>
<p></p>
<p>I agree that collecting data and using it for management purposes to help ensure fairness is an important aspect of Administrative oversight.</p>
<p></p>
<p>Susan's point about sharing the information with the Neutral, so they can self-assess regarding possible bias, points to another valuable use of the data.</p>
<p></p>
<p>I expect this information might provide additional insights if the individual neutral is provided the data on his/her activities and, then for comparison, the data on all activities of all the neutrals as a whole.</p>
<p></p>