ExamSoft Flags One-Third of California Bar Exam Test Takers For Cheating (eff.org) 82
The California Bar released data last week confirming that during its use of ExamSoft for the October Bar exam, over one-third of the nearly nine-thousand online examinees were flagged by the software. The Electronic Frontier Foundation is concerned that the exam proctoring software is incorrectly flagging students for cheating "due either to the software's technical failures or to its requirements that students have relatively new computers and access to near-broadband speeds." From the report: This is outrageous. It goes without saying that of the 3,190 applicants flagged by the software, the vast majority were not cheating. Far more likely is that, as EFF and others have said before, remote proctoring software is surveillance snake oil -- you simply can't replicate a classroom environment online, and attempting to do so via algorithms and video monitoring only causes harm. In this case, the harm is not only to the students who are rightfully upset about the implications and the lack of proper channels for redress, but to the institution of the Bar itself. While examinees have been searching for help from other examinees as well as hiring legal counsel in their attempt to defend themselves from potentially baseless claims of cheating, the California Committee of Bar Examiners has said "everything is going well" and called these results "a good thing to see" (13:30 into the video of the Committee meeting).
That is not how we see it. These flags have triggered concern for hundreds, if not thousands, of test takers, most of whom had no idea that they were flagged until recently. Many only learned about the flag after receiving an official "Chapter 6 Notice" from the Bar, which is sent when an applicant is observed (supposedly) violating exam conduct rules or seen or heard with prohibited items, like a cell phone, during the exam. In a depressingly ironic introduction to the legal system, the Bar has requested that students respond to the notices within 10 days, but it would appear that none of them have been given enough information to do so, as Chapter 6 Notices contain only a short summary of the violation. These summaries are decidedly vague: "Facial view of your eyes was not within view of the camera for a prolonged period of time"; "No audible sound was detected"; "Leaving the view of the webcam outside of scheduled breaks during a remote-proctored exam." Examinees do not currently have access to the flagged videos themselves, and are not expected to receive access to them, or any other evidence against them, before they are required to submit a response. The report goes on to say that some of these flags are technical issues with ExamSoft. For example, Lenovo laptops appear to have been flagged en masse for an issue with the software's inability to access the internal microphone.
Other flags are likely due to the inability of the software to correctly recognize the variability of examinees' demeanors and expressions. "We implore the California Bar to rethink its plans for remotely-proctored future exams, and to work carefully to offer clearer paths for examinees who have been flagged by these inadequate surveillance tools," the EFF says in closing. "Until then, the Bar must provide examinees who have been flagged with a fair appeals process, including sharing the videos and any other information necessary for them to defend themselves before requiring a written response."
That is not how we see it. These flags have triggered concern for hundreds, if not thousands, of test takers, most of whom had no idea that they were flagged until recently. Many only learned about the flag after receiving an official "Chapter 6 Notice" from the Bar, which is sent when an applicant is observed (supposedly) violating exam conduct rules or seen or heard with prohibited items, like a cell phone, during the exam. In a depressingly ironic introduction to the legal system, the Bar has requested that students respond to the notices within 10 days, but it would appear that none of them have been given enough information to do so, as Chapter 6 Notices contain only a short summary of the violation. These summaries are decidedly vague: "Facial view of your eyes was not within view of the camera for a prolonged period of time"; "No audible sound was detected"; "Leaving the view of the webcam outside of scheduled breaks during a remote-proctored exam." Examinees do not currently have access to the flagged videos themselves, and are not expected to receive access to them, or any other evidence against them, before they are required to submit a response. The report goes on to say that some of these flags are technical issues with ExamSoft. For example, Lenovo laptops appear to have been flagged en masse for an issue with the software's inability to access the internal microphone.
Other flags are likely due to the inability of the software to correctly recognize the variability of examinees' demeanors and expressions. "We implore the California Bar to rethink its plans for remotely-proctored future exams, and to work carefully to offer clearer paths for examinees who have been flagged by these inadequate surveillance tools," the EFF says in closing. "Until then, the Bar must provide examinees who have been flagged with a fair appeals process, including sharing the videos and any other information necessary for them to defend themselves before requiring a written response."
Messing with lawyers (Score:5, Funny)
Re: (Score:3)
Messing with lawyers doesn't seem a sound strategy. There are good chances they'll find at least one willing to sue them to death. Libel, privavy laws, etc.
Yes - seems to me that if they fail to sue the bar or fail in suing the bar then they deserved to fail in the bar in which case they should probably abandon the bar and just take the fail into a bar.
Re: (Score:2)
Re: (Score:2)
Messing with lawyers doesn't seem a sound strategy. There are good chances they'll find at least one willing to sue them to death. Libel, privavy laws, etc.
Uh, lawyers should be the first group in the universe keenly aware of just how easy it is for anyone to sue everyone for anything these days.
There's a good chance that hollow threats, have been hollowed out by the Greedy Kettle and the Rich pot. Careful of that banana peel you laid down, Esquire. You could slip and fall...
Re: (Score:2)
Re: (Score:2)
Messing with lawyers doesn't seem a sound strategy. There are good chances they'll find at least one willing to sue them to death. Libel, privavy laws, etc.
But they are not lawyers. They failed the bar exam so they're still peasants like the rest of us.
Re: (Score:3)
"Messing with lawyers doesn't seem a sound strategy. There are good chances they'll find at least one willing to sue them to death."
That's why they have to get them NOW, before they become lawyers.
Obligitory XKCD (Score:2, Insightful)
https://xkcd.com/2385/ [xkcd.com]
Just like this - this seems like a basic test of throwing a sample injustice in the face of these lawyers, and challenging them to work together and, if not identify and fix the underlying injustice, then at least sue them for 'damages'.
Ryan Fenton
Re: (Score:2, Insightful)
It is possible that a significant number of those flagged were in fact cheating. We see it all over when remote has replaced in-person. Take a look at what's happened to the chess world since they've moved to online-only tournaments.
Re: (Score:2)
Re: Obligitory XKCD (Score:2)
My thoughts exactly.
Cheating is rampant in colleges and universities around the world. Why would anyone expect anything different from the Bar ?
Re: (Score:2)
Or is your argument alike "Kill them all, God will know his own"? If it's o.k. with you, that if 33% of a certain population are suspected of cheating you randomly select 33% of them and have them fail the exam? The Romans had a similar tactic to suppress mutiny in the Roman Army: If you had an uprising within a formation, you had the formation marched up in order and count through them. Every t
Re: (Score:2)
"Or is your argument alike "Kill them all, God will know his own"?"
In fact, that's most possibly the underlying case. Not necessarily a searched-for situation but an "emergent property" that happens to nicely suit those with authority.
A bar is basically a monopoly-seeking structure for the profit of those within. Part of this profit comes from the scarcity of its members. This all means it's not about those that fail, but about those that pass. As long as those that pass are short in number and "good en
Re: (Score:2)
While I am also a lawyer, I am speaking from the perspective of an Economics professor here . . .'
>A bar is basically a monopoly-seeking structure for the profit of
>those within. Part of this profit comes from the scarcity of its
>members.
While this is true for many, maybe even most, such associations (control of med school admission numbers, taxis, hair dresser licensing, and so forth), to say this about law in the US is simply ignorance.
I suspect that law is different in this regard due to it bei
Re: (Score:2)
I would agree, though I find the increasing prevalence of adhesive contracts and many forms of ADR to be quite alarming. Much as it might speed things up and reduce costs to rely on such things, there is plenty of important, yet unremunerative work that ought to get done.
Some sort of overhaul for the system might be worthwhile along the model of universal health care? That is, if you have a meaningful need for legal services, they could be had at no cost to you in order to minimize tilted playing fields, b
Re: (Score:2)
"While this is true for many, maybe even most, such associations (control of med school admission numbers, taxis, hair dresser licensing, and so forth), to say this about law in the US is simply ignorance."
So your arguement is "Yes, what you say is, as a general matter, correct, but *my* particular case is the exception."
Usual argument, isn't it?
What you are saying is, at best, that in the case of lawyers' bar the check and balances are stronger than in other cases, because the wannabe lawyers will sue the
Re: (Score:2)
No, I'm saying *as an economist* that such things are the general state of things, but that, presumably for reasons peculiar to law, the *objectively measurable* outcome is different.
There is a public safety need in some cases (hair braiding being a notable exception) for competency examination. This is not the "chokepoint" in medicine (rather, it is in reduced admissions), but it is in law (near general admission).
And, still speaking as an economist, the measured externalities show that there are *too ma
Re: (Score:2)
If it's o.k. with you, that if 33% of a certain population are suspected of cheating you randomly select 33% of them and have them fail the exam?
That isn't what is happening so you are literally trying to cheat at this discussion about cheating.
Re: (Score:2)
Everything else is prejudices thrown around and a bag full of confirmation bias.
Re: (Score:3)
You're applying the wrong threshold. The real question is, are you sure ALL of the 33% cheated?
If the answer is no, it should trouble you that none of the 33% have been given an adequate opportunity to refute the accusation.
Re: (Score:3)
Perhaps, but it is unlikely that all of the ones flagged were cheating given that technical flaws in the proctoring have been identified. One would think a state BAR would understand the need to sign and date a notice that offers a hard deadline for response. One would also think that when a legal response is required, that they would need to provide more information than "we think you cheated" if they expect more than "No I didn't" in response.
Re: (Score:1)
That's very probable, but TFS also states that there were clearly technical issues for some people:
The real question (that's unlikely to get a satisfactory answer) is how many real cheaters were caught vs. how many non-cheaters were flagged erroneously. As William Blackstone once said:
Re: (Score:3)
There is nothing in the linked xkcd crap that has any relationship whatsoever to the story.
It very much is. The test taker's unfair fail ( or in the bar exam case, accusation that could lead to a fail), can be remedied by the exercising of the same skills the test was intended to evaluate. In both cases, successfully challenging the unfairness shows is an example of the test taker having at least part of the skills the original test was intended to evaluate.
skills and knowledge != resources (Score:2)
It's still an inequitable outcome, even if the students could theoretically sue.
Rich students who were "failed" will have time and money to fight it. Poor students might be working several jobs, and will have much less ability to bring resources to bear to fight.
As such, this should not be allowed to stand. It's unjust to put all the cost of righting an injustice on the victims.
--PM
Re: (Score:3)
Well, I *think* the point of the xkcd strip is that the test is that you need to hack into the department server and change your grade.
If this is correct, then I suppose the equivalent would be that you need to successfully sue the testing company to get your results corrected. In that both should be impossible requirements I suppose that's reasonable, though neither is fair, and neither would produce the kind of professionals that one would want to have.
In other words, only 1/3 took schooling to heart (Score:5, Funny)
An AI morality tale. (Score:2)
They're lawyers (Score:2, Redundant)
What did you expect?
Re: (Score:1)
Re: (Score:2)
148 is his pronoun? I don't get it. 101010 or 1000101 I can see, maybe "111 of 1001" if he's a Trekkie.
Re: (Score:2)
Re: (Score:2)
"There are 10 kinds of people, those who understand binary numbers and those who don't."
Keeping me from taking a test (Score:2)
I was taking an online class over the summer for a certification, and the test is only offered in person or proctored online. With the whole covid situation I don't want to take an in person exam (I don't even know if the testing centers are open) and with all the issues with remote proctoring I am very hesitant to go that route, especially since my one laptop with a camera is about 10 years old at this point, and I don't even think the camera works anymore.
Re: (Score:2)
This. I don't even have a webcam, I haven't needed one. The only use I have ever had for anything webcam like is video calls and that's what my phone is for.
Only a third cheated? (Score:4, Funny)
With baby lawyers involved, I'd have expected the percentage to be much higher.
Re: Only a third cheated? (Score:2)
Knowing what utter crap most software is, we can assume that it failed to detect 2/3rd of cheaters and falsely identified at least half.
I'm a software engineer, and I really hate software. We should be using computers to make our lives easier and more enriching. Not for putting up roadblocks and Orwellian fuckery.
Re: (Score:2)
If I was allowed to moderate a discussion I'm part of, I'd give you a point for that.
Will hurt poor most of all (Score:5, Interesting)
Re: (Score:2)
Re: (Score:2)
*upward mobility
Re: (Score:2)
We know that the software included facial recognition and we know that facial recognition has a higher failure rate for black people (why is an open debate).
Re: (Score:2)
Another simple thing; poorer students are more unlikely to have a dedicated silent room in which to take the exam, and far more likely to have disturbing noise in the background which will, eventually, cause them to instinctively look over their shoulder and therefore away from the camera.
Re: (Score:2)
I'm sorry to say that, in my experience, the resources necessary to acquire a law degree preclude it as an option for most (not quite all) people of limited means. Those who remain are quite likely to have access to decent computers and high-speed Internet.
Your point does stand, though, that most forms of injustice tend to disfavor the already worse off. That is one reason I advocate for liberty, justice and rule of law, as those things have been traditionally defined (not necessarily as they have recentl
2/3 is not a "vast majority" (Score:2)
While it's a majority, it's certainly not "vast". Such widespread indicates a commonplace acceptance of deceit and abuse in the profession.
Re: (Score:2)
FTFY:2/3 is not a "vast majority" (Score:2)
Such widespread indicates a commonplace acceptance of deceit and abuse in society.
Re: (Score:2)
I've worked with a lot of lawyers. In my anecdotal experience, very few will overtly break the letter of the law or of the relevant codes of ethics. There are statues with teeth, including Rule 11 of the FRCP and its equivalent in other jurisdictions, that can and will be deployed in the event of blatant or repeated violations. I've seen it happen, though not always as frequently as I'd like.
However, many will do their best to find creative ways to bend them. Sometimes in the interests of justice as t
Disabled students especially expect fun (Score:1)
I'm a diabetic. My cell phone has my blood sugar display and history from my continuous blood sugar sensor, and alerts me for blood sugar issues. Checking the status during a long exam is very useful, since my blood sugar can sometimes crash in a long exam.
Explaining that to an automated helpline which says "go to the website" and, if ever connected to a human successfully, has them say "go to the website" with an Indian or Pakistani accent is incredibly frustrating.
an ada lawsuit will force changes with that (Score:2)
an ada lawsuit will force changes with that
Re: (Score:2)
eh, do a search for "ada bar exam lawsuits", apparently there are problems in some places.
Re: (Score:2)
an ada lawsuit will force changes with that
I admit I haven't used Ada much but how can it be used to write tort software. Wouldn't Python b better?
I'm confused. (Score:2)
3190 hours of review time... (Score:2)
Lets assume you can do 1 review an hour. To get all the reviews done would take a year and a half of 1 persons time for a normal work year. Granted, a more cursory review would let them do it quicker but open themn up to more challenges. The whole thing sounds like the typical rush out a solution and hope it works and if it doesn't we'll just figure out what to do then.
I would not be surprised if this turns into a class action if the CA Bar digs their heels in. Personally, if I were running the reviews I
Re: (Score:2)
an hour each? really?
I'd guess closer to a minute for the overwhelmingly majority, which would be summarily dismissed . . .
And the time to grade the twelve hours of frantic writing per test (four three hour sessions) is *already* measured in hours . . .
This should surprise anyone (Score:1)
Re: (Score:3)
They are young and want to be lawyers. Lawyers tend to have low ethics and must be willing to lie to judges and juries to get their guilty clients off.
I'm just guessing here, but you don't know many lawyers, or work with them, do you? Being one, I know hundreds both professionally and personally, and most wouldn't tolerate or engage in unethical behavior in any form, no matter how slight. Criminal defense attorneys included. I've worked in a prosecutor's office. I respected the other side of the aisle, even if not their clients.
Lawyers do not tend to have low ethics. Some lawyers have low or none. But they're by far the exception. If you want to kn
Re: (Score:2)
I know hundreds both professionally and personally, and most wouldn't tolerate or engage in unethical behavior in any form, no matter how slight
You and I both know they don't consider lying to be unethical as long as they don't get caught.
I wouldn't do anything illegal or unethical first because it's wrong
I have no doubt you would do something illegal. I can probably troll through your past posts and find one where you say you did something illegal. As far as ethical, you probably don't have any ethics as you are a lawyer.
I see lots of bugs in software. All coders must be inept and have not only low competency,... They have no ethics. Right?
Silly human, I am not a coder. I am the person who gets called in when a coder's code has suddenly stopped working correctly and the reason has to be determined ASAP because the company is loosi
Re: (Score:2)
Not a lawyer myself but have worked with many, and I've observed the same basic bell curve I've seen elsewhere. Most are decent. Some are exceptional, and some are scoundrels. But most are decent.
I will say that when I was more closely involved with the profession, judges in the federal and state courts in my area (mostly Cleveland and Detroit) were VERY reluctant to sanction any lawyer until he or she had demonstrated repeated, egregious and willful violations of law, rules of conduct or procedure, or c
Re: (Score:2)
People who voluntarily take on six figures of un-dischargable debt in order to go into the sleaziest profession known to man (even pimps and pushers are at least honest about what they do) might consider cheating on the gateway test to their incredibly lucrative profession? Say it isn't so! I may have palpitations . . .
Oh I'd say that's not enough. (Score:2)
I doubt anyone but a few is not cheating in some way. It's The American Way(TM), after all. Gotta cheat to have the right mindset for working in a profit-maximizing organization.
Re: (Score:2)
The American Way as I always understood it is to survive and thrive through frugality, intelligence, hard work, honestly, and often a tiny bit of luck. And it's still possible for most of us, though less so for some than others, and at least arguably less so than at many points in the past.
However, as I noted above, bending the rules - as opposed to outright breaking them, which offers little additional benefit but lots of additional risk - seems common in this and many other professions. It often works
It's a feature (Score:1)
Isn't cheating a desirable trait for lawyers? Surely that correlates with being the most successful in that "profession."
Doctors or engineers would be outraged... (Score:2)
Lawyers, on the other hand? A cheating percentage that high is a job qualification, not a cause for outrage.
If these systems work, post the algorithms! (Score:2)
Why should anyone trust a blackbox with no accountability? Why aren't these students being provided the evidence and a full breakdown on how X decided Y? The lack of information and transparency in these systems is a major issue, and one that if I were still in school would be the root case for refusing to use them.
100% cheating detection rate! (Score:2)
It's easy to detect 100% of cheaters when you flag everything and ignore the false positive rate.
A Likely Target (Score:2)
My opinion (Score:1)