Thursday, March 13, 2014

An Assessment of the New Orleans Pretrial Services Program is No Assessment at All

I was recently forwarded a new study put out by the National Institute of Corrections.  The study was authored by Tara Boh Klute and Lori Eville and was titled “An Assessment of the New Orleans Pretrial Services Program.”  As I read this study and thought about its purpose more questions were being raised in my mind than answered.  That being said, below are the three big questions and issues that surface for me in this assessment report.

First, if this report is supposed to be an assessment of the VERA Institute’s efforts in managing the New Orleans pretrial services program than why is the National Institute of Corrections conducting the assessment?  Doesn’t the NIC focus on prison populations?   I would think that managing a post-conviction prison population is very different than managing a pre-trial jail population.  Comparing the two is like comparing apples and oranges.  It is like asking your gardener to assess how well your house is built.  You would think that an organization looking to assess a pretrial release program would at minimum have expertise and experience in what they were trying to assess.  That is very obviously not the case here.

Second, in order for an assessment to be a true assessment you must have a set of meaningful and “relevant” criteria in which you are using to measure the success or failure of what you are assessing.  Now, this study does have a set of criteria which it is using to measure the success of the VERA pretrial program, the problem however, is that the criteria being used are meaningless and irrelevant to actually measuring the success or effectiveness of the program.  This so called assessment is measuring whether or not VERA’s pretrial release program meets the criteria of the National Association of Pretrial Services Programs.  In other words, they are measuring themselves against their own philosophical standards as opposed to truly measuring their effectiveness in getting defendants back to court.  Doing a study that says that VERA is doing a great job at following NAPSA’s standards is meaningless if following NAPSA’s standards for pretrial release does nothing to guarantee a defendant’s appearance in court…and by the way, they don’t.   In fact, one doesn’t have to look hard to find a study that shows that public sector pretrial service programs are the least effective way to get a defendant back to court.  Conducting an assessment in this way would be like grading a student’s test based on whether he brought a pencil and paper to the test as opposed to whether he got the answers correct.

Lastly, this so called assessment in my opinion is a complete and utter waste of tax dollars.  The city of New Orleans has spent hundreds of thousands of taxpayer dollars on the VERA Institute and its ineffective pretrial release program.  What makes this wasteful use of tax dollars even more egregious is that there is a more effective, less expensive private sector option that guarantees its performance to the courts.  That option, of course, is commercial bail.  It makes no sense at all for the NIC to assess the effectiveness of a program such as this.  The City of New Orleans and its taxpaying residents are the ones who have paid for VERA and its failed programs.  They should be the ones demanding results and objective measurement not politically driven meaningless assessments.  When you hire someone to do a job for you, do you assess how good the job is by how well the vendor did at meeting his own personal standards or do you measure success by how well the vendor did at meeting your standards?  The people of New Orleans deserve better.  They deserve to see who the VERA institute is letting out of jail for FREE.  They deserve to see how these individuals are being assessed and ranked by VERA.  They deserve to see VERA’s true measure of success…how many defendants are showing up for court and how many are committing crimes while out on FREE release.  These are the criteria that matter; these are the criteria that should be assessed.

It is time that the public sector starts meeting the same criteria and standards that the private sector has to meet.  Instead of paying taxpayer dollars to conduct a meaningless assessment that has no purpose other than to pat an ineffective organization on the back for “trying their best” as opposed to “getting the job done” is wasteful and worse, shameful.

Remember that I am only scratching the surface here.  There is much more to talk about with this assessment, but that will be left for another blog post.  I look forward to your comments.  Below is a link to the assessment.

An Assessment of New Orleans Pretrial Services

Thursday, March 6, 2014

Pretrial Risk Assessment: Computers vs. Man, Fantasy vs. Reality

As technology has advanced over the years, many debates and questions have been asked around the topic of machine versus man.  These questions include things like can computers do a task better than a man? Can computers make thoughtful decisions that involve experience and emotion?  Can man be as effective and as accurate as a computer without the ability to process and analyze millions of data points?  While no one has been able to completely answer these questions, and trust me when I say that I do not plan on answering them in this blog post, I will, however, give my thoughts on the current man vs. machine debate that is happening in the criminal justice system.

A few weeks back I watched a “Ted Talk” video about risk assessment.  In this video, the speaker talked about risk assessments and their likeness to an approach used by the professional baseball team the Oakland Athletics called “moneyball.”  The moneyball philosophy (just in case you never saw the Hollywood movie made on the topic) is based on a deep analysis of data and statistics.  The theory is that you can use this data to tell you which players you should draft and/or acquire based on their statistical performance in certain situations.  For example, should you pay millions of dollars for a big name superstar that hits 50 home runs a year or do you spend a few hundred thousand dollars on a lesser known player that consistently hits for average with runners in scoring position.  This analytical approach to baseball has been very beneficial to the Oakland A’s organization allowing them to compete with teams that have much higher salaries and cost structures.

The question that I would like to raise is does this type of “moneyball” statistical analysis or “risk assessment” as it is referred to in the criminal justice system work like it does in baseball?  Is it a fair analysis to equate the two together and through the transitive property assume that both are effective?  I would argue no, it is not the same thing.

The concept of pretrial release is not something that can be distilled down to common statistics across a broad range of different environments.  Every defendant is different; every state is different; every county is different; every criminal act is different; and every reason a person commits a crime in the first place is different.  Criminal Justice is not that simple.  Additionally, the people who make the ultimate decisions in the criminal justice system are judges.  And judges don’t just become judges; they earn a judgeship through experience and practice.  Sitting in front of thousands of unique defendants is more valuable to understanding how criminals think and act than just looking at questionnaires and statistics.

The same goes for the bail bond industry.  Bail bond agents deal with thousands upon thousands of defendants.   These are defendants that they understand and are familiar with.  Why?  Because they have grown up and live in the communities that they serve.  They know the people; they know the families; they know the employers and the businesses.  They have a world of local knowledge and insight that no computer program or public sector pretrial risk assessment can quantify and make sense of.  Call it gut or call it experience, but the bail bond industry brings an intangible component to their own decision making process that is not merely about risk assessment but rather about something more important “risk management.”  Ultimately when a bail agent decides to financially secure a defendant’s release, they are focused on managing the risk of the defendant and ensuring they show up for court as opposed to just assessing the risk and trying to play a guessing game of whether that person will show up or commit another crime.

Based on the uniqueness of the criminal justice from locality to locality and the reality of the pretrial world, how can, as the speaker in the video explains, a computer program that spits out a 7 question risk assessment be enough information to accurately claim that a defendant is dangerous to the community and should not be let out of jail pending his trial.  In my opinion it can’t for the very reasons I have mentioned above.  Computers are not people.  They are tools that people use to make a complex task easier.  In the world of baseball that might work.  In the world of the criminal justice system and pretrial release, where public safety is at stake, I think that these computer modeled risk assessments are a shiny new toy for the public sector pretrial release community to rally around.  Why, because they have nothing else.

If statistics and data are so important, than why doesn't the public sector pretrial community just look at the data that is already in existence.  For example, there is plenty of data on pretrial release successes and failures that have been published by the Bureau of Justice Statistics.  The problem is that data doesn’t support pretrial release through a publicly funded pretrial release agency, but rather shows that release through a financially secured commercial bail bond is the most effective way to get defendants back to court. And when that happens, we all know that the public is protected, victims get a chance at justice and the system has accountability.

Summing things up, should the criminal justice system rely on a moneyball approach to pretrial release?  I guess the answer to that question is that the jury is still out, but as far as I know, the Oakland A’s still haven’t won a championship and public sector pretrial release still hasn't outperformed private sector financially secure release…so you decide. I look forward to your comments.