Documented Issues

In their January 15 response on the Feedback section of the LJ Index, Round Two, Lance and Lyons assert that HAPLR "corrected" scores based on the disclosure of faulty data by a prominent library.  They also say they won’t respond to undocumented claims.  How about documented ones?  I have documented certain issues with the correlation matrix for the LJ Index measures and the number of outliers in their Public Internet User data.  

"Correcting" Results

Lance and Lyons assert that I "corrected" their scores based on the disclosure by a prominent library that its Public Internet Use session data were incorrect.  That “prominent library” they referred to in the Feedback section did not voluntarily disclose their erroneous data.  It did so because on November 23, I questioned the data on the LJ Index site.  I noted that very visitor would have had to use a public internet terminal more than 4 times for the reported number to be correct.  I used IMLS edit checks to do so.  I did not “correct” the scores; I demonstrated how changing this one outlier measure affects the scores for ALL libraries in the over $30 Million expenditure category. 

Correlation Matrix

 LJ Index's Lance has said that a proper index consists of variables correlated between 0.60 and 0.80. The LJ Index web site rejects reference queries because the measure did not pass this test.  Attendance and public internet use ALSO fail the test and yet they are included. This is not an undocumented claim.  Anyone can go to the LJ site, download the data and prepare a correlation matrix using Excel.  I have included a Correlation Spreadsheet on this site and users are welcome to see for themselves.

Clearly only visits and circulation (correlated at 0.72) meet the test Lance defined as the test for a "proper" Index. The other correlations are all below 0.60. I do not agree with Lance’s criteria of 0.60 to 0.80, but you would think LJ would. Why violate LJ violate Lance’s rules for an Index without comment?

Outliers Noted

Lyons and Lance say that they cannot identify outliers without time consuming and costly research, but I identified them quite easily by simply dividing Public Internet Use Per Capita by Visits per Capita.  It takes but a few moments with Excel.  Column I in the table below is the result of dividing Public Internet Users per Capita by Visits per Capita and results in Public Internet Users per Visit.  The 70 libraries listed below all reported Public Internet Use sessions at a higher rate than the amount defined by the IMLS edit check criterion of 0.9 per visit.  They were also included in the LJ Index with these outlier numbers. 

IMLS has developed “edit check” parameters for the “Public Internet Terminal Use” question in the national data.  The edit checks alert state data coordinators when numbers are clearly out of range. A second level of edit checking is supposed to occur at the federal level as well.  The process did not work as intended, however.  The edit check is supposed to be triggered when nearly every visitor logs a Public Internet Terminal Use.  Somehow, the federal data retained this unlikely data despite these edit checks.   

Let's put that in perspective.  At 0.9 Public Internet Use sessions per Visit, 9 out of every 10 visitors had to use the Internet equipped computers in the library every time they visited.  According to the data in the table below, some visitors used them two to five times on every visit! 

Despite repeated requests, LJ has not responded by listing these outliers.  Here they are.

23 January 2010 

Home Ratings Order Samples Press FAQ Misc

April 2010

© haplr-index.com
Webmaster: thennen@haplr-index.com