Lertap 5 - Classical item and test analysis with Microsoft Excel

Lertap, the Laboratory of Educational Research and Test Analysis Package, has been around since 1973. Lertap 1 and Lertap 2 ran on mainframe computers. Lertap 3 and 4 ran on the first batch of microcomputers, including the Apple ][, the Osborne I, and the original IBM PC.

Now well into its fifth generation, Lertap runs as an Excel application. There are editions for Windows computers, and for Macintoshes. It's a solid, time-tested package used for the conventional classical analysis and scoring of cognitive tests and affective scales, with a variety of extra features, many of them graphics-based. Some user testimonials may be seen here.

Here's how it looks when used with Excel 2010:

Lertap 5 is a self-contained system; no tools other than Excel are required in order to use it. Test results are recorded in an Excel worksheet called "Data". Analysis specifications go into the "CCs" worksheet using the simple Lertap control language. Results are obtained by clicking on options found on the Lertap Excel tab, such as "Interpret", "Elmillon", and "Histograms". As options are selected and run, tables and graphs are created in a series of reports. Each report is an Excel worksheet.

By and large Lertap's reports are concise. On-screen links to context-sensitive help are available to provide detailed, web-based explanations of the terms used in the reports. Those who like supporting documents will find that Lertap has a rich variety, from its printed manual, to an extensive list of technical papers and sample datasets.

Data may be entered in a variety of ways. For example, they may be "typed" directly into the Data worksheet, or they may be brought in from a file created by a scanner. Lertap has a converter for one of the data input formats used by ASC's Iteman and Xcalibre programs, and it's able to write output files ready for subsequent input to Xcalibre, Iteman, and Bilog-MG. Of course, with its Excel base, the data in Lertap's worksheets may be directly read by programs such as SPSS and SAS.

Links to examples of Lertap in action:

• A set of web-based PowerPoint slides with an overview of system operation.
• A formative classroom chemistry quiz.
• A survey used as part of an honours student's research.
• A well-developed short mathematics test used in a large school district.
• Results from an international high-stakes achievement test (with a cut score, and venue breakouts).
• Investigating the possibility of cheating.
• Checking the performance of trial items.
• Screening tests used by an engineering school (low reliability but good classification consistency).
• Development of a mental tenacity scale (uses a principal-components analysis).
• Looking at parallel-forms reliability; a scholastic aptitude test with constructed-response items.

Features of special note:

• It's Excel. It's alive. Interactive. Colorful. Add your own charts and graphs. Copy and paste to Word and PowerPoint. Can even be fun for a rainy day.
• Allows data for up to 16,300 items and over 1,000,000 students (standard Excel 2010 column and row dimensions).
• Handles dichotomous and polytomous items; permits complex scoring patterns.
• Creates concise reports, many ready for direct printing. See, for example, the series of Stats reports.
• Two criteria may be used in item analysis: internal and external.
• Superior at flagging faulty item options.
• Includes elementary but effective data integrity check ("Freqs").
• Produces extensive graphics, including unique "packed plots".
• "DIF", differential item functioning, fully supported, with graphs.
• Group differences analyses feature ANOVAs, response plots, and box plots.
• Experimental features include 2PL IRT estimates.
• Forms complete interitem R matrix (SMCs, tetrachorics, eigenvalues).
• Easy export to Xcalibre, now one of the best IRT programs (also exports to Bilog-MG).
• Allows data sampling (to "halve and hold").
• Incorporates a cheat-checker: "response similarity analysis".
• Is programmable, allowing users to patch in their own code modules.
• Complies with all AERA/APA/NCME standards for testing, including those pertaining to the use of cut scores.
• Satisfies third-party accreditation program needs, including ANSI and NCCA.
• Support for instructors and classes. (Free class sets of Lertap often available on a limited-term basis.)
• A more complete data and item analysis system than similar products, backed by years of application in various fields and countries, including large-scale achievement assessments.
• Ever-friendly support desk 24/7 (subject to brief temporary closures when fishing).
• Inexpensive perpetual licenses. (In fact, all licenses are perpetually perpetual.)

Author's note on Edition 10 of Lertap 5

Mid-July 2012 saw the release of Lertap 5.10 for use with Excel 2010. There were three things I particularly wanted to address in this edition: providing access to more immediate on-line help for users, getting Lertap to provide more warning flags for items which may have problems, and to make "packed plots" easier to get.

All of Lertap's reports now have a small h at the top which provides a link to context-sensitive, web-based help. Thus, for example, for someone looking at a "Stats_b" report, and wondering about the meaning of its ? column, a click on the h will get the appropriate help topic to display straight away (assuming internet access; otherwise the standard in-built help may be accessed for the same information, but it's a bit more cumbersome to use). Speaking of the Stats_b report, it now includes the value of coefficient alpha at the base of the item summary lines, just above the scatterplot of item difficulty by discrimination. (You can see it here.)

I have over the years tried to make Lertap's output easier to understand. Back when we first started large achievement assessment programs, it seemed that most everyone could understand and make use of the statistics produced by Frank Baker's FORTAP program from Wisconsin, a program which provided the impetus for "DIEitem", the Spanish-language forerunner of Lertap written in 1972. For years we lived with reports which closely resembled today's "Stats_f" output. As time passed and more Lertappers emerged, it became quite clear that a more concise report would be well received. This became the "Stats_b" report. More time passed. Excel became more powerful, its charts were easier to program, and the "Stats_ul" reports, with companion quintile-like plots, emerged as an alternative, or a complement, to Stats_b. These have become extremely popular, even Iteman now has them.

Now I think there may be too much information. The output available presently may be good, and often pretty, but gee, do I (or you) really need to look at all of it?

I think perhaps not. Often I work with people who want to know what factors have served to produce a reliability figure less than expected or desired. I'm of the opinion that they do not need or even want to look at the stats for each item, especially not if it means taking in each and every item table in Stats_f, and/or the detailed response endorsement proportions seen in Stats_ul's item option tables. No. Consequently I have now trained Lertap to be a better flagger. There's a flag man or woman lurking behind each Stats report. His/her job is to add something to the report for those items which seem to have problems. The idea is that a user will now be able to scan a report, any report, be it Stats_f, Stats_b, or Stats_ul, and there will be little flags and/or special colours attached to items with potential problems.

Stats_b has always been good at raising and waving flags; it has always had that ? mark column just for this very purpose. Now Stats_f reports have flags too, and, in Stats-ul, items with bad discrimination are also flagged. Now a user can quickly spot items which may be bringing reliability down, there's not a need to necessarily study each of the item tables.

In this same vein, sort of a "gestalt" vein, Lertap's ability to make "packed" quantile plots has been considerably improved, and items with bad discrimination are even flagged in the plots (red coloring is used). Have a wee look:

Packing such as this has been available since Edition 9 (5.9.1 and later); even Edition 8 can do it. But now, in Edition 10, these packed plots are much easier to create. Not only that, but "bad" items will use red flagging. In the samples above, one of the twelve items was mis-keyed, and has been flagged. Can you spot it? (It's item 11.)

Next time someone brings some test output to look at, and two cups, glasses, or bottles of something to plonk on my desk, I'll sit with them, going through Stats_b, looking especially at its scatterplot of item difficulty and discrimination, then moving into packed plots. This output is generally easy to understand; I'll print it (in color), and use the results as a way-in to thinking about how test reliability might be improved -- with some practice, the packed plots (in particular) greatly help to hone in on item problems. The packing brings out the gestalt -- you can pretty well tell right away whether or not it's going to be possible to "fix" the test (when only a few bad response patterns are seen in the plots), or whether it'll be back to the drawing board, making recourse to texts and other references on good item writing techniques.

Student and 30-Day Trial Copy Downloads

Lertap software and documentation are exclusively distributed worldwide on behalf of Curtin University of Technology by Assessment Systems Corporation

• Demo/Student Version

  A limited Demo/Student version is available. This version implements all the functions of the commercial version, but is  limited to the analysis of 40 examinees. Note that this is different from the Student License of the full Lertap, which is in the purchase menu below.

  These are available for both PC and Mac, and may be obtained at: http://lertap.curtin.edu.au/Software.htm



• 30-Day Trial Copy for use with Windows Excel 2002 and 2003

  A fully-functioning 30-day trial copy is available for users of Windows Excel 2002 and 2003. If you decide to purchase a Lertap license, we can unlock your copy by email.

  • After downloading the file, unzip it and then double-click on SETUP.EXE.
  • If you do not have a program to unzip files, PC users can obtain one at  www.winzip.com.
  • WARNING - Lertap may not work properly unless it is installed by the installation program (SETUP.EXE) that is supplied as part of the download.
  
  

  Download the 30-day trial copy for use with Windows Excel 2002 & 2003 (15.1 MB).

  
  Note 1: please check these notes on Excel 2003 and 2007 if you’re not sure which Windows version of Excel you have.
  Note 2: the Excel 2003 version of Lertap does not support DIF, differential item functioning. Users interested in Lertap's DIF support should download either the Excel 2007 or the Excel 2010 version
  Note 3: this version will not work with Calc, the OpenOffice spreadsheet program.



• 30-Day Trial Copy for use with Windows Excel 2007

  A fully-functioning 30-day trial copy is available for users of Windows Excel 2007. If you decide to purchase a Lertap license, we can unlock your copy by email.

  • After downloading the file, unzip it and then double-click on SETUP.EXE.
  • If you do not have a program to unzip files, PC users can obtain one at www.winzip.com.
  • WARNING - Lertap may not work properly unless it is installed by the installation program (SETUP.EXE) that is supplied as part of the download.
  
  

  Download the 30-day trial copy for use with Windows Excel 2007 (13.9 MB).

  
  

  Note 1: please check these notes on Excel 2003 and 2007 if you’re not sure which Windows version of Excel you have.
  Note 2: this version will not work with Calc, the OpenOffice spreadsheet program.



• 30-Day Trial Copy for use with Windows Excel 2010 and Excel 2013

  A fully-functioning 30-day trial copy is available for users of Windows Excel 2010 and Excel 2013. If you decide to purchase a Lertap license, we can unlock your copy by email.

  • After downloading the file, unzip it and then double-click on SETUP.EXE.
  • If you do not have a program to unzip files, PC users can obtain one at www.winzip.com.
  • WARNING - Lertap may not work properly unless it is installed by the installation program (SETUP.EXE) that is supplied as part of the download.
  
  

  Download the 30-day trial copy for use with Windows Excel 2010 & 2013 (28.1 MB).

  
  Note 1: please check these notes on Excel 2003 and 2007 if you’re not sure which Windows version of Excel you have. Excel 2010 and 2013 are much like Excel 2007 for comparison purposes.
  Note 2: this version will not work with Calc, the OpenOffice spreadsheet program.



• 30-Day Trial Copy for use with Macintosh Excel X and Excel 2011

  Please write directly to the author (larry@lertap.com) to obtain a copy of the 30-day version of Lertap for use with Macintosh Excel 2011

Licenses

• Add to cart
• Ask a question about this product