Assessment Systems Corporation - Blog

I spent yesterday presenting at the Minnesota Assessment Conference.  My congratulations and thanks to the folks at the Minnesota Department of Education for again organizing a wonderful event with a huge range of topics, from technical psychometric issues all the way to practical district-level topics.  ASC's plans for upcoming conferences include the CLEAR conference (Sept.  23-25), the European AEA conference (Nov. 4-6), and the Maryland Assessment Conference (Nov. 17-19).  We are also holding a workshop on Sept. 4-5 in Tokyo, Japan.

 

I often get inquiries from around the world about where the next ASC workshop is going to take place.  Given today's economy, traveling long distances is not always an option, but assessment professionals still want to learn more about IRT and CAT.  For this reason, we have released a recording of our workshop in June 2009.  Now anyone can attend a workshop, for a fraction of the cost of attending in person.  Visit the webpage here to learn more.

 

FastTEST Web (www.fasttestweb.com) continues to be improved.  Development in July focused on the addition of a Reporting module.  Its purpose is to make test development more efficient by providing informative reports, such as a detailed assessment of the item bank to indicate where more questions are needed, or the current status of tests in your testing program.  The next addition will be computerized adaptive testing (CAT), which will allow anyone to easily create and deliver adaptive tests.

 

The new Iteman 4.1 is slated for release in August.   The most recent additions include a complete reliability analysis and a wizard-type interface for setting up multiple runs.  If you haven't looked at Iteman 4 yet, I encourage you to check it out.  It is unique in that it is not designed just for statistical analysis, but to actually produce customized reports of high enough quality to provide to stakeholders - in mere seconds.

This post finds me writing from Arnhem, The Netherlands, home of Cito and host to the IACAT 2010 conference.  I'm looking forward to hearing some of the excellent CAT research that is happening around the world.  The conference has 130 participants from approximately 45 countries - and the organization is only one year old!

 

FastTEST Web (www.fasttestweb.com) has seen several important additions over the last month.  Perhaps the most unique is an API functionality, which allows other software systems to interface efficiently.  For example, if your organization has its own test delivery engine, but wishes to utilize FastTEST Web's powerful item banking, you can now easily make use of both.  The next addition is to increase the number of reports available, both on the item bank itself and on examinee data.

 

ASC has also continued to work on the new Iteman 4.  Version 4.1 will be released soon, which includes additional output (both graphical and statistical), as well as the capability to work with polytomously scored items (rating scale or partial credit).  This version will even be able to work with mixed-format tests, such as a test composed primarily of multiple-choice items with a few open-response items at the end.  It is a free upgrade from version 4.0, so make sure to update your copy.

 

We will also soon be releasing two more updates.  ScoreAll 2.0 provides an additional level of detail in scoring conventional tests as well as updated interface.  POSTSIM 3.0 cements its spot as the foremost software for computerized adaptive testing (CAT) research by now providing hybrid simulations, a new methodology developed by David Weiss.  This methodology is especially useful because it is designed for sparse matrices, which typically occurs in CAT pilot testing.

May, 2010

As of May 5, FastTEST Web is now available to the public!  Please visit www.fasttestweb.com to see some of the features and watch a video of it in action.  We continue to add new features every day - so if there is something you would like to see in an online test development system, please contact me.

The buzz on Iteman 4 continues to grow!  It has been out for approximately a month now, and orders continue to roll in.  Iteman 4 is designed not just to analyze test data, but to produce formal reports - saving you the time of copying and pasting tables and figures for each item.  Remember that there are discounts available both for multiple users and for multiple years (e.g., purchasing 3 years up front rather than the annual renewal).

We continue to add new features to Iteman 4, including the choice of quantile plot groups, and support for polytomous and mixed-format tests.  In addition, Xcalibre is being re-engineered in the same spirit as Iteman 4.

The first IACAT election is complete; results are posted at http://www.iacat.org/news.html#elect2010.  The Board is now working on establishing a vision and future for the organization.  Of course, the most immediate future is the conference in June.  A program will be posted shortly.

After more than a year of development, Iteman 4 is now ready for release!  A second beta version was announced to our mailing list in late March.  Given the positive feedback, the complete version will be released to the public during the week of April 5.  An announcement will be sent to the mailing list with a link to the Iteman 4 webpage.  Upgrade discounts are available.

FastTEST Web has also been released to beta users, but improvements continue to be made before the first public version is released.  We are already adding enhancements requested by users, such as item import options, an embedded equation editor, and a split-screen item format (to present a reading passage on one half and the test question on the other half).  Many thanks to our partners at 4ROI (www.4roi.com) for implementing these.

We have been receiving inquiries about FastTEST Web for years, so this is highly anticipated.  What sets it apart from other systems is that it is not just a online test delivery engine; it is a test development system that is designed for measurement professionals, including certification/licensure managers, training managers, district assessment coordinators, psychometricians, and university professors.

Another workshop is now being planned: October 2010, outside Shanghai, China.  Stay tuned for more information on our website or via an email announcement.

Lately, I have been busy with the first IACAT election.  If you have not already voted, there is still some time.  Polls close on April 7.  Results will be released soon after that.  Board meetings will then be able to take place at the IACAT conference at Arnhem in June.  ASC psychometricians will be at the IACAT conference to present our research.  We hope to see you there!

I just got back from another workshop on IRT and CAT, this one at the Universita G. d'Annunzio, Chieti, Italy.  Many thanks to Aristide Saggino and Roberta Romanelli for doing such a great job organizing this workshop!  It was truly an international affair, with participants from the US, Canada, UK, Belgium, Portugal, Switzerland, Italy, Malta, Croatia, Turkey, Saudi Arabia, Iran, China, Brunei, and Korea.  Plus, besides getting to experience the Abruzzo region, many of us were lucky enough to spend a day or two visiting Rome.

 

Several participants inquired about future workshops, and the possibility of organizing one for their home country.  ASC currently has two future workshops tentatively planned: Tokyo (Sept 4-5, 2010) and Brasilia (June 2011).

 

Iteman 4

Iteman 4 is nearing release.  We are currently finishing the manual and the installation package for the program.  Once those are finished, it will be placed on the website.  Check back for news.

 

IACAT News for March

The first IACAT elections are being held in the middle of March 2010.  Visit www.iacat.org for more information.

IACAT Deadline

First thing to note this month: the deadline for proposals to present at IACAT 2010 is February 15, 2010.

 

New research

I added a good amount of research to the ASC research page, http://www.assess.com/research.php.  There are now links to 15 published articles, technical reports, white papers, and conference presentations.  Please peruse if you have not already.

 

In the works

ASC is currently in the final stages of two huge projects.  Iteman 4 provides extensive classical analysis of multiple-choice tests.  The previous version, Iteman 3.6, produced simple ASCII text output, but version 4 will produce professional reports, complete with colored figures and tables.  The other project is FastTEST Web, which will provide a web-based solution for item banking, test assembly, examinee management, and online testing.  If you are interested in either of these products, for purchase or as an advanced beta tester, please contact me at nthompson *at* assess.com.

 

CITAS

Most of the software ASC sells is targeted towards researchers and other users that are psychometrically sophisticated, and have the budgets to buy sophisticated software.  An important addition to our arsenal last year was CITAS: the classical item and test analysis spreadsheet.  CITAS is designed to provide a detailed classical psychometric analysis of your test, but with two notably differences from other software.  First, it is free, so it fits into everybody's budget.  Second, it is designed so that anyone can use it, because it requires no "running" the program by knowing what buttons to click or special codes to write.  All you have to do is type in your data (the item responses), and the output automatically populates.  This requires no expertise in either software or psychometrics, making it ideal for people on the front lines of education, such as school teachers, professors, or training directors.

One of the most important developments in psychometrics during 2009 was the founding of the International Association for Computerized Adaptive Testing (IACAT).  IACAT's mission is to encourage scholarly efforts and education regarding adaptive testing.  Learn more at www.iacat.org.

IACAT's first meeting will be in June 2010.  The call for papers (released in November 2009) is found below.

___________________________________________________________________

The International Association for Computerized Adaptive Testing (IACAT) is a nascent organization dedicated to advancing computerized adaptive testing (CAT) through research and education.  IACAT is planning to hold its first annual conference June 7-9, 2010.  The conference will take place in Arnhem, The Netherlands, and is hosted by the Research Center for Examination and Certification (RCEC: www.rcec.nl), a partnership between Cito (www.cito.com) and the University of Twente (www.universiteittwente.nl/en).

IACAT welcomes proposals for conference presentations at this time.  Presentations will be in one of two formats: a 20-minute session and a traditional poster format.  We also welcome proposals for organized symposia involving related papers and discussion.  Proposals must include title, authors, and a description of 300 words or less.  To submit a proposal, please visit the conference webpage at http://www.rcec.nl/iacat/en/.

CAT research generally falls into two categories: theoretical research on algorithms and applications of CAT.  Both types of research are welcome at the conference; please also specify the category when submitting.  Proposals will be evaluated on technical quality, practical applicability, advancement of knowledge, and perceived interest to participants.

The Conference is organized by a committee consisting of:

Clifford Donath, Donath Group, USA

Theo Eggen, Cito, University of Twente, Netherlands

Nathan Thompson, Assessment Systems Corporation,USA

David Weiss, University of Minnesota, USA

Birgit Olthof, RCEC, University of Twente, Netherlands

ASC now has a blog, as you have probably surmised if you are here!  This blog will be used to discuss both news at ASC (new software, tips on interpreting output, etc.) as well as topics of interest to the testing community in general (stories in the news, upcoming conferences, etc.).  Check back every month to see if I have posted anything of interest to you or your colleagues.  Moreover, if there is anything you would like to see discussed, you are welcome to email me at nthompson *at* assess.com.

Thanks for joining us here!