Assessment Systems Corporation - Blog

Here are some brief updates and news for April 2012: new website and updates to Xcalibre 4, CATSim, and FastTest Web!

New ASC Website

To reflect ASC's growing stature in the testing industry, we have released a new website that better reflects our new role as a comprehensive, world-class testing organization.  In addition to massively expanding our psychometric software capabilities, we now offer full consulting services, a network of secure testing centers, and software to completely manage the entire assessment process.  Check out the new site at www.assess.com!  We have also upgraded our new webstore (shopping cart system) and added a sleek, modern layout to be more customer-friendly; visit the store at http://www.assess.com/xcart/home.php. In addition, we are in the process of integrating Google Wallet functionality for enhanced credit card security and convenience. 

ASC presents workshop to SADC in Botswana

ASC presented a workshop on item response theory (IRT) and computerized adaptive testing (CAT) to member countries of the Southern African Development Council.  The workshop took place March 4-9 in Botswana, and provided five days of background and hands-on instruction regarding the application of these modern methods to solving issues in educational assessment, with a focus on ASC's landmark software, Iteman 4 and Xcalibre 4.  Such a combination of ASC's consulting services and leading-edge software can help your organization move to the next level; contact solutions@assess.com to learn more.

FastTEST Web updates

FastTest Web (FTW), our platform for item authoring and internet-based assessment, is being upgraded almost continually to stay at the forefront of testing technology.  In March, we released functionality that allows tests to be published as a public link, ideal for low stakes exams or pilot testing, as an alternative to the high-security registration process currently in place.  This approach is integrated with new functionality for managing demographic information on examinees; you can request that examinees enter such information before starting the test.  Additionally, we rebuilt the item authoring tool to be more user-friendly and WYSIWYG.  As always, we've also released a number of small enhancements.  The next major release will include an advanced Asset Manager for multimedia and the capability for testlets, groups of items that share a reading passage or other common stimulus.

Xcalibre 4 updated

ASC's cutting-edge software for item response theory has been substantially updated.  This is a free update for current license holders, and includes several bug fixes as well as improved functionality:

• User can specify bounds for scaled scores
• Lz person fit statistic now computed
• Test response function (TRF) values are now output
• Chi-square item fit statistics enhanced
• Subgroup analysis based on DIF group membership
• Rebuilt ranking of ability estimates to be much faster
• Observed item response function (fit line) now includes multi-key items
• Many more small improvements!

Current users can upgrade at our support page; new users can download a free version here.  Don't forget that graduate students can obtain free licenses from the APM student grants program!

CATSim updated

CATSim, the software that provides the research necessary for computerized adaptive testing (CAT) has also been upgraded.  The update focuses primarily on bug fixes.  Current users can upgrade for free at our support page; new users can download a free version here.

ASC to present at 2012 SIOP Conference

ASC will present a mini-workshop on the development of computerized adaptive tests (CATs) at the Society for Industrial-Organizational Psychology (SIOP) conference.  The session is scheduled for 1:30 P.M. on Friday, April 27.  More information is available at the SIOP website.  Please join us and learn about our CAT solutions and assessments!

ASC Update - February 2012

Here are some brief updates and news for February 2012...

ASC to conduct IRT/CAT Workshop in Botswana - 15 seats open!

ASC will be conducting an applied workshop on item response theory (IRT) and computerized adaptive testing (CAT), March 5-9 at the Grand Palm resort in Gaborone, Botswana.  The workshop is hosted by the Botswana Examinations Council for the benefit of countries in the South African Development Community (SADC).  Approximately 15 seats remain open for researchers and practitioners from Africa.  The cost is approximately $275; participants are responsible for their travel and lodging expenses.  If you are interested in attending, please email solutions@assess.com.

CATSim update

CATSim, the essential program for CAT simulation research, is now available as Version 1.0.6.  Changes include (1) when the classification termination option is used, a new table reports all examinee classifications in addition to the "high-confidence" classifications, and the examinee report differentiates between the two types of classifications; (2) with variable termination and constraints, a new termination type reports that examinees exhausted the item bank before other termination options were reached; and (3) an error using the graded response model with "reversed" item parameters was fixed.  Current license holders can download the no-cost update from http://www.assess.com/xcart/pages.php?pageid=10.  If you do not yet have CATSim, a free demo version is available at http://www.assess.com/xcart/product.php?productid=555.

Tech Report: Comparison of Xcalibre 4 and BILOG-MG

A technical report was recently released which provides a comparison of Xcalibre 4 and BILOG-MG, the world's two leading software programs for item response theory (IRT) analysis.  The two are compared in terms of parameter recovery, which is an appropriate methodology to evaluate the accuracy of IRT parameter estimates.  For the majority of conditions studied, Xcalibre 4 produced more accurate calibrations.  A full copy of the report is at http://www.assess.com/research.php.

FastTEST Web implements powerful new lockdown browser

FastTEST Web, the world's most advanced system for internet-based item banking and test delivery, has implemented a new "lockdown" delivery system, WebLock.  WebLock is a special browser that greatly enhances test security by 1) preventing the examinee from accessing any internet sites, 2) preventing the examinee from accessing any information/programs on their computer, and 3) checking for specific software that is a security concern, such as screen-capture programs.  These advantages make it ideal for high-security tests in fields like education, certification, and employment.  Moreover, this is in addition to other important FTW security advantages like role-based access, examinee passwords, and the use of an internal adaptive testing engine.  Learn more at http://www.fasttestweb.com/secure-online-testing.php.

The third conference of the International Association for Computerized Adaptive Testing (IACAT) will be held at the Sydney Convention Centre (www.scec.com.au), Australia from Sunday 12 to Tuesday 14 August 2012. This conference follows the successful first conference held in the Netherlands in June 2010 and the second conference held in Monterey in October 2011. It is jointly organized by the Australian Medical Council (AMC) (www.amc.org.au) and Excel Psychological and Educational Consultancy (EPEC) (www.epecat.com).

The theme of the conference is "CAT - the past, present and future" and the program is designed to facilitate the exchange of ideas, research and applications of CAT. Besides invited keynote addresses and symposia by global leaders in the field, there will be pre-conference workshops at the introductory and advanced levels.

The program committee invites you to join us in Sydney. Sydney is a vibrant, multi-cultural and cosmopolitan city which, as the largest city in Australia, has much to offer.

Proposal abstracts can already be submitted to IACATconference2012@amc.org.au Abstracts should be submitted in 12pt Times New Roman MS Word (max of 500 words) and include a title, name and email address of the presenter(s), institutional affiliation and a topic area.

Information about accommodation and registrations will soon be available through a designated conference website. For further information at this stage contact IACATconference2012@amc.org.au or visit www.iacat.org.

I have just returned from a workshop on item response theory (IRT) and computerized adaptive testing (CAT), held January 23-25 in Brasilia, Brazil.  The workshop was a resounding success, with attendees from all across Brazil as well as a few international attendees.  I'd like to thank CESPE (http://www.cespe.unb.br/) and INEP (http://www.inep.gov.br/) for making it possible.  Muito obrigado!

We've all seen "all of the above" or "none of the above" items in tests we have taken, mostly in low-stakes situations... because these items are frowned upon by psychometricians and testing experts.  The item options are no longer independent, and an item of that format will give away the answer more often than standard items, especially because the correct answer usually is AOTA or NOTA.  Therefore, these items should be modified to format more in concordance with best practices.

Here are three approaches to rearranging items with "All of the above" or related issues.  Consider the following question as an example.

Which of the following is a city in Florida?

A. Miami

B. Tampa

C. Jacksonville

D. All of the above*

1.  Make an AOTA into a single answer by setting one of the answers as correct and flipping the rest to incorrect.  Example:

Becomes...

Which of the following is a city in Florida?

A. Milwaukee

B. Tampa*

C. Phoenix

D. Chicago

2. Make answers into combinations by actually writing out the terms with more than one in each option.  This is similar to the previous suggestion, but usually makes the item more difficult.

Which of the following lists two cities in Florida?

A. Miami and Tampa

B. Tampa and Milwaukee

C. Chicago and Milwaukee

D. Phoenix and Chicago

3.  If you can't think of any plausible incorrect answers (sometimes the case), try flipping the stem and options to get something like:

Miami, Tampa, and Jacksonville are cities in__________.

A. Florida

B. Alabama

C. California

D. Georgia

Social Media has become an integral part of life and business over the past 5 years.  In fact, one of ASC's clients is a certification agency for social media specialists: http://nismonline.org/.  ASC recognizes this important step in world connectivity and has been a part of social media for a few years.  In addition to this blog, here are a few important links:

LinkedIn (ASC): http://www.linkedin.com/company/assessment-systems-corporation

Facebook (FastTest Web): http://www.facebook.com/pages/FastTEST-Web/123132391080511

Twitter (FastTest Web): http://twitter.com/FastTESTWeb

We encourage you to connect with ASC via these platforms, especially LinkedIn!  Of course, we will continue to send out news and announcements via our email list, www.assess.com, and www.fasttestweb.com.