Assessment Systems Corporation - Blog

Happiness and work

2012-02-03T15:13:35Z

http://www.ted.com/talks/shawn_achor_the_happy_secret_to_better_work.html Interesting talk on psychology and work - with some unintentional commentary on the Rasch model around 4:00. (removing some data points simply because you don't like them, with the excuse of measurement error)

January 2012 Newsletter

2012-02-02T15:56:32Z

Xcalibre 4 updates
Xcalibre 4is the most user-friendly software available for item response theory (IRT) analysis. An update has been recently released, which includes a number of bug fixes and enhancements, including an addition of distractor (quantile) plots into the output. These plots provide an excellent method for evaluating multiple-choice distractors by combining IRT and classical test theory. We have also added a new fit plot with standard errors to the report document and percentiles to the scores spreadsheet. Current license holders can download the no-cost update from http://www.assess.com/xcart/pages.php?pageid=10. If you do not yet have Xcalibre 4, a free demo version is available at http://www.assess.com/xcart/product.php?productid=569.

Iteman 4 updates
We have also released an update for Iteman 4, the leading software for analysis with classical test theory (CTT). This update focuses on bug fixes and maintenance. If you have a current license, you can download the update at http://www.assess.com/xcart/pages.php?pageid=10. If you do not yet have Iteman 4, a free demo version is available at http://www.assess.com/xcart/product.php?productid=541.

Conferences page on ASC website updated for 2012
ASC provides a number of resources to professionals in the field of testing and psychometrics, including a list of major conferences in the field. We have updated that list for 2012 at http://www.assess.com/conferences.php... see where you want to go! ASC has plans to be at the SIOP, IACAT, and ICE conferences.

Connect with ASC on LinkedIn
LinkedIn is an essential social networking site for professionals of all fields. It marks an important advancement in professional connectivity. ASC has had a page on LinkedIn for a number of years; if you aren't already following us, please visit http://www.linkedin.com/company/assessment-systems-corporation. We also update our news at the ASC blog.

Free training videos released
ASC hosts one or two workshops every year (one workshop was just recently held, January 23-25 in Brazil) and presents at a number of conferences, but we realize that not everyone is able to attend. Therefore, ASC is publishing a set of tutorial videos on our software and important issues in psychometrics, like item response theory and computerized adaptive testing. The videos are available for free at http://www.assess.com/tutorials.php. Two have been completed so far: Running Iteman 4 and Interpreting Iteman 4 Output.

ASC to present at 2012 SIOP Conference
ASC will present a mini-workshop on the development of computerized adaptive tests (CATs) at the Society for Industrial-Organizational Psychology (SIOP) conference. The session is scheduled for 1:30 PM on Friday, April 27. More information is available at the SIOP website.

Important links
ASC Website (Home)
ASC blog
ASC LinkedIn
FastTest Web - Secure Online Adaptive Testing
FastTest Web Facebook page
FastTest Web Twitter page
Student grants for free software

Official Announcement: IACAT 2012 in Sydney

2012-01-30T17:57:46Z

The third conference of the International Association for Computerized Adaptive Testing (IACAT) will be held at the Sydney Convention Centre (www.scec.com.au), Australia from Sunday 12 to Tuesday 14 August 2012. This conference follows the successful first conference held in the Netherlands in June 2010 and the second conference held in Monterey in October 2011. It is jointly organized by the Australian Medical Council (AMC) (www.amc.org.au) and Excel Psychological and Educational Consultancy (EPEC) (www.epecat.com).

The theme of the conference is "CAT - the past, present and future" and the program is designed to facilitate the exchange of ideas, research and applications of CAT. Besides invited keynote addresses and symposia by global leaders in the field, there will be pre-conference workshops at the introductory and advanced levels.

The program committee invites you to join us in Sydney. Sydney is a vibrant, multi-cultural and cosmopolitan city which, as the largest city in Australia, has much to offer.

Proposal abstracts can already be submitted to IACATconference2012@amc.org.au Abstracts should be submitted in 12pt Times New Roman MS Word (max of 500 words) and include a title, name and email address of the presenter(s), institutional affiliation and a topic area.

Information about accommodation and registrations will soon be available through a designated conference website. For further information at this stage contact IACATconference2012@amc.org.au or visit www.iacat.org.

Workshop in Brazil, January 23-25

2012-01-27T18:36:46Z

I have just returned from a workshop on item response theory (IRT) and computerized adaptive testing (CAT), held January 23-25 in Brasilia, Brazil. The workshop was a resounding success, with attendees from all across Brazil as well as a few international attendees. I'd like to thank CESPE (http://www.cespe.unb.br/) and INEP (http://www.inep.gov.br/) for making it possible. Muito obrigado!

Fixing "all of the above" items

2012-01-19T22:50:11Z

We've all seen "all of the above" or "none of the above" items in tests we have taken, mostly in low-stakes situations... because these items are frowned upon by psychometricians and testing experts. The item options are no longer independent, and an item of that format will give away the answer more often than standard items, especially because the correct answer usually is AOTA or NOTA. Therefore, these items should be modified to format more in concordance with best practices.

Here are three approaches to rearranging items with "All of the above" or related issues. Consider the following question as an example.

Which of the following is a city in Florida?

A. Miami

B. Tampa

C. Jacksonville

D. All of the above*

1. Make an AOTA into a single answer by setting one of the answers as correct and flipping the rest to incorrect. Example:

Becomes...

Which of the following is a city in Florida?

A. Milwaukee

B. Tampa*

C. Phoenix

D. Chicago

2. Make answers into combinations by actually writing out the terms with more than one in each option. This is similar to the previous suggestion, but usually makes the item more difficult.

Which of the following lists two cities in Florida?

A. Miami and Tampa

B. Tampa and Milwaukee

C. Chicago and Milwaukee

D. Phoenix and Chicago

3. If you can't think of any plausible incorrect answers (sometimes the case), try flipping the stem and options to get something like:

Miami, Tampa, and Jacksonville are cities in__________.

A. Florida

B. Alabama

C. California

D. Georgia

ASC on Social Media

2012-01-18T15:58:45Z

Social Media has become an integral part of life and business over the past 5 years. In fact, one of ASC's clients is a certification agency for social media specialists: http://nismonline.org/. ASC recognizes this important step in world connectivity and has been a part of social media for a few years. In addition to this blog, here are a few important links:

LinkedIn (ASC): http://www.linkedin.com/company/assessment-systems-corporation

Facebook (FastTest Web): http://www.facebook.com/pages/FastTEST-Web/123132391080511

Twitter (FastTest Web): http://twitter.com/FastTESTWeb

We encourage you to connect with ASC via these platforms, especially LinkedIn! Of course, we will continue to send out news and announcements via our email list, www.assess.com, and www.fasttestweb.com.

Free software for students

2011-12-30T18:51:53Z

Did you know that grants are available to purchase ASC software for graduate students? If you are a student, or a faculty member who is supervising graduate students, please visit http://www.assess.com/xcart/pages.php?pageid=5 for more information. We also make our software available free for a semester to students in your classes or graduate students who need our software for their research for up to four months.

Requirements for adaptive testing

2011-12-15T20:50:59Z

"Adaptive testing" has become a buzzphrase in testing circles over the past decade, especially in K-12 assessment. Unfortunately, very few people who hear the phrase, are interested in adaptive tests, or are even using adaptive tests, have any idea what goes into them. ASC gets a number of inquiries regarding the development of adaptive tests from organizations or individuals that are not familiar with the fundamentals of adaptive testing or the basic requirements that must be met to even think about developing adaptive tests. For example, you must have a pilot sample of at least 100 examinees, preferably 500 - and CAT is still often not feasible unless you are testing thousands of examinees. Therefore ASC psychometricians have published two important resources on the topic: a journal article describing the development of adaptive tests (http://pareonline.net/pdf/v16n1.pdf) and a white paper describing the basic requirements for a testing program to even consider adaptive testing (http://www.fasttestweb.com/ftw-docs/CAT_Requirements.pdf). These are essential readings for anyone involved with modern assessments.

FastTest Web as a Custom Enterprise Assessment System

2011-12-15T20:42:07Z

FastTest Web is the industry-leading platform for web-based development and delivery of tests and assessments. Unlike many competitors, it is designed by testing experts for testing experts - with a focus on modern testing methodology like IRT and adaptive testing. While it is primarily available as a public "brand" system via a software-as-a-service model, it is also important to know that it is available as a customized enterprise-style system. That is, we could completely customize system, adding functionality as well as important branding aspects for your organization, then host it on branded URL (yourcompany.fasttestweb.com). Best of all, we can do so affordably. Visit www.fasttestweb.com to learn more!

IACAT 2012 conference

2011-12-14T00:14:33Z

The IACAT 2012 conference has been planned for August 13-15 in Sydney Australia. IACAT 2011 (in October) was an excellent experience; in addition to the wonderful research and networking, many attendees enjoyed a visit to the Monterey Bay Aquarium, walks on the Pacific Coast, and enjoying all the other things to do in Monterey. Visit www.iacat.org for more information and a Call for Proposals (forthcoming). I'm certainly looking forward to Sydney!

"Person free" measurement

2011-12-09T18:13:17Z

If the Rasch idea of centering a scale on the item distribution produces "person-free measurement," does the centering of the scale on the person distribution produce "item-free measurement?" If so, what does each mean? I suppose that person-free measurement means that the measurement is only with regards to the items - but then you are making the assumption that the original calibration test form used to establish the scale is representative of the content universe. Meanwhile, item-free measurement means that we are placing people on the scale defined by the original calibration sample, which we are assuming is representative of the population. Which interpretation would best fit your needs?

2011: The first half

2011-06-13T17:23:00Z

The first half of 2011 has been quite busy at ASC. I have had the opportunity to visit Malaysia, India, and China for workshops on IRT and adaptive testing. It is exciting to see the growing interest for advanced psychometrics in those countries, and I am thankful to my friends for the chances to visit. ASC has also presented at two conferences this spring, the Association of Test Publishers and the National Council for Measurement in Education. Papers are available at our research page, http://www.assess.com/research.php.

ASc has also made major strides in its software. FastTEST Web (www.fasttestweb.com) has ever-expanding functionality, with a focus on item response theory and computerized adaptive testing. It remains the only publicly available platform for fully adaptive tests based on IRT. Another important release was Xcalibre 4.1 (http://www.assess.com/xcart/product.php?productid=569&cat=0&page=1&featured), which provides superior output for IRT analysis, as well as being the only IRT analysis program that is point-and-click. That is, no DOS-style command code is needed, which makes IRT much more accessible to researchers and practitioners in a wide variety of fields.

For more details on our software releases, workshops, and research, visit the ASC news page at http://www.assess.com/news.php.

August, 2010

2010-08-06T20:02:48Z

I spent yesterday presenting at the Minnesota Assessment Conference. My congratulations and thanks to the folks at the Minnesota Department of Education for again organizing a wonderful event with a huge range of topics, from technical psychometric issues all the way to practical district-level topics. ASC's plans for upcoming conferences include the CLEAR conference (Sept. 23-25), the European AEA conference (Nov. 4-6), and the Maryland Assessment Conference (Nov. 17-19). We are also holding a workshop on Sept. 4-5 in Tokyo, Japan.

I often get inquiries from around the world about where the next ASC workshop is going to take place. Given today's economy, traveling long distances is not always an option, but assessment professionals still want to learn more about IRT and CAT. For this reason, we have released a recording of our workshop in June 2009. Now anyone can attend a workshop, for a fraction of the cost of attending in person. Visit the webpage here to learn more.

FastTEST Web (www.fasttestweb.com) continues to be improved. Development in July focused on the addition of a Reporting module. Its purpose is to make test development more efficient by providing informative reports, such as a detailed assessment of the item bank to indicate where more questions are needed, or the current status of tests in your testing program. The next addition will be computerized adaptive testing (CAT), which will allow anyone to easily create and deliver adaptive tests.

The new Iteman 4.1 is slated for release in August. The most recent additions include a complete reliability analysis and a wizard-type interface for setting up multiple runs. If you haven't looked at Iteman 4 yet, I encourage you to check it out. It is unique in that it is not designed just for statistical analysis, but to actually produce customized reports of high enough quality to provide to stakeholders - in mere seconds.

June 2010 - Iteman 4.1; FastTEST Web update

2010-06-07T06:12:11Z

This post finds me writing from Arnhem, The Netherlands, home of Cito and host to the IACAT 2010 conference. I'm looking forward to hearing some of the excellent CAT research that is happening around the world. The conference has 130 participants from approximately 45 countries - and the organization is only one year old!

FastTEST Web (www.fasttestweb.com) has seen several important additions over the last month. Perhaps the most unique is an API functionality, which allows other software systems to interface efficiently. For example, if your organization has its own test delivery engine, but wishes to utilize FastTEST Web's powerful item banking, you can now easily make use of both. The next addition is to increase the number of reports available, both on the item bank itself and on examinee data.

ASC has also continued to work on the new Iteman 4. Version 4.1 will be released soon, which includes additional output (both graphical and statistical), as well as the capability to work with polytomously scored items (rating scale or partial credit). This version will even be able to work with mixed-format tests, such as a test composed primarily of multiple-choice items with a few open-response items at the end. It is a free upgrade from version 4.0, so make sure to update your copy.

We will also soon be releasing two more updates. ScoreAll 2.0 provides an additional level of detail in scoring conventional tests as well as updated interface. POSTSIM 3.0 cements its spot as the foremost software for computerized adaptive testing (CAT) research by now providing hybrid simulations, a new methodology developed by David Weiss. This methodology is especially useful because it is designed for sparse matrices, which typically occurs in CAT pilot testing.

May 2010 - FastTEST Web, Iteman 4

2010-05-05T18:23:00Z

May, 2010

As of May 5, FastTEST Web is now available to the public! Please visit www.fasttestweb.com to see some of the features and watch a video of it in action. We continue to add new features every day - so if there is something you would like to see in an online test development system, please contact me.

The buzz on Iteman 4 continues to grow! It has been out for approximately a month now, and orders continue to roll in. Iteman 4 is designed not just to analyze test data, but to produce formal reports - saving you the time of copying and pasting tables and figures for each item. Remember that there are discounts available both for multiple users and for multiple years (e.g., purchasing 3 years up front rather than the annual renewal).

We continue to add new features to Iteman 4, including the choice of quantile plot groups, and support for polytomous and mixed-format tests. In addition, Xcalibre is being re-engineered in the same spirit as Iteman 4.

The first IACAT election is complete; results are posted at http://www.iacat.org/news.html#elect2010. The Board is now working on establishing a vision and future for the organization. Of course, the most immediate future is the conference in June. A program will be posted shortly.