Psychometrician

The Psychometrician position is located within WIDA’s Assessment Department and will report to the Psychometrics Manager.  This position will be responsible for 1) supporting the psychometric operational and research activities associated with WIDA’s language assessments, 2) supporting the monitoring and evaluating of operational psychometric projects and analyses conducted by vendors, and 3) supporting the authoring and review of research reports and psychometric deliverables. 

The position will play a key role in supporting the psychometric operations and continuous improvement of the MODEL assessment program, including automated scoring initiatives. Additional responsibilities include conducting and reviewing test security analyses for ACCESS for ELLs, supporting item bank maintenance and quality assurance activities, providing psychometric quality control for ACCESS and Alternate ACCESS assessment programs, and contributing to operational and research projects that support the validity, reliability, and fairness of WIDA assessments. 

Employees in this position can expect to receive benefits such as generous vacation, holidays and paid time off; competitive insurance and saving accounts; and retirement benefits. Learn more: https://hr.wisc.edu/benefits/new-employee-benefits-enrollment/. 

Remote work is an option for this position on a voluntary basis; however, it is important to note that this role is headquartered in Madison, WI, and the tasks of the position require occasional work-related travel. If working remotely out-of-state, travel back to UW-Madison would not be reimbursable, and other business travel will require cost comparisons between your home and Madison, WI.  

UW Travel Policy and Remote Work Policies outline business travel requirements for fully remote employees, and WIDA is committed to supporting employees with their business travel and minimizing or eliminating out-of-pocket expenses whenever possible. Please review UW Remote Work Policy UW5087, in particular, Section III, Part G which addresses an employee's Headquarters Location. We will be happy to support and address questions about remote work or travel during the interview process. 

WIDA operates in a Windows-based environment and a Dell PC laptop will be issued for use in this role. Software and virtual connection tools that are critical to our work dictate that WIDA staff work on a PC.   

Education Requirements 

Ph.D. in Educational Measurement or related field with an emphasis on psychometrics or quantitative methods.  

Qualifications 

Required 

• Minimum 1 year of experience in psychometrics theory, research, and statistics. Examples may include classical test theory, item response theory, scaling and equating, sampling, generalizability theory, linear modelling, or structural equation modeling. 
• Experience with data manipulation and statistical/psychometrics analyses using Winsteps, SAS, R, or other related software packages.  
• Familiarity with multistage adaptive testing, Rasch model, and graded response IRT models. 
• Ability to write technical and research reports.  
• Ability to work independently and collaboratively within a team. 
• Experience collaborating with external vendors or partners. 

Preferred 

• Programming skills, including proficiency in Python or similar programming languages used for psychometric modeling, statistical analysis, and assessment research. 
• Experience applying artificial intelligence (AI) methods to assessment and measurement, such as natural language processing (NLP), automated scoring, machine learning, or related technologies. 
• Test security related analysis, training or experience 
• Experience working with K–12 educational assessments, language proficiency assessments, or large-scale standardized testing programs