|
|
Estimating Parameters for the BNM Model
A handful of programs are available for estimating parameters for unidimensional, polytomous models. The DOS version of MULTILOG, available from Scientific Software, is discussed in the examples that follow.
It is often necessary to estimate parameters for a large number of scales or experimental groups. For that purpose, we prefer the DOS version of MULTILOG because it is very easy to modify the input files and create MS-DOS batch files to conduct multiple runs. The simplest way to estimate parameters for your data using MULTILOG is to copy your data set into the directory where your copy of MULTILOG resides.
Data File Your data file must be saved in ASCII format (using Notepad or a DOS editor for example). An ID number should be provided in the data set for each respondent. To see how to prepare data for MULTILOG please click here.
Running MULTILOG
Running MULTILOG to estimate item and person parameters involves a multi-step process,
which is described throughly in the user's manual. Because this procedure is time
consuming particularly if you have many scales to calibrate,
we have developed an alternative method using DOS batch files.
Below is an example of a DOS batch file that estimates parameters for our example dataset, AGR1_CAL.dat.
Note that this batch file requirs two additional text files with extensions *.in1 and *.in2 that must be edited by the user before running MULTILOG. These file will be described below.
Batch File For Estimating Item and Person Parameters
DEL *.SAV
DEL *.LIS
DEL *.LOG
DEL *.OUT
DEL *.PRM
DEL *.TEN
INFORLOG agr1_cal < agr1_cal.IN1
MULTILOG agr1_cal
DEL AGR1_CAL.TEN
DEL AGR1_CAL.LOG
REN AGR1_CAL.LIS AGR1_CAL.LS1
DEL *.LIS
REN AGR1_CAL.SAV AGR1_CAL.PRM
DEL *.SAV
INFORLOG AGR1_CAL < AGR1_CAL.IN2
MULTILOG AGR1_CAL
REN AGR1_CAL.LIS AGR1_CAL.LS2
Description of the batch file
The first six lines
DEL *.SAV
DEL *.LIS
DEL *.LOG
DEL *.OUT
DEL *.PRM
DEL *.TEN
delete old input and output files in the MULTILOG directory.
The next line
INFORLOG agr1_cal < agr1_cal.IN1
runs the program INFORLOG, which creates the syntax file required by MULTILOG to conduct parameter estimation. INFORLOG is designed to be an interactive program in which the user responds to questions concerning parameter estimation by typing commands when prompted. This process can be automated using DOS redirection commands. In the line above, the '<' symbol redirects input from a text file containing our answers to questions for parameter estimation. The .in1 file must be edited to match your data specifications. The .in1 will be discussed shortly.
The next line
MULTILOG agr1_cal
runs the MULTILOG program to estimate item parameters using the *.TEN file created by INFORLOG Next an input file for MULTILOG is created and saved in the MULTILOG directory. Rather than starting from scratch, we suggest modifying one of the example input files that we provide. For example:
The next six lines perform several tasks
DEL AGR1_CAL.TEN
DEL AGR1_CAL.LOG
REN AGR1_CAL.LIS AGR1_CAL.LS1
DEL *.LIS
REN AGR1_CAL.SAV AGR1_CAL.PRM
DEL *.SAV
- The first line deletes the input file used for item parameter estimation.
- The next line deletes the log file created by MULTILOG.
- The third line saves the output from the item parameter estimation with a new extension so that it is not overwritten when person parameters are estimated.
- The fourth line deletes the old item parameter output files.
- The fifth line saves the item parameter file created by MULTILOG (*.sav) with a new extension (*.prm).
- The last line deletes the old item parameter file.
The last three lines estimate person parameters using input file *.in2 which has commands similar to those in *.in1.
INFORLOG AGR1_CAL < AGR1_CAL.IN2
MULTILOG AGR1_CAL
REN AGR1_CAL.LIS AGR1_CAL.LS2
Again, DOS redirection commands are used to run INFORLOG and MULTILOG successively. However, in this case, the person parameter file created by MULTILOG (*.LIS) is renamed *.ls2 extension to distinguish it from the *.ls1 file containing item parameter output.
Below is a sample *.in1 file. If you are following our continuing on-line example with the agreeableness scale, what follows below is the syntax in the
agr1_cal.in1 file.
CALIBRATION OF AGREEABLENESS NOMINAL MODEL
>PRO IN RA NI=10 NE=1500 NCHAR=4 NG=1;
>TEST ALL NO NC=(5(0)10) HI=(5(0)10);
>EST NC=50;
>SAVE;
>END;
5
12345
1111111111
2222222222
3333333333
4444444444
5555555555
(4A1,10A1)
*Please note that in the command lines, each line must end with a semicolon.
Additional input files for our examples are provided here.
CALIBRATION OF AGREEABLENESS NOMINAL MODEL
This FIRST LINE in the *.in1 file (in this example, "agr1_cal.in1") which serves as a title for this particular "run" of MULTILOG.
>PRO IN RA NI=10 NE=1500 NCHAR=4 NG=1;
The SECOND LINE denotes how the program (PRO) is to handle data where:
IN = individual responses recorded as vectors in the data file (one line represents one individual's responses to items 1 through k). Alternatively, one could use PA for a response pattern matrix or TA for a fixed-effects table of counts instead of a matrix of individual responses. A discussion of these alternative data files are discussed in the MULTILOG manual.
RA = indicates that marginal maximum likelihood item parameter estimation is to be carried out due to the fact that we do not know the theta for the given items. Alternatively, FI could indicate fixed-theta item parameter estimation or SCORE for the computation of individual scores for the respondents.
NI = number of items, k (for this sample data file we have 10 items).
NE = number of examinees (in this case, 1500).
NCHAR = number of characters in the ID field for each respondent (4 characters for our sample data files, e.g., 1001, 1002…).
NG = numbers of groups - for our discussion, we will only be dealing with a single group at a time.
>TEST ALL NO NC=(5(0)10) HI=(5(0)10);
The THIRD LINE denotes how the model will be estimated. In the text above, ALL indicates that all of the items in the data file will be examined. To delineate particular items to include, one would list them as follows: "IT=(1,2,3,4,6,7,8,9,10)" where the 5th item is excluded from the analyses.
Because we wish to use the Nominal model, "NO" is indicated. The number of categories (options) for each of the items must be specified as well. Since all of our 10 items have 5 options each we type "NC=(5,5,5,5,5,5,5,5,5,5)". One shortcut that is possible in this case is "NC=(5(0)10)" which indicates the same thing -10 items, each with 5 options. If we had 5 items of 4 options followed by 6 items of 5 options in our measure we would indicate "NC=(4,4,4,4,4,5,5,5,5,5,5)". Items keyed as "high" on the measure must be delineated with the Nominal model. This is done by stating which option for each item is "keyed": "HI=(5,5,5,5,5,5,5,5,5,5)". Alternatively, we could use a shortcut instead, "HI=(5(0)10). If our measure used all 4's for the "keyed" option except for the last item, where 5 is used, we could indicate "HI=(4,4,4,4,4,4,4,4,4,5)".
>EST NC=50;
The FOURTH LINE in this example indicates that a maximum number of 50 cycles should be used in the estimation procedure. If after completing one "run" of MULTILOG we find that more cycles are necessary to better estimate the parameters, we may wish to repeat this procedure with more cycles, e.g., 100. There are a number of other options available that are not discussed here but in the MULTILOG manual.
>SAVE;
The FIFTH LINE indicates that parameters will be saved as described in the previous commands above. If we had indicated RA or FI in the second line, we will obtain item parameters; if we had indicated SCORE, we obtain a score for the respondents.
>END;
The SIXTH LINE indicates to INFORLOG that we have finished the syntax.
5
12345
1111111111
2222222222
3333333333
4444444444
5555555555
These lines depend on the response format of the scale or measure. Here the value of 5 indicates that there are 5 separate characters that determine the responses the individuals can choose from. The next line indicates that those values are 1, 2, 3, 4, and 5. The next five lines are basically those options mapped to each of the 10 items that are in our measure. These values obviously change depending on the nature of the scale or measure in question.
(4A1,10A1)
The LAST LINE provides a FORTRAN statement indicating 4 characters for the ID field followed by the responses for the 10 items.
If you are using the MS DOS Editor (activated by typing "edit [filename]" at the DOS prompt corresponding to the appropriate directory), the above file should look like this:

To follow the strategy outlined above, the *.in2 files need to be edited to provide person parameters from MULTILOG. Below is an example that you may download here for our on-going example with the agreeableness scale:
SCORING AGREEABLENESS SCALE NOMINAL MODEL
>PRO SCORE IN RA NI=10 NE=1500 NCHAR=4 NG=1;
>TEST ALL NO NC=(5(0)10) HI=(5(0)10);
>START;
Y
>SAVE;
>END;
5
12345
1111111111
2222222222
3333333333
4444444444
5555555555
(4A1,10A1)
SCORING AGREEABLENESS SCALE NOMINAL MODEL
This FIRST LINE in the *.in1 file (in this example, "agr1_cal.in2") which serves as a title for this particular "run" of MULTILOG.
>PRO SCORE IN RA NI=10 NE=1500 NCHAR=4 NG=1;
In the SECOND LINE we wish to obtain scoring information on the respondents; thus, as provided in our explanation of *.in1, we indicate that scoring (SCORE) is to be carried out. Also, individual responses are provided in the data file (IN), there are 10 items (NI-10), we have 1500 examinees (NE=1500), there are 4 characters for our data files representing the ID field for each respondent (NCHAR=4), and we are dealing with a single group (NG=1).
>TEST ALL NO NC=(5(0)10) HI=(5(0)10);
Because we are "scoring" the same scale, the THIRD LINE can be identical to our *.in1 file. Note that in this syntax however, we have used the "shortcuts" where "(5(0)10)" the first time indicates that the number of categories for each of the ten items is 5. The second "(5(0)10)" phrase indicates that for all ten items, the 5th option is "keyed".
>START;
The FOURTH LINE is an optional command that is unique to obtaining the person parameters (scoring) in that this will override default starting values for the item parameters. These parameters will be "read in" by MULTILOG based on another file (created by the first portion of the batch file -first run of INFORLOG).
Y
The FIFTH LINE contains a "Y" to indicate to INFORLOG that the parameters are contained in another file obtained in the initial run of MULTILOG (to estimate the person parameters).
The SIXTH LINE is blank because INFORLOG is looking for a format statement for these parameters; however, none is needed if these parameters come from another set of output from MULTILOG.
>SAVE;
The SEVENTH LINE indicates that scoring estimates will be saved in a file based on the commands denoted in the previous five lines.
>END;
The eighth line indicates an end to the command syntax.
5
As before, this line (in this case the NINTH LINE) provides INFORLOG with the number of characters denoting the various options.
12345
1111111111
2222222222
3333333333
4444444444
5555555555
As before with the *.in1 file, those characters that are used in the actual response vectors for the individuals.
(4A1,10A1)
The FINAL LINE provides the FORTRAN statement for the data.
In the MS DOS Editor, this file appears as follows:

Running MULTILOG
Now we are finished with editing the two input files that our batch file requires. Note that your file names cited in the batch file will differ depending on your data, we have chosen to use the "agr1_cal" data set for this example. To run the batch file, ensure that it is saved in the same directory as MULTILOG. Simply type the name of that batch file at the prompt, e.g., "agr1_cal".
OUTPUT FILES
If everything runs correctly, the following output files are provided (note that additional files may have been created and deleted following the commands issued in the batch file):
agr1_cal.log
agr1_cal.lis
agr1_cal.sav
Agr1_cal.ls1
Agr1_cal.ls2
Agr1_cal.ten
Agr1_cal.prm
*.LOG
This file merely provides a "log" of the commands submitted to INFORLOG and should be exactly the same as what was in the *.in2 file (an earlier *.log file was created following the submission of *.in1 to INFORLOG, but deleted before INFORLOG was run with *.in2). This file is of little significance.
*.LIS
This file provides details of the last (second) run of MULTILOG, the first portion is provided to assist troubleshooting (e.g., "was the data read in correctly"). The second portion, provides the values used by MULTILOG in the scoring of the individuals. Finally, the last portion of this file provides person parameters. Note that the *.lis files are renamed as *.ls1 and *.ls2, discussed later.
*.SAV
This file is a tab delimited (data separated by a tab) matrix of the person parameters (estimated theta, standard error, and respondent ID).
*.LS1
The batch file provided renames the first *.lis file as *.ls1. This file is useful in ensuring that the data were read in and the model specified correctly.
View of the agr1_cal.ls1 file in the MS DOS Editor:

The *.ls1 file also provides a report of the estimation procedure with the estimated
item parameters.
View of the agr1_cal.ls1 file in the MS DOS Editor:

Note that it is important to check that the estimation procedure converged (i.e., the largest
change in any parameter in the last cycle was smaller than the convergence criterion). If convergence
was not successful, the user may wish to increase the number of cycles permitted, specify a prior
distribution, or eliminate problematic items (as identified by CTT).
In this case, the estimation procedure did not converge and a second run must be completed with
an increase in the number of cycles permitted (e.g., agr1_bn2.in1).
These parameters can be plotted with the MODFIT program (explained later in this tutorial). Remember, MULTILOG's estimates for
aik and
cik
contain a scaling factor of 1.7. Therefore, the user must divide the
aik and
cik
parameters by 1.7, then compute bik by dividing the new
cik by the new aik
and multiplying by -1. Below are the plots for the Agreeableness scale.

Plots of Agreeableness items
Previous Next
Below are the plots for the Conscientiousness scale.
Plots of Conscientiousness items
Previous Next
Back
|
|
|