[R-lang] Re: lmer: Significant fixed effect only when random slopeisincluded

René Mayer mayer@psychologie.tu-dresden.de
Thu May 12 23:49:24 PDT 2011


Hi Jorrig,
I also had the impression that you don't have enough observations  
within each cell to run this mixed logistic regression. I don't know  
if you need random effects to interpret your hypothesis. As you have  
more subjects than within subject observations a marginal model may be  
an option: geeglm( PRO ~ cAGTOP + cAGVIS, id = SUBJ, binomial);  
library(geepack), of course here you don't have the crossed random  
effect of Item.
René


Zitat von "Levy, Roger" <rlevy@ucsd.edu>:

> Hi Jorrig,
>
> I don't think we have got enough information from you to interpret  
> the conditions when they are named a-f...
>
> I take it pronoun use is the response not a predictor?
>
> Also: if there are only 2 responses per subject per condition, then  
> there must be *huge* inter-subject variation in order to support a  
> subject random effect.  I'd think that plotting the  
> subject-by-condition means would still be useful in these  
> circumstances.
>
> Roger
>
>
> On May 12, 2011, at 2:45 AM, J. Vogels wrote:
>
>> The condition means are as follows (1=pronoun; 0=no pronoun):
>>
>>      a          b          c          d          e          f
>>   0 0.12765957 0.11956522 0.91304348 0.86046512 0.74725275 0.77528090
>>   1 0.87234043 0.88043478 0.08695652 0.13953488 0.25274725 0.22471910
>>
>> so many of them around 90%. Would that be a problem?
>>
>> The subject-by-condition means are not very informative, because  
>> each subject has only 2 responses per condition. So the means here  
>> are either 0%, 50%, or 100%.
>> Starting with the model with the full fixed effects structure and  
>> the random slope for cAGVIS, a likelihood-ratio test indicates that  
>> removing the interaction term between the fixed effects does not  
>> result in a significant difference (p=.096). A second test shows  
>> that further removing cAGVIS results in a difference in fit that  
>> does not reach the .05 significance level either (p=.056). However,  
>> comparing a model with only cAGTOP as predictor to the first model  
>> does give a significant result (p=.031), suggesting that cAGVIS has  
>> some influence.
>>
>> @Ian & Daniel: I used the anova() function in R to do model  
>> comparisons. When the model did not converge, I started by removing  
>> the interaction terms for the random slopes, first in the item  
>> effects, then in the subject effects, as suggested by Florian  
>> Jaeger in his blog. Next, I tried either removing cAGVIS or cAGTOP,  
>> again first for the items. The first model that converged had  
>> by-subjects random slopes for cAGVIS and cAGTOP, and no by-items  
>> random slopes.
>>
>> Jorrig
>>
>>
>>
>> > -----Original Message-----
>> > From: ling-r-lang-l-bounces@mailman.ucsd.edu [mailto:ling-r-lang-l-
>> > bounces@mailman.ucsd.edu] On Behalf Of Levy, Roger
>> > Sent: woensdag 11 mei 2011 20:17
>> > To: ling-r-lang-l@mailman.ucsd.edu
>> > Subject: [R-lang] Re: lmer: Significant fixed effect only when random
>> > slopeisincluded
>> >
>> > Just as a brief follow-up: since this is categorical data, given the
>> > magnitudes of some of the coefficients in question I would indeed worry
>> > a bit.  It looks like some condition means may be close to 100%, but
>> > that different subjects may have really dramatically different behavior
>> > and that this factor may dominate everything else. Also, getting close
>> > to 100% can mess with the Z statistic.  What do your condition-mean and
>> > subject-by-condition mean tables look like?  And what do likelihood-
>> > ratio tests for your fixed effects tell you in these models?  You may
>> > be fine and the random-slope model you included in your first message
>> > may indeed be a good one for interpreting your data, but eyeballing
>> > these tables would be useful as well.
>> >
>> > Roger
>> >
>> >
>> > On May 11, 2011, at 6:04 AM, Finlayson, Ian wrote:
>> >
>> > > What was the random effect structure of this first converging model?
>> > As I said earlier the significant interaction seems fine (as does your
>> > explanation), but I’m just curious about how you carried out backward
>> > stepwise elimination when the full model didn’t converge.
>> > >
>> > > Ian
>> > >
>> > >
>> > > From: ling-r-lang-l-bounces+ifinlayson=qmu.ac.uk@mailman.ucsd.edu
>> > [mailto:ling-r-lang-l-bounces+ifinlayson=qmu.ac.uk@mailman.ucsd.edu] On
>> > Behalf Of Jorrig Vogels
>> > > Sent: 11 May 2011 11:56
>> > > To: ling-r-lang-l@mailman.ucsd.edu
>> > > Subject: [R-lang] Re: lmer: Significant fixed effect only when random
>> > slopeisincluded
>> > >
>> > > Hello Ian,
>> > >
>> > > In the full model, I included random slopes for cAGTOP and cAGVIS and
>> > their interaction for both subjects and items. However, this did not
>> > converge. The first model that converged showed a significant
>> > interaction between cAGTOP1 and cAGVIS, but not between cAGTOP2 and
>> > AGVIS.
>> > >
>> > > Jorrig
>> > >
>> > >
>> > > From: Finlayson, Ian
>> > > Sent: Wednesday, May 11, 2011 12:36 PM
>> > > To: Jorrig Vogels ; ling-r-lang-l@mailman.ucsd.edu
>> > > Subject: RE: [R-lang] lmer: Significant fixed effect only when random
>> > slope isincluded
>> > >
>> > > Hello,
>> > >
>> > > I assume that you only have one random slope because the removal of
>> > the other two (cAGTOP, and its interaction with cAGVIS) didn’t
>> > significantly harm fit. Were the fixed effects for the interaction
>> > significant in the full model?
>> > >
>> > > FWIW, I have seen this happen before and it seems perfectly
>> > reasonable to me that an effect may only become significant after
>> > controlling for some of the noise.
>> > >
>> > > Ian
>> > >
>> > > From: ling-r-lang-l-bounces@mailman.ucsd.edu [mailto:ling-r-lang-l-
>> > bounces@mailman.ucsd.edu] On Behalf Of Jorrig Vogels
>> > > Sent: 11 May 2011 10:50
>> > > To: ling-r-lang-l@mailman.ucsd.edu
>> > > Subject: [R-lang] lmer: Significant fixed effect only when random
>> > slope isincluded
>> > >
>> > > Dear R users,
>> > >
>> > > I have a logit mixed model with two categorical predictors (two types
>> > of salience measures) and a categorical dependent variable (pronoun
>> > used Y/N). One predictor has 2 levels, and the other has 3. I centered
>> > the 2-level predictor, and transformed the 3-level predictor into two
>> > binary predictors using contrast (sum) coding. I determined the random-
>> > effects structure by starting from a full model, and eliminating step
>> > by step all terms without a significant contribution to the model.
>> > >
>> > > In the final model, I end up with random intercepts for subjects and
>> > items, and a by-subject random slope for my 2-level predictor. In this
>> > model, I get significant interactions between the fixed factors, which
>> > I had not expected to be significant by just looking at the data.
>> > Removing the random slope from the model completely eliminates these
>> > interactions, but model comparison suggests the random slope should be
>> > included. I have attached the two model summaries below.
>> > >
>> > > Now my question is: is it normal to find such a large influence of
>> > random effects on the fixed effects structure? How do I know the
>> > interaction effects are not spurious? And what exactly do these
>> > findings mean? Participants varied greatly in their reaction to
>> > predictor B, but when this variation is accounted for, predictor B
>> > affects pronoun use, but differently for each level of predictor A?
>> > >
>> > >
>> > > Jorrig Vogels
>> > > PhD candidate
>> > > Tilburg Univ., Netherlands
>> > >
>> > > ================================================================
>> > >
>> > > Model with random slope:
>> > >
>> > > Generalized linear mixed model fit by the Laplace approximation
>> > > Formula: PRO ~ cAGTOP * cAGVIS + (1 + cAGVIS | SUBJ) + (1 | ITEM)
>> > >    Data: vislingag
>> > >    AIC   BIC logLik deviance
>> > > 318.4 361.4 -149.2    298.4
>> > > Random effects:
>> > > Groups Name        Variance Std.Dev. Corr
>> > > SUBJ   (Intercept) 49.6457  7.0460
>> > >         cAGVIS      21.8342  4.6727   0.663
>> > > ITEM   (Intercept)  1.3205  1.1491
>> > > Number of obs: 544, groups: SUBJ, 48; ITEM, 12
>> > >
>> > > Fixed effects:
>> > >                Estimate Std. Error z value Pr(>|z|)
>> > > (Intercept)      -2.578      1.217  -2.117  0.03422 *
>> > > cAGTOP1          -6.627      0.913  -7.259 3.90e-13 ***
>> > > cAGTOP2           9.868      1.502   6.569 5.05e-11 ***
>> > > cAGVIS           -1.699      1.008  -1.685  0.09207 .
>> > > cAGTOP1:cAGVIS   -3.223      1.170  -2.755  0.00587 **
>> > > cAGTOP2:cAGVIS    3.120      1.371   2.275  0.02289 *
>> > > ---
>> > > Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>> > >
>> > > Correlation of Fixed Effects:
>> > >             (Intr) cAGTOP1 cAGTOP2 cAGVIS cAGTOP1:
>> > > cAGTOP1      0.075
>> > > cAGTOP2     -0.041 -0.867
>> > > cAGVIS       0.535  0.108  -0.059
>> > > cAGTOP1:AGV  0.074  0.562  -0.346   0.128
>> > > cAGTOP2:AGV -0.049 -0.480   0.528  -0.054 -0.668
>> > >
>> > >
>> > > Model without random slope:
>> > >
>> > > Generalized linear mixed model fit by the Laplace approximation
>> > > Formula: PRO ~ cAGTOP * cAGVIS + (1 | SUBJ) + (1 | ITEM)
>> > >    Data: vislingag
>> > >    AIC   BIC logLik deviance
>> > > 324.3 358.7 -154.2    308.3
>> > > Random effects:
>> > > Groups Name        Variance Std.Dev.
>> > > SUBJ   (Intercept) 21.63217 4.65104
>> > > ITEM   (Intercept)  0.61539 0.78447
>> > > Number of obs: 544, groups: SUBJ, 48; ITEM, 12
>> > >
>> > > Fixed effects:
>> > >                Estimate Std. Error z value Pr(>|z|)
>> > > (Intercept)    -1.41142    0.77639  -1.818   0.0691 .
>> > > cAGTOP1        -4.59707    0.52139  -8.817   <2e-16 ***
>> > > cAGTOP2         7.13115    0.84489   8.440   <2e-16 ***
>> > > cAGVIS         -0.35538    0.40416  -0.879   0.3792
>> > > cAGTOP1:cAGVIS -0.59940    0.58255  -1.029   0.3035
>> > > cAGTOP2:cAGVIS -0.08268    0.56682  -0.146   0.8840
>> > > ---
>> > > Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>> > >
>> > > Correlation of Fixed Effects:
>> > >             (Intr) cAGTOP1 cAGTOP2 cAGVIS cAGTOP1:
>> > > cAGTOP1      0.070
>> > > cAGTOP2     -0.040 -0.867
>> > > cAGVIS       0.000  0.061  -0.036
>> > > cAGTOP1:AGV  0.038  0.082  -0.012   0.102
>> > > cAGTOP2:AGV -0.020  0.008  -0.037   0.037 -0.575
>> >
>> > --
>> >
>> > Roger Levy                      Email: rlevy@ucsd.edu
>> > Assistant Professor             Phone: 858-534-7219
>> > Department of Linguistics       Fax:   858-534-4789
>> > UC San Diego                    Web:   http://idiom.ucsd.edu/~rlevy
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>
> --
>
> Roger Levy                      Email: rlevy@ucsd.edu
> Assistant Professor             Phone: 858-534-7219
> Department of Linguistics       Fax:   858-534-4789
> UC San Diego                    Web:   http://idiom.ucsd.edu/~rlevy
>
>
>
>
>
>
>
>
>
>
>




More information about the ling-r-lang-L mailing list