[R-lang] Re: lmer: Significant fixed effect only when random slopeisincluded
René Mayer
mayer@psychologie.tu-dresden.de
Thu May 12 23:49:24 PDT 2011
Hi Jorrig,
I also had the impression that you don't have enough observations
within each cell to run this mixed logistic regression. I don't know
if you need random effects to interpret your hypothesis. As you have
more subjects than within subject observations a marginal model may be
an option: geeglm( PRO ~ cAGTOP + cAGVIS, id = SUBJ, binomial);
library(geepack), of course here you don't have the crossed random
effect of Item.
René
Zitat von "Levy, Roger" <rlevy@ucsd.edu>:
> Hi Jorrig,
>
> I don't think we have got enough information from you to interpret
> the conditions when they are named a-f...
>
> I take it pronoun use is the response not a predictor?
>
> Also: if there are only 2 responses per subject per condition, then
> there must be *huge* inter-subject variation in order to support a
> subject random effect. I'd think that plotting the
> subject-by-condition means would still be useful in these
> circumstances.
>
> Roger
>
>
> On May 12, 2011, at 2:45 AM, J. Vogels wrote:
>
>> The condition means are as follows (1=pronoun; 0=no pronoun):
>>
>> a b c d e f
>> 0 0.12765957 0.11956522 0.91304348 0.86046512 0.74725275 0.77528090
>> 1 0.87234043 0.88043478 0.08695652 0.13953488 0.25274725 0.22471910
>>
>> so many of them around 90%. Would that be a problem?
>>
>> The subject-by-condition means are not very informative, because
>> each subject has only 2 responses per condition. So the means here
>> are either 0%, 50%, or 100%.
>> Starting with the model with the full fixed effects structure and
>> the random slope for cAGVIS, a likelihood-ratio test indicates that
>> removing the interaction term between the fixed effects does not
>> result in a significant difference (p=.096). A second test shows
>> that further removing cAGVIS results in a difference in fit that
>> does not reach the .05 significance level either (p=.056). However,
>> comparing a model with only cAGTOP as predictor to the first model
>> does give a significant result (p=.031), suggesting that cAGVIS has
>> some influence.
>>
>> @Ian & Daniel: I used the anova() function in R to do model
>> comparisons. When the model did not converge, I started by removing
>> the interaction terms for the random slopes, first in the item
>> effects, then in the subject effects, as suggested by Florian
>> Jaeger in his blog. Next, I tried either removing cAGVIS or cAGTOP,
>> again first for the items. The first model that converged had
>> by-subjects random slopes for cAGVIS and cAGTOP, and no by-items
>> random slopes.
>>
>> Jorrig
>>
>>
>>
>> > -----Original Message-----
>> > From: ling-r-lang-l-bounces@mailman.ucsd.edu [mailto:ling-r-lang-l-
>> > bounces@mailman.ucsd.edu] On Behalf Of Levy, Roger
>> > Sent: woensdag 11 mei 2011 20:17
>> > To: ling-r-lang-l@mailman.ucsd.edu
>> > Subject: [R-lang] Re: lmer: Significant fixed effect only when random
>> > slopeisincluded
>> >
>> > Just as a brief follow-up: since this is categorical data, given the
>> > magnitudes of some of the coefficients in question I would indeed worry
>> > a bit. It looks like some condition means may be close to 100%, but
>> > that different subjects may have really dramatically different behavior
>> > and that this factor may dominate everything else. Also, getting close
>> > to 100% can mess with the Z statistic. What do your condition-mean and
>> > subject-by-condition mean tables look like? And what do likelihood-
>> > ratio tests for your fixed effects tell you in these models? You may
>> > be fine and the random-slope model you included in your first message
>> > may indeed be a good one for interpreting your data, but eyeballing
>> > these tables would be useful as well.
>> >
>> > Roger
>> >
>> >
>> > On May 11, 2011, at 6:04 AM, Finlayson, Ian wrote:
>> >
>> > > What was the random effect structure of this first converging model?
>> > As I said earlier the significant interaction seems fine (as does your
>> > explanation), but I’m just curious about how you carried out backward
>> > stepwise elimination when the full model didn’t converge.
>> > >
>> > > Ian
>> > >
>> > >
>> > > From: ling-r-lang-l-bounces+ifinlayson=qmu.ac.uk@mailman.ucsd.edu
>> > [mailto:ling-r-lang-l-bounces+ifinlayson=qmu.ac.uk@mailman.ucsd.edu] On
>> > Behalf Of Jorrig Vogels
>> > > Sent: 11 May 2011 11:56
>> > > To: ling-r-lang-l@mailman.ucsd.edu
>> > > Subject: [R-lang] Re: lmer: Significant fixed effect only when random
>> > slopeisincluded
>> > >
>> > > Hello Ian,
>> > >
>> > > In the full model, I included random slopes for cAGTOP and cAGVIS and
>> > their interaction for both subjects and items. However, this did not
>> > converge. The first model that converged showed a significant
>> > interaction between cAGTOP1 and cAGVIS, but not between cAGTOP2 and
>> > AGVIS.
>> > >
>> > > Jorrig
>> > >
>> > >
>> > > From: Finlayson, Ian
>> > > Sent: Wednesday, May 11, 2011 12:36 PM
>> > > To: Jorrig Vogels ; ling-r-lang-l@mailman.ucsd.edu
>> > > Subject: RE: [R-lang] lmer: Significant fixed effect only when random
>> > slope isincluded
>> > >
>> > > Hello,
>> > >
>> > > I assume that you only have one random slope because the removal of
>> > the other two (cAGTOP, and its interaction with cAGVIS) didn’t
>> > significantly harm fit. Were the fixed effects for the interaction
>> > significant in the full model?
>> > >
>> > > FWIW, I have seen this happen before and it seems perfectly
>> > reasonable to me that an effect may only become significant after
>> > controlling for some of the noise.
>> > >
>> > > Ian
>> > >
>> > > From: ling-r-lang-l-bounces@mailman.ucsd.edu [mailto:ling-r-lang-l-
>> > bounces@mailman.ucsd.edu] On Behalf Of Jorrig Vogels
>> > > Sent: 11 May 2011 10:50
>> > > To: ling-r-lang-l@mailman.ucsd.edu
>> > > Subject: [R-lang] lmer: Significant fixed effect only when random
>> > slope isincluded
>> > >
>> > > Dear R users,
>> > >
>> > > I have a logit mixed model with two categorical predictors (two types
>> > of salience measures) and a categorical dependent variable (pronoun
>> > used Y/N). One predictor has 2 levels, and the other has 3. I centered
>> > the 2-level predictor, and transformed the 3-level predictor into two
>> > binary predictors using contrast (sum) coding. I determined the random-
>> > effects structure by starting from a full model, and eliminating step
>> > by step all terms without a significant contribution to the model.
>> > >
>> > > In the final model, I end up with random intercepts for subjects and
>> > items, and a by-subject random slope for my 2-level predictor. In this
>> > model, I get significant interactions between the fixed factors, which
>> > I had not expected to be significant by just looking at the data.
>> > Removing the random slope from the model completely eliminates these
>> > interactions, but model comparison suggests the random slope should be
>> > included. I have attached the two model summaries below.
>> > >
>> > > Now my question is: is it normal to find such a large influence of
>> > random effects on the fixed effects structure? How do I know the
>> > interaction effects are not spurious? And what exactly do these
>> > findings mean? Participants varied greatly in their reaction to
>> > predictor B, but when this variation is accounted for, predictor B
>> > affects pronoun use, but differently for each level of predictor A?
>> > >
>> > >
>> > > Jorrig Vogels
>> > > PhD candidate
>> > > Tilburg Univ., Netherlands
>> > >
>> > > ================================================================
>> > >
>> > > Model with random slope:
>> > >
>> > > Generalized linear mixed model fit by the Laplace approximation
>> > > Formula: PRO ~ cAGTOP * cAGVIS + (1 + cAGVIS | SUBJ) + (1 | ITEM)
>> > > Data: vislingag
>> > > AIC BIC logLik deviance
>> > > 318.4 361.4 -149.2 298.4
>> > > Random effects:
>> > > Groups Name Variance Std.Dev. Corr
>> > > SUBJ (Intercept) 49.6457 7.0460
>> > > cAGVIS 21.8342 4.6727 0.663
>> > > ITEM (Intercept) 1.3205 1.1491
>> > > Number of obs: 544, groups: SUBJ, 48; ITEM, 12
>> > >
>> > > Fixed effects:
>> > > Estimate Std. Error z value Pr(>|z|)
>> > > (Intercept) -2.578 1.217 -2.117 0.03422 *
>> > > cAGTOP1 -6.627 0.913 -7.259 3.90e-13 ***
>> > > cAGTOP2 9.868 1.502 6.569 5.05e-11 ***
>> > > cAGVIS -1.699 1.008 -1.685 0.09207 .
>> > > cAGTOP1:cAGVIS -3.223 1.170 -2.755 0.00587 **
>> > > cAGTOP2:cAGVIS 3.120 1.371 2.275 0.02289 *
>> > > ---
>> > > Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>> > >
>> > > Correlation of Fixed Effects:
>> > > (Intr) cAGTOP1 cAGTOP2 cAGVIS cAGTOP1:
>> > > cAGTOP1 0.075
>> > > cAGTOP2 -0.041 -0.867
>> > > cAGVIS 0.535 0.108 -0.059
>> > > cAGTOP1:AGV 0.074 0.562 -0.346 0.128
>> > > cAGTOP2:AGV -0.049 -0.480 0.528 -0.054 -0.668
>> > >
>> > >
>> > > Model without random slope:
>> > >
>> > > Generalized linear mixed model fit by the Laplace approximation
>> > > Formula: PRO ~ cAGTOP * cAGVIS + (1 | SUBJ) + (1 | ITEM)
>> > > Data: vislingag
>> > > AIC BIC logLik deviance
>> > > 324.3 358.7 -154.2 308.3
>> > > Random effects:
>> > > Groups Name Variance Std.Dev.
>> > > SUBJ (Intercept) 21.63217 4.65104
>> > > ITEM (Intercept) 0.61539 0.78447
>> > > Number of obs: 544, groups: SUBJ, 48; ITEM, 12
>> > >
>> > > Fixed effects:
>> > > Estimate Std. Error z value Pr(>|z|)
>> > > (Intercept) -1.41142 0.77639 -1.818 0.0691 .
>> > > cAGTOP1 -4.59707 0.52139 -8.817 <2e-16 ***
>> > > cAGTOP2 7.13115 0.84489 8.440 <2e-16 ***
>> > > cAGVIS -0.35538 0.40416 -0.879 0.3792
>> > > cAGTOP1:cAGVIS -0.59940 0.58255 -1.029 0.3035
>> > > cAGTOP2:cAGVIS -0.08268 0.56682 -0.146 0.8840
>> > > ---
>> > > Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>> > >
>> > > Correlation of Fixed Effects:
>> > > (Intr) cAGTOP1 cAGTOP2 cAGVIS cAGTOP1:
>> > > cAGTOP1 0.070
>> > > cAGTOP2 -0.040 -0.867
>> > > cAGVIS 0.000 0.061 -0.036
>> > > cAGTOP1:AGV 0.038 0.082 -0.012 0.102
>> > > cAGTOP2:AGV -0.020 0.008 -0.037 0.037 -0.575
>> >
>> > --
>> >
>> > Roger Levy Email: rlevy@ucsd.edu
>> > Assistant Professor Phone: 858-534-7219
>> > Department of Linguistics Fax: 858-534-4789
>> > UC San Diego Web: http://idiom.ucsd.edu/~rlevy
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>
> --
>
> Roger Levy Email: rlevy@ucsd.edu
> Assistant Professor Phone: 858-534-7219
> Department of Linguistics Fax: 858-534-4789
> UC San Diego Web: http://idiom.ucsd.edu/~rlevy
>
>
>
>
>
>
>
>
>
>
>
More information about the ling-r-lang-L
mailing list