C.S.
Only you know the tone she is taking and since you are sharing with us, it must be in an upsetting tone. Hopefully, what is meant is that if you don't behave you have to stay home and not go out with us (parents).
Try to think of the things you do with her and the things she likes to do. My mother in law loves to decorate the home. She has left me pot racks and magazines to try to light my fire. The magazines are actually in the trash and the pot rack is in the garage. Likewise, she has always told my daughter that when she is old enough to stay with her, they would do all sorts of fun stuff and go shopping together. While my daughter loves her grandmother, she doesn't enjoy shopping, doesn't enjoy home interior departments, is full of energy. Are you fitting the 5 year olds bill?
I would take caution in mentioning to here, but at some point you can say something that politely lets her know that staying home with grandma is not so bad and hopefully she realizes what she is saying.
Sometimes people just don't think about how their comments are not helping or not appreciated. Just his morning at practice a parent apologized for saying something 3 weeks ago at a game. I had no clue what she said and she reminded me...well, at that point I remembered. I just told her it was not problem and told her my daughter wasn't offended. Sometimes it just takes a while to click. Hopefully that's the case.