E.B.
I think I would go with a bigger agency, one that can send serval nannies to your house so that you can interview them and choose one you like. Also, you can do a separate background check through a private investigation firm, which I highly recommend.
I don't think I would really trust a women, that I never met, to send someone to my house to watch my child. Is she licensed to run a nanny company? Please do as much "investigating" as you can, it will put your mind at ease.