D.H.
One problem with giving him every weekend, though, is that most get-togethers or parties are on the weekends, which means you won't be able to take her with you. Anytime you have weekend plans, there will be issues about him having to give up his time if you want to take her with you. Most non-custodial parents get every-other weekend. But if you want her to have more time with him, can he spend a few hours with her in the evenings during the week?
My husband and I both had visitation issues to deal with. I never had any court-ordered visitation for my son - he went to his dad's every-other weekend, and if his dad wanted and evening during the week, that was fine. He also took him on vacation with him a couple times, and if either of us had a family get-together on the other's weekend, we worked it out.
My husband had to get a court-order for his visitation, including getting the kids for our wedding. His ex had a tendency to change her mind when he'd go over to pick them up.
Most holidays we worked out - mother's day with mom, father's day with dad, Christmas eve with us (for all of them) and Christmas day with their other parent.
good luck