Yes, you need to address this with your doctor. It depends on where the nerve impingement is taking place and why. Yes, it can be a benign, chronic condition, at the least... or something more. Since it is bothersome, the Doctor may do additional testing/x-rays/mri, if warranted, and possibly determine that Physical Therapy would be beneficial for you. You don't want to mess with numbness - it's concerning! In general, people will run to a dr. because of a severe pain but... with numbness, it just doesn't seem as concerning because it doesn't really hurt and "it doesn't happen all the time". Yes, it's concerning - you have to determine the source of the cause and severity to make that judgement call. It's good that you are being proactive on this.
If they determine Physical Therapy would be beneficial - I would go for it. It is amazing what they can teach you about the mechanics of your body!
Good luck.