We have loved the Eye Wonder books, by the publisher Dorling Kindserly (often displayed as just, DK Books). They are a bit old for 4, but we've been looking at them since my daughter was 4, too. They are picture encyclopedias about particular topics. We just started with looking at the photographs, and maybe reading some of the main points, and are now (she is six) reading all the captions.
When she gets older, the same publisher has a more advanced picture encyclopedia (Eye Witness), which gets into more detail. The younger ones might be, for instance, "Ocean," but the older kids' books are "Seashore," "Fish," "Whales" "Dolphins" and so on... with more room to get into details about each.