A div is a block element and will stretch to fill all the space available to it. It is not sized by its content but by how much room there is available in its containing block.
You placed an image inside the div but the div is not sized by the image as the div is already as wide as the space that is available. (Imagine that the image is 100px and the div fills all 300px space available and then you understand why a border on the div does not tightly wrap the image.)
If you made the image width:100% then it would fill the div and then the border would appear around the image.
When you changed the div to display:inline-block that means that the div will now shrink to fit its content. An inline-block element is only as wide as its content if no width is applied (the same applies to floats).
So the answer to your question is that elements with different
display properties will behave according to the specification for that type of display. When you use display:block the element will fill all the horizontal space available to it. On the other hand display:inline:block is a shrink to fit algorithm and has no width other than the width of the content it holds. There are various other values for the display property and all have differences in one way or another so need to be studied and learned
(Incidentally you could have just applied the white border to the image itself rather than a parent div.)