It's a bug. What adobes hold up getting these bugs fixed I cannot say... sigh.
If you save the metadata (CMD+S) before opening in PS is seems to prevent it. Also in the 3.2 RC I don't believe Ive been seeing it. Though the speed issues have still not been corrected.