Carlos Lee fielding is a minor mismatch and is no big deal so I just don't buy that excuse. Lee also gets just as many chances to field a ball as any LF does so it isn't the same thing at all. Kickers in the NFL don't have to play offense, the goalie in hockey isn't expected to play the wing, pitchers in baseball shouldn't be hitting. They just do not get enough reps to stay effective as hitters and it increases their odds of getting hurt. This is one of those things that stays around because of the history of the game. Catchers still get 400+ PA as a starter in baseball, pitchers get like 75 a year, the comparison isn't really valid at all. If pitchers actually hit 400 times a year and just stunk at it I'd say you are correct, the fact that they don't get enough AB to effectively keep their hitting skills is the entire point.