I'm admitting a sin: I've never read Catcher in the Rye, but I am aware if its sexual connotations and will probably pick it up in the wake of Mr. Salinger's death. The only part of the comment that could potentially be sexual harassment is the Holden Caufield comment. So, to figure out what Tony's potential context was, I did what any good student who hasn't read his homework would do: I checked Spark Notes, and it reads: "Moreover, he is disturbed by the fact that he is aroused by women whom he doesn’t respect or care for, like the blonde tourist he dances with in the Lavender Room, or like Sally Hayes, whom he refers to as “stupid” even as he arranges a date with her. Finally, he is disturbed by the fact that he is aroused by kinky sexual behavior—particularly behavior that isn’t respectful of one’s sex partner, such as spitting in one’s partner’s face. Although Holden refers to such behavior as “crumby,” he admits that it is pretty fun, although he doesn’t think that it should be." If this is what Tony meant by her being a "Holden Caufield fantasy", I have no problem with some time off.