Seems like Terence Stamp was a little unhappy about his experience on the set of Episode I. AICN reports the following from teletext.co.uk:
He told Lorraine Kelly on GMTV: "Actors prefer to work with actors. When I arrived on set for Episode 1 George Lucas said, 'I've given Natalie (Portman) the day off'.
"So he pointed to a piece of paper on a post and said pretend that's her. They couldn't afford me again."
I guess we won't be seeing Chancellor Valorum in Episode II, eh?