You might want to get in touch with Andrew who developed the Wordpress Connector and see if he has had any experience with that many pages. It's a relatively new module so may not have been tried on a site that big.
I did contact Andrew and he was kind enough to look into the problem. Well according to him the the default Wordpress XML-RPC API does not allow to fetch any sub-set of posts hence we can't get limited number of post at a time. He adviced increasing the memory for PHP to few GB but again no luck.
However as a work around I deleted my post and brought it down to 600 post. Now atleast these post show up in External Content >> WordpressPostSource. But again the actual import fails. I tried queuing it and got an error, I have attached the file.