TwoJStudios, thanks for the reply and the ideas.
siredesigns - I hadn't seen that, it does look pretty similar, but the idea I had in my mind was for it to be completely theme-independent. i.e. You'd be able to use any theme you wanted to display your different items - hence the API.
There'd be extensive docs to show people how to do this, naturally.
I like what Freelance Folder have done there, I must admit, but I'm hoping that my plugin would be significantly more customisable and have other features including a genuine CRM, To-Do list, project management (i.e. break a project down into several different sub-items (linked to the To-Do list) and then you can assign other WordPress users to these tasks (if you have more than 1 person) and you can set 'due by' dates for these things etc.) There'd also be a time-tracking tool in there to help you invoice properly.
The private communications that TwoJStudios mentions is a nice idea although I'm not 100% certain on having a file manager. Being able to upload files to different projects would be a no-brainer and even allowing clients, once they've logged in, to be able to upload files and make comments. A full filesystem would probably be complete overkill, imo.
Interesting discussion guys, thanks so far.