i love this dream!! It is completely do-able!! I completely understand that practical side as i have the same side.. Yet, yes it is doable.. just start planning little bits at a time.. Which for most of us, is saving the $$.. put little bits away at a time, that you will not notice.. that is how i started, and figure out how much you need.. in most countries you need less than you think, even with a family of 4... also.. you always have the option to teach english online since you are a native english speaker (of course this is just a backup, but I need 4 backup plans!).
but it all starts with a dream:)