I looked into this as well. Basically, depending on where the company is located, accepting online payments becomes a huge issue. A lot of countries (my home country included) just do not have good financial infrastructure to be able to handle online payments.
There's also the administrative overhead and weird tax laws that come into play. Basically, if I start something elsewhere, it would be the opposite of "lean."
What does it take for a foreigner to own a LLC-like company in Ireland (LLC is a see-through entity so individual just pays standard income tax)? For e.g. how easy is it to set it up compared to a Delaware Corp, which can be done in a single day?
Currently you need a minimum of two directors to setup a company, so it probably isn't great for this. Ireland isn't really a great location for traditional companies, but great for multinationals due to the double taxation loopholes. There are EU countries with lower corporate tax rates, but they don't have the loopholes.
In the UK you can register a company online [0], however you need a UK address. There are various formation agents that will register the company and provide an address for a small fee (<£100) [1]. If you want a business bank account I think you'll need to do it in person though. You will probably want to get an accountant too which will be ~ £1k/year, although you can do it yourself.
It's also really, really easy and cheap to set up an international business corporation (IBC) in well known offshore locations like the British Virgin Islands or Gibralter. It's just because of the payments issue that I suggested Ireland.