HE is incredibly slow, and unlikely to ever be fast enough for common workflows. As in, an HE implementation of an algorithm that runs in seconds in Python on a regular machine might run in minutes or tens of minutes.
Not to mention, to avoid side-channel attacks, an HE scheme still needs to always run the longest possible sequence of operations regardless of input (otherwise, information about the input data is leaked through timing). So an HE version of a quick-sort scheme would always run in O(n^2), otherwise it would leak details about the contents of the list. In some cases it would even have to run in the same amount of time regardless of the size of the list, to avoid leaking information about that.
To be fair, the performance problem you're talking about should affect latency rather than throughput. If you can batch lots of operations (not all controlled by the same user) then you can do things as fast as you can without leaking (much) information.
HE hides the execution complexity from the system doing the computation, so no, it can't do someone else's computation and just wait in between to avoid leaking information, it's designed so that the computing operation order and quantity is simply independent on the input data, i.e. the worst case complexity, and a valid HE scheme would have mathematical proof that it's impossible for the system to find a way to do it faster than the worst case.
Is HE any different from taking the encrypted input data, sticking the program you want to run on the end, and returning that? If anything, it sounds less efficient than that.
HE is designed for running a secret algorithm with secret input data and obtaining secret output data, all on an untrusted computer.
It works a little bit like this: you write your algorithm using only HE computing primitives, then you encrypt it and the input data and send those to the HE runtime, who you don't trust, and who DOESN'T have any way to decrypt your message. The HE runtime executes your encrypted program and sends back the still encrypted output. Once you receive your output, you decrypt it using the key you have kept secret.
That's not how it works, that's what it does. I provided an example of how it could work, but it had better be more efficient than that or it's a silly idea.
One note, first of all: I made one mistake, the algorithm is not secret in HE (the HE runtime has to know it). Only the input data is guaranteed to be secret.
I'm not sure what you mean. This is how HE works, at a high level. The whole point is that you're doing public operations on secret data on a machine that can never know what the data was, nor what response it gave you. Furthermore, the point of HE is also that you know that your data remained secret regardless of how the HE runtime was implemented, as you just send it encrypted information.
Making it more efficient by giving it access to the decrypted data would defeat the whole point. Making it more efficient by optimizing based on the actual data would allow your data to leak through timing side-channels, again defeating the whole point.
Basically, HE refers to computing schemes where DECRYPT(ALGORITHM(ENCRYPT(X)) == ALGORITHM(X). You send [ALGORITHM, ENCRYPT(X)] to the runtime, and it gives you Y. DECRYPT(Y) is then the result of your computation.
As far as we know, this is indeed hopelessly slow, and there is no guarantee that we will ever find fast secure HE primitives.
Not to mention, to avoid side-channel attacks, an HE scheme still needs to always run the longest possible sequence of operations regardless of input (otherwise, information about the input data is leaked through timing). So an HE version of a quick-sort scheme would always run in O(n^2), otherwise it would leak details about the contents of the list. In some cases it would even have to run in the same amount of time regardless of the size of the list, to avoid leaking information about that.