Dhvani Patel

FHE Notes

Note: There is an overall general lack of good educational and developer resources. The central community in the space is FHE.org.

What

  • Fully Homomorphic Encryption (FHE) lets you compute directly on encrypted data so that the results, when decrypted, match what you would have gotten by computing on the raw data—without ever revealing the data itself. In short: you can run programs on secrets without seeing them.
  • Computation on encrypted data is possible because of homomorphism. This means the structure of the data is preserved after it’s encrypted to allow certain operations.
  • Most of the schemes are based on the hardness problems of learning with errors (LWE) and ring learning with errors (RLWE), which are part of a broader field of cryptography known as lattice-based cryptography.
    • Note: Lattice operations are highly parallelizable which is good for hardware acceleration, and they are also assumed to be post-quantum secure.
  • Diagrams
    • FHE flow: x is encrypted with a public key, a circuit for f is evaluated on Enc(x) with a public evaluation key, and the secret key decrypts Enc(f(x)) to f(x)
    • A homomorphism δ from A to B where δ(x *A y) = δ(x) *B δ(y), with the example δ(x) = e^x turning addition into multiplication

Types

  • Partially Homomorphic Encryption
    • only simple additions and multiplications (not both)
    • ElGamal method for multiplication
    • Paillier method for addition/subtraction
    • Note: faster to use these than FHE if your application only needs it
  • Somewhat Homomorphic Encryption
    • hard coded for statistics
  • Leveled Fully Homomorphic Encryption
    • hard coded for machine learning
  • Fully Homomorphic Encryption
    • general purpose

History

  • Originally called privacy homomorphism, introduced as a property of RSA in 1978.
  • First FHE scheme by Craig Gentry in 2009; paper.
  • BFV introduced in 2012 which operated on integers. CKKS introduced in 2016 which operated on floating point values (good for ML).
  • CHIMERA recently came out in 2025 which allows switching schemes. This is useful when you have different constraints for different environments. eg Balvi project scheme switched from TFHE on device to CKKS on the FHE server.
  • One constraint is existing schemes usually operate over only small primes which limits certain applications. In June 2025, a paper was published with a new scheme that operates over large primes and is faster than tfhe-rs by 2000x.
  • 4 different generations of FHE so far; the space is slowly maturing
    • Zama’s timeline of about 40 years of homomorphic encryption, from privacy homomorphisms in 1978 through four generations of schemes
    • Slide: FHE is maturing, listing communities and standards (fhe.org, homomorphicencryption.org, open-source projects, iDash) and government projects (DARPA DPrive, NIST and EU standardization)

Decryption Setup

  • There are many different kinds of use cases and setups for using FHE
  • Provider/Consumer are the same:
    • A single consumer wants some computation done on their own secret data.
      • They can encrypt their secret using their public key, get a third party to apply the computation on the encrypted data, and then decrypt the result using the corresponding private key.
      • They cannot run this locally either because it is expensive or the computation is private, eg AI model.
      • eg I want to run an AI model over my private journal
    • Multiple consumers want some computation done on their own respective secret data
      • Each consumer will encrypt via threshold encryption. This means decryption of the result requires the participation of all consumers.
      • eg FrogZone
      • This requires coordination and liveness of all consumers during decryption and becomes a bottleneck in the system when there are lots of consumers and they are geographically far away from each other
  • Provider/Consumer are different:
    • A single provider wants to allow consumers to use their secret data without revealing it
      • The provider can encrypt their data and send it to the consumer
      • Consumers can run algorithms on the encrypted data, and request the provider decrypt the result.
    • Multiple providers want to allow consumers to use their secret data without revealing it
      • Use threshold encryption to encrypt and decrypt
    • eg hospitals can encrypt their data and allow for data analysis
  • One challenge is how do you know that an encrypted output is from a specific FHE program, and not something else?
    • You can use zkSNARKS here to prove computation
    • This further impacts performance though this is also an active area of research.
  • Alternative “FHE” like setup using TEE:
    • Without FHE at all:
      • Instead of using threshold encryption, everyone encrypts their private data using the TEE key.
      • Everyone sends their data to the TEE where the program decrypts the data, runs a computation, and sends back the result to everyone
      • With a TEE, even the runner of the TEE cannot see the decrypted data. They can however turn off the TEE. EigenCloud’s vision is to make it so TEE’s are run blockchain style.
    • With FHE but as a decryption mechanism:
      • Instead of using threshold encryption, everyone encrypts their private data using the TEE key.
      • The FHE computation runs on some accelerated hardware, and outputs (encrypted_result, program_proof)
      • The program running in the TEE checks the program proof and then whatever other conditions to allow decryption
      • In normal threshold decryption, all players need to follow a certain protocol to decrypt. However, they can all collude and change it. Eg in FrogZone, players could all just decrypt the entire board.
      • If the decryption condition is programmatic, then this is not possible.
      • This is similar to witness encryption but via a TEE vs cryptography.

Libraries

Treemap of FHE library usage: OpenFHE 39%, FHEW/TFHE/TFHE-rs 37%, Concrete 27%, SEAL 27%, Lattigo 24%, HElib 10%, and smaller libraries

  • A comprehensive list can be found on this Github list: https://github.com/jonaschn/awesome-he.
  • OpenFHE is the OG library that has been most used in the past with TFHE libraries getting more traction recently.
  • Zama is an active company in the FHE space and offers the following main libraries:
    • tfhe-rs: implementation of TFHE in Rust; latest performance.
    • concrete: compiler to convert regular Python programs to FHE programs
    • concrete-ml: concrete but specific for ML programs
    • fhevm: framework for writing smart contracts that use FHE
    • Note: these are not free to use commercially, you must pay for a license. Even to contribute to the open source, you have to sign a legal agreement.
  • Poulpy is a very new library that is structured to be modular compared to the previous ones.

Phantom VM

  • Phantom VM is a VM built using FHE-circuits that executes encrypted programs on encrypted inputs.
  • It is very new, and based on their prior work implementing FHE-based RAM; this old demo repo no longer works, but has been incorporated into the VM.
  • Note: It is extremely slow and constrained.

    • On average a cycle takes approx. 655ms on 32 cores (benchmarked on AWS r6i.metal instance) for a ~1KB encrypted program and a ~4KB encrypted memory.

    • ~0.6s for a single compute cycle, on a machine that is $8/hour
    • Running the template simple program takes 20 minutes on my M3, 36GB ram laptop.
  • Note: Currently, the program and input both need to be encrypted with the same private key, and threshold encryption is not supported. This makes it impractical to use if the developer wants the program to be private from the user, and the user their inputs private from the developer. See discussion here.

Performance

  • One of the key time consuming steps in FHE is the procedure of bootstrapping.
    • To prevent dictionary attacks on the cipher text, a small noise is added to the plaintext. ie Enc(x + noise).
    • Therefore, after each addition operation and multiplication operation, this noise grows. It grows linearly for addition and multiplicatively for multiplication.
    • To clean up noise that comes up, a method called bootstrapping is used which is a clever way to remove the noise by reencrypting and decrypting under a derived key.
    • However, bootstrapping has been historically slow, though recent schemes have made a lot of progress here.
    • One way to avoid bootstrapping is if you start with a large error bounds and parameters. The larger you go, the higher the fixed amount of operations you can do before the plaintext becomes unrecoverable.
  • The trend is that performance is rapidly improving. The vectors for improvement are: better schemes, better software engineering, better hardware.
    • Chart of the speed of computing on encrypted data on IBM’s HElib, falling from infeasible in 2010 to 0.0001 seconds per bit in 2014
    • Slide: HE is getting faster 8 times every year, with bootstrapping time falling from 1800s in 2011 to 0.5s in 2021
    • Craig Gentry’s slide contrasting TFHE (latency, exact bits, blockchain) with CKKS (floating point, bandwidth, AI models)
  • While FHE is embarrassingly parallel, in practice it is hard because of memory bottlenecks and synchronization of threads.
  • A lot of effort right now is being put towards better hardware with some companies like Cornami having already produced chips. Optalysys is another interesting company here making optical computing chips. Others include Belfort, Niobium, and Fabric.
    • FHE fits well into the “circuit model” so possible to make custom chips for it versus SMPC.
    • Big boost from AI hardware because the issues are similar
  • One fundamental limitation in performance that seems to be at the cryptographic level is that the FHE program has to run in worst-case time vs the regular program will run in average time.
    • All programs have a best, worst, and average case run time. This is dependent on any inputs to the program.
    • To not leak any information about the inputs, an FHE program must run through all if branches and loops must run through their max iteration counts. Therefore, the FHE program will always run in worst case time.
    • Maybe the way you can get around this is using obfuscation of the program (ie encrypted program), but that needs more research.
  • Another related challenge which impacts performance is you don’t know if the program terminated. What libraries like Phantom do for this is add an infinite loop to the end of the program. When no registers change after some time period, you can safely assume the program terminated. The Balvi project had a similar approach.
  • Likely after 1-2 years, things will be way more production ready.

Applications

  • There are small niche deployments: https://www.jeremykun.com/fhe-in-production/.
  • Private information retrieval (PIR) has been a popular area of research for application of FHE
    • PIR lets a user retrieve an item from a database without the server learning which item was requested.
    • This can be easily achieved via FHE with a basic scheme by taking a homomorphic dot product between a query vector and the database vector, and then summing up the values.
  • Healthcare has had many POCs but few things in actual production.
    • One reason is due to lack of maturity, and being able to do certain things without FHE, which is simpler.
    • eg Truveta Consortium has $20M+ from hospitals in the US for analytics on private data
    • Slide: FHE is not yet mature, comparing visibility, compliance, and integration for Wiz, Datavant, and cryptography products
  • AI recently is becoming an active area of research due to better models from training on private data.
  • Compared to SMPC, FHE can be better for enterprise because there’s no TPRM (third-party risk mgmt), and the data doesn’t move. You don’t need to create a hole in your firewall for sensitive data
  • FHE will need to open up new opportunities for B2B vs just check a security box
  • One UX challenge is communicating to users that none of their private data is being sent to perform a computation.