This is magic

[arunpatala]

I tried one of the dflash models for speculative decoding with sglang and i immediately got more than 2.5x speedup. Even at the largest batch sizes.
It almost felt like magic. What a nice idea and good use of diffusion models as speculative models.

Just wanted to thank for this great repo.