Is it more energy efficient to remove the ballast?

No, removing the ballast does not make your LED conversion consume less electricity. The ballast is an essential component which converts and regulates power delivered to a fluorescent tube. LEDs also require this power conversion and regulation; it can not be avoided. While it is true that the ballast is not 100% efficient  (nothing is), the LED Tube Lights which require removing the ballast have, simply put, a ballast inside the tube itself. As a result, those products suffer from the exact same inefficiencies as the existing ballast.