If you go much higher the balloon will leak heat through black body radiation faster than you’ll be able to heat it. And even if you insulated the balloon in a way that didn’t add too much weight you’d eventually get to the top of the atmosphere and hot air balloons won’t go into space because they don’t have any propulsion systems.

If you fill it with helium, it will go into space, and then it will immediately explode.