A family forced to move into a temporary home have been left in a property blighted by damp.