to run with the hare and hunt with the hounds; to tread a fine line

curiosity killed the cat

to do a moonlight flit, to levant (to move out of a place without telling the owner and without paying one's rent, to disappear with one's possessions); to take French leave, to abscond, to do a runner

(literary) to paint the town red, to go out on the town

(figuratively) it's like closing the stable door after the horse has bolted

Alternative spelling of mettre la clé sous le paillasson

(law) for there to be danger in delay

(figuratively) to carry coals to Newcastle

(figuratively) blithely, lightly, rather carelessly

Alternative form of remuer le couteau dans la plaie