Goop, I think your lack of any makeable low hand is the only thing that really kills you here, and could help justify the fold.
I have no problem with the SB completion for the reasons you mentioned. Yeah, that's not a hand you'd play UTG, but the pot/implied odds certainly justify seeing a flop.
You got a flop that was just good enough to keep you interested, but you could quite conceivably be up against a made straight, or set of jacks. I like the pot bet, and wouldn't really read too much into the flat call from an admittedly flaky opponent.
By the turn there are 9 BBs in the pot (IIRC), but now the added low draw really hurts your hand, since some of the cards that will nut you could very well lose you half the pot. You have to consider that in your odds calcs. You have 9 outs to the flush, but 6 of them put out a low (and the 4c would pair the board, which could be worse for you if he has JJ). 2 aces and 2 queens should give you a scoop (most likely, assuming he doesn't have AA or QQ in which case you'd be toast), but the rest of the cards in the deck hurt your hand.
I think a fold was a wise move, variance or no variance issues. Calling the turn appears to be a -EV move regardless.