Yes, I've always had some latency too, it just doesn't bother me as much as some users. It isn't anywhere close to a second, though (I'm sure that would bother me).
Everything I've read says the best way to take advantage of the smaller audio buffers in JB is to do the audio on the native side with SLES, to remove dalvik and gc from the equation. Also, using a one-read one-write buffer strategy instead of thread locks, and using a specific format for output to avoid an extra trip through Android's resampler, and doing audio processing from a specific audio callback thread that has higher priority than other threads. These are all things I have planned for the audio plugin I'm writing.