Quilted jacket with knit panels