When it’s a little higher, all of the other air molecules around will have had similar experiences and will also be moving a little slower. When a ball or air molecules are bouncing, they exchange “kinetic energy” for “gravitational potential energy“, but overall the total energy stays the same. Temperature drops as you go up, but hot air still rises through cold air. The warmer layer has more energy than the colder layer just beneath it (so it’s hotter and lighter), and this stops the air layers from mixing. The cooler air doesn’t have the extra energy needed to rise through the warmer air above it, so the air layers stack up like oil on water.