I may have found the reason why the response profiles I measured for Robin’s Miles stars varied from star to star.
I replaced the 23 micron slit in the LISA with a 50 micron slit and repeated the exercise. This time the response profiles of all of the stars were almost identical with only minor differences arising from the subjective smoothing process. Comparing raw uncorrected spectra of each star with the narrow and wide slits, it is clear that I was progressively losing the red end of the spectrum with the narrow slit compared to the wide slit. The effect was worse with stars of later spectral type.
My measurements of Miles stars were all made in January and February when the seeing was consistently bad, 4” or worse as measured from photometry on the same nights. With my setup, >4” seeing gives a star size at the slit >31 microns. With a 23 micron slit this means the star image is considerably larger than the slit which means I am losing light.
The wavelength-dependence of diffraction indicates that for my setup the seeing disk is about 5 microns wider for light at 7400A compared to 3900A. So as well as losing light, progressively more light is likely to be lost towards the red end of the spectrum, which is what I see. With the wide slit, the full seeing disk passes through the slit so no light is lost and the spectrum is unaffected. The response profiles for all the Miles stars should be the same, as indeed they are.
On nights with seeing better than 3”, the full star image passes through a 23 micron slit and the effect will be much less obvious which agrees with my experience.
I don’t recall seeing this mentioned as a possible source of error compared to the problem Christian Buil has mentioned with atmospheric dispersion.