The major difference between the .js file that was sometimes being downloaded and the ones that weren't was that the ones being downloaded had a "content-disposition" header and the the others didn't. In theory, a content-disposition of "attachment" with a file name means that this file should be downloaded, not sent to whatever is responsible for the MIME type represented in the Content-Type header. In practice, browsers know that web servers often get the http headers wrong, so they look for other evidence of the intent. Most of the time they look at the content-type and only download the file if its value is "application/octet-stream". They apparently also look at the context in which is it being called.
That behaviour is easily reproduced, by going directly to the appropriate folder of the App Catalog's ClientSideAssets library and clicking on the file name. That file prompted the user to download it, while other .js files in the same folder did not.
I won't bore you with everything that we tried to figure this out. As some point we were convinced that this happened to with application customizers but did not happen to webparts. As it turns out, the problem was with our naming convention. using the words "modern application customizer" in the name rather than "modern webpart" made a difference.