It started with an intermittent problem from a client. In some cases their users were being prompted to download a JavaScript file that had our product's name somewhere in the file name.
That file was a localization file that SPFx uses. When you create an SPFx webpart or application customizer, there is a "loc" folder in which you put files that contain the character strings that vary according to language. You can find more details about SPFx localization here. Each file in that folder will have a name corresponding to the locale, for example "en-us.js" and "fr-fr.js". These files will then be packaged under a longer name, consisting of
(the name in the project)_en-us_(some long hex string).js
It turns out that this file was the one that was sometimes downloaded. Looking more deeply at why it was being downloaded sometimes when other JavaScript files weren't, it turns out that there was a difference in the http headers that were being sent.
The major difference between the .js file that was sometimes being downloaded and the ones that weren't was that the ones being downloaded had a "content-disposition" header and the the others didn't. In theory, a content-disposition of "attachment" with a file name means that this file should be downloaded, not sent to whatever is responsible for the MIME type represented in the Content-Type header. In practice, browsers know that web servers often get the http headers wrong, so they look for other evidence of the intent. Most of the time they look at the content-type and only download the file if its value is "application/octet-stream". They apparently also look at the context in which is it being called.
In the case of the client, the download usually came when the user had just logged out and logged back in again. Their authentication was not the usual one, so it involved some redirection to and from their on premise identity management. If the caching of this JavaScript file had expired but other files were in the cache, it could very well be the first file being fetched from the SharePoint Online domain, so no context is available for the browser to take into account. In that case, it seemed, it respected the incorrect Content-Disposition header. Also, they were not using and did not want to use any CDN to serve these files, unlike most of our other clients with a similar architecture. Files served by a CDN tend to have the correct http headers.
That behaviour is easily reproduced, by going directly to the appropriate folder of the App Catalog's ClientSideAssets library and clicking on the file name. That file prompted the user to download it, while other .js files in the same folder did not.
I won't bore you with everything that we tried to figure this out. As some point we were convinced that this happened to with application customizers but did not happen to webparts. As it turns out, the problem was with our naming convention. using the words "modern application customizer" in the name rather than "modern webpart" made a difference.
The difference was the name of the JavaScript file, specifically the length of the name. The file name was already pretty long, but the difference between the words "webpart" and "application customizer" took it over some magic threshold. Our naming convention sabotaged the attempt to reproduce the problem with a "hello, world" test. If looks like a file with a name of 81 characters has the correct http header, but one of 87 has the wrong one. I'm still not sure what is the exact limit, and whether it is a limit on the file name or the path or the full URL. None of those lengths seem close to some magic number, but it doesn't matter since shortening the part of the file name that we are responsible for fixes the problem.