It isn't mentioned anywhere that I've seen, but $filename *is* case-sensitive on Windows.
It means that while C:\Windows\TEMP\php123.tmp may have been uploaded, C:\Windows\Temp\php123.tmp was not.
I found this out because I was using realpath() on the filename which 'fixed' the case (my Temp folder is in titlecase, not uppercase - thank you Vista).
Anyways, the problem was that PHP used %TEMP% to determine the destination for the uploaded file, and %TEMP% used the all-capitals version of the path. Changing it to use titlecase instead + restarting Apache fixed the problem.