Yes you unquestionably will since you now have multiple connections competing for bandwidth. It's not a connectivity interrupt, rather reduced throughput share by all clients, which isn't a big deal for that brief period for a download but could cause stuttering on streaming calls and other video playback, if the needed (average) bitrate is higher than the fraction that client gets - made worse by using wifi.
You can use QOS to deprioritize the pi, but then you don't get accurate speedtest results except "real world" of what it sees. Why do you need to run an hourly test?
If you are having performance problems, certainly a log can prove it to an ISP, but not necessarily the cause, would probably still require an on-site analysis by their tech and if just their network congestion, won't do much good. On the other hand if the internet is just disconnecting, you can log that with far less interruption than a speed test might cause.