Speed tests seem grossly inaccurate

tgreen · May 2020

The Fing desktop seems to be way off.
This is testing from the exact same machine. I ran Fing before and after the speedtest.net one to be sure it wasn't just a hiccup in the connectivity
Is this a known issue or am I looking at something wrong?
Sure Calgary (5 hr drive) is a bit farther than Edmonton (2 hr drive), but it's not like this is overseas or something and these are significantly different in results

Image: https://us.v-cdn.net/6031733/uploads/editor/on/0jby8qlqwi64.jpg

kltaylor · May 2020

Hi @tgreen welcome to the community and thanks for asking your question.

In regards to speed test results, the addage 'your mileage may vary' is more-than-likely at play here. Through my professional career in and IT-related field, I've learned to not trust Speedtest too much.

With that being said, your screenshots do seem to be off, quite a bit off. Is this a situation where you're testing on your wireless device versus a desktop using an Ethernet connection?

I use a third-party site almost exclusively called testmy.net. I like their UI and their API approach to help you obtain a real speed test with data loads, and not just with speed. You can also set it to continuously test your connection over a period of time to see where peaks and valley's happen, which can be a precursor to an interesting conversation with your ISP.

tgreen · May 2020

This is all from the same machine and same hardwired connection (wireless is disabled)

Never see a discrepancy this off with Ookla, but using different tests, it seems to group differently in DL, but UL all seem to align
These agree the DL being high
speedtest.net --> 253.8 Mbps DL and 15.0 Mbps UP
speedtest.shaw.ca (my ISP) --> 268.7 Mbps DL and 14.5 Mbps UL (but this uses Ookla, so I would expect these to align)
fast.com --> 220 Mbps DL
dslreports.com --> 239.9 Mbps DL and 15.2 Mbps UL

These one sit in the middle
speedsmart.net --> 151.7 Mbps DL and 16.7 Mbps UL

These agree closely with Fing
Fing tool --> 39.9 MbpsDUL and 14.5 Mbps UL
testmy.net --> 23.8 Mbps DL and 12.6 Mbps UL
bandwidthplace.com --> 32.4 Mbps DL and 14.0 Mbps UL
highspeedinternet.com --> 25.7 Mbps DL and 12.2 Mbps UL
performance.cira.ca --> 26 Mbps DL and 11 Mbps UL

Strange. It's not like one is MB/s and the other Mb/s either

kltaylor · May 2020

Thanks for compiling that much information, it's certainly useful. I'm going to ping @Robin on this and see if he can help shed a little more light on this for us.

tgreen · May 2020

kltaylor said:

You can also set it to continuously test your connection over a period of time to see where peaks and valley's happen, which can be a precursor to an interesting conversation with your ISP.

Which sounds like this may be happening this week as I've been looking to document the ISP speeds over time to have this discussion with Shaw. Cable, so there are most definitely noticeable area load issues at certain times of the day

kltaylor · May 2020

I'm a firm believer in collecting as much raw data that you can before presenting it to the ISP, as they seem to have a literal script to recite when anyone makes a remark about their internet speeds.

tgreen · May 2020

Absolutely. Not to mention they have Ookla as their test tool - which shows things are good. I don't really notice an issue most of the time (and the kids stream all day while stuck at home it seems). I'm paying for a 300 Mbps package though, so seeing 10% of that is not very impressive

kltaylor · May 2020

With not knowing where you are geo-located, I know that Ookla is owned by Comcast ... yeah, no bias there at all.

Robin · May 2020

Hi @tgreen
Can you please check if M-Lab speed test is the same from link below?
https://speed.measurementlab.net/#/

That is the partner we use for speed measurement, so if Fing results are not in line with the ones reported by M-Lab, we definitely have to fix something.

Our Speed test is provided by M-Labs. They confirmed that they have recently rolled out an upgraded of their infrastructure & migration of machines to a new platform. This is being done to improve the overall Speedtest performance going forward. By doing this, initially they have cut of the multi-stream download Speedtest that we rely on for our Speedtests. However, they have greatly improved the single stream download Speedtest (previously the single stream download could measure up to 250 Mbps, where now it can measure up to 500+ Mbps). With speeds above this, the measured speed reported will be lower than before, due to the single stream limits.

To note for Fingbox users: this will also affect Fingbox Speedtests. We have released a new firmware version for the Fingbox which will allow single stream download to work again, firmware version 1.14.13 and onwards. If you have not yet received the new release, please power-cycle the Fingbox, this will automatically update the firmware on the Fingbox.

M-Labs are working on re-introducing the mutli speed download speedtest, to include it again in the future. Currently, we have no time-frame on when this will be re-introduced. Unfortunately at this stage, there is no further steps we can take to expedite this process.

tgreen · May 2020

Thanks for the reply @Robin
That M-Lab site does agree with the one half of the speed tests.
Be interesting to know why there is such a large discrepancy here, not that it's a Fing issue, just an interesting topic

sarob · May 2020

I have 0.5Gbps connection with Comcast in the SF Bay Area Orbi using Speedtest shows 588Mbps down. Comcast similar results. Fing using Mlab reports between 350-200Mbps down. It has been broken like this for about 6 months. I just stopped trusting it.

tgreen · May 2020

sarob said:

I have 0.5Gbps connection with Comcast in the SF Bay Area Orbi using Speedtest shows 588Mbps down. Comcast similar results. Fing using Mlab reports between 350-200Mbps down. It has been broken like this for about 6 months. I just stopped trusting it.

Interesting. It seems this doesn't agree with Ookla ( I presume this is what you mean when you say Speedtest), but which is correct? I used a pile of testers and seemed to have a pretty clear divide here. Perhaps these other services also use Mlab and that is part of the issue then

Busta999 · May 2020

My Ubiquiti UniFi Dream Machine is reporting 931MB down and 133MB up, Fing reports 470MB/130MB, I HAVE A 900/100 service.

Fing seems more than a bit off here.

Speed tests seem grossly inaccurate

Answers