In our first chapter of “The problem with User-Agent strings” we are taking a deep dive into history and start with the basics of the User-Agent string.
Our first stop is in 2018, with the release of Firefox 60. There is nothing significant about this specific release—I could have chosen pretty much any version—but we have to start somewhere.
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Firefox/60.0So what do we have here? At the back of the User-Agent string we can see our browser, Firefox, version 60. It uses the Gecko rendering engine, and we can tell it’s Windows 10 because it uses version 10 of the Windows NT kernel. We see that it uses WOW64, which means it runs as a 32-bit application on a 64-bit version of the operating system.
Now, how reliable is this information?
That is a difficult question because all browsers lie about their identity, and the User-Agent string is usually chock full of lies.
The first blatant lie we see is the Gecko version. That is not an actual version number but a date, and it says January 1st, 2010. That is eight years too old. Why? Mozilla stopped updating that date because web developers started using hacks to work around bugs in specific versions of the Gecko engine. That caused problems down the line after the bug was fixed.
But the rabbit hole goes much deeper than that. Notice the Mozilla/5.0? That is a lie and is there for a backward compatibility issue. Not a recent one, but an issue that occurred all the way back in 1995.
Even though this is the User-Agent string from Mozilla Firefox, the Mozilla part does not mean it is Mozilla Firefox. Mozilla was the internal code name for the original Netscape Navigator browser and means Mosiac Killer. The name was re-used for Mozilla, the open-source foundation that spun out of Netscape and eventually released Firefox.
The Mozilla in the User-Agent string tells us that this is Netscape Navigator 5, which was never released. But the actual version is not the point. It is fake. It is part of our User-Agent string because back in the early 90s, there were two popular browsers: the original NCSA Mosaic and the more advanced Netscape Navigator.
Mosaic/0.9
Mozilla/1.0 (Win3.1)Netscape pioneered some new additions to HTML that were not available in Mosaic. That led to developers creating two versions of their websites, one for Mosaic and one enhanced version that uses the new additions that Netscape supported. They used the User-Agent string to determine who got to see which version of the website – commonly referred to as browser sniffing. If it started with Mozilla, it was Netscape, and it got the better version.
Now, enter a third browser: Microsoft Internet Explorer.
Mozilla/1.0 (compatible; MSIE 1.0; Windows 95)Internet Explorer was initially based on Mosaic but supported new HTML features like Netscape. And how do you get those websites to give you a better version built with those new HTML features? You had to lie. The website checked if you were Mozilla, so you had to tell the website that you were Mozilla. And that is what Internet Explorer did.
And that is what every browser has done since 1995.
Yes. It’s 2024, and we’re still using Mozilla in our User-Agent strings because of a backwards compatibility hack that is almost 30 years old and probably already was irrelevant 25 years ago.
Now, look at the User-Agent string from Microsoft Edge 12, released in 2015. It is the successor to Internet Explorer 11.
Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10162It says it is Netscape Navigator 5 and uses WebKit, KHTML, and Gecko as rendering engines. But wait—it is Chrome. No, you’re mistaken; it’s Safari. Well, actually, it is Microsoft Edge 12.
It contains more lies than accurate information. Once again, the reason is backward compatibility. It is almost identical to the Chrome User-Agent string:
Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36However Edge 12 is not based on Chromium but on EdgeHTML, an evolution of the Trident rendering engine used in Internet Explorer 11. So it makes more sense to mention Internet Explorer in the User-Agent string, right? Well, no.
In 2001, Microsoft released Internet Explorer 6, which was frankly amazing. Yes it was. It won Microsoft the browser wars and caused Netscape to give up altogether. And then… nothing much happened. Microsoft decided their browser was good enough.
It took Microsoft 5 years to release version 7 of their browser, but the web had changed during that time. Netscape transformed into Mozilla Firefox, and Apple launched its new browser, Safari. Slowly but surely, the market share of Internet Explorer started to drop.
At the same time, there was a push to get browsers and web developers to use proper standards. Internet Explorer was clearly on the back foot there. The other browsers had surpassed them. Instead of just building a website for Internet Explorer, web developers were suddenly developing for standards first and using hacks to make it work on Internet Explorer.
Between 2007 and 2015, Microsoft improved their browser but never seemed to catch up with the other browsers. Not only that, but the competition also got fiercer with the release of Google Chrome.
The nail in the coffin was that websites kept giving Internet Explorer users those old hacks to make it work, even when it wasn’t necessary anymore. That is the whole reason why Microsoft needed a clean break from Internet Explorer.
So Edge lied. It claimed to be Chrome and Safari, all to ensure that it would get identical versions of the websites as those other browsers. These lies were to prevent websites from giving Edge the old hacky Internet Explorer versions.
In the end, Microsoft gave up and started using Chromium – an open-source version of Chrome.
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3763.0 Safari/537.36 Edg/75.0.131.0Ironically, the User-Agent string doesn’t mention the name Edge anymore. Instead, it now contains Edg. Microsoft dropped the “e” to ensure it got a clean break from the hacks that web developers used to get their websites working on the old Edge browsers.
So, in a way, there is this Cold War-style arms race between browsers and web developers. Web developers must work around browser issues and use the User-Agent string to target a specific browser. Eventually, those issues will be fixed, and the browser needs to work around that workaround by lying in its user-agent string. And we end up with a never-ending spiral of lies.
These lies sound worse than they are. They are attempts to circumvent naive and simple browser sniffing methods. But with enough knowledge about these lies, we can tell what lies are and what is factual information. That is what browser detection libraries do, including WhichBrowser, which I wrote. They sift through the lies to get to the accurate information.
Continue with A chapter of unnecessary detail →