Terence Eden’s Blog on Nostr: A quick look inside the HSTS file ...
A quick look inside the HSTS file
https://shkspr.mobi/blog/2024/01/a-quick-look-inside-the-hsts-file/
You type in to your browser's address bar example.com and it automatically redirects you to the https:// version. How does your browser know that it needed to request the more secure version of a website?
The answer is... A big list. The HTTP Strict Transport Security (HSTS) list is a list of domain names which have told Google that they always want their website served over https. If the user tries to manually request the insecure version, the browser won't let them. This means that a user's connection to, for example, their bank cannot be hijacked. A dodgy WiFi network cannot force the user to visit an insecure and fraudulent version of a site.
After about a decade of use, the list is now 14MB in size, with around 130,000 entries in it. You can view the list online or download it.
The format is relatively straightforward:{ "name": "example.com", "policy": "bulk-1-year", "mode": "force-https", "include_subdomains": true },
When the list is updated, Chrome creates a trie with Huffman coding compression - so it doesn't have to parse that monster file each time.A rummage inside
The most popular (over 1,000 entries) TLDs / Public Suffixes are:RankTLDEntries1com43,2362tk19,0223de5,2164org4,7315gov4,5076net4,4107ga4,3268nl2,6719cf2,45810ml2,27111co.uk2,13912fr1,71413ru1,51614eu1,28315com.br1,22616gq1,22517io1,21518com.au1,20219it1,10320cz1,004
After .com, the free .tk domain names absolutely dominate. I wonder how many of them are fraudulent?
There are 2,676 .uk domain names - only 537 of which aren't on .co.uk.
Going a bit further, there are 418 IDNs (which start with xn--).
And about 187 have "porn" in the domain.
You can't really extrapolate much from this as a data set. Lots of the domains seem to have expired or otherwise no longer work. Reading around https://hstspreload.org it notes that because this list is hard-coded into Chrome it can take months before a site is added. Similarly, removal can take a long time as well.
I can't help feeling that there should be a better way to manage all this though.
https://shkspr.mobi/blog/2024/01/a-quick-look-inside-the-hsts-file/
#CyberSecurity #dns #domains #https #security
https://shkspr.mobi/blog/2024/01/a-quick-look-inside-the-hsts-file/
You type in to your browser's address bar example.com and it automatically redirects you to the https:// version. How does your browser know that it needed to request the more secure version of a website?
The answer is... A big list. The HTTP Strict Transport Security (HSTS) list is a list of domain names which have told Google that they always want their website served over https. If the user tries to manually request the insecure version, the browser won't let them. This means that a user's connection to, for example, their bank cannot be hijacked. A dodgy WiFi network cannot force the user to visit an insecure and fraudulent version of a site.
After about a decade of use, the list is now 14MB in size, with around 130,000 entries in it. You can view the list online or download it.
The format is relatively straightforward:{ "name": "example.com", "policy": "bulk-1-year", "mode": "force-https", "include_subdomains": true },
When the list is updated, Chrome creates a trie with Huffman coding compression - so it doesn't have to parse that monster file each time.A rummage inside
The most popular (over 1,000 entries) TLDs / Public Suffixes are:RankTLDEntries1com43,2362tk19,0223de5,2164org4,7315gov4,5076net4,4107ga4,3268nl2,6719cf2,45810ml2,27111co.uk2,13912fr1,71413ru1,51614eu1,28315com.br1,22616gq1,22517io1,21518com.au1,20219it1,10320cz1,004
After .com, the free .tk domain names absolutely dominate. I wonder how many of them are fraudulent?
There are 2,676 .uk domain names - only 537 of which aren't on .co.uk.
Going a bit further, there are 418 IDNs (which start with xn--).
And about 187 have "porn" in the domain.
You can't really extrapolate much from this as a data set. Lots of the domains seem to have expired or otherwise no longer work. Reading around https://hstspreload.org it notes that because this list is hard-coded into Chrome it can take months before a site is added. Similarly, removal can take a long time as well.
I can't help feeling that there should be a better way to manage all this though.
https://shkspr.mobi/blog/2024/01/a-quick-look-inside-the-hsts-file/
#CyberSecurity #dns #domains #https #security