Would have been fun if the NTP server would return dates in the futures for requests from Netgear equipment, making certificates invalid and giving people an awful experience.
Apparently they didn't just do that, they also had them make requests way more frequently than they needed to (I guess particularly surprising given that they'd already have to be fairly negligent to have the server hard-coded):
> We learned that these packets appeared to be legitimate, well-formed Simple Network Time Protocol (SNTP) version 1 queries, albeit at an inexplicably high rate from each client host. For instance, during one trace, many clients produced about one query per second. This would be highly unusual for a properly constructed SNTP client, since an application which uses SNTP is merely interested in setting its own clock relatively accurately so that its host has some reasonable notion of the current time. One query per second is ridiculous, and is far from best practice for NTP client behavior.
On another note, back when I ran a web hosting business we hosted a few NTP servers in the pool. It’s such a simple thing to give back, and worth anyone who can make a stable contribution doing so.
If you do run an NTP server, please make sure it's not vulnerable to DDoS amplification (monlist, readvar, etc need to be disabled) and apply some rate limiting to make it less useful for reflection attacks. And be proactive about monitoring its traffic volume.
If you see high packet rate from a specific IP address or prefix, it's very likely not them abusing your service, but rather you attacking them by responding to spoofed requests.
Note that in the past I've had companies writing embedded linux based firmware using ntppool for time sync request their own vendor zones, however a lot of those requests were ignored so it's unclear if that's still expected. In the end they ended up just using the default ntppool domains since they never got their own vendor zones.
100k/s is consistent with on the order of lower 10s of millions of devices, with most checking every 10 or so minutes, and a few misconfigured ones hammering the server at a high frequency.
CNAME'ing pool-ntp.tesla.com to something they do not control is already quite risky as it would allow someone to e.g. request pool-ntp.tesla.com certificate though it might take quite a few tries.
I thought about trying this, but MPIC makes it very very very difficult (the round-robin has some geolocation magic baked in regarding what server it connects you to).
> That points to a glaring hole in the modern-day automated web PKI, not Tesla's dangling DNS record.
It's not. They control a long-term high-value asset (the domain tesla.com). They decided to delegate part of that asset to a large number of "random" people that they do not have a contract or agreement with.
Being able to issue certs for cloud IPs has nothing to do with this since it is not a long term asset, and if it is you probably don't delegate it to random people to control unless you do not value that asset.
This is why IP certificates are limited to a max lifetime of 6 days.
> IP address certificates allow server operators to authenticate TLS connections to IP addresses rather than domain names. Let’s Encrypt supports both IPv4 and IPv6. IP address certificates must be short-lived certificates, a decision we made because IP addresses are more transient than domain names, so validating more frequently is important.
I'd try to contact Assetnote. Most (sadly not all) managed vuln scan companies are pretty sensitive to scanning stuff that doesn't belong to their client and could expose them to liabilities because they don't have permission.
Oh, and be sure to include "you're scanning a pool address so you're probably scanning a lot of other sites that don't belong to your customer". They should know it's potentially not one little web site.
Thankfully it doesn't seem to be much traffic, but still... weird. You'd hope at somepoint the weird responses would get looked at in some log, but I won't hold my breath for that haha.
Tangential, but I love the design of your blog. That's so freakishly accurate to old GNOME 2 Ubuntu, amazing work.
Do a Tesla move and say that's a new service and they need to pay 10000 a month to keep using the service otherwise you reserve the right to mess up with the responses. :P of course, just a joke, but that's what they do
So many of the news on HN, such as present one, can be actual stories/scenes from a cyberpunk game/movie these days, that we can safely assume this (otherwise imaginary) future has already arrived.
As a bug bounty researcher, my systems would do the same thing if they ended up georouted to this IP. *.tesla.com is marked as in scope on https://bugcrowd.com/engagements/tesla, and my agents will probe anything under there as it is presumed to have explicit authorization.
Not sure if there is a great solution, but I'm inclined to say that attack traffic like this is the new normal. In fact, the attack volume they got is quite small compared to the volume I have seen on other tech company subdomains - the new normal is probably much worse.
How do you suggest I determine the information is bad, if the domain is hosted on tesla.com, and Tesla says I am authorized to test it? Should I inspect all 1,368 subdomains on tesla.com by hand, and then do the same for 400+ bug bounty programs?
Maybe i'm old fashioned, but personally I think the onus should be on the person sending out unauthorized malicious requests to figure out how to not do that.
Any responsible bug bounty researcher reviewing the DNS zone by hand would spot the CNAME and remove it from the target list. You don't get to wash your hands of that because your chatbot did it.
Even before AI, I can't imagine a single bug bounty researcher doing that. Pre-AI, everyone ran a tool like subfinder to enumerate subdomains, httpx to resolve them, nuclei to scan them, etc. There's no human review involved there at the subdomain level.
And I don't know anyone that would really look at the intermediary of a CNAME even during a manual test. Maybe if it was obviously a third party service.
You're now confident that the other 399+ domains you mentioned are not under any sort of duress because they're controlled by people who are away of what's happening?
I feel confident that no system exposed to the internet should have a problem with 50,000 requests per month! If they do, they probably shouldn't run a public NTP server, or have a public IP address at all.
Okay, so you're just confident in exactly how many requests everyone else in the world will be making if you try to pentest resources unbeknownst to their owners.
Are you confident you are not viewing too many pages on HN? What if many other people are also trying to read this thread?
I think this line of reasoning doesn't make any sense. The internet is not an inherently safe network regardless of what we wish for; we can't wish away the bad activity, and it's only going to increase. The activity that helps prevent the bad activity from working is a net positive.
> Are you confident you are not viewing too many pages on HN? What if many other people are also trying to read this thread?
I think it's pretty disingenuous to compare viewing a couple of pages once a day with running scripting tools against over 400 websites.
> I think this line of reasoning doesn't make any sense. The internet is not an inherently safe network regardless of what we wish for; we can't wish away the bad activity, and it's only going to increase. The activity that helps prevent the bad activity from working is a net positive.
Oh good, no one has ever claimed "it's for your own good" when doing something selfish without consent.
We can quantify the impact of the scripting tools pretty easily - 1.5k requests per day! I just went on Reddit and viewed a few posts, and it caused about 500 HTTP requests in DevTools. Luckily HN is not that bloated, but I just don't see the number of requests as meaningful even if it was orders of magnitude higher.
As the OP said, they don't do anything when the server isn't vulnerable, and serving a 404 page is incredibly cheap.
When I engage a security assessor on behalf of a client, I am required to provide detailed scope and attest to in scope assets (including IP blocks and public hostnames), as well as that I have legal authority for them to be tested. This is validated by my executive sponsor.
It is your responsibility to do your due diligence as a security researcher versus “spray and pray” to ensure you are not exceeding the scope beyond your intended target.
Dump the subdomains, resolve them, and review where they resolve to in order to understand the footprint and attack surface boundaries before engaging scanning or agentic red team harnesses. Automate as much as possible for building the state graph of the target, but a human must remain in the loop to sanity check. To not do this means you could be attacking hyperscaler object storage, a CDN, a partner SaaS frontend, ticketing systems, mail systems, etc (ie anything someone may CNAME off the root domain but that is outside of their organization’s control).
I reckon there's a decent argument to be made that an authorization to scan *.tesla.com definitively does NOT extend to any hosts resolved via a CNAME chain that goes foo.tesla.com -> bah.not-tesla.com -> host-that-never-authorized-attacking.
I'm curious about your use of agents for security bug bounties. Do you use self hosted models? GLM 5.2? Do the economics of self-hosting make it worth it? Or if you use 3rd party hosted models, don't you run into safeguards that try to prevent hacking? (unless convincing them it's a genuine ethical bug bounty program works, but it doesn't in my experience)
I just checked my own webserver logs (I also run a sever in the ntp pool) and I too see some hits in my webserver logs.
They look to all be log4j vuln scanning activity (CVE-2021-44228), and the volume isn't that high (a few a day, and not every day). They just have some overzealous vuln scanning. And yes, they shouldn't have the NTP pool under their DNS name.
I've had all sorts of strange things happen because of my ntp pool membership, this one is pretty far on the benign end of things.
I've been consistently attacked by ShadowServer who have the following sponsors,
Akamai,
APNIC Foundation,
Arctic Security,
AusCERT,
Avast,
Backblaze,
Canadian Center for Cyber Security,
CERT.AT,
CERT.br,
CERT.LV,
CIRA,
CIRCL,
Craig Newmark Philanthropies,
CSIRT.LI,
CSIS Security Group,
DFN‑CSIRT,
Digital Trust Center,
EURid,
HelseCERT,
ICANN,
Identity Digital,
KPN,
Mastercard,
NASK (CERT.pl),
NCSC Ireland,
NICS,
Nihon Cyber Defence,
Nucleus Security,
Orange Polska,
Precursor Security,
Protect.ngo,
Public Interest Registry (PIR),
Red Hat,
SURFcert,
SWITCH,
Team Cymru,
Trend Micro,
Trivest AG,
Tucows,
Verisign,
VulnCheck,
I don't care what they say they're doing, I hate how corporations can act with impunity with these types of things while everyone else would get a felony for it.
Calling vuln scanning from a non-profit a felony is a bit of a stretch. Many for-profit companies do similar vuln scanning and then threaten companies with security "scorecards". That is borderline extortion.
If the non-profit was walking down the road and rattling everybody’s door lock to see which are unlocked, and having a look around the windows to see if any are open, would that be a crime?
Because that is exactly what all of these vulnerability scanning companies are doing, and all of us sort of just… let them.
If the neighborhood was constantly being canvased by criminals checking doorknobs, so your concerned neighbor went over to your house to check your doorknob, and then let you know if you accidentally left it open, would you also accuse your neighbor of being a criminal trying to break in?
This is why the checking doors / neighborhood analogy isn't a good one.
Having one person with poor computer security negatively impacts everyone. Hacked sites turn into phishing landing pages, exploit kit hosting, stolen data dumps, and launching off points for attacks on everyone else. The vuln scanning ShadowServer is doing is meant to be a public good, which is why they share the info with ISPs and governments.
Security is too intertwined to stand by and say other people's vulnerabilities aren't your problem.
It is a good analogy because thieves stealing from one house successfully gives them resources and incentivizes them stealing from the same area again.
And even if you remove the analogy, ShadowServer means good, but good intentions doesn’t necessarily make their action moral or legal.
Yes, compromised servers can be used by hackers as means to commit crimes. But when these groups scan the entire internet, they do cause harm as well, as shown in the original linked article. Much less harm than a black hat, but they still waste time and resources from innocent third parties.
It’s fair to ask if the harm they cause is worth the good they do.
> It’s fair to ask if the harm they cause is worth the good they do.
Researcher disclosures, even with POCs, have moved the industry to action incumbents would have rather buried. I’m thinking of CPU and memory exploits, and stuff like log4j, as examples.
Frontier AI is enabling the cyber arms race more than anything past, and certainly more than some bot slowly crawling web servers for old vulnerabilities.
If we’re talking harm, it should be in the broader context of internet history, imo.
I would assume the fastest way to actually make this stop would be to setup a bunch of honeypot exploits, trigger their detection and someone will figure out what they did wrong.
Other than not, with these huge companies you have 0 recourse.
Meta not only hasn't noticed, but is currently sending about 11 requests per second to my site. I've also seemingly trapped one of those TV proxy scraper nets as I'm getting absolutely hammered by requests from all over the place now. I get maybe 10 legit visitors per day, and I'm currently blocking 406,787 IPs from things that have fallen into my honeypot.
I've tweaked my site to return empty status responses a configurable amount of time but the traffic has been so intense that Traefik is now struggling, so I'm going to have to figure out something else. I was returning over-capacity errors and I think that was a mistake, I've swapped to 400 range status codes now. I don't want to use Cloudflare so I'm not sure what to do after this.
The people at these companies are either incompetent or malicious.
Most distributed bots won't follow a 301. If 404/400 don't work, just 444 them, they're not worth giving back a response, especially on a personal website.
Haha I did debate going much worse with the junk images but wanted to err on the side of caution in case I subject possible clients to something like goatse.
Unlikely, but the lazy wonks could enable stratum 1 time services with the cars GPS/cellular receiver and RTC. Having done that for equipment in the past, we all know it is literary only $8.43/unit in parts. Also, setting ntp time as the tertiary fall back has been around for over a decade. =3
I wonder if source IP can be ping triangulated, then total ping count displayed as "Tesla MAU: +/-x% today", "Suspected ownership changes this month: xxx cars" by apparent home location changes, then residential and Tesla-unrelated locations excluded server side, and plotted on the map, then finally the whole system exposed to the public Internet and shared to SpaceX fans.
"Hey Texas Model S #345 just left KXYZ, moving at >100mph towards the pad. Everyone get cameras out!"
Often, the folks you want to ban are not the hosts running the scans.
One's best bet is to play possum, and use your clients last login IPs falling in your service area geo-IP ranges for a firewall white-list. Then redirect the other traffic for a black hole route.
If the nuisance hosts assume they have driven the host offline, they will eventually give up and move on. =3
It’s just laziness on Tesla’s part because working your bare bones crew to failure isn’t the exception but the SOP. Same why you can’t reach anyone about Supercharger issues, this is by design to run as lean as possible for profits.
I see this in cyber clients that are perpetually short staffed and simply accept the degraded system because the consequences are minimal. If nothing important breaks, nothing changes.
People are trying to compromise Tesla and because this guy provides NTP services, and Tesla set their NTP up wrong, it appears to other people like his machine is part of Tesla.
Got it. The way it was phrased it made it sound like Assetnote was the party actually sending the exploits (which made me assume it was intentional testing going to the wrong target), not that they were coming in from unknown senders in the outside world.
50,000 requests in the span of ... 21 days? It might be interesting if it were 50 million in those 21 days (even then, 27 RPS is ... nothing). Drop a report to AWS T&S, block, drop, reject, and move on. This is like less than background noise at this point.
Edit: tamping down a bit of my prickliness because it looks like this individual is a relative newcomer to running internet-facing services. This is actually a pretty good intro to that: the place as a whole is a cesspool and any conceivable “attack”, scan, probe, pentest is, has, or will be happening at all times. Some you can mitigate yourself, others you’ll need to bring in a specialist service (see DDoS sinks and mitigation services, for example) or contact someone’s abuse address, others you just have to ignore because it’s just not directed at you personally, or just not worth more than three seconds’ thought beyond a firewall rule. This is the latter. Maybe interesting if this is the first time you’re seeing something like this, but for more grizzled operators like myself, it doesn’t even register as notable anymore.
In case of the very likely title only reader posting:
> Speculation: Assetnote pulled in everything it could find under tesla.com, including pool-ntp.tesla.com, which CNAMEs to pool.ntp.org, which can resolve to my machine — 67.215.249.229. The asset inventory saves this as a Tesla asset, and starts throwing exploits at me, a stranger.
> Not a vuln in Tesla, and I'm not asking for anything, but I just wanted to let you know that you may unintentionally be being a nuisance.
This happens EVERY day to EVERY web server out there. I have a personal site that gets thousands of requests per day from bots.
Running a public server (like NTP) means you will get tons of strange requests. Moreso if you run a web server on the same IP because bots will scrape certificate transparency logs. The entire IPv4 space is scanned continuously.
This may sound harsh, but you cannot stop it. It is whack-a-mole. Filter it and move on, go outside and touch grass, seriously. This is not worth being upset over.
I treat these as an opportunity to tune my filters and firewall rules.
You don’t think there’s a difference between “hackers try to attack everything“ and “Tesla decided that I personally need to be tested as one of their systems due to a lazy misconfiguration“ are different?
Are you asking if I think Elon had this person's website on an 80" screen in a conference room whilst they collude how to send traffic to their site?
No, I do not.
I get thousands of these "security scanner" requests on a low-traffic site weekly, if not daily. Not a day goes by where someone doesn't try to kick in my door. Are you prepared to waste valuable time in your life to separate the white hat from the black hat requests?
I would have added the IP addresses to my firewall's IP blacklist and forgotten about it the next day. It's really all you can do.
> Are you asking if I think Elon had this person's website on an 80" screen in a conference room whilst they collude how to send traffic to their site?
i cannot fathom how you could possibly arrive at this reading of the gp's comment, what are you talking about
I'm aware how much crap there is on the Internet, I just think the specific nature of this (legitimate commercial vuln scanner thinks I'm Tesla) is funny
It does bring attention to: how many other organizations are doing this?
Tesla is a large enterprise.
They almost certainly subscribe to some overpriced SaaS garbage which is manned by offshore drones who by definition do not care because they're not paid enough to care.
Unfortunately this isn't the 80s anymore where you can ring up a system administrator at a university and get a human on the other end.
I understand that Tesla is treating your NTP server, a volunteer server and part of the greater volunteer pool of NTP servers, as their own infrastructure.
However, I can't tell from the article if the scans originate with:
A.) IT staff at Tesla that are scanning exposed services on what they perceive, or claim wrongly, as their own network for vulnerabilities.
B.) Somehow a rogue operator (read botnet)
C.) A rogue operator who is using the cars themselves to run exploit scans?
Please read the article, it's not the volume of the NTP requests, they're actively sending exploit/attempt to compromise payloads. They're probing things in a way that you would ordinarily only do to your own internal infrastructure.
"They tried all kinds of exploits against me: path traversal, webshell uploads, probing software internals, probing WordPress and other CMS management endpoints, SSRF, Log4Shell, and a lot more."
I don't disagree with you, I have tons of things that have public interfaces (as mundane as a fully patched wordpress where the wp-admin login is accessible to external blog writers), we get tens of thousands of random shit anything per day. But the problem here is that Tesla is treating NTP pool operators like they are their internal infrastructure. Also because the attribution of the 'attacks' is fairly well known.
I don't go complaining on the internet about the absolute shitflood of compromised routers on broadband ISPs in Indonesia probing my stuff 24x7x365 because I know it would be futile. But if I found one specific american company that was repeatedly probing my stuff all the time? Maybe I'd escalate it.
It's HTTP requests, not NTP requests, and the volume isn't the problem, it's that Assetnote is sending live exploit payloads /at all/ to a stranger on Tesla's behalf
Ask a lawyer about sending an unpleasant letter to the scanner, detailing the situation and demanding that they cease & desist. That clobbers their "we didn't know" defense, and their Legal Dept. will likely order them to stop ASAP.
Remember in 2003 when netgear hardcoded a university's NTP server into a ton of their products? Well....
https://www.google.com/search?&q=university+ntp+server+netge...
https://pages.cs.wisc.edu/~plonka/netgear-sntp/
Would have been fun if the NTP server would return dates in the futures for requests from Netgear equipment, making certificates invalid and giving people an awful experience.
Apparently they didn't just do that, they also had them make requests way more frequently than they needed to (I guess particularly surprising given that they'd already have to be fairly negligent to have the server hard-coded):
> We learned that these packets appeared to be legitimate, well-formed Simple Network Time Protocol (SNTP) version 1 queries, albeit at an inexplicably high rate from each client host. For instance, during one trace, many clients produced about one query per second. This would be highly unusual for a properly constructed SNTP client, since an application which uses SNTP is merely interested in setting its own clock relatively accurately so that its host has some reasonable notion of the current time. One query per second is ridiculous, and is far from best practice for NTP client behavior.
This was a great read, thank you for posting!
I love articles like that.
I’m pretty sure this way they’ve hardcoded the NTPs is actually against the ToS for use of the NTP pool too.
The way a vendor embedding NTP is _meant_ to do so is documented here: https://www.ntppool.org/en/vendors.html
On another note, back when I ran a web hosting business we hosted a few NTP servers in the pool. It’s such a simple thing to give back, and worth anyone who can make a stable contribution doing so.
If you do run an NTP server, please make sure it's not vulnerable to DDoS amplification (monlist, readvar, etc need to be disabled) and apply some rate limiting to make it less useful for reflection attacks. And be proactive about monitoring its traffic volume.
If you see high packet rate from a specific IP address or prefix, it's very likely not them abusing your service, but rather you attacking them by responding to spoofed requests.
> The way a vendor embedding NTP is _meant_ to do so is documented here: https://www.ntppool.org/en/vendors.html
Note that in the past I've had companies writing embedded linux based firmware using ntppool for time sync request their own vendor zones, however a lot of those requests were ignored so it's unclear if that's still expected. In the end they ended up just using the default ntppool domains since they never got their own vendor zones.
I remember going through the same thing. No response. This was a long time ago, though.
> The pool is currently keeping the time of an estimated 5-15 million systems accurate.
When was this page last updated? I would expect that number to be in the hundreds of millions these days, perhaps even billions.
To quote from the page:
> Do not use the standard pool.ntp.org names as a default configuration in your system
Apple's default is time.apple.com, ubuntu has their own pool, android as well, same for windows.
All the big linux distros also run their own ntp servers.
The ntppool.org stats show about 100k queries per second https://status.ntppool.org/
100k/s is consistent with on the order of lower 10s of millions of devices, with most checking every 10 or so minutes, and a few misconfigured ones hammering the server at a high frequency.
It certainly isn't billions of devices.
Yes, they should absolutely be using a vendor zone instead of a CNAME under tesla.com.
“You must absolutely not use the default pool.ntp.org zone names as the default configuration in your application or appliance.”
ref: https://www.ntppool.org/en/vendors.html
TIL. I'll modify my firmware tonight.
CNAME'ing pool-ntp.tesla.com to something they do not control is already quite risky as it would allow someone to e.g. request pool-ntp.tesla.com certificate though it might take quite a few tries.
I thought about trying this, but MPIC makes it very very very difficult (the round-robin has some geolocation magic baked in regarding what server it connects you to).
Out of curiosity, how is MPIC relevant? Not that familiar with it, but CNAME would resolve to your server regardless no?
pool-ntp.tesla.com --CNAME--> pool.ntp.org --GeoDNS--> thousands of possible servers, biased heavily by user location
Oops should have RTFA before commenting, I thought the CNAME was only to your servers.
Wouldn't the same apply to pool.ntp.org then?
Maybe running a web server on the same IP as an NTP server is a bad idea.
a .tesla.com certificate might well enable more shenanigans than a .pool.ntp.org cert.
Hope there are no sensitive *.tesla.com cookies out there...
The chance of being issued a certificate in this instance, while theoretically possible, is infinitesimally small.
That points to a glaring hole in the modern-day automated web PKI, not Tesla's dangling DNS record.
Hell, they issue certificates to IP addresses now. For cloud systems, ownership of an IP could be a few hours.
This has almost certainly been deemed an acceptable risk.
> That points to a glaring hole in the modern-day automated web PKI, not Tesla's dangling DNS record.
It's not. They control a long-term high-value asset (the domain tesla.com). They decided to delegate part of that asset to a large number of "random" people that they do not have a contract or agreement with.
Being able to issue certs for cloud IPs has nothing to do with this since it is not a long term asset, and if it is you probably don't delegate it to random people to control unless you do not value that asset.
This is why IP certificates are limited to a max lifetime of 6 days.
> IP address certificates allow server operators to authenticate TLS connections to IP addresses rather than domain names. Let’s Encrypt supports both IPv4 and IPv6. IP address certificates must be short-lived certificates, a decision we made because IP addresses are more transient than domain names, so validating more frequently is important.
https://letsencrypt.org/2026/01/15/6day-and-ip-general-avail...
I'd try to contact Assetnote. Most (sadly not all) managed vuln scan companies are pretty sensitive to scanning stuff that doesn't belong to their client and could expose them to liabilities because they don't have permission.
Yeah, I'll try to do this, but I can't find anything better than the generic contact form.
Try search on LinkedIn. Often turns up folks who work there who you can reach out to.
Oh, and be sure to include "you're scanning a pool address so you're probably scanning a lot of other sites that don't belong to your customer". They should know it's potentially not one little web site.
Thankfully it doesn't seem to be much traffic, but still... weird. You'd hope at somepoint the weird responses would get looked at in some log, but I won't hold my breath for that haha.
Tangential, but I love the design of your blog. That's so freakishly accurate to old GNOME 2 Ubuntu, amazing work.
Do a Tesla move and say that's a new service and they need to pay 10000 a month to keep using the service otherwise you reserve the right to mess up with the responses. :P of course, just a joke, but that's what they do
I really like this website's theme. I noticed this yesterday when using a very old iOS version: Shiny UI is morale-boosting.
So many of the news on HN, such as present one, can be actual stories/scenes from a cyberpunk game/movie these days, that we can safely assume this (otherwise imaginary) future has already arrived.
As a bug bounty researcher, my systems would do the same thing if they ended up georouted to this IP. *.tesla.com is marked as in scope on https://bugcrowd.com/engagements/tesla, and my agents will probe anything under there as it is presumed to have explicit authorization.
Not sure if there is a great solution, but I'm inclined to say that attack traffic like this is the new normal. In fact, the attack volume they got is quite small compared to the volume I have seen on other tech company subdomains - the new normal is probably much worse.
One solution is to not set up and run a computer program that relies on bad information to perform automated cyber-attacks on third parties.
How do you suggest I determine the information is bad, if the domain is hosted on tesla.com, and Tesla says I am authorized to test it? Should I inspect all 1,368 subdomains on tesla.com by hand, and then do the same for 400+ bug bounty programs?
Maybe i'm old fashioned, but personally I think the onus should be on the person sending out unauthorized malicious requests to figure out how to not do that.
Any responsible bug bounty researcher reviewing the DNS zone by hand would spot the CNAME and remove it from the target list. You don't get to wash your hands of that because your chatbot did it.
Even before AI, I can't imagine a single bug bounty researcher doing that. Pre-AI, everyone ran a tool like subfinder to enumerate subdomains, httpx to resolve them, nuclei to scan them, etc. There's no human review involved there at the subdomain level.
And I don't know anyone that would really look at the intermediary of a CNAME even during a manual test. Maybe if it was obviously a third party service.
Yes, unless you think that trying to do a bug bounty is a good excuse to participate in DoS.
The OP says they have received 50,000 requests in about a month. What service is being denied by 0.01 requests per second?
You're now confident that the other 399+ domains you mentioned are not under any sort of duress because they're controlled by people who are away of what's happening?
I feel confident that no system exposed to the internet should have a problem with 50,000 requests per month! If they do, they probably shouldn't run a public NTP server, or have a public IP address at all.
Okay, so you're just confident in exactly how many requests everyone else in the world will be making if you try to pentest resources unbeknownst to their owners.
Are you confident you are not viewing too many pages on HN? What if many other people are also trying to read this thread?
I think this line of reasoning doesn't make any sense. The internet is not an inherently safe network regardless of what we wish for; we can't wish away the bad activity, and it's only going to increase. The activity that helps prevent the bad activity from working is a net positive.
> Are you confident you are not viewing too many pages on HN? What if many other people are also trying to read this thread?
I think it's pretty disingenuous to compare viewing a couple of pages once a day with running scripting tools against over 400 websites.
> I think this line of reasoning doesn't make any sense. The internet is not an inherently safe network regardless of what we wish for; we can't wish away the bad activity, and it's only going to increase. The activity that helps prevent the bad activity from working is a net positive.
Oh good, no one has ever claimed "it's for your own good" when doing something selfish without consent.
We can quantify the impact of the scripting tools pretty easily - 1.5k requests per day! I just went on Reddit and viewed a few posts, and it caused about 500 HTTP requests in DevTools. Luckily HN is not that bloated, but I just don't see the number of requests as meaningful even if it was orders of magnitude higher.
As the OP said, they don't do anything when the server isn't vulnerable, and serving a 404 page is incredibly cheap.
When I engage a security assessor on behalf of a client, I am required to provide detailed scope and attest to in scope assets (including IP blocks and public hostnames), as well as that I have legal authority for them to be tested. This is validated by my executive sponsor.
It is your responsibility to do your due diligence as a security researcher versus “spray and pray” to ensure you are not exceeding the scope beyond your intended target.
Dump the subdomains, resolve them, and review where they resolve to in order to understand the footprint and attack surface boundaries before engaging scanning or agentic red team harnesses. Automate as much as possible for building the state graph of the target, but a human must remain in the loop to sanity check. To not do this means you could be attacking hyperscaler object storage, a CDN, a partner SaaS frontend, ticketing systems, mail systems, etc (ie anything someone may CNAME off the root domain but that is outside of their organization’s control).
I reckon there's a decent argument to be made that an authorization to scan *.tesla.com definitively does NOT extend to any hosts resolved via a CNAME chain that goes foo.tesla.com -> bah.not-tesla.com -> host-that-never-authorized-attacking.
Pretty hard to implement in practice!
% dig www.tesla.com +short
www.tesla.com.edgekey.net.
e1792.dscx.akamaiedge.net.
<akamai IP>
I'm curious about your use of agents for security bug bounties. Do you use self hosted models? GLM 5.2? Do the economics of self-hosting make it worth it? Or if you use 3rd party hosted models, don't you run into safeguards that try to prevent hacking? (unless convincing them it's a genuine ethical bug bounty program works, but it doesn't in my experience)
Can't you report them to the FBI, someone is clearly trying to hack your server
> rg -zFI pool-ntp.tesla.com access.log*
Neat! Didn’t know about this command that’s very helpful
I just checked my own webserver logs (I also run a sever in the ntp pool) and I too see some hits in my webserver logs.
They look to all be log4j vuln scanning activity (CVE-2021-44228), and the volume isn't that high (a few a day, and not every day). They just have some overzealous vuln scanning. And yes, they shouldn't have the NTP pool under their DNS name.
I've had all sorts of strange things happen because of my ntp pool membership, this one is pretty far on the benign end of things.
I've been consistently attacked by ShadowServer who have the following sponsors,
Akamai, APNIC Foundation, Arctic Security, AusCERT, Avast, Backblaze, Canadian Center for Cyber Security, CERT.AT, CERT.br, CERT.LV, CIRA, CIRCL, Craig Newmark Philanthropies, CSIRT.LI, CSIS Security Group, DFN‑CSIRT, Digital Trust Center, EURid, HelseCERT, ICANN, Identity Digital, KPN, Mastercard, NASK (CERT.pl), NCSC Ireland, NICS, Nihon Cyber Defence, Nucleus Security, Orange Polska, Precursor Security, Protect.ngo, Public Interest Registry (PIR), Red Hat, SURFcert, SWITCH, Team Cymru, Trend Micro, Trivest AG, Tucows, Verisign, VulnCheck,
I don't care what they say they're doing, I hate how corporations can act with impunity with these types of things while everyone else would get a felony for it.
Report it https://www.cisa.gov/reporting-cyber-incident
Calling vuln scanning from a non-profit a felony is a bit of a stretch. Many for-profit companies do similar vuln scanning and then threaten companies with security "scorecards". That is borderline extortion.
If the non-profit was walking down the road and rattling everybody’s door lock to see which are unlocked, and having a look around the windows to see if any are open, would that be a crime?
Because that is exactly what all of these vulnerability scanning companies are doing, and all of us sort of just… let them.
If the neighborhood was constantly being canvased by criminals checking doorknobs, so your concerned neighbor went over to your house to check your doorknob, and then let you know if you accidentally left it open, would you also accuse your neighbor of being a criminal trying to break in?
Yes.
I have personally done this before, the correct sequence of moves is:
1. Call your neighbor, ask for permission. 2. Check the door 3. Lock the door
If you don’t have their phone number, you are not on good enough terms to touch their lock.
This is why the checking doors / neighborhood analogy isn't a good one.
Having one person with poor computer security negatively impacts everyone. Hacked sites turn into phishing landing pages, exploit kit hosting, stolen data dumps, and launching off points for attacks on everyone else. The vuln scanning ShadowServer is doing is meant to be a public good, which is why they share the info with ISPs and governments.
Security is too intertwined to stand by and say other people's vulnerabilities aren't your problem.
It is a good analogy because thieves stealing from one house successfully gives them resources and incentivizes them stealing from the same area again.
And even if you remove the analogy, ShadowServer means good, but good intentions doesn’t necessarily make their action moral or legal.
Yes, compromised servers can be used by hackers as means to commit crimes. But when these groups scan the entire internet, they do cause harm as well, as shown in the original linked article. Much less harm than a black hat, but they still waste time and resources from innocent third parties.
It’s fair to ask if the harm they cause is worth the good they do.
> It’s fair to ask if the harm they cause is worth the good they do.
Researcher disclosures, even with POCs, have moved the industry to action incumbents would have rather buried. I’m thinking of CPU and memory exploits, and stuff like log4j, as examples.
Frontier AI is enabling the cyber arms race more than anything past, and certainly more than some bot slowly crawling web servers for old vulnerabilities.
If we’re talking harm, it should be in the broader context of internet history, imo.
I would assume the fastest way to actually make this stop would be to setup a bunch of honeypot exploits, trigger their detection and someone will figure out what they did wrong.
Other than not, with these huge companies you have 0 recourse.
I did something like this a few weeks ago on my photography site: https://robertmay.photography/journal/meta-has-tried-to-scra...
Meta not only hasn't noticed, but is currently sending about 11 requests per second to my site. I've also seemingly trapped one of those TV proxy scraper nets as I'm getting absolutely hammered by requests from all over the place now. I get maybe 10 legit visitors per day, and I'm currently blocking 406,787 IPs from things that have fallen into my honeypot.
I've tweaked my site to return empty status responses a configurable amount of time but the traffic has been so intense that Traefik is now struggling, so I'm going to have to figure out something else. I was returning over-capacity errors and I think that was a mistake, I've swapped to 400 range status codes now. I don't want to use Cloudflare so I'm not sure what to do after this.
The people at these companies are either incompetent or malicious.
Return a HTTP 301 pointing to https://facebook.com?
Might make them scan themselves instead.
Most distributed bots won't follow a 301. If 404/400 don't work, just 444 them, they're not worth giving back a response, especially on a personal website.
Since it's a photography site, route 'em over to goatse. That might get someone's attention.
Haha I did debate going much worse with the junk images but wanted to err on the side of caution in case I subject possible clients to something like goatse.
I actually think this is brilliant. Let the scanner IP get access to the most unpatched Wordpress, maybe even generate some garbage ERP numbers.
Once the asset scanner detects the vulns, everyone will kick into high gear to patch this.
(OOP here). I wonder if switching my replies from 299 to 200 OK would be enough for some of them.
Unlikely, but the lazy wonks could enable stratum 1 time services with the cars GPS/cellular receiver and RTC. Having done that for equipment in the past, we all know it is literary only $8.43/unit in parts. Also, setting ntp time as the tertiary fall back has been around for over a decade. =3
I wonder if source IP can be ping triangulated, then total ping count displayed as "Tesla MAU: +/-x% today", "Suspected ownership changes this month: xxx cars" by apparent home location changes, then residential and Tesla-unrelated locations excluded server side, and plotted on the map, then finally the whole system exposed to the public Internet and shared to SpaceX fans.
"Hey Texas Model S #345 just left KXYZ, moving at >100mph towards the pad. Everyone get cameras out!"
It'll be gone by lunchtime that day.
Often, the folks you want to ban are not the hosts running the scans.
One's best bet is to play possum, and use your clients last login IPs falling in your service area geo-IP ranges for a firewall white-list. Then redirect the other traffic for a black hole route.
If the nuisance hosts assume they have driven the host offline, they will eventually give up and move on. =3
Tangential; I love the style drop down.
> unintentionally be being a nuisance.
As opposed to intentionally being a nuisance?
It’s just laziness on Tesla’s part because working your bare bones crew to failure isn’t the exception but the SOP. Same why you can’t reach anyone about Supercharger issues, this is by design to run as lean as possible for profits.
I see this in cyber clients that are perpetually short staffed and simply accept the degraded system because the consequences are minimal. If nothing important breaks, nothing changes.
I bet we'll find that its openAI agent benchmarks off on their own again
I'd report this to AWS instead since Tesla doesn't seem aware. It's AWS IPs as the source.
I might try Searchlight Cyber first (it's their service doing this, presumably without their knowledge), and then if that doesn't work go to AWS.
Yeah it looks like someone thinks they're "hacking" Tesla when really it's hitting volunteer IPs.
Give Tesla the wrong time.
Is this basically a free pen test, or am I misunderstanding the setup?
People are trying to compromise Tesla and because this guy provides NTP services, and Tesla set their NTP up wrong, it appears to other people like his machine is part of Tesla.
Got it. The way it was phrased it made it sound like Assetnote was the party actually sending the exploits (which made me assume it was intentional testing going to the wrong target), not that they were coming in from unknown senders in the outside world.
nope, it really is assetnote's intentional testing going to the wrong target
> They tried all kinds of exploits against me ... probing WordPress and other CMS management endpoints
This is standard bot crawler traffic. Anyone who runs a home server sees attempts to load wp paths all the time
Oh absolutely, I just think the specific nature of this (legitimate commercial vuln scanner thinks I'm Tesla) is funny
50,000 requests in the span of ... 21 days? It might be interesting if it were 50 million in those 21 days (even then, 27 RPS is ... nothing). Drop a report to AWS T&S, block, drop, reject, and move on. This is like less than background noise at this point.
Edit: tamping down a bit of my prickliness because it looks like this individual is a relative newcomer to running internet-facing services. This is actually a pretty good intro to that: the place as a whole is a cesspool and any conceivable “attack”, scan, probe, pentest is, has, or will be happening at all times. Some you can mitigate yourself, others you’ll need to bring in a specialist service (see DDoS sinks and mitigation services, for example) or contact someone’s abuse address, others you just have to ignore because it’s just not directed at you personally, or just not worth more than three seconds’ thought beyond a firewall rule. This is the latter. Maybe interesting if this is the first time you’re seeing something like this, but for more grizzled operators like myself, it doesn’t even register as notable anymore.
In case of the very likely title only reader posting:
> Speculation: Assetnote pulled in everything it could find under tesla.com, including pool-ntp.tesla.com, which CNAMEs to pool.ntp.org, which can resolve to my machine — 67.215.249.229. The asset inventory saves this as a Tesla asset, and starts throwing exploits at me, a stranger.
> Not a vuln in Tesla, and I'm not asking for anything, but I just wanted to let you know that you may unintentionally be being a nuisance.
Huh, are we sure this is just NTP shenanigans and not some super duper dangerous AI agents "escaping" the "containment"?
This seems like a gross exaggeration.
This happens EVERY day to EVERY web server out there. I have a personal site that gets thousands of requests per day from bots.
Running a public server (like NTP) means you will get tons of strange requests. Moreso if you run a web server on the same IP because bots will scrape certificate transparency logs. The entire IPv4 space is scanned continuously.
This may sound harsh, but you cannot stop it. It is whack-a-mole. Filter it and move on, go outside and touch grass, seriously. This is not worth being upset over.
I treat these as an opportunity to tune my filters and firewall rules.
You don’t think there’s a difference between “hackers try to attack everything“ and “Tesla decided that I personally need to be tested as one of their systems due to a lazy misconfiguration“ are different?
Are you asking if I think Elon had this person's website on an 80" screen in a conference room whilst they collude how to send traffic to their site?
No, I do not.
I get thousands of these "security scanner" requests on a low-traffic site weekly, if not daily. Not a day goes by where someone doesn't try to kick in my door. Are you prepared to waste valuable time in your life to separate the white hat from the black hat requests?
I would have added the IP addresses to my firewall's IP blacklist and forgotten about it the next day. It's really all you can do.
> Are you asking if I think Elon had this person's website on an 80" screen in a conference room whilst they collude how to send traffic to their site?
i cannot fathom how you could possibly arrive at this reading of the gp's comment, what are you talking about
I'm aware how much crap there is on the Internet, I just think the specific nature of this (legitimate commercial vuln scanner thinks I'm Tesla) is funny
It does bring attention to: how many other organizations are doing this?
Tesla is a large enterprise.
They almost certainly subscribe to some overpriced SaaS garbage which is manned by offshore drones who by definition do not care because they're not paid enough to care.
Unfortunately this isn't the 80s anymore where you can ring up a system administrator at a university and get a human on the other end.
That said: cool looking website btw.
To OP robinpie:
I understand that Tesla is treating your NTP server, a volunteer server and part of the greater volunteer pool of NTP servers, as their own infrastructure.
However, I can't tell from the article if the scans originate with:
A.) IT staff at Tesla that are scanning exposed services on what they perceive, or claim wrongly, as their own network for vulnerabilities.
B.) Somehow a rogue operator (read botnet)
C.) A rogue operator who is using the cars themselves to run exploit scans?
C would be the most alarming and concerning.
> it has received ~8,000 requests from two of your scanning hosts
If it were 8000 requests per second, this might be worthy of some investigation.
But 8000 ntp requests alone consume far less than 1 us cent of compute + bandwidth. This isn't worth lifting a finger over.
Please read the article, it's not the volume of the NTP requests, they're actively sending exploit/attempt to compromise payloads. They're probing things in a way that you would ordinarily only do to your own internal infrastructure.
"They tried all kinds of exploits against me: path traversal, webshell uploads, probing software internals, probing WordPress and other CMS management endpoints, SSRF, Log4Shell, and a lot more."
If you host a webserver on the internet it is normal to receive that kind of traffic all the time. Source: I host a server on my Comcast connection.
But it shouldn't be normal, that a car company tries to automatically hack private servers.
I don't disagree with you, I have tons of things that have public interfaces (as mundane as a fully patched wordpress where the wp-admin login is accessible to external blog writers), we get tens of thousands of random shit anything per day. But the problem here is that Tesla is treating NTP pool operators like they are their internal infrastructure. Also because the attribution of the 'attacks' is fairly well known.
I don't go complaining on the internet about the absolute shitflood of compromised routers on broadband ISPs in Indonesia probing my stuff 24x7x365 because I know it would be futile. But if I found one specific american company that was repeatedly probing my stuff all the time? Maybe I'd escalate it.
Oh absolutely, I just think the specific nature of this (legitimate commercial vuln scanner thinks I'm Tesla) is funny
It's HTTP requests, not NTP requests, and the volume isn't the problem, it's that Assetnote is sending live exploit payloads /at all/ to a stranger on Tesla's behalf
Isn't this technically a crime, since they're actively attempting to access a computer system they don't own?
yup just report them to the FBI
In today’s world, a crime is only a crime if you get charged. Tesla has enough power to not get charged.
Tesla isn’t doing the scanning though, instead somebody thinks they are scanning Tesla, but Tesla points them to someone else.
The scanner is likely illegal.
The pointing is… so stupid nobody thought to make a law about it.
Ask a lawyer about sending an unpleasant letter to the scanner, detailing the situation and demanding that they cease & desist. That clobbers their "we didn't know" defense, and their Legal Dept. will likely order them to stop ASAP.
It's not 8000 requests. It's 8000 attempts to exploit various software on OP's server.
is this not something you can report to the FBI or something? is trying to hack someone servers not illegal?