Tons of sites are currently offline to hopefully prevent an attack until they can update PHP; just the things I've run into this morning: OBS, TicketTailor, tons of wikis...
I thought they were down earlier, but apparently they were just pre-announcing being down in about 30 minutes, until they finish patching their servers.
I've seen a number of notifications on X and Bluesky of people taking their sites offline, though. Supposedly Cloudflare is providing all customers a WAF rule that will mitigate the exploits they've been able to see so far (though they mention, this is not full protection—patching is still required).
I just moved my last PHP site, my personal blog, to a static site generator. It looks like my timing was lucky!
The only significant thing lost of comments, but they have been getting more spammy over the years. Its actually better now too in many ways and I fixed various issues during the move.
Which is why suggestions like OP’s are never useful or actually serious. I honestly consider them snobby. Like the person who tells you you really ought to check your engine oil, radiator fluid, and brakes everytime you get in a car. It would prevent some awful issues sure, but it’s not gonna happen for 99% of people.
Having static site generator still require an admin endpoint that end users can use to author, publish or otherwise manage the content. Pushing that to a local tool means now you need to somehow synchronize that content across all of the end users devices without git. Not to mention if there is no server side component that means the client needs to have full permission to publish content on the server. Until recently ssh was the only option. Maybe someone could vibe an iroh app that gives you a vpn'd server side app without that now.
Did you try to implement it seriously? Naturally you need an admin endpoint, it just does not have to be reachable for everybody from the internet, just via your VPN. Big difference.
The person you're replying to didn't mention a static site generator. You could simply use Vinyl(fka Varnish), and after you pulled your PHP down, your boring brochure site would still be up.
The first site I put up professionally was a handrolled static generator written in PHP for a state website. Users just edited text files, triggered a rebuild, and then there was a sync to prod if staging looked fine. They could even add new sections by creating new subdirectories, and Imagemagick would automatically conjure new menu buttons and the code would put them in the right place.
> Pushing that to a local tool means now you need to somehow synchronize that content across all of the end users devices without git.
This is not hard. Rsync? Rsyncd?
> Not to mention if there is no server side component
Why would there be no server-side component? What does that even mean? Just give them a script to trigger if you don't want them to have permission to do anything. Give them another script to revert if you don't want to get any emergency calls.
This is not advanced technology. Weird to talk about it like it's impossible.
> never useful or actually serious
So strange. That site was so simple to deal with that it stayed up for a decade after I left. Ancient technology.
Generate static pages from your content system. Even for wordpress there's some projects.
If you need adaptations, AI should be helpful.
You can sell this also to bosses: Delivery of static sites consumes drastically less resources.
Which kind of makes sense, I think a lot of Dutch websites have been taken offline. I think it’s due to recent regulations that have been introduced.
I think, given the (supposed) seriousness of this issue, it’s an adequate response. I suspect Cloudflare et al will have rules added to their managed WAFs within a few hours after release as well.
That said: it seems like the core issue is that they can’t release PHP publicly without revealing the exploit, wouldn’t it have made more sense to coordinate in the background with these organizations, and important distros, so that this could have been patched without revealing the exploit?
It creates an absolutely enormous political snarl to decide who is "worthy" of getting advance notice on something like this, and to correctly determine who is capable of keeping the secret in the presence of highly motivated active attackers who may have capabilities all the way up to and including "state level actors".
Basically, it's only possible in our imaginations where we don't have to get into the nitty-gritty of exactly who gets what notifications. It isn't possible in reality for something this large and consequential.
But they say it's a stopgap and still advise patching. It makes sense, since their tweet also says "are updating mitigations as new vectors show up" so it seems like there are many ways to exploit this, whatever it is.
Yes our university is doing the same right now! They where thinking of putting the whole network behind VPN access to protect the poor sysadmins in the Institutes that have not updated their website in years…
I know nothing about the Belgian Center for Cybersecurity, but the fact that they are running a platform that has been infamous for its security issues for 25 years, and been considered essentially obsolete for 15 years, immediately makes me not take them seriously.
This is like a Health Ministry recommending leech therapy to its employees.
That's reducing things too far. "What car manufacturer hasn't had car crashes?" All cars are equally safe?
There's a vast difference between different languages and CMSs and how likely it is for critical flaws to occur. PHP has a horrible track record and continues to surprise us negatively.
Given the % of traffic it serves and over how long, I actually wonder if you have any proof that PHP actual track record in on the bad side because I think it will probably be on the better side actually.
I fought hard about four years ago at a previous company to upgrade us from PHP 5 to 8 because I think 5 was going EOL or something like that (could be wrong about that now). Based on my experience in software, I have to imagine the vast majority of normal developers do not give a damn about security in being in security patch lifecycles on software.
> I have to imagine the vast majority of normal developers do not give a damn about security in being in security patch lifecycles on software.
I just don't think it's the developer's responsibility.
Depending on the size of the team of course, but the work that developer's do comes from someone else. Empower me to make decisions and I'll make them, but don't blame me for the choices other people make if you won't empower me.
I think I get where you're coming from. I've worked mostly on small teams (sub 30) so it's always been on us. The engineering leadership never prioritizes security unless it's something as big as the log4J stuff because it was on customers' minds.
That said, if they're software packages from something like Node or PyPI, I do think that it falls on the developer. If the point is that leadership needs to give devs the ability to act on that, that's very fair.
Exactly. It's not the bricklayer's responsibility when mortar fails. It's the person who specified that mortar.
So I'd agree it isn't not a coder or developer's responsibility when they're given a bill of materials and told how to build, but it is an engineer's responsibility. Comp should recognize that as well.
I've worked at a company that delivered on-premise software, and the runners of said software were... risk-averse, but in a strange way in that they didn't really want to upgrade software if it didn't need to. So the software was stuck on PHP 5.something, also because newer versions required a RHEL update which they also didn't like to do.
I don't know. Still, at a time like this I would expect a 99% static public website to be up, running and serving all comers. This is a complete mission fail, both in terms of using PHP and in terms of not having a fall-back plan in place.
Due respect, neither of those make sense. PHP is boring software, and this website routinely suggests (1) use it if it is fit for purpose and (2) don't rewrite for the sake of it. For most sites, the engineering effort to have a static backup of their dynamic website (say, generated by Wordpress or Drupal) just isn't possible. They just need to power down and patch.
[EDIT: I see I may have erroneously extended your criticism to sites other than NCSC, for which it does make sense. I would still defend using PHP.]
Because websites with news, continuously updated advice, job openings, etc aren't all that static? This type of site hugely benefits from having a easily understandable CMS that you can teach to more or less everyone in 2 days or less. I suffered through a few of those back in my time in the military, where an instructor would teach a class of 20-30 how to build new pages in the CMS
Now are there more powerful tools out there that result in more scalable websites? Sure. My personal sites are built around static site generators for that exact reason. But none of better tools are as easy to understand for non-programmers as wordpress or one of the other click-and-drag CMS tools.
Not all hacks are used to steal data. It's an authoritative website. If this hack is used to host a fake PHP fix stuffed with credential stealers then you have a problem.
Sometimes, they just want to send spam. Ages ago, someone got a hold of our company's webserver and all they used it for was sending a ridiculous volume of emails. Our ISP spotted the traffic spike and pulled our server off the network.
Oh I absolutely agree they should take it offline until the fix is ready, I just mean that the NCSC itself is very much still operational even if their public website is temporarily offline. Like a computer with the monitor switched off.
PHP is going to publish the patch in 5 minutes, right? On their website which runs on... Perl Catalyst? Ruby on Rails? Django? What could possibly go wrong?
Because you need a certain amount of notice and in this case drama to be created so people know to update ASAP, before the exploit is out in the wild. As soon as these patches are out it becomes a lot easier for any actors to figure out exactly what they are patching, and how to exploit it.
Announcing when this patch is going to be released should leave a much shorter vulnerable window with the exploit out in the wild than releasing it immediately and hoping people update before they're a target.
Once the fix is released, people will reverse-engineer it and use the exploit on pages that haven't updated yet.
So they need to give time for infra people in all different time zones to wake up, learn about the announcement, and plan how to patch it.
The intended course of action is 1. take down website a few hours before the fix drops 2. apply the fix 3. only make website public again after it has been fixed
> Today, a remote code execution vulnerability was found in PHP, called php2shell.
[...] Since explain xkcd is affected by this exploit, I have decided to take the wiki down until tomorrow.
PHP is worse in this aspect with its long tail of legacy and staying close to C. It's absolutely possible that memory-unsafe Python interpreters could have something vaguely similar, but it's much less likely. Plus there are Python interpreters in memory-safe languages.
In any case, memory-unsafe codebases will continue to be a significant danger. Be it in interpreters or anywhere else. Just like they have been for the past 20+ years.
>Safely ahead in a digitally resilient Netherlands.
Things being offline seems like a daily occurrence recently. Is it a co-incidence? Is it AI? Are Russia/Iran doing an e-blitzkrieg?
If it's a combination of all three, surely nation states are going to unleash everything possible right now and do as much damage before the world adapts and becomes resilient?
If the latter, would be interesting to put an actual cost on all this downtime and whether it's actually making any impact, or is just noise/free training.
Tons of sites are currently offline to hopefully prevent an attack until they can update PHP; just the things I've run into this morning: OBS, TicketTailor, tons of wikis...
OBS seems fine
Only 'parts' of their website are down currently (e.g. download pages, linked from the home page: https://obsproject.com/welcome-redirect returns a 502).
TicketTailor looks up to me?
I thought they were down earlier, but apparently they were just pre-announcing being down in about 30 minutes, until they finish patching their servers.
I've seen a number of notifications on X and Bluesky of people taking their sites offline, though. Supposedly Cloudflare is providing all customers a WAF rule that will mitigate the exploits they've been able to see so far (though they mention, this is not full protection—patching is still required).
I don't know, have you done anything to impress them?
That's why (mostly) static sites should be delivered as static HTML.
I just moved my last PHP site, my personal blog, to a static site generator. It looks like my timing was lucky!
The only significant thing lost of comments, but they have been getting more spammy over the years. Its actually better now too in many ways and I fixed various issues during the move.
I'm using Remark42; some downsides but somehow, spam has not been a big issue for me. https://remark42.com
I've started using atproto as comments on my static blog:
https://sequoia.pub/comments
How do you deal with end users wanting to edit their content? They have no clue what Markdown or Git is.
Which is why suggestions like OP’s are never useful or actually serious. I honestly consider them snobby. Like the person who tells you you really ought to check your engine oil, radiator fluid, and brakes everytime you get in a car. It would prevent some awful issues sure, but it’s not gonna happen for 99% of people.
Having static site generator still require an admin endpoint that end users can use to author, publish or otherwise manage the content. Pushing that to a local tool means now you need to somehow synchronize that content across all of the end users devices without git. Not to mention if there is no server side component that means the client needs to have full permission to publish content on the server. Until recently ssh was the only option. Maybe someone could vibe an iroh app that gives you a vpn'd server side app without that now.
Use CityDesk: https://www.joelonsoftware.com/2001/10/12/what-does-citydesk...
(So old.....)
Did you try to implement it seriously? Naturally you need an admin endpoint, it just does not have to be reachable for everybody from the internet, just via your VPN. Big difference.
The person you're replying to didn't mention a static site generator. You could simply use Vinyl(fka Varnish), and after you pulled your PHP down, your boring brochure site would still be up.
The first site I put up professionally was a handrolled static generator written in PHP for a state website. Users just edited text files, triggered a rebuild, and then there was a sync to prod if staging looked fine. They could even add new sections by creating new subdirectories, and Imagemagick would automatically conjure new menu buttons and the code would put them in the right place.
> Pushing that to a local tool means now you need to somehow synchronize that content across all of the end users devices without git.
This is not hard. Rsync? Rsyncd?
> Not to mention if there is no server side component
Why would there be no server-side component? What does that even mean? Just give them a script to trigger if you don't want them to have permission to do anything. Give them another script to revert if you don't want to get any emergency calls.
This is not advanced technology. Weird to talk about it like it's impossible.
> never useful or actually serious
So strange. That site was so simple to deal with that it stayed up for a decade after I left. Ancient technology.
> Users just edited text files, triggered a rebuild, and then there was a sync to prod if staging looked fine.
Thanks for proving my point.
> This is not hard. Rsync? Rsyncd?
How do I do that on an iPad or an Android?
> Just give them a script to trigger if you don't want them to have permission to do anything.
uhhh, what permissions does that "script" have? or is that your /wp-admin equivilent?
> So strange.
Indeed.
Generate static pages from your content system. Even for wordpress there's some projects. If you need adaptations, AI should be helpful. You can sell this also to bosses: Delivery of static sites consumes drastically less resources.
Exactly!
Unfortunately my static HTML is pre-rendered by a PHP engine before being served.
That's not a problem - as long as the PHP scripts are not reachable by the general public you should be fine, generally speaking.
These particular vulnerabilities affect PHP scripts accepting POST requests as far as I've been able to gather so far.
You might want to use Cecil ( https://github.com/Cecilapp/Cecil )
It's a static web site generator, powered by php :)
Which kind of makes sense, I think a lot of Dutch websites have been taken offline. I think it’s due to recent regulations that have been introduced.
I think, given the (supposed) seriousness of this issue, it’s an adequate response. I suspect Cloudflare et al will have rules added to their managed WAFs within a few hours after release as well.
That said: it seems like the core issue is that they can’t release PHP publicly without revealing the exploit, wouldn’t it have made more sense to coordinate in the background with these organizations, and important distros, so that this could have been patched without revealing the exploit?
It creates an absolutely enormous political snarl to decide who is "worthy" of getting advance notice on something like this, and to correctly determine who is capable of keeping the secret in the presence of highly motivated active attackers who may have capabilities all the way up to and including "state level actors".
Basically, it's only possible in our imaginations where we don't have to get into the nitty-gritty of exactly who gets what notifications. It isn't possible in reality for something this large and consequential.
Cloudflare have already deployed WAF managed rules to help mitigate the vuln: https://x.com/dok2001/status/2112811965735202970
But they say it's a stopgap and still advise patching. It makes sense, since their tweet also says "are updating mitigations as new vectors show up" so it seems like there are many ways to exploit this, whatever it is.
A WAF can block known _exploits_ it doesn't prevent the vulnerability from being abused by a new / other / slightly changed exploit.
Also a WAF isn't magical. I remember the whole 8KiB AWS WAF bypass thing.
You also just described system patching.
I expect that Cloudflare already knows some details of the problem and will have the rules up within minutes or seconds.
Yes our university is doing the same right now! They where thinking of putting the whole network behind VPN access to protect the poor sysadmins in the Institutes that have not updated their website in years…
All it takes is one small leak. Which seems to be what happened here.
Ah, so you’re saying that proper orchestration was happening, it’s just that someone leaked it?
> Which kind of makes sense, I think a lot of Dutch websites have been taken offline.
Europa.eu and all related subdomains were down very early CEST morning. They run Drupal.
Also the Belgian Centre for Cybersecurity is down https://ccb.belgium.be/
Dark day for Drupal and Wordpress, heh
I know nothing about the Belgian Center for Cybersecurity, but the fact that they are running a platform that has been infamous for its security issues for 25 years, and been considered essentially obsolete for 15 years, immediately makes me not take them seriously.
This is like a Health Ministry recommending leech therapy to its employees.
What platform hasn't had security issues?
Anyway.
CCB is overtly a marketing organization, not an IT security one.
IIRC they don't even host their own website, The Chancellery of the Prime Minister does it for them.
> What platform hasn't had security issues?
That's reducing things too far. "What car manufacturer hasn't had car crashes?" All cars are equally safe?
There's a vast difference between different languages and CMSs and how likely it is for critical flaws to occur. PHP has a horrible track record and continues to surprise us negatively.
Given the % of traffic it serves and over how long, I actually wonder if you have any proof that PHP actual track record in on the bad side because I think it will probably be on the better side actually.
This attitude is what needs to change if we are going to be serious about security. Everything needs to be checked.
my thoughts and prayers to the stubborn php 4-6 developers who refused to upgrade since 7 and now they have breaking changes with the new 9.2
I fought hard about four years ago at a previous company to upgrade us from PHP 5 to 8 because I think 5 was going EOL or something like that (could be wrong about that now). Based on my experience in software, I have to imagine the vast majority of normal developers do not give a damn about security in being in security patch lifecycles on software.
> I have to imagine the vast majority of normal developers do not give a damn about security in being in security patch lifecycles on software.
I just don't think it's the developer's responsibility.
Depending on the size of the team of course, but the work that developer's do comes from someone else. Empower me to make decisions and I'll make them, but don't blame me for the choices other people make if you won't empower me.
I think I get where you're coming from. I've worked mostly on small teams (sub 30) so it's always been on us. The engineering leadership never prioritizes security unless it's something as big as the log4J stuff because it was on customers' minds.
That said, if they're software packages from something like Node or PyPI, I do think that it falls on the developer. If the point is that leadership needs to give devs the ability to act on that, that's very fair.
Exactly. It's not the bricklayer's responsibility when mortar fails. It's the person who specified that mortar.
So I'd agree it isn't not a coder or developer's responsibility when they're given a bill of materials and told how to build, but it is an engineer's responsibility. Comp should recognize that as well.
That's the problem with the tech industry. Nobody gives a fuck.
When a doctor makes a mistake they get to practice medicine in Peru for the rest of their life.
I've worked at a company that delivered on-premise software, and the runners of said software were... risk-averse, but in a strange way in that they didn't really want to upgrade software if it didn't need to. So the software was stuck on PHP 5.something, also because newer versions required a RHEL update which they also didn't like to do.
Probably easier to back port the patch to 4-6 code than upgrade to 9.2.
Once it's out it's likely to be an easy "disable this damnable feature nobody needs" as usual.
I feel like I'm reading a conversation between bots.
There's no such thing as PHP 6. There's no such thing as PHP 9.2.
They're taking about Drupal versions, not PHP
Ah, my mistake.
Well, the public website of the NCSC is offline. Who knows what crazy proprietary stuff they have.
I don't know. Still, at a time like this I would expect a 99% static public website to be up, running and serving all comers. This is a complete mission fail, both in terms of using PHP and in terms of not having a fall-back plan in place.
Due respect, neither of those make sense. PHP is boring software, and this website routinely suggests (1) use it if it is fit for purpose and (2) don't rewrite for the sake of it. For most sites, the engineering effort to have a static backup of their dynamic website (say, generated by Wordpress or Drupal) just isn't possible. They just need to power down and patch.
[EDIT: I see I may have erroneously extended your criticism to sites other than NCSC, for which it does make sense. I would still defend using PHP.]
Why would you use boring tech to dynamically create sites that are inherently static?
Because websites with news, continuously updated advice, job openings, etc aren't all that static? This type of site hugely benefits from having a easily understandable CMS that you can teach to more or less everyone in 2 days or less. I suffered through a few of those back in my time in the military, where an instructor would teach a class of 20-30 how to build new pages in the CMS
Now are there more powerful tools out there that result in more scalable websites? Sure. My personal sites are built around static site generators for that exact reason. But none of better tools are as easy to understand for non-programmers as wordpress or one of the other click-and-drag CMS tools.
Not all hacks are used to steal data. It's an authoritative website. If this hack is used to host a fake PHP fix stuffed with credential stealers then you have a problem.
> Not all hacks are used to steal data
Sometimes, they just want to send spam. Ages ago, someone got a hold of our company's webserver and all they used it for was sending a ridiculous volume of emails. Our ISP spotted the traffic spike and pulled our server off the network.
Oh I absolutely agree they should take it offline until the fix is ready, I just mean that the NCSC itself is very much still operational even if their public website is temporarily offline. Like a computer with the monitor switched off.
PHP 4 maybe? lol
PHP is going to publish the patch in 5 minutes, right? On their website which runs on... Perl Catalyst? Ruby on Rails? Django? What could possibly go wrong?
It's scheduled to release at 14:00 UTC so around an hour from now.
Whoops, sorry, that's right. Thanks!
Why is this fix “scheduled” rather than ASAP?
Because you need a certain amount of notice and in this case drama to be created so people know to update ASAP, before the exploit is out in the wild. As soon as these patches are out it becomes a lot easier for any actors to figure out exactly what they are patching, and how to exploit it.
Announcing when this patch is going to be released should leave a much shorter vulnerable window with the exploit out in the wild than releasing it immediately and hoping people update before they're a target.
Once the fix is released, people will reverse-engineer it and use the exploit on pages that haven't updated yet.
So they need to give time for infra people in all different time zones to wake up, learn about the announcement, and plan how to patch it.
The intended course of action is 1. take down website a few hours before the fix drops 2. apply the fix 3. only make website public again after it has been fixed
As far as I understand it, releasing this fix also broadcasts the vulnerability.
Scheduled release gives the option to turn things off on a set schedule while applying patches.
e: turns out several people answered this. Whoops.
Who knows? Not any official explanation for this.
php.net does run php, but one would assume they are already on the patched version.
https://github.com/php/web-php
Given their site is mostly static and currently getting a lot of traffic, wouldn't it make sense to have it mostly be static html?
Maybe change URL to https://web.archive.org/web/20261021115632/https://www.ncsc.... ?
Lots of obsolete PHP still being used by the Dutch government: https://basisbeveiliging.nl/museum/NL/
Finnish state services too.
https://yle.fi/a/74-20252600
Also explainxkcd: https://www.explainxkcd.com/wiki/index.php/Main_Page
> Today, a remote code execution vulnerability was found in PHP, called php2shell. [...] Since explain xkcd is affected by this exploit, I have decided to take the wiki down until tomorrow.
Never enough blame PHP/s
It's easy to harp on PHP but I wonder if we're going to get this kind of vulnerabilities in languages like Python or Go soon too.
PHP is worse in this aspect with its long tail of legacy and staying close to C. It's absolutely possible that memory-unsafe Python interpreters could have something vaguely similar, but it's much less likely. Plus there are Python interpreters in memory-safe languages.
In any case, memory-unsafe codebases will continue to be a significant danger. Be it in interpreters or anywhere else. Just like they have been for the past 20+ years.
The way things are going, I imagine there are going to be vulnerabilities found in everything
Probably not. PHP is so badly built these kinds of things happen frequently.
All my favorite private trackers such as empornium are down :(
Not sure if that's the name or a typo but I'm not gonna lookup that
bit the bullet, it is what you think it is.
>The National Cyber Security Center (NCSC)
>Safely ahead in a digitally resilient Netherlands.
Things being offline seems like a daily occurrence recently. Is it a co-incidence? Is it AI? Are Russia/Iran doing an e-blitzkrieg?
If it's a combination of all three, surely nation states are going to unleash everything possible right now and do as much damage before the world adapts and becomes resilient?
If the latter, would be interesting to put an actual cost on all this downtime and whether it's actually making any impact, or is just noise/free training.
so is Australian one https://www.cyber.gov.au. It was still working ~hour ago just returning 403 on alerts subsection https://www.cyber.gov.au/alert/critical-vulnerabilities-in-p... but now its whole thing.
PHP strikes again.