This is the first I've actually heard of the name change... I used to use Varnish quite a bit, and had a decent grasp of VCL, for Drupal deployments. But I think Varnish 6 or 7 was when I started dropping off managing the caching layer as almost every project chose to offload caching to Cloudflare.
This is helpful, but it is yet to be seen how downstream picks it up. Wikidata[0] has renamed it and marked the Vinyl repo as the preferred one. Gentoo[1] renamed the package and switched to Vinyl. Homebrew[2] is now tracking Varnish Software (downstream of Vinyl). Fedora[3] has switched to Varnish Software as well. At endoflife.date[5], we renamed to vinyl and switched tracking as well. Wikipedia[6] has renamed Varnish (Software) -> Vinyl Cache.
Did not know this had happened, but does seem PHK (the original author of Varnish) is now with of the Vinyl Cache project, so this is not just a typical fork.
I'm not "refusing to add TLS support" I insist that the certificate is safely isolated in a separate process for security reasons. There are many ways to skin that cat.
the whole point of varnish software keeping a public version of "vinyl cache" as "varnish cache" with TLS is to give people a way to access a FOSS version with native TLS.
I think TLS is table-stakes now, and has been for the last 10 years, at least.
haproxy supports both the offload (client) and onload (backend) use case. This is the main reason for why I personally prefer it. I can not comment on how well hitch works in comparison, because I have not used it for years.
fwiw; Varnish Software still maintains and supports hitch, but we can't say we see a bright future for it. Both the ergonomics and the performance of not being integrated into Varnish are pretty bad. It was the crutch we leaned as it was the best thing we could make available.
I would recommend migrating off within a year or two.
To claim "the ergonomics and the performance of not being integrated into Varnish are pretty bad" you would need to show some numbers.
In my view, https://vinyl-cache.org/tutorials/tls_haproxy.html debunks the "ergonomics are bad" argument, because using TLS backends is literally no different than using non-TLS.
On performance, the fundamentals have already been laid out in https://vinyl-cache.org/docs/trunk/phk/ssl.html - crypto being so expensive, that the additional I/O to copy in and out another process makes no difference.
We've been pushing 1.5Tbps with TLS in lab settings. I've yet to see any other HTTP product being able to saturate these kind of networking. There is lots to be said about threading, but it is able to push a lot bandwidth.
And yes, I think the ergonomics are bad. Having varnish lose visibility into the transport means ACLs are gone, JA3 and similar are gone and the opportunity to defend from DoS are much more limited.
Crypto used to be expensive in 2010. It is no longer that expensive. All the serialization, on the other hand, that is expensive and latency is adding up.
Every single HTTP server in use out there has TLS support. The users expectation is that the HTTP server can deal with TLS.
Thanks for the info, but I'm a bit confused, sorry.
The reason for hitch was that tls and caching are a different concern, and the current recommendation is to use haproxy, which also isnt integrated into varnish/vinyl.
But you say that the reason to migrate off hitch is that its not integrated?
But what happend to separation of concerns, then? Is the plan to integrate tls termination into vinyl? Is this a change of policy/outlook?
So because perbu was clearly talking with his varnish software hat on, here's the perspective from someone working on Vinyl Cache FOSS only:
I already commented on the separation of concerns in the tutorial, and the unpublished project which one person from uplex is working on full time will have the key store in a separate process. You might want to read the intro of the tutorial if you have not done so.
But the main reason for why the new project will be integrating TLS more deeply has not been mentioned: It is HTTP/3, or rather QUIC. More on that later this year.
Varnish Software released hitch to facilitate TLS for varnish-cache.
Now that Varnish has been renamed, Varnish Software will keep what has been referred to as a downstream version or a fork, which has TLS built in, basically taking the TLS support from Varnish Enterprise.
This makes Hitch a moot point. So, I assume it'll receive security updates, but not much more.
Wrt. separation of concerns. Varnish with in-core TLS can push terabits per second (synthetic load, but still). Sure, for my blog, that isn't gonna matter, but having a single component to run/update is still valuable.
In particular using hitch/haproxy/nginx for backend is cumbersome.
Totally agree. But, if i may, the docs on varnish and tls are hella confusing. I just re-read the varnish v9 docs, and its not clear at all that/if it supports tls termination.
Literally every doc, from the install guide to the "beef in the sandwich" talks about it NOT supporting tls termination... then one teeny para in "extra features in v9.0" mentions 'use -A flag'...
This is cool! But also, worth mentioning. Sure I know its an open source project so you don't owe anyone anything, but also one with a huge company behind it - and this is a huge change of stance and also, sounds cool.
This is the first I've actually heard of the name change... I used to use Varnish quite a bit, and had a decent grasp of VCL, for Drupal deployments. But I think Varnish 6 or 7 was when I started dropping off managing the caching layer as almost every project chose to offload caching to Cloudflare.
This is helpful, but it is yet to be seen how downstream picks it up. Wikidata[0] has renamed it and marked the Vinyl repo as the preferred one. Gentoo[1] renamed the package and switched to Vinyl. Homebrew[2] is now tracking Varnish Software (downstream of Vinyl). Fedora[3] has switched to Varnish Software as well. At endoflife.date[5], we renamed to vinyl and switched tracking as well. Wikipedia[6] has renamed Varnish (Software) -> Vinyl Cache.
[0]: https://www.wikidata.org/wiki/Q1602447
[1]: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=679937b...
[2]: https://github.com/Homebrew/homebrew-core/pull/273280
[3]: https://src.fedoraproject.org/rpms/varnish/c/59f403810b746e0...
[4]: https://repology.org/project/varnish/packages
[5]: https://github.com/endoflife-date/endoflife.date/pull/9792
[6]: https://en.wikipedia.org/wiki/Vinyl_Cache
Did not know this had happened, but does seem PHK (the original author of Varnish) is now with of the Vinyl Cache project, so this is not just a typical fork.
I appreciate the mysql/mariadb comparison.
It finally made it click for me.
What's the deal with Antirez and PHK refusing to add TLS support?
I'm not "refusing to add TLS support" I insist that the certificate is safely isolated in a separate process for security reasons. There are many ways to skin that cat.
Aside: Loved your bit talking about money and varnish in Gift Community[1]. And thanks for the Beerware License, I've started using it!
[1]: https://www.youtube.com/watch?v=tOn-L3tGKw0
Varnish Enterprise has https support.
the whole point of varnish software keeping a public version of "vinyl cache" as "varnish cache" with TLS is to give people a way to access a FOSS version with native TLS.
I think TLS is table-stakes now, and has been for the last 10 years, at least.
just use the tool that does the job.
TLS in -> hitch or caddy Cache -> varnish/vinyl TLS out -> haproxy
Connect them up with Unix sockets, if you like.
because the topic keeps coming up, I now wrote the tutorial which we should have had years ago: https://vinyl-cache.org/tutorials/tls_haproxy.html
Thanks for this. You dont mention hitch though. Is that now deprecated/discouraged?
It hasn't seen much action in a while, but maybe thats cos it works?
haproxy supports both the offload (client) and onload (backend) use case. This is the main reason for why I personally prefer it. I can not comment on how well hitch works in comparison, because I have not used it for years.
fwiw; Varnish Software still maintains and supports hitch, but we can't say we see a bright future for it. Both the ergonomics and the performance of not being integrated into Varnish are pretty bad. It was the crutch we leaned as it was the best thing we could make available.
I would recommend migrating off within a year or two.
To claim "the ergonomics and the performance of not being integrated into Varnish are pretty bad" you would need to show some numbers. In my view, https://vinyl-cache.org/tutorials/tls_haproxy.html debunks the "ergonomics are bad" argument, because using TLS backends is literally no different than using non-TLS. On performance, the fundamentals have already been laid out in https://vinyl-cache.org/docs/trunk/phk/ssl.html - crypto being so expensive, that the additional I/O to copy in and out another process makes no difference.
But, again, if you have numbers, show them.
We've been pushing 1.5Tbps with TLS in lab settings. I've yet to see any other HTTP product being able to saturate these kind of networking. There is lots to be said about threading, but it is able to push a lot bandwidth.
And yes, I think the ergonomics are bad. Having varnish lose visibility into the transport means ACLs are gone, JA3 and similar are gone and the opportunity to defend from DoS are much more limited.
Crypto used to be expensive in 2010. It is no longer that expensive. All the serialization, on the other hand, that is expensive and latency is adding up.
Every single HTTP server in use out there has TLS support. The users expectation is that the HTTP server can deal with TLS.
Thanks for the info, but I'm a bit confused, sorry.
The reason for hitch was that tls and caching are a different concern, and the current recommendation is to use haproxy, which also isnt integrated into varnish/vinyl.
But you say that the reason to migrate off hitch is that its not integrated?
But what happend to separation of concerns, then? Is the plan to integrate tls termination into vinyl? Is this a change of policy/outlook?
Thanks!
So because perbu was clearly talking with his varnish software hat on, here's the perspective from someone working on Vinyl Cache FOSS only:
I already commented on the separation of concerns in the tutorial, and the unpublished project which one person from uplex is working on full time will have the key store in a separate process. You might want to read the intro of the tutorial if you have not done so.
But the main reason for why the new project will be integrating TLS more deeply has not been mentioned: It is HTTP/3, or rather QUIC. More on that later this year.
Varnish Software released hitch to facilitate TLS for varnish-cache.
Now that Varnish has been renamed, Varnish Software will keep what has been referred to as a downstream version or a fork, which has TLS built in, basically taking the TLS support from Varnish Enterprise.
This makes Hitch a moot point. So, I assume it'll receive security updates, but not much more.
Wrt. separation of concerns. Varnish with in-core TLS can push terabits per second (synthetic load, but still). Sure, for my blog, that isn't gonna matter, but having a single component to run/update is still valuable.
In particular using hitch/haproxy/nginx for backend is cumbersome.
TLS is a primary concern on the internet today.
Totally agree. But, if i may, the docs on varnish and tls are hella confusing. I just re-read the varnish v9 docs, and its not clear at all that/if it supports tls termination.
Literally every doc, from the install guide to the "beef in the sandwich" talks about it NOT supporting tls termination... then one teeny para in "extra features in v9.0" mentions 'use -A flag'...
This is cool! But also, worth mentioning. Sure I know its an open source project so you don't owe anyone anything, but also one with a huge company behind it - and this is a huge change of stance and also, sounds cool.
https://github.com/varnish/varnish/issues/33
It should be fixed fairly quick.
we're not at all a huge company. we're 80 people with a lot on our plates trying to juggle this on top of everything else.
but I truly appreciate the feedback. I'll reach our to the team working on this and see if I can make this a bit clearer.
I initially read this as "we" being "Varnish Software", but maybe that was wrong.
in my experience this has a lot more moving parts than it should.
Terminate tls and you have your cache.