As an aside, it is really interesting to see a computational package that, while supporting multiple GPU vendors, was first vetted on AMD, not NVidia. It is encouraging to see ROCM finally shaking off its reputation for poor support.
>the reference implementation from Physically Based Rendering (Pharr, Jakob, Humphreys)
I'd like to know a little about the process you went through for the port. That book * sounds like an excellent resource to start from but what was it like using it and the code?
I've done lots of manually refactoring of the initial Prototype in Trace.jl (by Anton Smirnov, who I think ported an earlier version of the pbrt book).
This helped familiarizing myself with the math and infrastructure and the general problems a raytracer faces and lay the ground work for the general architecture and what to pay attention to for fast GPU execution. One key insight was, that its possible to not need to have an UberMaterial, but instead use a MultiTypeSet for storing different materials and lights, which allows fast and concretely typed iterations.
Then I found that pbrt moved away from the initial design and I used claude code to port large parts of the new C++ code to Julia. This lead to a pretty bad port and I had lots of back and forth to fix bugs, improve the GPU acceleration, make the code more concise and "Julian" and correct the AIs mistakes and bogus design decisions ;) This polish isn't really over yet, but it works well enough and is fast enough for a beta release!
Is the material description part of the language the same as in PBRT?
I'm asking because I had a lot of trouble trying to describe interfaces between materials, only to find out that what I wanted to do was not possible in PBRT without modifying the code. Apparently, in PBRT a material can only have one other material touching it. So, for example rendering a glass filled with water and ice is not possible without hacks. From a user's point of view this is a bit of a let-down, of course.
Nope, we made a complete high level Julia interface and I plan to have the Makie API be the main user facing scene description, which can be more descriptive than pbrt I think!
Sorry, I was on my phone. This doesn't seem to be a problem of the description language, but rather how the integrator and materials work internally, so this works the same way in Julia currently.
I do think though, that its more approachable to add experimental features like this in the Julia version.
Would certainly be an interesting project! I do want to over time get further away from the pbrt-v4 architecture and get to something much more modular and easy to extend.
I feel like the overlaps resolve should happen at scene creation time, to not have an expensive priority stack at raytracing time - then it would be just a matter of better tracking the media at boundary crossing. But haven't really thought this through of course ;)
I think it was a problem with the language as well as how they handle it internally. It was basically the algorithm that dictates how the language works, and consequently there was no way to have one material touch more than one other material. But I might misremember.
Anyway, I'm looking at this from the user's perspective. I wanted to do some physics-based ray-tracing with lenses and pbrt is what I ended up trying. As such, I really needed the multi-material aspect to work correctly. Also, it would be nice to be able to describe surfaces using a z=f(x,y) kind of formulation, or a way to place a hook in the renderer.
It's definitely an architectural problem as well. I do wonder if we could extend that though, without too much trouble for the general architecture - after all, the material does not necessarily need to represent all the outside materials and instead the ray only needs to be able to go from one medium to another.
I'm happy to chat about possible extensions in that direction, although to be fair I wont have much time in the next weeks to sit down on anything like this.
But, I do really hope that this can become a playground for ray tracing experiments in general!
I think maybe the easiest way to tackle the problem is to have the language describe surfaces instead of solid objects, and let every surface have a normal and two materials. This might be the most natural representation for a ray tracer.
honestly the AMD-first bit surprised me - usually ROCm support is an afterthought or just broken outright.
curious about BVH traversal specifically. dynamic dispatch patterns across GPU backends can get weird fast. did KernelAbstractions hold up there or were there vendor-specific fallbacks needed for the heavier acceleration structure work?
Well I'm a bit of an AMD "fanboy" and really dislike NVIDIA's vendor lock in.
I'm not sure what you mean by dynamic dispatch across GPU backends - nothing should be dynamic there and most easier primitives map quite nicely between vendors (e.g. local memory, work groups etc).
To be honest, the BVH/TLAS has been pretty simple in comparison to the wavefront infrastructure. We haven't done anything fancy yet, but the performance is still really good. I'm sure there are still lots of things we can do to improve performance, but right now I've concentrated on getting something usable out.
Right now, we're mostly matching pbrt-v4 performance, but I couldn't compare to their NVIDIA only GPU acceleration without an NVIDIA gpu. I can just say that the performance is MUCH better than what I initially aimed for and it feels equally usable as some of the state of the art renderers I've been using. A 1:1 comparison is still missing though, since it's not easy to do a good comparison without comparing apples to oranges (already mapping materials and light types from one render to another is not trivial).
pbrt-v4 parity is a solid baseline - that codebase already leans hard on NVIDIA so a fair comparison was always going to be messy. surprised wavefront was the harder bit though, i'd have expected BVH tuning to be the nightmare.
To be fair I was suprised too. But I made a relatively simple straight port from the AMD rays sdk plus some input from the pbrt-v4 CPU bvh code and it just worked relatively well out of the box...
This is the main intersection function which is quite simple: https://github.com/JuliaGeometry/Raycore.jl/blob/sd/multityp...
I'm not even using local memory, since it was already fast enough ;)
But I think we can still do quite a lot, large parts of the construction code are still very messy, and I want to polish and modularize the code over time.
I don't hear nearly as much about Julia as I used to. A few years ago the view was that it was about to replace Python as the language of choice for data science. Seems like that didn't happen?
I think the hype has slowed down, but all growth statistics haven't.
Personally, I think Julia is the only language where I can implement something like Makie without running into a maintenance nightmare, and with Julia GPU programming is actually fun and high level and composes well, which I miss in most other languages.
So, I dont really care about it replacing python or not.
I do think for replacing python Julia will need to solve compilation latency, shipping AOT binaries and maybe interpret more of the glue code, which currently introduces quite a lot of compilation overhead without much gains in terms of performance.
I don't know about everyone else, but slow Julia compilation continues to cause me ongoing suffering to this day. I don't think they're ever going to "fix" this. On a standard GitHub Actions Windows worker, installing the public Julia packages I use, precompiling, and compiling the sysimage takes over an hour. That's not an exaggeration. I had to juice the worker up to a custom 4x sized worker to get the wall clock time to something reasonable.
It took me days to get that build to work; doing this compilation once in CI so you don't have to do it on every machine is trickier than it sounds in Julia. The "obvious" way (install packages in Docker, run container on target machine) does not work because Julia wants to see exactly the same machine that it was precompiled on. It ends up precompiling again every time you run the container on other machines. I nearly shed a tear the first time I got Julia not to precompile everything again on a new machine.
R and Python are done in five minutes on the standard worker and it was easy; it's just the amount of time it takes to download and extract the prebuilt binaries. Do that inside a Docker container and it's portable as expected. I maintain Linux and Windows environments for the three languages and Julia causes me the most headaches, by far. I absolutely do not care about the tiny improvement in performance from compiling for my particular microarch; I would opt into prebuilt x86_64 generic binaries if Julia had them. I'm very happy to take R's and Python's prebuilt binaries.
Versus Python, it seems to fork into the "thinkers" vs "doers" camp. Julia provides a level of abstraction that some people find comforting. I thought I could use it as a sort of open source Matlab for a lot of thinky, 1-based index code I had lying around. It didn't meet my needs. And "spend half an hour waiting for a Jupyter notebook to boot up" is real. Great for some but it's not compatible with the way I work.
Elsewhere someone used the term "janky" and perhaps it's the fact that there are so many incredibly smart people around it that makes it so janky. By way of example, somebody needed to check disk space and the architect told him to shell out to Python.
Remember when LLVM first came out and it got kudos for the quality of its error messages? Well if you miss the old-school 1980s GCC experience the nonsense that eventually comes out of the Julia compiler after an hour will relight that flame.
Want to use greek letters and other symbols that don't appear on your keyboard as variable names? You've found your people.
As someone who currently uses dabbles in both. That prediction seems a bit unrealistic. Julia is a fantastic language but it has some trade offs that need to be considered. Probably the most well known is `time to first x`. Julia like Python is used comfortably in notebooks but loading libraries can take a minute, compared to Python where it happens right away. It may lead you to not reach for it when you want to do quick testing of something especially plotting. You can mitigate this somewhat by loading all the libraries you'll ever need at startup (preferably long before you are ready to experiment) but that assumes you already know what libraries you'll need for what you're wanting to try.
What prediction? Maybe I need to rephrase what I said: My prediction is, that if Julia ever wants to have a shot at replacing Python, it absolutely has to solve the first time to first x problem!
That's what I mean by shipping fully ahead of time compiled binaries and interpreting more glue code - which both have the potential to solve the first time to x problem.
IMO it just had too many rough edges. Very slow compilation, correctness issues (https://yuri.is/not-julia/), kinda janky tooling (not nearly as bad as pip tbf). Even basic language mistakes like implicit variable declaration and 1-based indexing (in 2012??).
Yes 1-based indexing is a mistake. It leads to significantly less elegant code - especially for generic code - and is no harder to understand than 1-based indexing for people capable of programming. Fight me.
> Yes 1-based indexing is a mistake. It leads to significantly less elegant code - especially for generic code - and is no harder to understand than 1-based indexing for people capable of programming.
Some would argue that 0-based indexing is significantly less elegant for numerical/scientific code, but that depends on whether they come from a MATLAB/Fortran or Python/C(++) background.
A decision was made to target the MATLAB/Fortran (and unhappy? Python/C++) crowd first, thus the choice of 1-based indexing and column-major order, but at the end of the day it's a matter of personal preference.
0-based indexing would have made it easier to reach a larger audience, however.
> and is no harder to understand than 1-based indexing for people capable of programming.
Aside from the fact that 1-based indexing is better for scientific code (see Fortran), I don’t think that it matters very often. I don’t think that any Julia program I’ve ever written would need to change if Julia adopted 0-based tomorrow. You don’t typically write C-style loops in Julia; you use array functions and operators, and if you need to iterate you write `for i in array ...`. If you really need the first or last element you write `a[begin]` or `a[end]`.
> the fact that 1-based indexing is better for scientific code (see Fortran)
It really isn't. "Scientific code" isn't some separate thing.
The only way it can help is if you're trying to write code that matches equations in a paper that uses 1-based indexing. But that very minor advantage doesn't outweigh the disadvantages by a wide margin. Lean doesn't make this silly mistake.
lol.
There's not much to fight since its a very personal problem how you want to write code. It's evident that all the capable programmers in the Julia community, have found satisfactory ways to get around it, so if you haven't yet, I don't see how that's a Julia problem ;)
I can only say I haven't had a single problem with one based indexing in 12 years of developing Julia code.
I also haven't run into many correctness issues compared to other languages I've been using. I think Yuri also has been using lots of packages which haven't been very mature. How on earth can you compare a 10 years old library with lots of maintainers with packages created in one year by one person? That's at least what Yuri's critic boils down to me.
On iOS Safari the videos are fullscreening themselves as I scroll. I've seen this on other blogs before but I don't know what causes it. Super annoying
Don't quote me on this, but I think there is a "playsinline" / "webkit-playsinline" attribute for the video element you need to add to avoid that, + if it's autoplay you need to set "muted" too. I've also had this happen and I think both/either of those solved it last time.
As an aside, it is really interesting to see a computational package that, while supporting multiple GPU vendors, was first vetted on AMD, not NVidia. It is encouraging to see ROCM finally shaking off its reputation for poor support.
well, I do hate vendor lock in with a passion ;) But yeah, a lot did happen, this likely wouldn't have been possible one or two years ago!
It's says:
>the reference implementation from Physically Based Rendering (Pharr, Jakob, Humphreys)
I'd like to know a little about the process you went through for the port. That book * sounds like an excellent resource to start from but what was it like using it and the code?
* https://pbrt.org/
I've done lots of manually refactoring of the initial Prototype in Trace.jl (by Anton Smirnov, who I think ported an earlier version of the pbrt book). This helped familiarizing myself with the math and infrastructure and the general problems a raytracer faces and lay the ground work for the general architecture and what to pay attention to for fast GPU execution. One key insight was, that its possible to not need to have an UberMaterial, but instead use a MultiTypeSet for storing different materials and lights, which allows fast and concretely typed iterations.
Then I found that pbrt moved away from the initial design and I used claude code to port large parts of the new C++ code to Julia. This lead to a pretty bad port and I had lots of back and forth to fix bugs, improve the GPU acceleration, make the code more concise and "Julian" and correct the AIs mistakes and bogus design decisions ;) This polish isn't really over yet, but it works well enough and is fast enough for a beta release!
That's an impressive accomplishment and a fantastic tool to explore.
Is the material description part of the language the same as in PBRT?
I'm asking because I had a lot of trouble trying to describe interfaces between materials, only to find out that what I wanted to do was not possible in PBRT without modifying the code. Apparently, in PBRT a material can only have one other material touching it. So, for example rendering a glass filled with water and ice is not possible without hacks. From a user's point of view this is a bit of a let-down, of course.
Context: https://news.ycombinator.com/item?id=45668543
Nope, we made a complete high level Julia interface and I plan to have the Makie API be the main user facing scene description, which can be more descriptive than pbrt I think!
Ok. Did you see this:
https://blog.yiningkarlli.com/2019/05/nested-dielectrics.htm...
And I'm curious how you solve it.
Sorry, I was on my phone. This doesn't seem to be a problem of the description language, but rather how the integrator and materials work internally, so this works the same way in Julia currently. I do think though, that its more approachable to add experimental features like this in the Julia version. Would certainly be an interesting project! I do want to over time get further away from the pbrt-v4 architecture and get to something much more modular and easy to extend. I feel like the overlaps resolve should happen at scene creation time, to not have an expensive priority stack at raytracing time - then it would be just a matter of better tracking the media at boundary crossing. But haven't really thought this through of course ;)
I think it was a problem with the language as well as how they handle it internally. It was basically the algorithm that dictates how the language works, and consequently there was no way to have one material touch more than one other material. But I might misremember.
Anyway, I'm looking at this from the user's perspective. I wanted to do some physics-based ray-tracing with lenses and pbrt is what I ended up trying. As such, I really needed the multi-material aspect to work correctly. Also, it would be nice to be able to describe surfaces using a z=f(x,y) kind of formulation, or a way to place a hook in the renderer.
It's definitely an architectural problem as well. I do wonder if we could extend that though, without too much trouble for the general architecture - after all, the material does not necessarily need to represent all the outside materials and instead the ray only needs to be able to go from one medium to another. I'm happy to chat about possible extensions in that direction, although to be fair I wont have much time in the next weeks to sit down on anything like this. But, I do really hope that this can become a playground for ray tracing experiments in general!
I think maybe the easiest way to tackle the problem is to have the language describe surfaces instead of solid objects, and let every surface have a normal and two materials. This might be the most natural representation for a ray tracer.
honestly the AMD-first bit surprised me - usually ROCm support is an afterthought or just broken outright.
curious about BVH traversal specifically. dynamic dispatch patterns across GPU backends can get weird fast. did KernelAbstractions hold up there or were there vendor-specific fallbacks needed for the heavier acceleration structure work?
Well I'm a bit of an AMD "fanboy" and really dislike NVIDIA's vendor lock in. I'm not sure what you mean by dynamic dispatch across GPU backends - nothing should be dynamic there and most easier primitives map quite nicely between vendors (e.g. local memory, work groups etc). To be honest, the BVH/TLAS has been pretty simple in comparison to the wavefront infrastructure. We haven't done anything fancy yet, but the performance is still really good. I'm sure there are still lots of things we can do to improve performance, but right now I've concentrated on getting something usable out. Right now, we're mostly matching pbrt-v4 performance, but I couldn't compare to their NVIDIA only GPU acceleration without an NVIDIA gpu. I can just say that the performance is MUCH better than what I initially aimed for and it feels equally usable as some of the state of the art renderers I've been using. A 1:1 comparison is still missing though, since it's not easy to do a good comparison without comparing apples to oranges (already mapping materials and light types from one render to another is not trivial).
pbrt-v4 parity is a solid baseline - that codebase already leans hard on NVIDIA so a fair comparison was always going to be messy. surprised wavefront was the harder bit though, i'd have expected BVH tuning to be the nightmare.
To be fair I was suprised too. But I made a relatively simple straight port from the AMD rays sdk plus some input from the pbrt-v4 CPU bvh code and it just worked relatively well out of the box... This is the main intersection function which is quite simple: https://github.com/JuliaGeometry/Raycore.jl/blob/sd/multityp... I'm not even using local memory, since it was already fast enough ;) But I think we can still do quite a lot, large parts of the construction code are still very messy, and I want to polish and modularize the code over time.
I don't hear nearly as much about Julia as I used to. A few years ago the view was that it was about to replace Python as the language of choice for data science. Seems like that didn't happen?
I think the hype has slowed down, but all growth statistics haven't. Personally, I think Julia is the only language where I can implement something like Makie without running into a maintenance nightmare, and with Julia GPU programming is actually fun and high level and composes well, which I miss in most other languages. So, I dont really care about it replacing python or not. I do think for replacing python Julia will need to solve compilation latency, shipping AOT binaries and maybe interpret more of the glue code, which currently introduces quite a lot of compilation overhead without much gains in terms of performance.
I don't know about everyone else, but slow Julia compilation continues to cause me ongoing suffering to this day. I don't think they're ever going to "fix" this. On a standard GitHub Actions Windows worker, installing the public Julia packages I use, precompiling, and compiling the sysimage takes over an hour. That's not an exaggeration. I had to juice the worker up to a custom 4x sized worker to get the wall clock time to something reasonable.
It took me days to get that build to work; doing this compilation once in CI so you don't have to do it on every machine is trickier than it sounds in Julia. The "obvious" way (install packages in Docker, run container on target machine) does not work because Julia wants to see exactly the same machine that it was precompiled on. It ends up precompiling again every time you run the container on other machines. I nearly shed a tear the first time I got Julia not to precompile everything again on a new machine.
R and Python are done in five minutes on the standard worker and it was easy; it's just the amount of time it takes to download and extract the prebuilt binaries. Do that inside a Docker container and it's portable as expected. I maintain Linux and Windows environments for the three languages and Julia causes me the most headaches, by far. I absolutely do not care about the tiny improvement in performance from compiling for my particular microarch; I would opt into prebuilt x86_64 generic binaries if Julia had them. I'm very happy to take R's and Python's prebuilt binaries.
Versus Python, it seems to fork into the "thinkers" vs "doers" camp. Julia provides a level of abstraction that some people find comforting. I thought I could use it as a sort of open source Matlab for a lot of thinky, 1-based index code I had lying around. It didn't meet my needs. And "spend half an hour waiting for a Jupyter notebook to boot up" is real. Great for some but it's not compatible with the way I work.
Elsewhere someone used the term "janky" and perhaps it's the fact that there are so many incredibly smart people around it that makes it so janky. By way of example, somebody needed to check disk space and the architect told him to shell out to Python.
Remember when LLVM first came out and it got kudos for the quality of its error messages? Well if you miss the old-school 1980s GCC experience the nonsense that eventually comes out of the Julia compiler after an hour will relight that flame.
Want to use greek letters and other symbols that don't appear on your keyboard as variable names? You've found your people.
As someone who currently uses dabbles in both. That prediction seems a bit unrealistic. Julia is a fantastic language but it has some trade offs that need to be considered. Probably the most well known is `time to first x`. Julia like Python is used comfortably in notebooks but loading libraries can take a minute, compared to Python where it happens right away. It may lead you to not reach for it when you want to do quick testing of something especially plotting. You can mitigate this somewhat by loading all the libraries you'll ever need at startup (preferably long before you are ready to experiment) but that assumes you already know what libraries you'll need for what you're wanting to try.
What prediction? Maybe I need to rephrase what I said: My prediction is, that if Julia ever wants to have a shot at replacing Python, it absolutely has to solve the first time to first x problem! That's what I mean by shipping fully ahead of time compiled binaries and interpreting more glue code - which both have the potential to solve the first time to x problem.
The prediction I was referring to was the one in the parent comment. (The one I was commenting under)
Ah sorry :D
IMO it just had too many rough edges. Very slow compilation, correctness issues (https://yuri.is/not-julia/), kinda janky tooling (not nearly as bad as pip tbf). Even basic language mistakes like implicit variable declaration and 1-based indexing (in 2012??).
Yes 1-based indexing is a mistake. It leads to significantly less elegant code - especially for generic code - and is no harder to understand than 1-based indexing for people capable of programming. Fight me.
> Yes 1-based indexing is a mistake. It leads to significantly less elegant code - especially for generic code - and is no harder to understand than 1-based indexing for people capable of programming.
Some would argue that 0-based indexing is significantly less elegant for numerical/scientific code, but that depends on whether they come from a MATLAB/Fortran or Python/C(++) background.
A decision was made to target the MATLAB/Fortran (and unhappy? Python/C++) crowd first, thus the choice of 1-based indexing and column-major order, but at the end of the day it's a matter of personal preference.
0-based indexing would have made it easier to reach a larger audience, however.
> and is no harder to understand than 1-based indexing for people capable of programming.
The same could be said the other way around ;-)
Aside from the fact that 1-based indexing is better for scientific code (see Fortran), I don’t think that it matters very often. I don’t think that any Julia program I’ve ever written would need to change if Julia adopted 0-based tomorrow. You don’t typically write C-style loops in Julia; you use array functions and operators, and if you need to iterate you write `for i in array ...`. If you really need the first or last element you write `a[begin]` or `a[end]`.
> the fact that 1-based indexing is better for scientific code (see Fortran)
It really isn't. "Scientific code" isn't some separate thing.
The only way it can help is if you're trying to write code that matches equations in a paper that uses 1-based indexing. But that very minor advantage doesn't outweigh the disadvantages by a wide margin. Lean doesn't make this silly mistake.
lol. There's not much to fight since its a very personal problem how you want to write code. It's evident that all the capable programmers in the Julia community, have found satisfactory ways to get around it, so if you haven't yet, I don't see how that's a Julia problem ;) I can only say I haven't had a single problem with one based indexing in 12 years of developing Julia code. I also haven't run into many correctness issues compared to other languages I've been using. I think Yuri also has been using lots of packages which haven't been very mature. How on earth can you compare a 10 years old library with lots of maintainers with packages created in one year by one person? That's at least what Yuri's critic boils down to me.
On iOS Safari the videos are fullscreening themselves as I scroll. I've seen this on other blogs before but I don't know what causes it. Super annoying
Ugh, yeah I had some super weird bugs like this in safari, still haven't found the source :(
Don't quote me on this, but I think there is a "playsinline" / "webkit-playsinline" attribute for the video element you need to add to avoid that, + if it's autoplay you need to set "muted" too. I've also had this happen and I think both/either of those solved it last time.