Sometime in the 80s, I implemented the core of the Mandelbrot Set calculation using assembly on an 8087. As the article mentions, the compilers did math very inefficiently on this stack architecture. For example, if you multiplied two numbers together and then added a third, they would push the first two numbers, multiply, pop the result, push the result back onto the stack (perhaps clearing the stack? after 40 years I don't remember), push the third number, add, pop the result. For the Mandelbrot loop this was even worse, as it never kept the results of the loop. My assembly kept all the intermediate results on the stack for a 100x speed up.
Running this code, the 8087 emitted a high-pitched whine. I could tell when my code was broken and it had gone into an infinite loop by the sound. Which was convenient because, of course, there was no debugger.
Ah, lots of supposedly solid state computer stuff, including CPUs, did that. I, too, used it for debugging. This wasn't very conscious on my part, but if some whine became unusual and constant, it was often a sign of something hanging.
As I got older, not only did computers stop doing that, my hearing also got worse (entirely normal for my age, but still), so that's mostly a thing of the past.
The sound usually comes from inductors and capacitors in the power supply circuitry, not the ICs themselves as they draw pulses of power in patterns at audible frequencies. Modern CPUs and GPUs will still whine audibly if given a suitable load; the amount of current they consume is amazingly high, dozens to hundreds of amps, and also changing extremely quickly.
I had a Radeon 5850 that did it. I ran someone's simple test unity project with vsync disabled, was getting around 3000 fps, and heard a tone that was probably 3000hz. Supposedly the 5090 FE's are pretty bad too.
> - Forth people defined the IEEE754 standard on floating point, because they knew how to do that well in software.
IEEE 754 was principally developed by Kahan (in collaboration with his grad student, Coonen, and a visiting professor, Stone, whence the name KCS draft), none of whom were involved with Forth in any way that I am aware. And the history is pretty clear that the greatest influence on IEEE 754 before its release was Kahan's work with Intel developing the 8087.
There were a couple interesting points about the market for 8087 chips -- Intel designed the motherboard for the IBM PC, and they included an 8086 slot and a slot for either an 8087 or 8089. IBM didn't populate the slot for the coprocessor chip as it would compete with their mainframes, but Intel went around marketing the chips to research labs. One of them ended up with Stephen Fried who founded Microway in 1981 to create software for the 8087 and sell the chips, and the company is still in business after 44 years of chasing high performance computing. That's how I first got started with computing - a Microway Number Smasher (TM) card in an IBM PC.
The 80287 (AKA 287) and 80387 (AKA 387) floating point microprocessors started to pick up some competition from Weitek 1167 and 4167 chips and Inmos Transputer chips, so Intel integrated the FPU into the CPU with the 80486 processor (I question whether this was a monopoly move on Intel's part). This was also the first time that Intel made multiple versions of a CPU - there was a 486DX and a 486SX (colloquially referred to as the "sucks" model at the time) which disabled the FPU.
The 486 was also interesting because it was the first Intel x86 series chip to be able to operate at a multiple of the base frequency with the release of the DX2, DX3, and DX4 variants which allowed for different clock rates of 50MHz, 66MHz, 75MHz, and 100MHz based on the 25MHz and 33MHz base clock rates. I had a DX2-66MHz for a while and a DX4-100. The magic of these higher clock rates came from the introduction of the cache memory. The 486 was the first Intel CPU to utilize a cache.
Even though Intel had superseded the 8087/287/387 floating point coprocessor by including the latest version in the 80486, they introduced the 80860 (AKA i860) which was a VLIW RISC-based 64-bit FPU that was significantly faster, and also was the first microprocessor to exceed 1 million transistors.
The history of the FPU dedicated for special purpose applications is that it eventually became superseded by the GPU. Some of the first powerful GPUs from companies like Silicon Graphics utilized a number of i860 chips on a card in a very similar structure to more modern GPUs. You can think of each of the 12x i860 chips on an SGI Onyx / RealityEngine2 like a Streaming Multiprocessor node in an NVIDIA GPU.
Obviously, modern computers run at significantly faster clock speeds with significantly more cache and many kinds of cache, but it's good to look at the history of where these devices started to appreciate where we are now.
The 2-bit-per-transistor ROM using four transistor sizes is wild. Were there other chips from this era experimenting with semi-analog storage, or was the 8087 unusually aggressive here?
Looking at the complexity and area of hardware floating point, I often wonder why we don't see more unified combined integer+floating point units, like done in the R4200 [1], which reused most of the integer datapath while just adding a smaller extra smaller 12-bit datapath for the exponent.
If you happen to know... what was the reasoning behind the oddball stack architecture? It feels like Intel must have had this already designed for some other purpose so they tossed it in. I can't imagine why anyone would think this arch was a good idea.
Then again... they did try to force VLIW and APX on us so Intel has a history of "interesting" ideas about processor design.
edit: You addressed it in the article and I guess that's probably the reason but for real... what a ridiculous hand-wavy thing to do. Just assume it will be fine? If the anecdotes about Itanium/VLIW are true they committed the same sin on that project: some simulations with 50 instructions were the (claimed) basis for that fiasco. Methinks cutting AMD out of the market might have been the real reason but I have no proof for that.
Stack-based architectures have an appeal, especially for mathematics. (Think of the HP calculator.) And the explanation that they didn't have enough instruction bits also makes sense. (The co-processor uses 8086 "ESCAPE" instructions, but 5 bits get used up by the ESCAPE itself.) I think that the 8087's stack could have been implemented a lot better, but even so, there's probably a reason that hardly any other systems use a stack-based architecture. And the introduction of out-of-order execution made stacks even less practical.
x86 has a general pattern of encoding operands, the ModR/M byte(s), which gives you either two register operands, or a register and a memory operand. Intel also did this trick that uses one of the register operand for extra opcode bits, at the cost of sacrificing one of the operands.
There are 8 escape opcodes, and all of them have a ModR/M byte trailing it. If you use two-address instructions, that gives you just 8 instructions you can implement... not enough to do anything useful! But if you're happy with one-address instructions, you get 64 instructions with a register operand and 64 instructions with a memory operand.
A stack itself is pretty easy to compile for, until you have to spill a register because there's too many live variables on the stack. Then the spill logic becomes a nightmare. My guess is that the designers were thinking along these lines--organizing the registers in the stack is an efficient way to use the encoding space, and a fairly natural way to write expressions--and didn't have the expertise or the communication to realize that the design came with some edge cases that were painfully sharp to deal with.
I didn't expect the microcode to be at the center of the chip. I'd expect it on the side and only talking to the microcode engine, making more room for data traffic between chip halves. Also, the microcode is huge.
The microcode was so huge that they had to use a semi-analog ROM that held two bits per transistor by using four transistor sizes.
As far as the layout, the outputs from the microcode ROM are the control signals that go to all parts of the chip, so it makes sense to give it a central location. There's not a lot of communication between the upper half of the chip (the bus interface to the 8086 and memory) and the lower half of the chip (the 80-bit datapath), so it doesn't get in the way too much. That said, I've been tracing out the chip and there is a surprising amount of wiring to move signals around. The wiring in the 8087 is optimized to be as dense as possible: things like running some parallel signals in silicon and some in polysilicon because the lines can get squeezed together just a bit more that way.
My compiler knowledge is limited, but I think that you end up with the same parse tree at a very early level of processing, whether you use Reverse-Polish notation or inline notation. So I don't think a language change would make a difference.
I remember failing an interview with the optimization team of a large fruit trademarked computer maker because I couldn't explain why the x87 stack was a bad design. TBF they were looking for someone with a masters, not someone just graduating with a BS. But, now I know... honestly, I'm still not 100% sure what they were looking for in an answer. I assume something about register renaming. memory, and cycle efficiency.
Having given a zillion interviews, I expect that they weren't looking for the One True Answer, but were interested in seeing if you discussed plausible reasons in an informed way, as well as seeing what areas you focused on (e.g., do you discuss compiler issues or architecture issues). Saying "I dunno" is bad, especially after hints like "what about ..." and spouting complete nonsense is also bad.
(I'm just commenting on interviews in general, and this is in no way a criticism of your response.)
I think I said something about the stack efficiency. I was a kid that barely understood out of order execution. Register renaming and the rest was well beyond me. It was also a long time ago, so recollections are fuzzy. But, I do recall is they didn't prompt anything. I suspect the only reason I got the interview is I had done some SSE programming (AVX didn't exist yet, and to give timing context AltiVec was discussed), and they figured if I was curious enough to do that I might not be garbage.
Edit: Jogging my memory I believe they were explicit at the end of the interview they were looking for a Masters candidate. They did say I was on a good path IIRC. It wasn't a bad interview, but I was very clearly not what they were looking for.
It's all about that 80-bit/82-bit floating point format with the explicit mantissa bit just to be extra different. ;) Not only is it a 1:15:1:63, it's (2(tag)):1:15:1:63, whereas binary64 is 1:11:0:52. (sign:exponent [biased]:explicit leading mantissa bit stored?:manitissa remaining)
Other pre-P5 ISA idiosyncrasies: Only the 8087 has FDISI/FNDISI, FENI/FNENI. Only the plain 287 has a functional FSETPM. Most everything else looks like a 387 ISA-wise, more or less until MMX arrived. That's all I know.
I'm curious what the CX-83D87 and Weiteks look like.
Keep up the good work!
PS: Perhaps sometime in the (near) future we might get almost 1:1 silicon "OCR" transcription of die scans to FPGA RTL with bugs and all?
> I'm curious what the CX-83D87 and Weiteks look like.
The Weitek's were memory mapped. (At least those built for x86 machines.).
This essentially increased bandwidth by using the address bus as a source for floating point instructions. Was really a very cool idea, although I don't know what the performance realities were when using one.
The operand fields of a WTL 3167 address have been
specifically designed so that a WTL 3167 address can
be given as either the source or the destination to a
REP MOVSD instruction. [
Single-precision vector arithmetic is accomplished by
applying the 80386 block move instruction REP
MOVSD to a WTL 3167 address involving arithmetic
instead of loading or storing.
haha - took me a while to figure out that's Mauro Bonomi's signature
iirc the 3167 was a single clocked, full barrel shift mac pipeline with a bunch (64?) of registers, so the FPU could be driven with a RISC-style opcode on every address bus clock (given the right driver on the CPU) ... the core registers were enough to run inner loops (think LINPACK) very fast with some housekeeping on context switch of course
this window sat between full PCB minicomputer FPUs made from TTL and the decoupling of microcomputer internal clocks & cache from address bus rates ...
Weitek tried to convert their FPU base into an integrated FPU/CPU play during the RISC wars, but lost
This is cool, but the renormalization and (Programmable and bidirectional) barrel shifter are of much more interest.
I had a 10Mhz XT, and ran a 8087-8 at a bit higher clock rate. I used it both for Lotus 1-2-3 and Turbo Pascal-87. It made Turbo Pascal significantly faster.
Sometime in the 80s, I implemented the core of the Mandelbrot Set calculation using assembly on an 8087. As the article mentions, the compilers did math very inefficiently on this stack architecture. For example, if you multiplied two numbers together and then added a third, they would push the first two numbers, multiply, pop the result, push the result back onto the stack (perhaps clearing the stack? after 40 years I don't remember), push the third number, add, pop the result. For the Mandelbrot loop this was even worse, as it never kept the results of the loop. My assembly kept all the intermediate results on the stack for a 100x speed up.
Running this code, the 8087 emitted a high-pitched whine. I could tell when my code was broken and it had gone into an infinite loop by the sound. Which was convenient because, of course, there was no debugger.
Thanks for bringing back this memory.
Ah, lots of supposedly solid state computer stuff, including CPUs, did that. I, too, used it for debugging. This wasn't very conscious on my part, but if some whine became unusual and constant, it was often a sign of something hanging.
As I got older, not only did computers stop doing that, my hearing also got worse (entirely normal for my age, but still), so that's mostly a thing of the past.
The sound usually comes from inductors and capacitors in the power supply circuitry, not the ICs themselves as they draw pulses of power in patterns at audible frequencies. Modern CPUs and GPUs will still whine audibly if given a suitable load; the amount of current they consume is amazingly high, dozens to hundreds of amps, and also changing extremely quickly.
I had a Radeon 5850 that did it. I ran someone's simple test unity project with vsync disabled, was getting around 3000 fps, and heard a tone that was probably 3000hz. Supposedly the 5090 FE's are pretty bad too.
The compilers available at the time that the 8087 was commonplace were overall horrible and easily beaten anyway.
On the other hand, skilled humans can do very very well with the x87; this 256-byte demo makes use of it excellently: https://www.pouet.net/prod.php?which=53816
- You can do the Mandelbrot set with integers. In Forth it's 6 lines.
- Coincidentally, Forth promotes a fixed point philosophy.
- Forth people defined the IEEE754 standard on floating point, because they knew how to do that well in software.
> - Forth people defined the IEEE754 standard on floating point, because they knew how to do that well in software.
IEEE 754 was principally developed by Kahan (in collaboration with his grad student, Coonen, and a visiting professor, Stone, whence the name KCS draft), none of whom were involved with Forth in any way that I am aware. And the history is pretty clear that the greatest influence on IEEE 754 before its release was Kahan's work with Intel developing the 8087.
There were a couple interesting points about the market for 8087 chips -- Intel designed the motherboard for the IBM PC, and they included an 8086 slot and a slot for either an 8087 or 8089. IBM didn't populate the slot for the coprocessor chip as it would compete with their mainframes, but Intel went around marketing the chips to research labs. One of them ended up with Stephen Fried who founded Microway in 1981 to create software for the 8087 and sell the chips, and the company is still in business after 44 years of chasing high performance computing. That's how I first got started with computing - a Microway Number Smasher (TM) card in an IBM PC.
The 80287 (AKA 287) and 80387 (AKA 387) floating point microprocessors started to pick up some competition from Weitek 1167 and 4167 chips and Inmos Transputer chips, so Intel integrated the FPU into the CPU with the 80486 processor (I question whether this was a monopoly move on Intel's part). This was also the first time that Intel made multiple versions of a CPU - there was a 486DX and a 486SX (colloquially referred to as the "sucks" model at the time) which disabled the FPU.
The 486 was also interesting because it was the first Intel x86 series chip to be able to operate at a multiple of the base frequency with the release of the DX2, DX3, and DX4 variants which allowed for different clock rates of 50MHz, 66MHz, 75MHz, and 100MHz based on the 25MHz and 33MHz base clock rates. I had a DX2-66MHz for a while and a DX4-100. The magic of these higher clock rates came from the introduction of the cache memory. The 486 was the first Intel CPU to utilize a cache.
Even though Intel had superseded the 8087/287/387 floating point coprocessor by including the latest version in the 80486, they introduced the 80860 (AKA i860) which was a VLIW RISC-based 64-bit FPU that was significantly faster, and also was the first microprocessor to exceed 1 million transistors.
The history of the FPU dedicated for special purpose applications is that it eventually became superseded by the GPU. Some of the first powerful GPUs from companies like Silicon Graphics utilized a number of i860 chips on a card in a very similar structure to more modern GPUs. You can think of each of the 12x i860 chips on an SGI Onyx / RealityEngine2 like a Streaming Multiprocessor node in an NVIDIA GPU.
Obviously, modern computers run at significantly faster clock speeds with significantly more cache and many kinds of cache, but it's good to look at the history of where these devices started to appreciate where we are now.
Story of how Intel-derived proposal was standardized as IEEE754: https://people.eecs.berkeley.edu/~wkahan/ieee754status/754st...
The 2-bit-per-transistor ROM using four transistor sizes is wild. Were there other chips from this era experimenting with semi-analog storage, or was the 8087 unusually aggressive here?
Intel also used the 2-bit-per-transistor ROM in the iAPX 432, their unsuccessful "micro-maninframe" chip.
Nowadays, flash uses multiple voltage levels to store four bits per cell (QLC, Quad Level Cell), which is a similar concept.
I wrote a whole blog post about the 2-bit-per-transistor technique, back in 2018: https://www.righto.com/2018/09/two-bits-per-transistor-high-...
Looking at the complexity and area of hardware floating point, I often wonder why we don't see more unified combined integer+floating point units, like done in the R4200 [1], which reused most of the integer datapath while just adding a smaller extra smaller 12-bit datapath for the exponent.
[1] https://en.wikipedia.org/wiki/R4200
The integer pipeline is often needed for address calculation near the same time as the floating point pipeline.
The R4200 FPU performance suffered for this reason.
Author here for your 8087 questions...
If you happen to know... what was the reasoning behind the oddball stack architecture? It feels like Intel must have had this already designed for some other purpose so they tossed it in. I can't imagine why anyone would think this arch was a good idea.
Then again... they did try to force VLIW and APX on us so Intel has a history of "interesting" ideas about processor design.
edit: You addressed it in the article and I guess that's probably the reason but for real... what a ridiculous hand-wavy thing to do. Just assume it will be fine? If the anecdotes about Itanium/VLIW are true they committed the same sin on that project: some simulations with 50 instructions were the (claimed) basis for that fiasco. Methinks cutting AMD out of the market might have been the real reason but I have no proof for that.
Stack-based architectures have an appeal, especially for mathematics. (Think of the HP calculator.) And the explanation that they didn't have enough instruction bits also makes sense. (The co-processor uses 8086 "ESCAPE" instructions, but 5 bits get used up by the ESCAPE itself.) I think that the 8087's stack could have been implemented a lot better, but even so, there's probably a reason that hardly any other systems use a stack-based architecture. And the introduction of out-of-order execution made stacks even less practical.
To expand on this a little bit more:
x86 has a general pattern of encoding operands, the ModR/M byte(s), which gives you either two register operands, or a register and a memory operand. Intel also did this trick that uses one of the register operand for extra opcode bits, at the cost of sacrificing one of the operands.
There are 8 escape opcodes, and all of them have a ModR/M byte trailing it. If you use two-address instructions, that gives you just 8 instructions you can implement... not enough to do anything useful! But if you're happy with one-address instructions, you get 64 instructions with a register operand and 64 instructions with a memory operand.
A stack itself is pretty easy to compile for, until you have to spill a register because there's too many live variables on the stack. Then the spill logic becomes a nightmare. My guess is that the designers were thinking along these lines--organizing the registers in the stack is an efficient way to use the encoding space, and a fairly natural way to write expressions--and didn't have the expertise or the communication to realize that the design came with some edge cases that were painfully sharp to deal with.
insightful footnote, thanks: https://web.archive.org/web/20190301193516/http://www.drdobb...
Make no mistake, this article is of extraordinary historical significance, even the list of constantans being hardwired....
I didn't expect the microcode to be at the center of the chip. I'd expect it on the side and only talking to the microcode engine, making more room for data traffic between chip halves. Also, the microcode is huge.
The microcode was so huge that they had to use a semi-analog ROM that held two bits per transistor by using four transistor sizes.
As far as the layout, the outputs from the microcode ROM are the control signals that go to all parts of the chip, so it makes sense to give it a central location. There's not a lot of communication between the upper half of the chip (the bus interface to the 8086 and memory) and the lower half of the chip (the 80-bit datapath), so it doesn't get in the way too much. That said, I've been tracing out the chip and there is a surprising amount of wiring to move signals around. The wiring in the 8087 is optimized to be as dense as possible: things like running some parallel signals in silicon and some in polysilicon because the lines can get squeezed together just a bit more that way.
I wonder, if C used Reverse-Polish notation for math operations, would compilers have been able to target the 8087 better than they did?
My compiler knowledge is limited, but I think that you end up with the same parse tree at a very early level of processing, whether you use Reverse-Polish notation or inline notation. So I don't think a language change would make a difference.
Converting to RPN is, roughly speaking, the easiest way to generate code for either register or stack architectures.
Once you have a parse tree, visiting it in post order (left tree, right tree, operation) produces the RPN.
Looks like a log multiply-adder ... maybe a 5 clock cycle? Also, on the microcode ... them FP divide algorithms are pretty intense.
Would be cool to hear a real designer compare to the Weitek 1064.
I remember failing an interview with the optimization team of a large fruit trademarked computer maker because I couldn't explain why the x87 stack was a bad design. TBF they were looking for someone with a masters, not someone just graduating with a BS. But, now I know... honestly, I'm still not 100% sure what they were looking for in an answer. I assume something about register renaming. memory, and cycle efficiency.
Having given a zillion interviews, I expect that they weren't looking for the One True Answer, but were interested in seeing if you discussed plausible reasons in an informed way, as well as seeing what areas you focused on (e.g., do you discuss compiler issues or architecture issues). Saying "I dunno" is bad, especially after hints like "what about ..." and spouting complete nonsense is also bad.
(I'm just commenting on interviews in general, and this is in no way a criticism of your response.)
I think I said something about the stack efficiency. I was a kid that barely understood out of order execution. Register renaming and the rest was well beyond me. It was also a long time ago, so recollections are fuzzy. But, I do recall is they didn't prompt anything. I suspect the only reason I got the interview is I had done some SSE programming (AVX didn't exist yet, and to give timing context AltiVec was discussed), and they figured if I was curious enough to do that I might not be garbage.
Edit: Jogging my memory I believe they were explicit at the end of the interview they were looking for a Masters candidate. They did say I was on a good path IIRC. It wasn't a bad interview, but I was very clearly not what they were looking for.
Very cool.
It's all about that 80-bit/82-bit floating point format with the explicit mantissa bit just to be extra different. ;) Not only is it a 1:15:1:63, it's (2(tag)):1:15:1:63, whereas binary64 is 1:11:0:52. (sign:exponent [biased]:explicit leading mantissa bit stored?:manitissa remaining)
Other pre-P5 ISA idiosyncrasies: Only the 8087 has FDISI/FNDISI, FENI/FNENI. Only the plain 287 has a functional FSETPM. Most everything else looks like a 387 ISA-wise, more or less until MMX arrived. That's all I know.
I'm curious what the CX-83D87 and Weiteks look like.
Keep up the good work!
PS: Perhaps sometime in the (near) future we might get almost 1:1 silicon "OCR" transcription of die scans to FPGA RTL with bugs and all?
> I'm curious what the CX-83D87 and Weiteks look like.
The Weitek's were memory mapped. (At least those built for x86 machines.).
This essentially increased bandwidth by using the address bus as a source for floating point instructions. Was really a very cool idea, although I don't know what the performance realities were when using one.
http://www.bitsavers.org/components/weitek/dataSheets/WTL-31...
This is nuts, in the best way.
The operand fields of a WTL 3167 address have been specifically designed so that a WTL 3167 address can be given as either the source or the destination to a REP MOVSD instruction. [
Single-precision vector arithmetic is accomplished by applying the 80386 block move instruction REP MOVSD to a WTL 3167 address involving arithmetic instead of loading or storing.
haha - took me a while to figure out that's Mauro Bonomi's signature
iirc the 3167 was a single clocked, full barrel shift mac pipeline with a bunch (64?) of registers, so the FPU could be driven with a RISC-style opcode on every address bus clock (given the right driver on the CPU) ... the core registers were enough to run inner loops (think LINPACK) very fast with some housekeeping on context switch of course
this window sat between full PCB minicomputer FPUs made from TTL and the decoupling of microcomputer internal clocks & cache from address bus rates ...
Weitek tried to convert their FPU base into an integrated FPU/CPU play during the RISC wars, but lost
This is cool, but the renormalization and (Programmable and bidirectional) barrel shifter are of much more interest.
I had a 10Mhz XT, and ran a 8087-8 at a bit higher clock rate. I used it both for Lotus 1-2-3 and Turbo Pascal-87. It made Turbo Pascal significantly faster.
You're in luck, I wrote about the 8087's shifter back in 2020 :-) https://www.righto.com/2020/05/die-analysis-of-8087-math-cop...